Substrate processing device, and information processing method in substrate processing device

Abstract

To efficiently recognize the posture of a component.SOLUTION: A substrate processing device acquires reference shape information about a two-dimensional shape of a 3D model in a virtual image that can be acquired by photographing the 3D model of a search object component about each of a plurality of first virtual camera positions located along a plurality of virtual spherical surfaces surrounding the 3d model of the search object component and corresponding to one or more initial setting ranges to the search object component regulated by setting range information on the basis of three-dimensional design information related to the search object component among one or more object components and the setting range information for regulating one or more initial setting ranges related to a virtual camera position to a preset object component. It calculates a numerical value indicating a coincidence degree between real shape information and reference shape information about a two-dimensional shape of an object in a real image obtained by acquiring the search object component about each first virtual camera position. It detects a virtual camera position at which the coincidence degree between the real shape information and the reference shape information is the largest among the plurality of first virtual camera positions.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for recognizing the posture of a component in a substrate processing apparatus that processes a substrate. Substrates to be processed in the substrate processing apparatus include, for example, semiconductor substrates, substrates for flat panel displays (FPDs) such as liquid crystal displays or organic electroluminescence (EL) displays, glass substrates for photomasks, substrates for optical disks, substrates for magnetic disks, and substrates for solar cells. [Background technology]

[0002] Some substrate processing apparatuses include a processing chamber, a substrate holder, a nozzle, a camera, an image processor, and a monitor (see, for example, Patent Document 1).

[0003] In this substrate processing apparatus, a substrate holding unit, a nozzle, and a camera are disposed in a processing chamber. The substrate holding unit holds the substrate in a horizontal position. The substrate holding unit rotates the substrate in a horizontal plane. The nozzle moves between a standby position off to the side of the substrate and a discharge position above the substrate by pivoting a drive arm to which the nozzle is fixed. The nozzle is disposed in the standby position when the substrate is attached to or detached from the substrate holding unit, and is disposed in the discharge position when the processing liquid is discharged from the nozzle toward the substrate. The camera is attached at a predetermined position in the processing chamber, and captures an image of a predetermined area including the nozzle that has moved to the discharge position. The image processing unit obtains second nozzle position information indicating the position of the nozzle based on an image from the camera, and outputs the second nozzle position information to a monitoring unit. The monitoring unit determines whether or not there is an abnormality in the position of the nozzle based on the correspondence between the first nozzle position information as information directly or indirectly indicating the arrangement position of the nozzle from the control unit and the second nozzle position information from the image processing unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 146456 Summary of the Invention [Problem to be solved by the invention]

[0005] However, there is room for improvement in substrate processing apparatuses in terms of efficiently recognizing the orientation of components. [Means for solving the problem]

[0006] A substrate processing apparatus according to a first aspect is a substrate processing apparatus for processing substrates, and includes: one or more photographing units that photograph an actual image capturing one or more target parts and a search target part that is any one of the one or more target parts; a search processing unit that executes a search process to search for a maximum matching virtual camera position among the plurality of virtual camera positions, the maximum matching virtual camera position having the greatest matching between the reference shape information and the actual shape information, based on reference shape information about a two-dimensional shape of a three-dimensional model in each of a plurality of virtual images that can be obtained by photographing a three-dimensional model of the search target part from a plurality of virtual camera positions, the reference shape information being generated based on three-dimensional design information related to the search target part, and actual shape information about a two-dimensional shape of an object in the actual images; and a storage unit that stores setting range information that specifies one or more initial setting ranges related to a virtual camera position that is preset for each of at least one target part among the one or more target parts, the search processing unit the first virtual camera position acquisition unit acquires the reference shape information generated for each of a plurality of first virtual camera positions, the reference shape information being generated for each of the plurality of first virtual camera positions, the reference shape information being generated for each of the plurality of first virtual camera positions, the reference shape information being generated for each of the plurality of first virtual camera positions, the reference shape information being generated for each of the plurality of first virtual camera positions, the reference shape information being generated for each of the plurality of first virtual camera positions, the reference shape information being generated for each of the plurality of first virtual camera positions, and a first detection unit detects, based on a result of the calculation by the first calculation unit, a high-matching virtual camera position among the plurality of first virtual camera positions, which has the highest match between the actual shape information and the reference shape information, and the search processing unit searches for the highest-matching virtual camera position based on the high-matching virtual camera position detected by the first detection unit, or the high-matching virtual camera position is the highest-matching virtual camera position.

[0007] A substrate processing apparatus according to a second aspect is a substrate processing apparatus according to the first aspect, wherein each of the one or more initial setting ranges includes a first numerical range which is a range of distance from the reference point, a second numerical range which is a range of latitude about a virtual sphere centered on the reference point and surrounding the three-dimensional model, and a third numerical range which is a range of longitude about the virtual sphere.

[0008] A substrate processing apparatus according to a third aspect is the substrate processing apparatus according to the second aspect, wherein the first shape information acquisition unit virtually sets T1 (T1 is a natural number of 2 or more) surface aggregates, each of which includes a plurality of virtual surfaces positioned along a virtual sphere surrounding the three-dimensional model with the reference point as a center, and has different distances from the reference point according to the first numerical range in the at least one initial setting range, based on the three-dimensional design information and the setting range information, and further sets T1 (T1 is a natural number of 2 or more) surface aggregates, each of which includes a plurality of virtual surfaces positioned along a virtual sphere surrounding the three-dimensional model with the reference point as a center, the distances from the reference point according to the first numerical range in the at least one initial setting range being different from each other, and Assuming a case where the three-dimensional model is photographed from each of M1×T1 first virtual camera positions which are the plurality of first virtual camera positions virtually set for each of M1 (M1 is a natural number equal to or greater than 2) virtual surfaces in each of the T1 surface collections corresponding to each of the ranges, the reference shape information generated for each of the M1×T1 first virtual camera positions is acquired, and the first calculation unit calculates a numerical value indicating a degree of agreement between the actual shape information and the reference shape information for each of the M1×T1 first virtual camera positions. the first detection unit detects, based on a calculation result by the first calculation unit, the high-matching virtual camera position, which is a first virtual camera position having the highest match between the actual shape information and the reference shape information, among the M1×T1 first virtual camera positions, and the search processing unit detects, based on a calculation result by the first calculation unit, a line that includes a high-matching virtual surface, which is a virtual surface on which the high-matching virtual camera position is virtually set, among the M1 virtual surfaces in each of the T1 surface sets, and that passes through the reference point and the high-matching virtual camera position on the side of the high-matching virtual camera position relative to the reference point, a division surface generation unit that generates M2×T2 virtual division surfaces by dividing each of T2 (T2 is a natural number equal to or greater than 2) virtual surfaces that intersect with the line and have mutually different distances from the reference point using the same rule, and generating M2 (M2 is a natural number equal to or greater than 2) virtual division surfaces for each of the T2 virtual surfaces; and assuming a case in which the three-dimensional model is photographed from each of M2×T2 second virtual camera positions that are M2×T2 virtual camera positions that are virtually set for each of the M2×T2 virtual division surfaces based on the three-dimensional design information,The system includes a second shape information generation unit that generates the reference shape information for each of the M2×T2 second virtual camera positions, a second calculation unit that calculates a numerical value indicating a degree of agreement between the actual shape information and the reference shape information for each of the M2×T2 second virtual camera positions, and a second detection unit that detects the virtual camera position having the highest degree of agreement between the actual shape information and the reference shape information among the high agreement virtual camera position and the M2×T2 second virtual camera positions, wherein the search processing unit searches for the maximum agreement virtual camera position based on the virtual camera position detected by the second detection unit, or the virtual camera position detected by the second detection unit is the maximum agreement virtual camera position.

[0009] A substrate processing apparatus according to a fourth aspect is the substrate processing apparatus according to the third aspect, wherein the search processing unit executes a first unit process for the search target part, and then executes one or more n-th unit processes (n is a natural number equal to or greater than 2), and the search processing unit sequentially executes a 1A process, a 1B process, a 1C process, and a 1D process in the first unit process, and the 1A process includes a first reference virtual camera position which is a virtual camera position which is first detected by the second detection unit and which is generated by the divided surface generation unit by dividing each of the T2 virtual surfaces. a first virtual division surface including a virtual division surface including the reference point, intersecting a straight line passing through the reference point and the first reference virtual camera position on the first reference virtual camera position side of the reference point, and having different distances from the reference point, the first virtual division surface being divided by the same rule to generate first M3 virtual division surfaces which are M3 (M3 is a natural number of 2 or more) virtual division surfaces for each of the first T3 virtual division surfaces, thereby generating first M3×T3 virtual division surfaces which are M3×T3 virtual division surfaces. The first processing is a process for generating a cutting surface by the second shape information generation unit, based on the three-dimensional design information, assuming a case in which the three-dimensional model is photographed from each of the first M3×T3 third virtual camera positions which are M3×T3 virtual camera positions virtually set for each of the first M3×T3 virtual dividing surfaces, and the first processing is a process for generating the reference shape information for each of the first M3×T3 third virtual camera positions by the second calculation unit, based on the three-dimensional design information, assuming a case in which the three-dimensional model is photographed from each of the first M3×T3 third virtual camera positions which are M3×T3 virtual camera positions virtually set for each of the first M3×T3 virtual dividing surfaces. the first D process is a process of detecting, by the second detection unit, a second reference virtual camera position which is a virtual camera position having the greatest degree of matching between the actual shape information and the reference shape information, among the first reference virtual camera position and the first M3×T3 third virtual camera positions, and the search processing unit performs an n-th A process, an n-th B process, an n-th C process, and an n-th D process in that order in the n-th unit process, and the n-th A process isthe n-1th T3 virtual dividing surfaces are generated by the dividing surface generating unit by dividing each of the n-1th T3 virtual dividing surfaces, and the n-th T3 virtual dividing surfaces are T3 virtual dividing surfaces that include a virtual dividing surface including an n-th reference virtual camera position that is the virtual camera position that is the n-th detected by the second detection unit, intersect with a straight line passing through the reference point and the n-th reference virtual camera position on the n-th reference virtual camera position side of the reference point, and have mutually different distances from the reference point, and the n-th T3 virtual dividing surfaces are generated by dividing each of the n-th T3 virtual dividing surfaces by the same rule, and n-th M3 virtual dividing surfaces are generated for each of the n-th T3 virtual dividing surfaces, which are M3 virtual dividing surfaces, and the nB process is virtually set for each of the n-th M3×T3 virtual dividing surfaces by the second shape information generating unit based on the three-dimensional design information. the nC process is a process of generating the reference shape information for each of the nth M3×T3 third virtual camera positions, assuming a case where the three-dimensional model is photographed from each of the nth M3×T3 third virtual camera positions, the nC process is a process of calculating, by the second calculation unit, a numerical value indicating a degree of agreement between the actual shape information and the reference shape information for each of the nth M3×T3 third virtual camera positions, the nD process is a process of detecting, by the second detection unit, a virtual camera position having the highest degree of agreement between the actual shape information and the reference shape information among the nth reference virtual camera position and the nth M3×T3 third virtual camera positions, and the virtual camera position detected by the second detection unit in the last nD process in the one or more nth unit processes is the highest degree of agreement virtual camera position.

[0010] A substrate processing apparatus according to a fifth aspect is a substrate processing apparatus according to the fourth aspect, wherein the search processing unit terminates execution of the one or more nth unit processes in response to the second detection unit detecting one of the reference virtual camera positions from the first reference virtual camera position to the nth reference virtual camera position consecutively a first predetermined number of times.

[0011] A substrate processing apparatus according to a sixth aspect is a substrate processing apparatus according to the fourth aspect, wherein the search processing unit terminates execution of the nth unit process one or more times in response to the nth unit process having been executed a second predetermined number of times that is set in advance.

[0012] A substrate processing apparatus according to a seventh aspect is a substrate processing apparatus according to any one of the third to sixth aspects, wherein the same rule includes a rule for dividing the surface to be divided into a plurality of surfaces by a plurality of line segments respectively connecting a center point of the surface to be divided to all of the vertices of the surface to be divided.

[0013] A substrate processing apparatus according to an eighth aspect is a substrate processing apparatus according to any one of the third to seventh aspects, wherein each of the plurality of virtual surfaces is a triangular surface, and the surface collection is a polyhedron composed of a plurality of triangular surfaces.

[0014] A substrate processing apparatus according to a ninth aspect is a substrate processing apparatus according to the eighth aspect, wherein the dividing surface generation unit divides the high-matching virtual surface into three virtual dividing surfaces as the M2 virtual dividing surfaces by three line segments respectively connecting three vertices in the high-matching virtual surface and the high-matching virtual camera position.

[0015] A substrate processing apparatus according to a tenth aspect is the substrate processing apparatus according to any one of the first to ninth aspects, further comprising a moving mechanism for moving the search target part between a first position and a second position, the one or more initial setting ranges including a first initial setting range for the first position, and the search processing unit, at a timing when the moving mechanism positions the search target part at the first position, performs, based on the three-dimensional design information and the setting range information, by the first shape information acquisition unit, acquiring the complex images of the three-dimensional model corresponding to the first initial setting range, assuming a case where the three-dimensional model is photographed from each of the plurality of first virtual camera positions corresponding to the first initial setting range, which is the at least one initial setting range. the first calculation unit calculates, for each of the plurality of first virtual camera positions according to the first initial setting range, a numerical value indicating a degree of match between the reference shape information and the actual shape information relating to the actual image obtained by photographing with the one or more photographing units at a timing when the moving mechanism positions the search target part at the first position, and the high-match virtual camera position having the greatest degree of match between the actual shape information and the reference shape information, among the plurality of first virtual camera positions according to the first initial setting range, based on a result of the calculation by the first calculation unit.

[0016] A substrate processing apparatus according to an eleventh aspect is the substrate processing apparatus according to the tenth aspect, wherein the one or more initial setting ranges further include a second initial setting range for the second position, and the search processing unit generates, by the first shape information acquisition unit, a plurality of first virtual camera positions corresponding to the second initial setting range, based on the three-dimensional design information and the setting range information, assuming a case in which the three-dimensional model is photographed from each of the plurality of first virtual camera positions corresponding to the second initial setting range, which is the at least one initial setting range, at a timing when the moving mechanism positions the search target part at the second position. the first calculation unit calculates a numerical value indicating the degree of match between the reference shape information and the actual shape information relating to the actual image obtained by photographing with the one or more photographing units at the timing when the moving mechanism positions the search target part at the second position for each of the multiple first virtual camera positions corresponding to the second initial setting range, and the first detection unit detects the high-match virtual camera position among the multiple first virtual camera positions corresponding to the second initial setting range, which has the greatest degree of match between the actual shape information and the reference shape information, based on the calculation result by the first calculation unit.

[0017] The substrate processing apparatus of the 12th aspect is the substrate processing apparatus of the 10th or 11th aspect, further comprising an abnormality detection unit that detects an abnormality in the search target part if the degree of match between the reference shape information and the actual shape information for the maximum degree of match virtual camera position detected by the search processing unit is smaller than a predetermined degree of match set in advance.

[0018] A substrate processing apparatus according to a thirteenth aspect is the substrate processing apparatus according to the tenth aspect, wherein the one or more initial setting ranges further include a second initial setting range for the second position, and the search processing unit, at a timing when the moving mechanism positions the search target part at the first position, acquires the reference shape information generated by the first shape information acquisition unit for each of the plurality of first virtual camera positions corresponding to both the first initial setting range and the second initial setting range based on the three-dimensional design information and the setting range information, assuming a case where the three-dimensional model is photographed from each of the plurality of first virtual camera positions corresponding to both the first initial setting range and the second initial setting range, which are the at least one initial setting range, and acquires the reference shape information generated by the first calculation unit for each of the plurality of first virtual camera positions corresponding to both the first initial setting range and the second initial setting range, For each of the multiple first virtual camera positions, a numerical value indicating the degree of match between the actual shape information relating to the actual image obtained by shooting by the one or more shooting units at the timing when the search target part is positioned at the first position by the moving mechanism is calculated, and the first detection unit detects the high-match virtual camera position among the multiple first virtual camera positions corresponding to both the first initial setting range and the second initial setting range based on the calculation result by the first calculation unit, which has the highest degree of match between the actual shape information and the reference shape information, and the substrate processing apparatus further includes an abnormality detection unit that detects an abnormality in the search target part if the maximum-match virtual camera position is a position belonging to the second initial setting range at the timing when the search target part is positioned at the first position by the moving mechanism.

[0019] A substrate processing apparatus according to a fourteenth aspect is a substrate processing apparatus according to any one of the first to eleventh aspects, and further includes an abnormality detection unit that detects an abnormality in the search target part by comparing actual information related to the posture of the search target part recognized based on the maximum matching virtual camera position with normal information related to the posture of the search target part based on the three-dimensional design information when the state of the search target part is normal.

[0020] A substrate processing apparatus according to a fifteenth aspect is a substrate processing apparatus according to any one of the first to fourteenth aspects, wherein the one or more target parts include two or more target parts, and the at least one target part includes the two or more target parts.

[0021] An information processing method in a substrate processing apparatus according to a sixteenth aspect is an information processing method in a substrate processing apparatus having one or more target parts, the information processing method comprising: a storage step of storing in a storage unit setting range information defining one or more initial setting ranges relating to a virtual camera position preset for each of at least one of the one or more target parts; a photographing step of acquiring an actual image capturing a search target part, which is any one of the one or more target parts, by photographing with a photographing unit; and a search step of executing a search process by a calculation unit to search for a maximum matching virtual camera position among the plurality of virtual camera positions, the maximum matching virtual camera position having the greatest matching between the reference shape information and the actual shape information, based on reference shape information about a two-dimensional shape of a three-dimensional model in each of a plurality of virtual images that can be acquired by photographing a three-dimensional model of the search target part from a plurality of virtual camera positions, the reference shape information being generated based on three-dimensional design information about the search target part, and actual shape information about a two-dimensional shape of an object in the actual image obtained in the photographing step. a first shape information acquisition step of acquiring the reference shape information generated for each of a plurality of first virtual camera positions based on the design information and the setting range information, assuming a case in which the three-dimensional model is photographed from each of a plurality of first virtual camera positions that are positioned along a plurality of virtual spheres that surround the three-dimensional model with a reference point of the three-dimensional model as a center and correspond to at least one of the one or more initial setting ranges for the search target part defined in the setting range information; a first calculation step of calculating a numerical value indicating a degree of agreement between the reference shape information and the actual shape information for each of the plurality of first virtual camera positions acquired by the first shape information acquisition step; and a first detection step of detecting, based on a calculation result by the first calculation step, a high-match virtual camera position having the highest degree of agreement between the actual shape information and the reference shape information among the plurality of first virtual camera positions,Alternatively, the high matching virtual camera position is the maximum matching virtual camera position. Effect of the Invention

[0022] According to the substrate processing apparatus of the first aspect, when the range of the position and orientation of the search target part is limited, the amount of calculation in the search process can be reduced by limiting the range in which the multiple first virtual camera positions are set. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, the orientation of the part can be recognized efficiently in the substrate processing apparatus.

[0023] According to the substrate processing apparatus of the second aspect, when the ranges in which the angular relationship and the distance between the photographing unit and the search target part change are limited, the range in which the multiple first virtual camera positions are set can be limited. This can reduce the amount of calculation in the search process. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, the orientation of the part can be efficiently recognized in the substrate processing apparatus.

[0024] According to the substrate processing apparatus of the third aspect, a second virtual camera position is set on each of a plurality of virtual divided surfaces generated by dividing a plurality of virtual surfaces including a high-match virtual surface on which a high-match virtual camera position detected by a first detection unit is set, and a numerical value indicating a degree of match between actual shape information and reference shape information is calculated for each second virtual camera position. Therefore, the plurality of virtual surfaces including the high-match virtual surface and the plurality of virtual divided surfaces are not unrelated surfaces, and an increase in at least one of the number and area of ​​the plurality of virtual divided surfaces can be reduced. This can reduce the amount of calculation for recognizing the orientation of the search target component captured in the actual image. As a result, the substrate processing apparatus can efficiently recognize the orientation of the component.

[0025] According to the substrate processing apparatus of the fourth aspect, when the unit process is repeatedly executed, a plurality of virtual dividing surfaces including a virtual dividing surface including a virtual camera position where the degree of coincidence between the actual shape information and the reference shape information is the highest and which are different in distance from the three-dimensional model of the search target part are divided by the same rule to generate a plurality of virtual dividing surfaces for setting the next virtual camera position. This makes it possible to reduce an increase in at least one of the number and area of ​​the virtual dividing surfaces after division, instead of the virtual dividing surfaces before division being unrelated to the virtual dividing surfaces after division. As a result, the amount of calculation required to recognize the orientation of the search target part captured in the actual image can be reduced. Thus, the orientation of the part can be recognized efficiently in the substrate processing apparatus.

[0026] According to the substrate processing apparatus of the fifth aspect, the amount of calculation required to recognize the orientation of the search target component captured in the actual image can be reduced, and thus the substrate processing apparatus can efficiently recognize the orientation of the component.

[0027] According to the substrate processing apparatus of the sixth aspect, the amount of calculation required to recognize the orientation of the search target component captured in the actual image can be reduced, and thus the substrate processing apparatus can efficiently recognize the orientation of the component.

[0028] According to the substrate processing apparatus of the seventh aspect, the surface can be easily divided.

[0029] According to the substrate processing apparatus of the eighth aspect, it is possible to easily set a surface aggregate including a plurality of virtual surfaces.

[0030] According to the substrate processing apparatus of the ninth aspect, the division of the imaginary surface can be easily performed.

[0031] According to the substrate processing apparatus of the tenth aspect, when the search target part moves between the first position and the second position, the range in which the multiple first virtual camera positions are set can be limited for the first position, which has a more limited range in which the orientation of the search target part changes. This can reduce the amount of calculation required to recognize the orientation of the search target part captured in the actual image. As a result, the orientation of the part can be recognized efficiently in the substrate processing apparatus.

[0032] According to the substrate processing apparatus of the eleventh aspect, when the search target part moves between the first position and the second position, the range in which the orientation of the search target part changes can be limited to the second position, which is a more limited range. This can reduce the amount of calculation required to recognize the orientation of the search target part captured in the actual image. As a result, the orientation of the part can be recognized efficiently in the substrate processing apparatus.

[0033] According to the substrate processing apparatus of the twelfth aspect, the amount of calculation required for the search process is reduced, making it possible to easily detect an abnormality in the search target component.

[0034] According to the substrate processing apparatus of the thirteenth aspect, when the search target part moves between a first position and a second position, an abnormality in the search target part located at the second position can be easily detected at the timing when the search target part is positioned at the first position.

[0035] According to the substrate processing apparatus of the fourteenth aspect, the amount of calculation required for the search process is reduced, thereby enabling efficient recognition of real-world information relating to the posture of the search target part captured in the actual image, and therefore efficient detection of abnormalities in the search target part.

[0036] According to the substrate processing apparatus of the fifteenth aspect, since the substrate processing apparatus includes information that specifies one or more initial setting ranges for the virtual camera position for each target part, the ranges in which the multiple first virtual camera positions are set are limited for each search target part. This can reduce the amount of calculation in the search process. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, the orientation of the part can be efficiently recognized in the substrate processing apparatus.

[0037] According to the information processing method in the substrate processing apparatus of the sixteenth aspect, when the range of the position and orientation of the search target part is limited, the amount of calculation in the search process can be reduced by limiting the range in which the multiple first virtual camera positions are set. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, the orientation of the part can be recognized efficiently in the substrate processing apparatus. [Brief description of the drawings]

[0038] [Figure 1] FIG. 1 is a side view illustrating an example of a schematic configuration of a substrate processing apparatus according to a first embodiment. [Diagram 2] FIG. 2 is a plan view illustrating an example of a schematic configuration of the substrate processing apparatus according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram showing an example of a functional configuration of the substrate processing apparatus according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating a specific example of the setting range information. [Diagram 5] FIG. 5 is a block diagram showing a specific example of a functional configuration related to a search process in the control unit. [Figure 6] FIG. 6 is a diagram showing a specific example of an actual image captured by a camera. [Figure 7] FIG. 7 is a diagram showing a specific example of a processing target region in an actual image. [Figure 8] FIG. 8 is a diagram showing a specific example of the processing target real image. [Figure 9] FIG. 9 is a diagram showing a specific example of an edge image. [Figure 10] FIG. 10 is a diagram for explaining the basic concept of the search process. [Figure 11] FIG. 11 is a diagram for explaining the basic concept of the search process. [Figure 12] FIG. 12 is a diagram for explaining the basic concept of the search process. [Figure 13] FIG. 13 is a diagram for explaining the basic concept of the search process. [Figure 14] FIG. 14 is a diagram for explaining the basic concept of the search process. [Figure 15] FIG. 15 is a diagram for explaining the basic concept of the search process. [Figure 16] FIG. 16 is a diagram for explaining the basic concept of the search process. [Figure 17] FIG. 17 is a diagram for explaining the basic concept of the search process. [Figure 18] FIG. 18 is a diagram for explaining an example of a manner in which T1 surface aggregates having mutually different distances from a reference point of a 3D model of a search target part are virtually set. [Figure 19] FIG. 19 is a diagram showing an example of a setting mode of one surface collection. [Figure 20] FIG. 20 is a diagram showing an example of a setting mode of the first virtual camera position on one virtual plane. [Figure 21] FIG. 21 is a diagram for explaining a specific example of a process for calculating a numerical value indicating the degree of match between actual shape information and reference shape information for one virtual camera position. [Figure 22] FIG. 22 is a diagram for explaining a specific example of a calculation process of a numerical value indicating the degree of match between actual shape information and reference shape information for one virtual camera position. [Diagram 23] FIG. 23 is a diagram for explaining a specific example of a process for calculating a numerical value indicating the degree of match between actual shape information and reference shape information for one virtual camera position. [Figure 24] FIG. 24 is a diagram for explaining a specific example of a process for calculating a numerical value indicating the degree of match between actual shape information and reference shape information for one virtual camera position. [Diagram 25] FIG. 25 is a diagram for explaining a specific example of a process for calculating a numerical value indicating the degree of match between actual shape information and reference shape information for one virtual camera position. [Figure 26] FIG. 26 is a diagram showing a specific example in which a matching target region is set in an edge image serving as actual shape information. [Figure 27] FIG. 27 is an image diagram showing an example of T2 virtual planes. [Figure 28] FIG. 28 is a diagram showing a specific example of dividing a high-match virtual surface into a plurality of virtual divided surfaces. [Figure 29] FIG. 29 is an image diagram showing an example of the first T3 virtual divided surfaces. [Diagram 30] FIG. 30 is a diagram showing a schematic example of how the first reference virtual dividing surface is divided into a plurality of virtual dividing surfaces in the first first case. [Diagram 31] FIG. 31 is a diagram showing a schematic example of how the first reference virtual split surface is set in the first second case. [Diagram 32] FIG. 32 is a diagram showing a schematic example of how the first reference virtual dividing surface is divided into a plurality of virtual dividing surfaces in the first second case. [Diagram 33] FIG. 33 is an image diagram showing an example of the second T3 virtually divided surfaces, which is a specific example of the n-th T3 virtually divided surfaces. [Diagram 34] FIG. 34 is a diagram showing a schematic diagram of a specific example of dividing the second reference virtual dividing surface into a plurality of virtual dividing surfaces in the second first case. [Diagram 35] FIG. 35 is a diagram showing a schematic example of how the second reference virtual split surface is set in the second case. [Diagram 36]FIG. 36 is a diagram showing a schematic diagram of a specific example of dividing the second reference virtual dividing surface among the second T3 virtual dividing surfaces into a plurality of virtual dividing surfaces in the second second case. [Figure 37] FIG. 37 is a flowchart showing a specific example of a schematic flow of an operation relating to processing of one or more substrates in the substrate processing apparatus. [Figure 38] FIG. 38 is a flowchart showing a specific example of a schematic flow of an operation relating to a process for detecting an abnormality in a substrate processing apparatus. [Figure 39] FIG. 39 is a flowchart showing a specific example of the flow of image processing in step S14 of FIG. [Diagram 40] FIG. 40 is a flowchart showing a specific example of the flow of the search process in step S15 of FIG. [Diagram 41] FIG. 41 is a flowchart showing a specific example of the process flow in the primary search process in step Sb1 of FIG. [Diagram 42] FIG. 42 is a flowchart showing a specific example of the process flow in the secondary search process in step Sb2 of FIG. [Diagram 43] FIG. 43 is a flowchart showing a specific example of the process flow in the secondary search process in step Sb2 of FIG. [Diagram 44] FIG. 44 is a flowchart showing a specific example of the process flow in the secondary search process in step Sb2 of FIG. [Diagram 45] FIG. 45 is a diagram illustrating an example of a state in which the chucks are in the open position, the guard is in the lowered position, and the nozzle is in the standby position. [Diagram 46] FIG. 46 is a diagram illustrating an example of a state in which the chucks are located at the closed position. [Figure 47] FIG. 47 is a diagram illustrating an example of a state in which the nozzle is in the discharge position and the guard is in the raised position. [Figure 48] FIG. 48 is a diagram illustrating another specific example of the setting range information. [Figure 49] FIG. 49 is a diagram illustrating another specific example of the setting range information. [Figure 50] FIG. 50 is a diagram showing a schematic diagram of another specific example of the setting range information. [Figure 51] FIG. 51 is a diagram illustrating an example of a schematic configuration of a substrate processing system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0039] There is a substrate processing apparatus that includes a processing chamber, a substrate holder, a nozzle, a camera, an image processor, and a monitor.

[0040] In this substrate processing apparatus, a substrate holder, a nozzle, and a camera are disposed in a processing chamber. The substrate holder rotates the substrate in a horizontal position. The nozzle moves between a standby position off to the side of the substrate and a discharge position above the substrate by pivoting a drive arm to which the nozzle is fixed. The nozzle is disposed in the standby position when the substrate is attached to or detached from the substrate holder, and is disposed in the discharge position when the processing liquid is discharged from the nozzle toward the substrate. The camera is disposed in a predetermined position in the processing chamber, and captures an image of a predetermined area including the nozzle that has moved to the discharge position. The image processor obtains second nozzle position information indicating the position of the nozzle based on an image from the camera, and outputs the second nozzle position information to a monitoring unit. The monitoring unit determines whether or not there is an abnormality in the position of the nozzle based on the correspondence between the first nozzle position information as information directly or indirectly indicating the position of the nozzle from the control unit and the second nozzle position information from the image processor.

[0041] In substrate processing apparatus, it may be necessary to recognize the posture of one or more of the various components, such as a nozzle, a chuck that holds the edge of the substrate in the substrate holding part, and a guard that rises and falls in the area surrounding the side of the substrate holding part, captured by a camera in order to detect abnormalities and operating conditions of the components.

[0042] In this case, it is conceivable to recognize the part's posture by a method that performs the following [Process A] to [Process E] in order.

[0043] [Process A] A virtual polyhedron consisting of multiple virtual faces along a sphere centered on the three-dimensional design shape of the part (also called a three-dimensional part model) represented by the three-dimensional CAD (Computer Aided Design) data of the part is set. Each virtual face is composed of, for example, a triangular face.

[0044] [Process B] A camera (also called a virtual camera) is virtually set at a predetermined position (also called a virtual camera position) on each virtual surface constituting the virtual polyhedron set in the above process A. In this way, multiple virtual cameras are virtually set so as to surround the 3D part model at the center.

[0045] [Process C] Generate a plurality of images (also called virtual images) acquired by virtually photographing the three-dimensional part model with a plurality of virtual cameras installed at a plurality of virtual camera positions in the above-mentioned process B. The virtual images can be generated, for example, by projecting the three-dimensional part model onto a virtual plane by a process such as rendering.

[0046] [Process D] A matching process using template matching or the like is performed between each of the multiple virtual images generated in Process C above and an image (also called a real image) obtained by actually photographing the part with a camera (also called a real camera). This detects the virtual camera position among the multiple virtual camera positions that provides the greatest degree of match (also called the degree of agreement) between the real image and the virtual image in terms of the shape, orientation, size, etc. of the captured part. The degree of match (degree of agreement) is also called the degree of similarity (degree of similarity).

[0047] [Process E] Using the detection results from process D above, the orientation of the part is recognized based on the position of the actual camera, thereby recognizing the orientation of the part captured in the actual image.

[0048] In the above method, for example, when the distance between the real camera and the part changes, in the above process A, a plurality of virtual polyhedrons are set whose distances from the center of the three-dimensional part model are different from each other.

[0049] Here, for example, in the above process A, a virtual polyhedron is constructed using a large number of very fine virtual surfaces, and in the above process B, the number of virtual camera positions is increased, thereby improving the accuracy of recognizing the orientation of the part captured in the actual image. However, the increased amount of calculations would result in a longer time required for the calculations.

[0050] Therefore, for example, a conceivable method is to perform the above process E after repeating the above processes A to D in order a predetermined number of times, and in the first process A, a virtual polyhedron is constructed from a plurality of relatively large virtual faces, and in the second and subsequent processes A, a virtual polyhedron is constructed from a large number of virtual faces that are relatively finer than those in the previous process A, and in the second and subsequent processes B, virtual cameras are virtually installed at predetermined virtual camera positions on each of the plurality of virtual faces of the polyhedron set in the immediately preceding process A that are close to the virtual camera position detected in the most recent process D. This method can improve the accuracy of recognizing the orientation of a part captured in an actual image, while reducing the amount of calculations and shortening the time required for calculations.

[0051] However, in this method, it is necessary to set a virtual polyhedron that is finer in stages in the above process A multiple times, and the virtual polyhedron set in the above process A from the second time onwards is unrelated to the virtual polyhedron set in the previous process A. Therefore, in order to maintain the accuracy of recognizing the orientation of the part captured in the real image, the part of the plurality of virtual surfaces for which the virtual camera position is virtually set in the above process B from the second time onwards will cover a relatively wide area located near the virtual camera position detected in the most recent process D. As a result, at least one of the number and area of ​​the part of the plurality of virtual surfaces for which the virtual camera position is set in the above process B from the second time onwards will increase to a certain extent, and the number of virtual camera positions set in the above process B from the second time onwards may increase, making it difficult to sufficiently reduce the amount of calculation.

[0052] Therefore, there is room for improvement in the substrate processing apparatus in terms of efficiently recognizing the orientation of components.

[0053] Hereinafter, an embodiment will be described with reference to the attached drawings. Note that the drawings are schematic, and for the convenience of explanation, configurations are omitted or simplified as appropriate. Furthermore, the size and positional relationship of the configurations shown in the drawings are not necessarily accurately described, and may be changed as appropriate.

[0054] In the following description, the same components are denoted by the same reference numerals, and their names and functions are also the same. Therefore, detailed descriptions thereof may be omitted to avoid duplication.

[0055] Furthermore, in the description below, even if ordinal numbers such as "first" or "second" are used, these terms are used for convenience to facilitate understanding of the contents of the embodiments, and are not limited to the ordering that may result from these ordinal numbers.

[0056] When an expression indicating a relative or absolute positional relationship (e.g., "in one direction," "along one direction," "parallel," "orthogonal," "center," "concentric," "coaxial," etc.) is used, the expression not only strictly indicates the positional relationship, but also indicates a state in which the relative positional relationship is displaced with respect to angle or distance within a range in which a tolerance or similar function is obtained, unless otherwise specified. When an expression indicating an equal state (e.g., "same," "equal," "homogeneous," etc.) is used, the expression not only strictly indicates a quantitatively equal state, but also indicates a state in which a difference exists within a range in which a tolerance or similar function is obtained, unless otherwise specified. When an expression indicating a shape (e.g., "square shape" or "cylindrical shape," etc.) is used, the expression not only strictly indicates the shape geometrically, but also indicates a shape having, for example, any of concaves and convexes and chamfers within a range in which a similar effect is obtained, unless otherwise specified. When an expression "comprises," "has," "includes," "includes," or "has" a component is used, the expression is not an exclusive expression that excludes the presence of other components. When the phrase "at least one of A, B, and C" is used, the phrase includes A only, B only, C only, any two of A, B, and C, and all of A, B, and C.

[0057] <1. Overall configuration of substrate processing equipment> Fig. 1 is a side view illustrating an example of a schematic configuration of a substrate processing apparatus 1 according to the first embodiment. Fig. 2 is a plan view illustrating an example of a schematic configuration of the substrate processing apparatus 1 according to the first embodiment.

[0058] The substrate processing apparatus 1 is an apparatus for processing a substrate (e.g., a semiconductor wafer) W. In the first embodiment, the substrate processing apparatus 1 is a single-wafer type apparatus for processing the substrates W one by one. A thin flat plate having a substantially disk shape is applied to the substrate W. In other words, the substrate W has a circular shape when viewed from above. When no particular condition is stated, the plan view refers to a plan view in which each part is viewed from above downward. The substrate processing apparatus 1 rotates the substrate W while supplying a processing liquid to perform a predetermined process on the substrate W. For example, a substrate cleaning apparatus that cleans the substrate W using a processing liquid is applied to the substrate processing apparatus 1.

[0059] The substrate processing apparatus 1 includes a housing CA. The housing CA isolates the inside of the housing CA from the atmosphere surrounding the housing CA.

[0060] The substrate processing apparatus 1 includes a spin chuck 3. The spin chuck 3 has a circular structure with a diameter larger than that of the substrate W in a plan view. The upper end of a rotating shaft 5 is connected to the lower surface of the spin chuck 3. The lower end of the rotating shaft 5 is connected to a motor 7. The spin chuck 3 rotates about a rotation center PL1 by being driven by the motor 7. The rotation center PL1 extends in the vertical direction.

[0061] The spin chuck 3 includes a plurality of chucks 9. In other words, the substrate processing apparatus 1 includes a plurality of chucks 9. More specifically, the spin chuck 3 includes a plurality of chucks 9 on the peripheral portion of the upper surface of the disk-shaped main body of the spin chuck 3. In the present first embodiment, the spin chuck 3 includes four chucks 9. The number of chucks 9 included in the spin chuck 3 is not limited to four, as long as the spin chuck 3 can stably rotate the substrate W about the rotation center PL1 while supporting the substrate W in a horizontal position.

[0062] The chuck 9 includes, for example, a lower support portion 11, a peripheral support portion 13, and a rotating magnet 15. The lower support portion 11 is a portion for supporting the substrate W by contacting the lower surface of the substrate W from below. For example, if the contact area of ​​the lower support portion 11 with the lower surface of the substrate W is small, the degree of mutual contamination between the substrate W and the chuck 9 can be reduced. The lower support portion 11 is attached to the upper surface side of the main body of the spin chuck 3 so as to be rotatable about a rotation center PL2. The rotation center PL2 extends in the vertical direction. The peripheral support portion 13 is erected on the upper surface of the lower support portion 11. For example, if the height of the peripheral support portion 13 from the upper surface of the lower support portion 11 is higher than the thickness of the substrate W, the peripheral support portion 13 can stably hold the peripheral edge of the substrate W. When viewed in a plan view, the peripheral support portion 13 is provided at a position away from the rotation center PL2 toward the outer edge of the lower support portion 11. In other words, the peripheral support part 13 is eccentric with respect to the rotation center PL2. The rotation magnet 15 is attached to the lower surface side of the spin chuck 3 at a position corresponding to the rotation center PL2. The rotation magnet 15 is connected to the lower surface support part 11. The rotation magnet 15 is provided to be rotatable around the rotation center PL2.

[0063] The substrate processing apparatus 1 includes a moving mechanism (also referred to as a chuck moving mechanism) 17 that moves the chuck 9 between an open position as a first position (also referred to as a 1A position) and a closed position as a second position (also referred to as a 2A position). In the first embodiment, the chuck moving mechanism 17 is disposed below the rotating magnet 15. The chuck moving mechanism 17 is disposed closer to the rotating shaft 5 than the chuck 9. The chuck moving mechanism 17 includes, for example, an air cylinder 19 and a driving magnet 21. The driving magnet 21 has an annular shape when viewed from above. The air cylinder 19 has a piston rod (also referred to as a rod or an operating shaft) positioned along the vertical direction. The driving magnet 21 is attached to the upper end of the rod of the air cylinder 19. The chuck moving mechanism 17 is operated in response to a chuck operation command from the control unit 45. The chuck moving mechanism 17 can move the driving magnet 21 closer to the chuck 9 by raising the driving magnet 21 with the air cylinder 19, and can move the driving magnet 21 in a direction away from the chuck 9 by lowering the driving magnet 21 with the air cylinder 19. The chuck moving mechanism 17 can move the driving magnet 21 between a raised position (also referred to as an upper position) and a lowered position (also referred to as a lower position).

[0064] The chuck 9 is provided with a biasing mechanism (not shown). When the driving magnet 21 descends and moves from the upper position to the lower position, the chuck 9 moves from the open position to the closed position. When the driving magnet 21 ascends and moves from the lower position to the upper position, the chuck 9 moves from the closed position to the open position. When the chuck 9 moves from the open position to the closed position, the peripheral support part 13 rotates around the rotation center PL2, and the peripheral support part 13 moves in a direction approaching the rotation center PL1 and abuts against the peripheral edge of the substrate W supported by the lower surface support part 11. This allows the multiple chucks 9 to clamp the substrate W at the closed position. When the chuck 9 moves from the closed position to the open position, the peripheral support part 13 rotates around the rotation center PL2, and the peripheral support part 13 moves in a direction away from the rotation center PL1. This causes the multiple chucks 9 to not clamp the substrate W at the open position. In this state, it is possible to carry the substrate W onto the spin chuck 3 from outside the casing CA, and to carry the substrate W out from the spin chuck 3 to outside the casing CA.

[0065] The substrate processing apparatus 1 includes a first sensor Z1 disposed near the rotation magnet 15 of the chuck 9. When the chuck 9 moves to the closed position or the open position, the first sensor Z1 changes a signal (also referred to as an output signal) output from the first sensor Z1. For example, when the chuck 9 moves to the closed position or the open position, the first sensor Z1 turns on the output signal.

[0066] The substrate processing apparatus 1 includes a guard 23. The guard 23 is disposed around the spin chuck 3. In other words, the guard 23 surrounds the sides of the spin chuck 3. The guard 23 is a portion for preventing the processing liquid from scattering from the substrate W supported and rotated by the spin chuck 3 to the surroundings. The guard 23 has a cylindrical shape. The guard 23 has an opening 23a formed in an upper portion. The inner diameter of the opening 23a is larger than the outer diameter of the spin chuck 3.

[0067] The substrate processing apparatus 1 includes a moving mechanism (also referred to as a card moving mechanism) 25 that moves the guard 23 between a lowered position as a first position (also referred to as a 1B position) and an elevated position as a second position (also referred to as a 2B position). More specifically, the guard moving mechanism 25 can raise and lower the guard 23 between the lowered position and the elevated position. The guard moving mechanism 25 includes, for example, an air cylinder 27 and a locking piece 29. The guard moving mechanism 25 is disposed, for example, on the outer periphery side of the guard 23. The guard moving mechanism 25 may be disposed on the inner periphery side of the guard 23 as long as it can raise and lower the guard 23. The air cylinder 27 has a piston rod (also referred to as a rod or an operating shaft) that is positioned along the vertical direction. The locking piece 29 is attached to the upper end of the rod of the air cylinder 27. The locking piece 29 is fixed to the outer periphery surface of the guard 23. The guard moving mechanism 25 is not limited to the above-mentioned configuration as long as it can raise and lower the guard 23.

[0068] The guard moving mechanism 25 moves the guard 23 between the lowered position and the raised position in response to a guard operation command from the control unit 45. The lowered position of the guard 23 is the position of the guard 23 when the guard 23 is lowered by the guard moving mechanism 25. In other words, the lowered position is a position where the upper end of the guard 23 is low. The raised position of the guard 23 is the position of the guard 23 when the guard 23 is raised by the guard moving mechanism 25. In other words, the lowered position is a position lower than the raised position, and the raised position is a position higher than the lowered position. When the guard 23 is in the lowered position, the position of the upper edge of the guard 23 is lower than the position of the substrate W supported by the spin chuck 3. When the guard 23 is in the raised position, the position of the upper edge of the guard 23 is higher than the position of the substrate W supported by the spin chuck 3. Therefore, when the guard 23 is in the raised position, the inner peripheral surface of the guard 23 can receive the processing liquid splashing from the substrate W supported and rotated by the spin chuck 3 to the surroundings. The substrate processing apparatus 1 includes, for example, a second sensor Z2 disposed on the inner periphery side of the guard 23. When the guard 23 moves to the lowered position, the second sensor Z2 changes a signal (output signal) output from the second sensor Z2. For example, when the guard 23 moves to the lowered position, the second sensor Z2 turns on the output signal.

[0069] The substrate processing apparatus 1 includes a drain port (not shown) located on the inner periphery of the guard 23. The drain port is a portion for recovering the processing liquid scattered from the substrate W supported and rotated by the spin chuck 3 and received on the inner periphery of the guard 23. The number of the guards 23 is not limited to one, and may be multiple. The number of the drain port is not limited to one, and may be multiple. When the number of the guards 23 is multiple, a drain port may be disposed for each guard 23. In this case, each guard 23 may be configured to be raised and lowered by the guard moving mechanism 25 to switch the combination of the guard 23 that receives the processing liquid scattered from the substrate W and the drain port that recovers the processing liquid received on the inner periphery of the guard 23. For example, it is possible to switch the combination of the guard 23 that receives the processing liquid scattered from the substrate W and the drain port that recovers the processing liquid received on the inner periphery of the guard 23 by raising and lowering each guard 23 by the guard moving mechanism 25 according to the type of processing liquid.

[0070] The substrate processing apparatus 1 includes a processing liquid supply mechanism 31 disposed on the outer periphery side of the guard 23. The processing liquid supply mechanism 31 includes, for example, a nozzle 33, a base unit 37, and a moving mechanism (also referred to as a nozzle moving mechanism) 35. In other words, the substrate processing apparatus 1 includes the nozzle 33. In the present first embodiment, the processing liquid supply mechanism 31 includes, for example, two nozzles 33. In the following description, when it is necessary to distinguish between the two nozzles 33, the nozzle 33 on the left side in FIG. 2 will be appropriately referred to as nozzle 33A, and the nozzle 33 on the right side will be appropriately referred to as nozzle 33B. The number of nozzles 33 in the processing liquid supply mechanism 31 may be one, or may be three or more. In the present first embodiment, it is assumed that the two nozzles 33 have the same configuration.

[0071] Nozzle 33 includes, for example, extension portion 33a, hanging portion 33b, and tip portion 33c. One end of extension portion 33a of nozzle 33 is attached to base portion 37. Extension portion 33a extends horizontally from base portion 37. The other end of extension portion 33a is connected to hanging portion 33b. Hanging portion 33b extends vertically downward from extension portion 33a. Tip portion 33c is located on the lower end side of hanging portion 33b. Tip portion 33c ejects a processing liquid from the lower surface side. Examples of processing liquid include photoresist liquid, SOG (Spin-on-Glass) liquid, developing liquid, rinsing liquid, pure water, and cleaning liquid.

[0072] The nozzle moving mechanism 35 can move the nozzle 33 between a standby position as a first position (also referred to as a 1C position) and a discharge position as a second position (also referred to as a 2C position). The nozzle moving mechanism 35 includes, for example, a motor 39, a rotating shaft 41, and a position detection unit 43. The motor 39 is disposed in a vertical position. The rotating shaft 41 is rotated around a rotation center PL3 by the motor 39. The rotating shaft 41 is connected to a base unit 37. The base unit 37 is rotated by the driving of the motor 39. The nozzle 33 is swung around the rotation center PL3 together with the base unit 37 by the nozzle moving mechanism 35. The position detection unit 43 detects the position (also referred to as a rotation position) of the rotating shaft 41 in the rotation direction around the rotation center PL3 of the rotating shaft 41. The position detection unit 43 detects the angle of the rotating shaft 41 around the rotation center PL3 in a plan view. The position detector 43 outputs a pulse signal in accordance with the rotational position of the rotating shaft 41 .

[0073] The substrate processing apparatus 1 includes a standby cup 44 disposed at a position separated from the guard 23 laterally in a plan view. The standby cup 44 is disposed so that the tip 33c of the nozzle 33 can be positioned above the standby cup 44 in a plan view. From another perspective, the standby cup 44 is disposed below the standby position of the nozzle 33. The standby cup 44 is a portion for preventing the tip 33c of the nozzle 33 from drying. The standby cup 44 is used for idle discharge of the processing liquid from the nozzle 33. The nozzle moving mechanism 35 drives the motor 39 to oscillate the nozzle 33. The nozzle moving mechanism 35 moves the tip 33c of the nozzle 33 between the standby position and a discharge position above the rotation center PL1 of the spin chuck 3 in response to a nozzle operation command from the control unit 45. The discharge position of the nozzle 33 is a position where the nozzle 33 is disposed when the processing liquid is discharged from the tip 33c of the nozzle 33 onto the substrate W.

[0074] The substrate processing apparatus 1 includes, for example, a third sensor Z3 disposed on the outer periphery side of the rotating shaft 41. When the nozzle 33 is located at the standby position, the third sensor Z3 changes a signal (output signal) output from the third sensor Z3. For example, when the nozzle 33 moves to the standby position, the third sensor Z3 turns on the output signal. The third sensor Z3 may be omitted to simplify the configuration. In this case, a protrusion (also called a rotation-side protrusion) may be provided on a part of the rotating shaft 41, and a protrusion (also called a fixed-side protrusion) may be provided on the outer periphery side of the rotating shaft 41 and fixed to the housing CA. For example, the position detection unit 43 may detect that the rotation-side protrusion of the rotating shaft 41 abuts against the fixed-side protrusion due to the rotation of the rotating shaft 41 around the rotation center PL3 of the rotating shaft 41, making it impossible to rotate the rotating shaft 41 around the rotation center PL3, thereby detecting that the nozzle 33 is located at the standby position. Here, the position of the nozzle 33 at the time when the pulse signal of the position detection unit 43 becomes unchanged may be treated as the standby position.

[0075] The substrate processing apparatus 1 includes a camera CM as an imaging unit. The camera CM captures an image (also called an actual image) capturing a search target part, which is any one of the one or more target parts, by imaging. Here, the one or more target parts are parts included in the substrate processing apparatus 1. The target parts are parts that can be targets of a specific process. The specific process can include, for example, a process for detecting an abnormality. In the first embodiment, the one or more target parts include two or more target parts. The two or more target parts as the one or more target parts include a plurality of chucks 9, guards 23, and nozzles 33 (specifically, four). In other words, the substrate processing apparatus 1 includes, for example, a plurality of chucks 9, guards 23, and nozzles 33 as the one or more target parts. The search target part is a part that is a target of a search process, which will be described later, among the one or more target parts. Any one of the one or more target parts may be any one of the one or more target parts. The camera CM is attached to, for example, a part of the housing CA. The position of the camera CM may be anywhere as long as the search target parts are within the field of view. The camera CM has, for example, a field of view (angle of view) that allows all of one or more target parts to be within the field of view.

[0076] The substrate processing apparatus 1 includes a control unit 45, an instruction unit 47, and a notification unit 49. The control unit 45 will be described in detail later. The instruction unit 47 is operated by an operator of the substrate processing apparatus 1. The instruction unit 47 is, for example, a keyboard or a touch panel. The instruction unit 47 instructs one or more target parts, a timing to check, a recipe, and the start of processing. When the control unit 45 detects an abnormality, the notification unit 49 notifies the operator of the abnormality. Examples of the notification unit 49 include a display, a lamp, and a speaker.

[0077] <2. Configuration of the control system in the substrate processing apparatus> FIG. 3 is a block diagram showing an example of a functional configuration of the substrate processing apparatus 1 according to the first embodiment.

[0078] The control unit 45 includes, for example, a calculation unit 45a and a storage unit 45b. The calculation unit 45a has, for example, an electronic circuit that functions as a processor such as a central processing unit (CPU), and an electronic circuit that functions as a memory such as a random access memory (RAM) that temporarily stores data for the processor to execute processing. The storage unit 45b has, for example, a portion that functions as a nonvolatile storage medium such as a hard disk or a flash memory. The storage unit 45b may have, for example, a portion that functions as one nonvolatile storage medium, or may have two or more portions that function as nonvolatile storage media.

[0079] The calculation unit 45a includes, for example, a plurality of functional components, an operation control unit 51, an image processing unit 59, a search processing unit 61, and an anomaly detection unit 63. The storage unit 45b stores, for example, recipe information 53, parameter information 55, design information 57, setting range information 58, and a program Pg1. In the calculation unit 45a, for example, the operation control unit 51, the image processing unit 59, the search processing unit 61, and the anomaly detection unit 63 may be functional processing units (also referred to as functional processing units) that are realized by a CPU reading and executing the program Pg1 in the storage unit 45b.

[0080] The operation control unit 51 controls the operations of the motors 7, 39, the air cylinders 19, 27, and the camera CM. The operation control unit 51 receives signals from the first sensor Z1, the second sensor Z2, the third sensor Z3, and the position detection unit 43. The operation control unit 51 controls the operation of each unit, for example, according to a recipe in the recipe information 53 stored in the storage unit 45b. For example, after an operator instructs each unit to start operating based on the recipe, the operation control unit 51 outputs various operation commands to each unit based on the recipe, and operates each unit such as the motor 7 at a predetermined timing.

[0081] The recipe information 53 is information indicating various recipes pre-stored in the storage unit 45b. The recipes prescribe various procedures for processing the substrate W. The operator can operate the instructing unit 47 to instruct the execution of a desired recipe.

[0082] The parameter information 55 is information such as confirmation timing for each of one or more target parts, which is stored in advance in the storage unit 45b. The confirmation timing may be, for example, a timing for confirming the operating state of the target part. The one or more target parts and the confirmation timing can be arbitrarily set by the operator operating the instruction unit 47. The operator can instruct, via the instruction unit 47, which parts are to be the target parts, which timing is to be the confirmation timing, and so on.

[0083] The timings requiring confirmation may include, for example, the timing when the chuck moving mechanism 17 positions the chuck 9 in the open position, which is the first position (more specifically, the 1A position), the timing when the chuck moving mechanism 17 positions the chuck 9 in the closed position, which is the second position (more specifically, the 2A position), the timing when the guard moving mechanism 25 positions the guard 23 in the lowered position, which is the first position (more specifically, the 1B position), the timing when the guard moving mechanism 25 positions the guard 23 in the raised position, which is the second position (more specifically, the 2B position), the timing when the nozzle moving mechanism 35 positions the nozzle 33 in the standby position, which is the first position (more specifically, the 1C position), the timing when the nozzle moving mechanism 35 positions the nozzle 33 in the discharge position, which is the second position (more specifically, the 2C position), and so on.

[0084] Here, the confirmation-requiring timings may include, for example, the timing when the chuck moving mechanism 17 moves the chuck 9 in response to a chuck operation command and completes the movement of the chuck 9, the timing when the nozzle moving mechanism 35 moves the nozzle 33 in response to a nozzle operation command and completes the movement of the nozzle 33, and the timing when the guard moving mechanism 25 raises and lowers the guard 23 in response to a guard operation command and completes the movement of the guard 23. The confirmation-requiring timings may include, for example, the timing when the chuck 9 is set to be located at the open position by a chuck operation command, the timing when the chuck 9 is set to be located at the closed position by a chuck operation command, the timing when the guard 23 is set to be located at the lowered position by a guard operation command, the timing when the guard 23 is set to be located at the raised position by a guard operation command, the timing when the nozzle 33 is set to be located at the standby position by a nozzle operation command, and the timing when the nozzle 33 is set to be located at the discharge position by a nozzle operation command.

[0085] Here, in the first embodiment, among the one or more target parts, a target part that is in a check-required timing becomes a search target part that is a target of a search process described later.

[0086] When the chuck 9 as the target part is in the confirmation timing based on, for example, a signal from the first sensor Z1 and control of the operation of the chuck moving mechanism 17, the above-mentioned operation control unit 51 notifies the search processing unit 61 of information identifying the chuck 9 as the target part (search target part) in the confirmation timing and information indicating the position (e.g., open position or closed position, etc.) in which the chuck 9 as the target part (search target part) in the confirmation timing should be in at the confirmation timing. For example, when four chucks 9 exist, for example, the first chuck as the name identifying the first chuck 9, the second chuck as the name identifying the second chuck 9, the third chuck as the name identifying the third chuck 9, and the fourth chuck as the name identifying the fourth chuck 9 are adopted as the information identifying the chucks 9.

[0087] For example, based on a signal from the second sensor Z2 and control of the operation of the guard moving mechanism 25, if the guard 23 as the target part is at a timing to require confirmation, the operation control unit 51 informs the search processing unit 61 of information identifying the guard 23 as the target part (search target part) at the timing to require confirmation and information indicating the position (for example, a lowered position or an raised position) where the guard 23 as the target part (search target part) at the timing to require confirmation should be at the timing to require confirmation.

[0088] If the nozzle 33 as the target part is at a timing to require confirmation, for example, based on a signal from the third sensor Z3, a pulse signal from the position detection part 43, and control of the operation of the nozzle moving mechanism 35, the operation control part 51 notifies the search processing part 61 of information identifying the nozzle 33 as the target part (search target part) at the timing to require confirmation, and information indicating the position (for example, a lowered position or an raised position) where the nozzle 33 as the target part (search target part) at the timing to require confirmation should be at the timing to require confirmation.

[0089] Here, for example, if there is only one target part, the operation control unit 51 does not need to inform the search processing unit 61 of information identifying the search target part as a target part that is in a confirmation timing period. The operation control unit 51 may inform the search processing unit 61 that it has output an operation command to each unit according to the recipe.

[0090] The design information 57 includes, for example, three-dimensional design information related to the components constituting the substrate processing apparatus 1. The design information 57 includes, for example, 3D-CAD (three-dimensional Computer Aided Design) data related to the components constituting the substrate processing apparatus 1. The 3D-CAD data is expressed, for example, by three axes whose coordinate axes are orthogonal to each other, and is expressed by information on positions and angles (also collectively referred to as position information) when the components are arranged in a three-dimensional space. The design information 57 includes design information related to one or more target components transferred from a host computer (not shown). In other words, the storage unit 45b stores three-dimensional design information related to each of the one or more target components. The host computer (not shown) may store, for example, 3D-CAD data related to all the components of the substrate processing apparatus 1 as three-dimensional design information. Here, if the design information 57 stored in the storage unit 45b is limited to the design information of one or more target components, rather than the design information related to all the components of the substrate processing apparatus 1, the storage capacity of the storage unit 45b can be saved.

[0091] The image processing unit 59 processes an actual image capturing the search target part obtained by photographing with the camera CM. The image processing unit 59 performs image processing on the actual image to extract information on the two-dimensional shape of the object captured in the actual image (also referred to as actual shape information). For example, the image processing unit 59 extracts the actual shape information by performing a process of extracting contours for all parts captured in all areas of the actual image or a predetermined partial area. The contour here may include not only the contour of the outer shape of the part but also an edge part located inside the outer shape of the part. Here, it can be said that the actual shape information is information indicating the two-dimensional shape of the object in the actual image. The actual shape information extracted by the image processing unit 59 is provided to the search processing unit 61.

[0092] The search processing unit 61 performs a process (also called a search process) of searching for a virtual camera position (also called a maximum matching virtual camera position) having the highest matching degree (also called a similarity degree) between the reference shape information and the actual shape information among a plurality of virtual camera positions (also called virtual camera positions) based on a plurality of reference shape information and actual shape information. Here, the plurality of reference shape information are information generated based on three-dimensional design information on the search target part stored in the storage unit 45b or the like, and are information on the two-dimensional shape of the 3D model of the search target part in each of a plurality of virtual images (also called virtual images) that can be acquired by photographing the three-dimensional model (also called a 3D model) of the search target part from a plurality of virtual camera positions. For example, one piece of reference shape information is information generated based on three-dimensional design information on one search target part, and is information on the two-dimensional shape of the 3D model of the search target part in a virtual image that can be acquired by photographing the 3D model of the search target part from a single virtual camera position. The 3D model of the search target part is a three-dimensional model of the search target part that is virtually generated based on the three-dimensional design information on the search target part. The virtual image may be generated by projecting a 3D model onto a virtual plane by a process such as rendering. The degree of agreement between the reference shape information and the actual shape information refers to, for example, the degree to which the reference shape information is similar to the actual shape information. The greater the degree of agreement, the closer the reference shape information is to the actual shape information, and the smaller the degree of agreement, the more the reference shape information differs from the actual shape information. The degree of agreement between the reference shape information and the actual shape information is expressed, for example, by a numerical value indicating the degree of agreement between the reference shape information and the actual shape information. The degree of agreement and the numerical value indicating the degree of agreement will be described further below. The information on the two-dimensional shape of the 3D model may be, for example, a figure.

[0093] The search processing unit 61 performs a search process on the search target part as a target part that has become a confirmation-required timing. In the search process, it is possible to search whether a search target part that has the highest degree of match with a two-dimensional shape of a 3D model of the search target part in a virtual image that can be acquired by shooting a 3D model of the search target part from which virtual camera position among a plurality of virtual camera positions is captured in the real image captured by the camera CM. In other words, it is possible to search whether a two-dimensional shape of the search target part captured in the real image captured by the camera CM matches a two-dimensional shape of a 3D model of the search target part captured in a virtual image that can be acquired by shooting a 3D model of the search target part from which virtual camera position. Therefore, according to the result of the search process, the orientation of the search target part based on the position of the camera CM can be recognized. In other words, according to the result of the search process, the orientation of the search target part captured in the real image can be recognized. In other words, in the search processing unit 61, the orientation of the search target part as a target part that has become a confirmation-required timing can be recognized based on the result of the search process.

[0094] The set range information 58 is information that specifies one or more initial setting ranges for a preset virtual camera position for at least one of the one or more target parts. The one or more initial setting ranges for the virtual camera position specify a range in which the virtual camera position is set in an initial stage of the search processing by the search processing unit 61. The set range information 58 is provided to limit the range in which the virtual camera position is set in an initial stage of the search processing by the search processing unit 61, focusing on the fact that the movement range of the target parts in the substrate processing apparatus 1 is limited.

[0095] According to this set range information 58, the number of virtual camera positions set in the initial stage of the search process by the search processing unit 61 can be reduced. Here, when the range of the position and orientation of the search target part is limited, the amount of calculation in the search process for searching for the maximum matching virtual camera position can be reduced by limiting the range in which multiple virtual camera positions are set. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, in the substrate processing apparatus 1, the orientation of the search target part can be efficiently recognized.

[0096] FIG. 4 is a diagram showing a specific example of the setting range information 58. As shown in FIG. 4, the setting range information 58 is, for example, information that specifies one or more initial setting ranges related to the virtual camera position for each of two or more target parts as at least one target part among two or more target parts as one or more target parts. In other words, the setting range information 58 includes, for example, information that specifies one or more initial setting ranges related to the virtual camera position for each target part. Here, the one or more target parts include two or more target parts, and at least one target part includes two or more target parts. In an example of the setting range information 58 in FIG. 4, as two or more target parts, a "first chuck" as a name specifying the first chuck 9, a "second chuck" as a name specifying the second chuck 9, and a "nozzle" as a name specifying the nozzle 33 are exemplified. In FIG. 4, illustration of one or more initial setting ranges related to the virtual camera position for other target parts is omitted.

[0097] In the first embodiment, in the setting range information 58, one or more initial setting ranges related to the virtual camera position for each target part include a first initial setting range for the first position and a second initial setting range for the second position. In other words, the setting range information 58 specifies an initial setting range for each position for each target part. For example, the first initial setting range for a target part may be an initial setting range corresponding to a position where the target part should be when the target part is located at the first position. For example, the second initial setting range for a target part may be an initial setting range corresponding to a position where the target part should be when the target part is located at the second position.

[0098] More specifically, for example, in the setting range information 58, one or more initial setting ranges related to the virtual camera position for each chuck 9 include a first initial setting range (also referred to as the 1A initial setting range) for the "open position" as the first position (1A position), and also include a second initial setting range (also referred to as the 2A initial setting range) for the "closed position" as the second position (2A position). For example, in the setting range information 58, one or more initial setting ranges related to the virtual camera position for the guard 23 include a first initial setting range (also referred to as the 1B initial setting range) for the "downward position" as the first position (1B position), and also include a second initial setting range (also referred to as the 2B initial setting range) for the "upward position" as the second position (2B position). For example, in the setting range information 58, one or more initial setting ranges relating to the virtual camera position relative to the nozzle 33 include a first initial setting range (also referred to as the 1C initial setting range) for the "waiting position" as the first position (1C position), and also include a second initial setting range (also referred to as the 2C initial setting range) for the "ejection position" as the second position (2C position).

[0099] 4, one or more initial setting ranges related to the virtual camera position for each of the first chuck as the first chuck 9 and the second chuck as the second chuck 9 include an initial setting range for the open position (1A initial setting range) and an initial setting range for the closed position (2A initial setting range). Also, in the setting range information 58 illustrated in FIG. 4, one or more initial setting ranges related to the virtual camera position for the nozzle 33 include an initial setting range for the standby position (1C initial setting range) and an initial setting range for the discharge position (2C initial setting range).

[0100] In the present embodiment, each of the one or more initial setting ranges in the setting range information 58 includes a numerical range (also referred to as a first numerical range) which is a range of distance, a numerical range (also referred to as a second numerical range) which is a range of latitude, and a numerical range (also referred to as a third numerical range) which is a range of longitude. Here, for example, the distance from a reference point of a 3D model of a search target part as a target part at a confirmation timing is applied to the distance. For example, the center of the 3D model is applied to this reference point. For example, the center of gravity of the 3D model is applied to the center of the 3D model. For example, the latitude of a virtual sphere (also referred to as a virtual sphere) that surrounds the 3D model with the reference point of the 3D model as the center is applied to the latitude. For example, the longitude of a virtual sphere that surrounds the 3D model with the reference point of the 3D model as the center is applied to the longitude.

[0101] In this case, when the ranges in which the angular relationship and distance between the camera CM and the search target part change are limited, the range in which the multiple virtual camera positions are set can be limited. This can reduce the amount of calculation required for the search process to search for the maximum matching virtual camera position. As a result, the amount of calculation required for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, the orientation of the part can be efficiently recognized in the substrate processing apparatus.

[0102] Each initial setting range defined in the setting range information 58 may be set in advance based on, for example, three-dimensional design information related to the components constituting the substrate processing apparatus 1 included in the design information 57. Each initial setting range may be arbitrarily set by, for example, the operator operating the instruction unit 47. The setting range information 58 may be stored in the storage unit 45b by, for example, the operator operating the instruction unit 47. When each initial setting range is set in advance, each target component is not in a confirmation-required timing, so the reference point of the 3D model may be the reference point of a three-dimensional model (3D model) of the target component based on the three-dimensional design information of the target component. Each initial setting range may be set based on, for example, the relationship between the 3D model of the target component in the design information 57 and the camera CM, such as the relative position and posture. Each initial setting range is set to a certain width according to, for example, the error in the mounting position and posture of the target component and the camera CM in the substrate processing apparatus 1. In other words, each initial setting range encompasses the range in which the camera CM can be positioned when the reference point of the 3D model of the target part is used as the reference point in a virtual space based on three-dimensional design information relating to the parts that make up the substrate processing apparatus 1 contained in the design information 57.

[0103] The anomaly detection unit 63 detects an anomaly in the search target part as a target part at a check-required timing in response to information from the search processing unit 61. For example, the anomaly detection unit 63 may detect an anomaly in the search target part if the degree of match between the reference shape information and the actual shape information for the maximum matching virtual camera position detected by the search processing unit 61 is smaller than a predetermined degree of match set in advance. In the first embodiment, as described above, the amount of calculation of the search process for searching for the maximum matching virtual camera position can be reduced, so that an anomaly in the search target part can be easily detected. This predetermined degree of match may be stored in the storage unit 45b, for example, by the operator operating the instruction unit 47. The predetermined degree of match may be defined by a numerical value indicating the degree of match.

[0104] The abnormality detection unit 63 causes the notification unit 49 to perform a notification operation in response to the detection of an abnormality in the search target part. Specifically, the abnormality detection unit 63 causes the notification unit 49 to perform a notification operation in response to the detection of an abnormality in the search target part. The notification operation may be, for example, an operation of notifying an operator of the occurrence of an abnormality. The notification unit 49 may also notify, together with the occurrence of an abnormality, information identifying the search target part in which the abnormality was detected. Here, examples of the information identifying the search target part in which the abnormality was detected include a character string or a voice indicating, for example, a name or symbol identifying each of the multiple chucks 9, a name or symbol identifying each of one or more guards 23, or a name or symbol identifying each of one or more nozzles 33.

[0105] <3. Specific examples of functional configurations related to search processing> Fig. 5 is a block diagram showing a specific example of a functional configuration related to search processing in the control unit 45. In Fig. 5, of the multiple functional configurations in the control unit 45, a storage unit 45b, an image processing unit 59, and a search processing unit 61 are shown.

[0106] <3-1. Image processing section> The image processing unit 59 has, as a plurality of functional processing units, for example, a processing target area extraction unit 591 and a contour extraction unit 592. The processing target area extraction unit 591 and the contour extraction unit 592 are realized, for example, in the calculation unit 45a by the CPU reading and executing the program Pg1 stored in the storage unit 45b.

[0107] The processing target area extraction unit 591 acquires an actual image capturing the search target part captured by the camera CM, and extracts from the actual image a portion related to the area to be processed (also referred to as the processing target area) as an image (also referred to as the processing target actual image). This can reduce the amount of computation required for the processing in the contour extraction unit 592 and the search processing in the search processing unit 61, for example. As a result, the orientation of the search target part captured in the actual image can be efficiently recognized.

[0108] 6 to 8 are diagrams for explaining the process of extracting a portion related to the processing target area from a real image. Fig. 6 shows a specific example of a real image Ir1 capturing a search target part captured by a camera CM. Fig. 7 shows a specific example of a processing target area R1 in the real image Ir1. Fig. 8 shows a specific example of a processing target real image Ir2 extracted from the real image Ir1.

[0109] For example, when the processing target area extraction unit 591 acquires the actual image Ir1 shown in FIG. 6, it extracts the portion of the actual image Ir1 relating to the processing target area R1 surrounded by a thick dotted line as shown in FIG. 7, thereby obtaining the actual processing target image Ir2 as shown in FIG. 8.

[0110] The processing target area R1 may be set in advance, for example, for each target part that is a search target part, so as to include an area in which the search target part may exist as a target part at a confirmation-required timing on the actual image. Furthermore, the processing target area R1 may be set in advance, for example, for each position of the target part that is a search target part, so as to include an area in which the search target part may exist as a target part at a confirmation-required timing on the actual image. For example, the processing target area R1 may be set by referring to an actual image obtained in advance by photographing with the camera CM, or may be set based on three-dimensional design information related to the parts constituting the substrate processing apparatus 1 and the photographing direction and angle of view of the camera CM.

[0111] The contour extraction unit 592 extracts information (actual shape information) about the two-dimensional shape of the object captured in the actual image by, for example, performing a process of extracting contours for all parts captured in the processing target actual image Ir2 obtained by the processing target region extraction unit 591. This allows the image processing unit 59 to obtain the actual shape information about the two-dimensional shape of the object captured in the actual image. The actual shape information obtained by the image processing unit 59 is provided to the search processing unit 61.

[0112] Here, the processing target real image Ir2 obtained by the processing target region extraction unit 591 may be regarded as a real image capturing the search target part. As described above, the contour may include not only the contour of the outer shape of the part, but also the edge part located inside the outer shape of the part. The process of extracting the contour may be realized by using, for example, a method of detecting edges such as the Canny method. Here, the contours of all parts captured in the processing target real image Ir2 may be extracted by further performing a process of expanding lines such as morphological transformation on the detected edges. The actual shape information may be, for example, an edge image showing the contour of an object captured in the actual image. As the edge image, for example, a binary image that can distinguish the contour of the object from other parts is adopted. Here, the edge image as the actual shape information can be said to be information showing the two-dimensional shape of the object in the actual image. In other words, the actual shape information may be, for example, information showing the two-dimensional shape of the object in the actual image.

[0113] Fig. 9 is a diagram showing a specific example of edge image Ir3 as actual shape information acquired by contour extraction unit 592. The edge image Ir3 illustrated in Fig. 9 is an example of an edge image obtained by performing processing to extract contours of all parts captured in the actual processing target image Ir2 illustrated in Fig. 8. The edge image Ir3 illustrated in Fig. 9 is a binary image in which the contours of the parts extracted by contour extraction unit 592 are shown in white and the other parts are shown in black.

[0114] <3-2. Search processing section> <3-2-1. An example of the basic concept of search processing> 10 to 17 are diagrams for explaining an example of the basic concept of the search processing performed by the search processing unit 61. FIG.

[0115] For example, as shown in FIG. 10, a 3D model 3dm of a search target part, which is a three-dimensional model of the search target part virtually generated based on three-dimensional design information related to the search target part (e.g., a chuck 9), is set as appropriate around the 3D model 3dm of the search target part, and information (reference shape information) about the two-dimensional shape of the 3D model 3dm in a virtual image (virtual image) that can be acquired by photographing the 3D model 3dm from each virtual camera position P1 is generated. That is, reference shape information is generated for each virtual camera position P1. As the reference shape information, for example, as shown in FIG. 11, an image (also referred to as a reference image) Iv1 showing the contour as the two-dimensional shape of the 3D model 3dm in the virtual image that can be acquired by photographing the 3D model 3dm from each virtual camera position P1 can be adopted. Here, it can be said that the reference image as the reference shape information is information showing the two-dimensional shape of the 3D model 3dm in the virtual image. In other words, the reference shape information may be, for example, information showing the two-dimensional shape of the 3D model 3dm in the virtual image. Fig. 11 shows an example of a reference image Iv1 in which the outline of the 3D model 3dm is drawn with a thick two-dot chain line for the sake of convenience. The reference image Iv1 shown in Fig. 11 is a binary image in which the outline of the 3D model 3dm as a two-dimensional shape is shown in black, and the other parts are shown in white.

[0116] Then, for example, a numerical value indicating the degree of match is calculated between each of a plurality of reference images Iv1 as a plurality of pieces of reference shape information obtained for a plurality of virtual camera positions P1 and an edge image Ir3 as actual shape information about a two-dimensional shape of an object captured in an actual image capturing a search target part as shown in Fig. 12. The edge image Ir3 illustrated in Fig. 12 is a binary image in which the contour of the object is shown in black and the other parts are shown in white. Here, for example, the reference image Iv1 as the reference shape information is rotated and translated relatively to the edge image Ir3 as the actual shape information, and a numerical value indicating the degree of match when the degree of match of the reference image Iv1 as the reference shape information with the edge image Ir3 as the actual shape information is maximized can be calculated as a numerical value indicating the degree of match between the edge image Ir3 as the actual shape information and the reference image Iv1 as the reference shape information. In other words, for each virtual camera position P1, a numerical value indicating the degree of matching between the edge image Ir3 as actual shape information and the reference image Iv1 as reference shape information when the degree of matching is maximum for the edge image Ir3 as actual shape information can be calculated as a numerical value indicating the degree of matching between the edge image Ir3 as actual shape information and the reference image Iv1 as reference shape information. In other words, for each virtual camera position P1, a numerical value indicating the degree of matching between the edge image Ir3 as actual shape information and the reference image Iv1 as reference shape information when the degree of matching is maximum for the edge image Ir3 as actual shape information is taken as the degree of matching between the edge image Ir3 as actual shape information and the reference image Iv1 as reference shape information.

[0117] 13, for example, a virtual camera position P1 that provides the highest degree of matching between the edge image Ir3 as actual shape information and the reference image Iv1 as reference shape information is detected from among the multiple virtual camera positions P1. This makes it possible to search for a search target part that has the highest degree of matching with the two-dimensional shape of the 3D model 3dm in a virtual image that can be acquired by photographing the 3dm of the 3D model from which virtual camera position among the multiple virtual camera positions P1, in an actual image capturing the search target part captured by photographing with the camera CM.

[0118] Now, as shown in FIG. 14, a case is assumed where a right-handed xyz coordinate system is set with the center of the 3D model 3dm as the origin as the reference point Po. For example, the center of gravity of the 3D model 3dm is applied to the center of the 3D model 3dm. In this case, a plurality of virtual camera positions P1 centered on the 3D model 3dm can be defined by an angle (also called latitude) α in the direction of rotation about the x axis, an angle (also called longitude) β in the direction of rotation about the z axis, and a distance D from the origin. In addition, the attitude of a virtual camera that captures the 3D model 3dm from the virtual camera position P1 can be defined by an angle (also called roll angle) γ in the direction of rotation about a straight line Ln1 drawn by a thin dashed line passing through the reference point Po and the virtual camera position P1.

[0119] Here, when the latitude α and longitude β that define the virtual camera position P1 are changed, the direction in which the 3D model 3dm is photographed from the virtual camera position P1 changes. For this reason, when the latitude α and longitude β that define the virtual camera position P1 are changed, the two-dimensional shape of the outline of the 3D model 3dm drawn by a thick two-dot chain line in the reference image Iv1 as the reference shape information may change, for example, as shown in FIG.

[0120] Furthermore, even if the distance D that defines the virtual camera position P1 is changed, the direction in which the 3D model 3dm is photographed from the virtual camera position P1 does not change. For this reason, if the distance D that defines the virtual camera position P1 is changed, for example, as shown in Fig. 16, the two-dimensional shape of the outline of the 3D model 3dm drawn by a thick two-dot chain line in the reference image Iv1 as the reference shape information may change in size without changing in shape.

[0121] Also, here, when the roll angle γ that determines the attitude of the virtual camera at the virtual camera position P1 is changed, for example, as shown in Figure 17, the two-dimensional shape of the outline of the 3D model 3dm drawn with a thick dotted line in the reference image Iv1 as reference shape information can change orientation without changing the shape or size.

[0122] In the first embodiment, in order to efficiently perform the search process, the search processing unit 61 performs a primary search process and then a secondary search process using the result of the primary search process. In other words, the search process includes a primary search process and a secondary search process that are performed in sequence.

[0123] In the primary search process, for example, a limited number of virtual camera positions P1 are set within the range of the initial setting range for the search target part defined in the setting range information 58. Here, for example, the range of the distance D from the reference point Po of the 3D model 3dm in which the virtual camera position P1 is set is limited to a first numerical range that is a distance range included in the initial setting range for the search target part defined in the setting range information 58. For example, the range of latitude α in which the virtual camera position P1 is set is limited to a second numerical range that is a latitude range included in the initial setting range for the search target part defined in the setting range information 58. For example, the range of latitude β in which the virtual camera position P1 is set is limited to a third numerical range that is a longitude range included in the initial setting range for the search target part defined in the setting range information 58.

[0124] In addition, in the primary search process, the latitude α and longitude β are further roughly varied to a certain extent, and a numerical value indicating the degree of match between the reference shape information and the actual shape information is calculated for a limited number of virtual camera positions P1, and the virtual camera position P1 (also called the high-match virtual camera position) at which the degree of match between the reference shape information and the actual shape information is greatest is detected.

[0125] In the secondary search process, a virtual camera position P1 where the degree of matching between the reference shape information and the actual shape information is greater is searched for in more detail based on the high-matching virtual camera position detected in the primary search process. In other words, in the secondary search process, the above-mentioned maximum-matching virtual camera position is searched for based on the high-matching virtual camera position. That is, the search processing unit 61 searches for the maximum-matching virtual camera position based on the high-matching virtual camera position.

[0126] When a numerical value indicating the degree of matching between the reference shape information and the actual shape information is calculated in the first search process and the second search process, for example, a roll angle γ at which the degree of matching is greater is also obtained, and the roll angle γ is also taken into consideration when calculating the numerical value indicating the degree of matching. As a result, the latitude α, longitude β, distance D, and roll angle γ for the maximum matching virtual camera position, which is the virtual camera position P1 at which the degree of matching between the reference shape information and the actual shape information is the greatest, can be acquired as a result of the search process. The orientation of the 3D model 3dm based on the maximum matching virtual camera position can be recognized by these latitude α, longitude β, distance D, and roll angle γ. Therefore, according to the result of the search process, the orientation of the search target part based on the position of the camera CM that captured the actual image capturing the search target part can be recognized. In other words, according to the result of the search process, the orientation of the search target part captured in the actual image can be recognized. Here, the orientation of the search target part may be recognized in the form of, for example, latitude α, longitude β, distance D, and roll angle γ.

[0127] <3-2-2. Functional configuration of the search processing unit> The search processing unit 61 has, as a plurality of function processing units, for example, a first search processing unit 611 and a second search processing unit 612. The first search processing unit 611 and the second search processing unit 612 are realized, for example, by the CPU in the calculation unit 45a reading and executing a program Pg1 stored in the storage unit 45b. The first search processing unit 611 performs processing related to the first search processing, and the second search processing unit 612 performs processing related to the second search processing.

[0128] <3-2-2-1. First search processing unit> The first search processing unit 611 includes, as a plurality of functional processing units, for example, a density determination unit 6111, a first shape information acquisition unit 6112, a first calculation unit 6113, and a first detection unit 6114. The density determination unit 6111, the first shape information acquisition unit 6112, the first calculation unit 6113, and the first detection unit 6114 are realized, for example, by the CPU in the calculation unit 45a reading and executing the program Pg1 stored in the storage unit 45b. In the first search processing unit 611, for example, the first shape information acquisition unit 6112, the first calculation unit 6113, and the first detection unit 6114 can realize the first search process.

[0129] <<Density determination unit 6111>> The density determination unit 6111 determines areas with low density of the contours of the components (also called low-density areas) for the edge image as the actual shape information acquired by the contour extraction unit 592, for example. The density determination unit 6111 calculates, for example, the proportion of the contours of the components in each area of ​​a predetermined size (also called unit determination area) in the edge image as the density of the contours of the components, and can determine a unit determination area whose density is equal to or less than a predetermined value as a low-density area. The predetermined size of the unit determination area can be set according to, for example, the size of the reference image as the above-mentioned reference shape information. The predetermined value can be set to a numerical value such as zero.

[0130] For example, when zero is applied to the predetermined value, a unit determination region in the edge image that does not include any contour of a component is determined to be a low-density region. The determination result in the density determination unit 6111 is provided to the first calculation unit 6113 and the second search processing unit 612. Here, it is estimated that the low-density region does not include the contour of the search target component. For this reason, for example, in the first calculation unit 6113 and the second calculation unit 6124 described later, the low-density region in the edge image as actual shape information is excluded from the calculation for calculating a numerical value indicating the degree of match between the actual shape information and the reference shape information, thereby reducing the amount of calculation required for the search process.

[0131] <<First shape information acquisition unit 6112>> The first shape information acquisition unit 6112 acquires reference shape information generated for each of the first virtual camera positions, assuming a case where the 3D model 3dm of the search target part is photographed from each of the first virtual camera positions, based on, for example, the three-dimensional design information and the setting range information 58 related to the search target part stored in the storage unit 45b or the like. The first virtual camera positions are located along a plurality of virtual spheres that surround the 3D model 3dm of the search target part with the reference point Po of the 3D model 3dm of the search target part as the center, and correspond to at least one initial setting range of one or more initial setting ranges for the search target part defined in the setting range information 58. Here, when the range of the position and orientation of the search target part is limited, the amount of calculation in the search process can be reduced by limiting the range in which the first virtual camera positions are set. This can reduce the amount of calculation for recognizing the orientation of the search target part captured in the actual image. As a result, the orientation of the search target part can be efficiently recognized in the substrate processing apparatus 1.

[0132] Here, the reference shape information is information generated based on three-dimensional design information regarding the search target part, and is information regarding a two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from a first virtual camera position. The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane by a process such as rendering. The reference point Po of the 3D model 3dm of the search target part may be set to a center point such as a center of gravity of the 3D model 3dm of the search target part. At least one of the one or more initial setting ranges for the search target part includes an initial setting range corresponding to a position where the search target part at the confirmation required timing should be at the confirmation required timing. The multiple virtual camera positions according to at least one of the one or more initial setting ranges for the search target part may be multiple virtual camera positions that are virtually set within the range of at least one of the one or more initial setting ranges for the search target part.

[0133] Here, for example, it is assumed that one or more initial setting ranges for the search target part defined in the setting range information 58 include the first initial setting range for the first position, and that the timing is such that the search target part is positioned at the first position by the moving mechanism. At this timing, the search processing unit 61 can acquire reference shape information generated for each of the multiple first virtual camera positions corresponding to the first initial setting range, for example, by the first shape information acquisition unit 6112, based on the three-dimensional design information and setting range information 58 for the search target part stored in the storage unit 45b or the like, assuming a case where a 3D model 3dm of the search target part is photographed from each of the multiple first virtual camera positions corresponding to the first initial setting range. Thereby, for example, when the search target part moves between the first position and the second position, it is possible to limit the range in which the multiple first virtual camera positions are set for the first position where the range in which the orientation of the search target part changes is more limited. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, in the substrate processing apparatus 1, it is possible to efficiently recognize the orientation of the search target part. Here, the first initial setting range corresponds to at least one of the one or more initial setting ranges for the search target part described above. In other words, at least one of the one or more initial setting ranges for the search target part described above is the first initial setting range. The multiple virtual camera positions corresponding to the first initial setting range for the search target part may be multiple virtual camera positions that are virtually set within the range of the first initial setting range for the search target part.

[0134] For example, if the part to be searched for at the confirmation-required timing is the first chuck and the chuck moving mechanism 17 is caused to position the first chuck in an open position as a first position, which is a first A position, the first initial setting range may be the first A initial setting range for the open position of the first chuck. For example, if the part to be searched for at the confirmation-required timing is the guard 23 and the guard moving mechanism 25 is caused to position the guard 23 in a lowered position as a first position, which is a first B position, the first initial setting range may be the first B initial setting range for the lowered position of the guard 23. For example, if the part to be searched for at the confirmation-required timing is the nozzle 33 and the nozzle moving mechanism 35 is caused to position the nozzle 33 in a standby position as a first position, which is a first C position, the first initial setting range may be the first C initial setting range for the standby position of the nozzle 33.

[0135] Here, for example, it is assumed that one or more initial setting ranges for the search target part defined in the setting range information 58 include the second initial setting range for the second position, and that the timing is at which the search target part is positioned at the second position by the moving mechanism. At this timing, the search processing unit 61 can acquire reference shape information generated for each of the multiple first virtual camera positions corresponding to the second initial setting range, for example, by the first shape information acquisition unit 6112, based on the three-dimensional design information and setting range information 58 for the search target part stored in the storage unit 45b, etc., assuming a case where a 3D model 3dm of the search target part is photographed from each of the multiple first virtual camera positions corresponding to the second initial setting range. Thereby, for example, when the search target part moves between the first position and the second position, it is possible to limit the range in which the multiple first virtual camera positions are set for the second position where the range in which the orientation of the search target part changes is more limited. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, in the substrate processing apparatus 1, it is possible to efficiently recognize the orientation of the search target part. Here, the second initial setting range corresponds to at least one of the one or more initial setting ranges for the search target part described above. In other words, at least one of the one or more initial setting ranges for the search target part described above is the second initial setting range. The multiple virtual camera positions corresponding to the second initial setting range for the search target part may be multiple virtual camera positions that are virtually set within the range of the second initial setting range for the search target part.

[0136] For example, if the part to be searched for at the confirmation-required timing is the first chuck and the chuck moving mechanism 17 is caused to position the first chuck in a closed position as the second position, which is the second A position, the second initial setting range may be the second A initial setting range for the closed position of the first chuck. For example, if the part to be searched for at the confirmation-required timing is the guard 23 and the guard moving mechanism 25 is caused to position the guard 23 in a raised position as the second position, which is the second B position, the second initial setting range may be the second B initial setting range for the raised position of the guard 23. For example, if the part to be searched for at the confirmation-required timing is the nozzle 33 and the nozzle moving mechanism 35 is caused to position the nozzle 33 in a discharge position as the second position, which is the second C position, the second initial setting range may be the second C initial setting range for the discharge position of the nozzle 33.

[0137] Here, for example, as shown in FIG. 4, if one or more initial setting ranges for the search target part each include a first numerical range which is a distance range, a second numerical range which is a latitude range, and a third numerical range which is a longitude range, the range in which the multiple first virtual camera positions are set can be limited when the range in which the angular relationship and the distance between the camera CM and the search target part change is limited. This can reduce the amount of calculation in the search process. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Therefore, in the substrate processing apparatus 1, the orientation of the search target part can be efficiently recognized.

[0138] Further, here, for example, as shown in FIG. 4, if the setting range information 58 includes information that specifies one or more initial setting ranges related to the virtual camera position for each target part, as described above, the range in which the multiple first virtual camera positions are set for each search target part as a target part at the confirmation timing can be limited. This can reduce the amount of calculation in the search process. As a result, the amount of calculation for recognizing the posture of the search target part captured in the actual image can be reduced. Therefore, in the substrate processing apparatus 1, the posture of the search target part can be efficiently recognized. Here, the fact that the setting range information 58 includes information that specifies one or more initial setting ranges related to the virtual camera position for each target part is synonymous with the fact that the setting range information 58 specifies one or more initial setting ranges related to the virtual camera position for at least one of the two or more target parts as the one or more target parts.

[0139] In the first embodiment, more specifically, the first shape information acquisition unit 6112 acquires reference shape information generated for each of M1×T1 first virtual camera positions (M1 and T1 are both natural numbers of 2 or more) assuming a case where a 3D model 3dm of the search target part is photographed from each of the M1×T1 first virtual camera positions, which are a plurality of first virtual camera positions, based on three-dimensional design information and setting range information 58 related to the search target part stored in the storage unit 45b, etc. The M1×T1 first virtual camera positions are a plurality of virtual camera positions that are virtually set for each of M1 virtual faces in each of the T1 face sets when T1 face sets having mutually different distances from the reference point Po of the 3D model 3dm of the search target part are virtually set. The T1 surface collections each include a plurality of virtual surfaces located along a virtual sphere that surrounds the 3D model 3dm of the part to be searched for, with the reference point Po of the 3D model 3dm as its center, and are a plurality of surface collections corresponding to a first numerical range that is a distance range in at least one of the one or more initial setting ranges for the part to be searched for defined in the setting range information 58. The M1 virtual surfaces in each of the T1 surface collections are a plurality of virtual surfaces corresponding to a second numerical range that is a latitude range and a third numerical range that is a longitude range in at least one of the one or more initial setting ranges for the part to be searched for defined in the setting range information 58.

[0140] Here, for example, as shown in Figure 14, by setting the right-handed xyz coordinate system with the reference point Po of the 3D model 3dm of the part to be searched defined in the three-dimensional design information of the part to be searched defined as the origin, as well as the angle (latitude) α in the direction of rotation about the x-axis, the angle (longitude) β in the direction of rotation about the z-axis, and the distance D from the origin, the position and orientation of each virtual surface, as well as the position of each first virtual camera, can be defined.

[0141] Here, in each of the M1 virtual surfaces in each of the T1 surface aggregates, for example, a first virtual camera position is virtually set at a predetermined position of the virtual surface. For example, the position of the center of the virtual surface is applied to the predetermined position. For example, the center of gravity of the virtual surface is applied to the center of the virtual surface. The multiple surface aggregates according to the first numerical range, which is a range of distance in at least one initial setting range of one or more initial setting ranges for the search target part defined in the setting range information 58, may be, for example, multiple surface aggregates in which the distance from the reference point Po of the 3D model 3dm of the search target part is included within the range of this first numerical range, or multiple surface aggregates located along multiple virtual spherical surfaces in which the distance from the reference point Po of the 3D model 3dm of the search target part is included within the range of this first numerical range. In addition, the multiple virtual surfaces according to the second numerical range, which is a range of latitude, and the third numerical range, which is a range of longitude, in at least one initial setting range of one or more initial setting ranges for the search target part defined in the setting range information 58, may be multiple virtual surfaces in which the latitude is included within the range of this second numerical range and the longitude is included within the range of this third numerical range. Furthermore, the plurality of virtual surfaces may be a plurality of virtual surfaces whose respective predetermined positions have latitudes within the second numerical range and whose respective predetermined positions have longitudes within the third numerical range, and each predetermined position of each of the plurality of virtual surfaces may be, for example, a center such as a center of gravity of the virtual surface.

[0142] In the first embodiment, for example, a triangular surface is applied to each of the multiple virtual surfaces constituting the surface assembly, and a polyhedron composed of multiple triangular surfaces is applied to the surface assembly. In other words, a spherical polyhedron composed of multiple triangular surfaces is applied to the surface assembly. This makes it easy to set a surface assembly including multiple virtual surfaces. Here, an equilateral triangle may be applied to the triangle. If the virtual surface is a triangular surface, the center of the virtual surface may be, for example, the center of gravity of the triangle or the incenter.

[0143] In this first embodiment, the first shape information acquisition unit 6112 sets multiple first virtual camera positions based on three-dimensional design information and setting range information 58 regarding the part to be searched stored in, for example, a memory unit 45b, and generates reference shape information for each of the multiple first virtual camera positions, thereby acquiring the reference shape information generated for each of the multiple first virtual camera positions.

[0144] Fig. 18 is a diagram for explaining an example of a manner in which T1 surface aggregates As1 having mutually different distances from a reference point Po of a 3D model 3dm of a search target part are virtually set. Fig. 19 is a diagram showing an example of a setting manner of one surface aggregate As1. Fig. 20 is a diagram showing an example of a setting manner of a first virtual camera position P11 on one virtual surface St1.

[0145] In Fig. 18, the outer edges of T1 virtual spheres Sv1, which are virtually generated based on three-dimensional design information related to the search target part, have different distances from the reference point Po of the 3D model 3dm of the search target part, and surround the 3D model 3dm with the reference point Po of the 3D model 3dm of the search target part as the center, are drawn with thin two-dot chain lines. In Fig. 18, three virtual spheres Sv1 are drawn among the T1 virtual spheres Sv1, and the other virtual spheres Sv1 are omitted. More specifically, as the three virtual spheres Sv1, a first virtual sphere Sv1 (also called a virtual sphere Sv11) whose distance from the reference point Po is a distance D1, a second virtual sphere Sv1 (also called a virtual sphere Sv12) whose distance from the reference point Po is a distance D2, and a T1th virtual sphere Sv1 (also called a virtual sphere Sv1T1) whose distance from the reference point Po is a distance DT1 are drawn.

[0146] In FIG. 19, in order to avoid complicating the drawing, only the origin as the reference point Po of the right-handed xyz coordinate system is shown, and the three axes of xyz are omitted, and the angle (latitude) α and the angle (longitude) β are also omitted. In addition, in FIG. 19, the angle (roll angle) γ in the rotation direction about the straight line passing through the origin as the reference point Po and the first virtual camera position P11, which defines the attitude of the virtual camera that captures the 3D model 3dm of the search target part from the first virtual camera position, is also omitted. In addition, in FIG. 19, the outline of the front side part of the surface aggregate As1 is drawn with a solid line, and the outline of the back side part of the surface aggregate As1 is drawn with a thin dashed line. In addition, in FIG. 19, the outline of the 3D model 3dm of the search target part is drawn with a thick dashed line for convenience. In FIG. 19, for the sake of convenience, two triangular imaginary surfaces St1 as an example of the M1 imaginary surfaces St1 are indicated with the symbol "St1" and hatched with oblique lines.

[0147] In the first shape information acquisition unit 6112, for example, as shown in FIG. 18, a surface aggregate As1 as shown in FIG. 19 is virtually set along each of the T1 virtual spheres Sv1. As a result, the T1 surface aggregates As1 are virtually set. Here, the T1 virtual spheres Sv1 are set at a predetermined pitch, for example. The predetermined pitch is, for example, 1 centimeter (cm). The T1 surface aggregates As1 have shapes in a similar relationship that are enlarged and reduced with respect to the reference point Po of the 3D model 3dm of the search target part as the center. In other words, the T1 surface aggregates As1 have different sizes, but have the same shape and the same attitude with respect to the 3D model 3dm. In the example of FIG. 19, the surface aggregate As1 is a polyhedron composed of a plurality of triangular virtual surfaces St1. FIG. 19 shows an example of a spherical polyhedron in which the surface aggregate As1 is composed of about 200 triangular virtual surfaces St1. The number of the multiple virtual surfaces St1 constituting the surface aggregate As1 is not limited to about 200, and is set to, for example, about 100 to 300. Here, each of the T1 surface aggregates As1 includes multiple virtual surfaces St1 located along a virtual sphere surrounding the 3D model 3dm of the search target part with the reference point Po of the 3D model 3dm as the center. Then, for example, in each surface aggregate As1, a first virtual camera position P11 is virtually set for each of two virtual surfaces St1 as an example of M1 in the surface aggregate As1. In FIG. 19, the first virtual camera position P11 set for each of two virtual surfaces St1 as an example of M1 in one surface aggregate As1 is indicated by a black circle. Here, for example, as shown in FIG. 20, in each of two virtual surfaces St1 as an example of M1, a first virtual camera position P11 is set at the center of the virtual surface St1 as a predetermined position of the virtual surface St1. In FIG. 20, similarly to FIG. 19, the first virtual camera position P11 is indicated by a filled-in circle.

[0148] When generating reference shape information for each of the M1×T1 first virtual camera positions P11, the first shape information acquisition unit 6112 sets the shooting direction of the virtual camera that shoots the 3D model 3dm of the search target part from the first virtual camera position P11 to a direction from the first virtual camera position P11 toward the reference point Po of the 3D model 3dm of the search target part. In addition, at this time, the first shape information acquisition unit 6112 sets the roll angle γ that defines the attitude of the virtual camera that shoots the 3D model 3dm of the search target part from the first virtual camera position P11 to, for example, zero (0) degrees. Here, for each first virtual camera position P11, a reference image Iv1 is generated as reference shape information, for example, as shown in FIG. 11.

[0149] <<First Calculation Unit 6113>> The first calculation unit 6113 calculates a numerical value indicating the degree of agreement between information about the two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information for each of the multiple first virtual camera positions P11. More specifically, the first calculation unit 6113 calculates a numerical value indicating the degree of agreement between information about the two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information for each of the M1×T1 first virtual camera positions P11 as the multiple first virtual camera positions P11 described above.

[0150] Here, for example, it is assumed that one or more initial setting ranges related to the virtual camera position for the search target part defined in the setting range information 58 include the first initial setting range for the first position, and are at a timing when the search target part is positioned at the first position by the moving mechanism. At this timing, the search processing unit 61 can calculate, by the first calculation unit 6113, a numerical value indicating the degree of agreement between information on the two-dimensional shape of the object captured in the actual image capturing the search target part (actual shape information) and the reference shape information for each of a plurality of first virtual camera positions P11 corresponding to the first initial setting range. Here, the actual image capturing the search target part is an actual image captured by the camera CM at the timing when the search target part is positioned at the first position by the moving mechanism.

[0151] For example, if the search target part at the confirmation timing is the first chuck and the chuck moving mechanism 17 is caused to position the first chuck at an open position as the first position, 1A position, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the chuck moving mechanism 17 is caused to position the first chuck at the open position. For example, if the search target part at the confirmation timing is the guard 23 and the guard moving mechanism 25 is caused to position the guard 23 at a lowered position as the first position, 1B position, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the guard moving mechanism 25 is caused to position the guard 23 at the open position. For example, if the search target part at the confirmation timing is the nozzle 33 and the nozzle moving mechanism 35 is caused to position the nozzle 33 at a standby position as the first position, 1C position, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the nozzle moving mechanism 35 is caused to position the nozzle 33 at the standby position.

[0152] Here, for example, it is assumed that one or more initial setting ranges related to the virtual camera position for the search target part defined in the setting range information 58 include the second initial setting range for the second position, and that the timing is when the search target part is positioned at the second position by the moving mechanism. At this timing, the search processing unit 61 can calculate, by the first calculation unit 6113, a numerical value indicating the degree of agreement between information on the two-dimensional shape of the object captured in the actual image capturing the search target part (actual shape information) and the reference shape information for each of the multiple first virtual camera positions P11 corresponding to the second initial setting range. Here, the actual image capturing the search target part is an actual image captured by the camera CM at the timing when the search target part is positioned at the second position by the moving mechanism.

[0153] For example, if the search target part at the confirmation required timing is the first chuck, and the first chuck is moved to the closed position as the second position, the second A position, by the chuck moving mechanism 17, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the first chuck is moved to the closed position by the chuck moving mechanism 17. For example, if the search target part at the confirmation required timing is the guard 23, and the guard 23 is moved to the raised position as the second position, the second B position, by the guard moving mechanism 25, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the guard 23 is moved to the raised position by the guard moving mechanism 25. For example, if the search target part at the confirmation required timing is the nozzle 33, and the nozzle 33 is moved to the discharge position as the second position, the second C position, by the nozzle moving mechanism 35, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the nozzle 33 is moved to the discharge position by the nozzle moving mechanism 35.

[0154] Here, a specific example of the calculation process in the first calculation unit 6113 of a numerical value indicating the degree of matching between the actual shape information and the reference shape information for one first virtual camera position P11 will be described.

[0155] 21 to 25 are diagrams for explaining a specific example of a process for calculating a numerical value indicating the degree of match between actual shape information and reference shape information for one first virtual camera position P11.

[0156] 21 shows an example of a reference image Iv1 as reference shape information for one first virtual camera position P11. The first calculation unit 6113 calculates a numerical value indicating the degree of coincidence of the two-dimensional shapes of the contours between the reference image Iv1 as the reference shape information acquired by the first shape information acquisition unit 6112 and the edge image Ir3 (FIG. 9) as the actual shape information acquired by the image processing unit 59, for example.

[0157] When calculating a numerical value indicating the degree of match between an edge image as actual shape information and a reference image as reference shape information for each first virtual camera position P11, the first calculation unit 6113 performs, for example, [Process 1a] a process of detecting the amount of deviation in the orientation of the two-dimensional shape of the contour corresponding to the deviation in roll angle γ between the edge image and the reference image, [Process 1b] a process of rotating the edge image in accordance with the deviation in roll angle γ, and [Process 1c] a process of detecting the position of the area where the degree of match between the rotated edge image and the reference image is greatest, in the order listed.

[0158] In the above process 1a, the first calculation unit 6113 detects the deviation amount of the orientation of the two-dimensional shape of the contour, for example, by using the rotation invariant phase only correlation (RIPOC) method between the edge image as the actual shape information and the reference image as the reference shape information. The deviation amount of the orientation of the two-dimensional shape of the contour is the deviation amount of the rotation direction of the contour on the image. Here, as shown in FIG. 22, for example, the first calculation unit 6113 divides the edge image Ir3 as the actual shape information input from the image processing unit 59 into a plurality of comparison target regions Re1, and calculates the deviation amount γ1 in the rotation direction that can maximize the degree of matching of the contour with the reference image by the RIPOC method for each comparison target region Re1. This deviation amount γ1 corresponds to the roll angle γ at which the degree of matching between the reference image as the reference shape information and the edge image as the actual shape information can be maximized.

[0159] The size of each comparison target region Re1 is set to, for example, 1 time or more and several times or less (for example, 3 times) in each of the vertical and horizontal directions based on the size of the reference image. In addition, the multiple comparison target regions Re1 are set so that adjacent comparison target regions Re1 partially overlap each other. In the example of FIG. 22, six comparison target regions Re1 are set in the edge image Ir3. In FIG. 22, the outer edge of the first comparison target region Re11, which is the first comparison target region Re1, is drawn with a thick dashed line. The outer edge of the second comparison target region Re12, which is the second comparison target region Re1, is drawn with a thin dashed line. The outer edge of the third comparison target region Re13, which is the third comparison target region Re1, is drawn with a thick dashed line. The outer edge of the fourth comparison target region Re14, which is the fourth comparison target region Re1, is drawn with a thin dashed line. The outer edge of the fifth comparison target region Re15, which is the fifth comparison target region Re1, is drawn with a thick two-dot dashed line. The outer edge of the sixth comparison target region Re16, which is the sixth comparison target region Re1, is drawn with a thin two-dot chain line. Here, the first calculation unit 6113 detects one comparison target region Re1 among the multiple comparison target regions Re1 that is likely to have the greatest degree of matching of the contour with the reference image, and a deviation amount γ1 in the rotation direction that is likely to have the greatest degree of matching of the contour between this one comparison target region Re1 and the reference image.

[0160] In the above process 1b, the first calculation unit 6113 rotates, for example, the edge image as the actual shape information so as to correct the deviation amount γ1 detected in the above process 1a. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the edge image as the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information. Here, the first calculation unit 6113 rotates, for example, the edge image of the comparison target area Re1 detected in the above process 1a so as to correct the deviation amount γ1 detected in the above process 1a. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the comparison target area Re1 constituting the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information. FIGS. 23 and 24 show, for example, a state in which the edge image of the fourth comparison target area Re14, which is the fourth comparison target area Re1 shown in FIG. 23, is rotated by the deviation amount γ1 to generate the rotated edge image Ir4 shown in FIG. 24. In FIG. 24, the outer edge of the edge image of the fourth comparison region Re14 before rotation is diagrammatically indicated by a thin two-dot chain line.

[0161] In the above process 1c, the first calculation unit 6113 performs template matching using a reference image as reference shape information, for example, on the rotated edge image related to the actual shape information generated in the above process 1b. Here, the first calculation unit 6113, for example, scans the reference image in the rotated edge image, and detects the position of an area where the degree of match (similarity) between each partial area in the rotated edge image and the reference image is the highest. This allows the position (also called a matching candidate position) of an area where the degree of match between the two-dimensional shape of the contour of the 3D model 3dm of the search target part in the reference image is the highest in the edge image as actual shape information to be detected.

[0162] Here, the degree of matching is not particularly limited, but may be, for example, the degree of matching of the reference image with respect to a partial region of the edge image after rotation when each partial region of the edge image after rotation is superimposed on the reference image. This degree of matching (similarity) may be expressed using a score (also called a matching score) indicating a known degree of matching (similarity), such as the sum of squared differences of pixel values, the sum of absolute values ​​of differences of pixel values, normalized cross-correlation, or zero-mean normalized cross-correlation. In addition, when performing the above template matching, a numerical value indicating the degree of matching (similarity) when the degree of matching (similarity) between the partial region in the edge image after rotation and the reference image is maximized is calculated as a numerical value indicating the degree of matching between the edge image as actual shape information and the reference image as reference shape information for the first virtual camera position P11. In other words, for each first virtual camera position P11, the degree of match between the reference image and a partial area in the rotated edge image is maximized, and this degree of match is regarded as the degree of match between the edge image as actual shape information and the reference image as reference shape information.

[0163] Here, the numerical value indicating the degree of agreement may be, for example, a matching score indicating the degree of agreement. For example, the greater the degree of agreement between the edge image as the actual shape information and the reference image as the reference shape information, the greater the matching score indicating the degree of agreement may be, or the smaller the matching score indicating the degree of agreement may be. In other words, there are cases where the greater the matching score indicating the degree of agreement, the greater the degree of agreement between the edge image as the actual shape information and the reference image as the reference shape information, and cases where the smaller the matching score indicating the degree of agreement, the greater the degree of agreement between the edge image as the actual shape information and the reference image as the reference shape information. In the following, unless a particular condition is described, the greater the degree of agreement between the edge image as the actual shape information and the reference image as the reference shape information, the greater the matching score indicating the degree of agreement. In FIG. 25, the outer edge of an example of a matching candidate position Pm0 in the edge image Ir4 is depicted by a thin two-dot chain line.

[0164] <<First detection unit 6114>> The first detection unit 6114 detects a high-matching virtual camera position, which is a first virtual camera position P11 having the highest degree of match between information about a two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information, among the multiple first virtual camera positions P11, based on the calculation result by the first calculation unit 6113. More specifically, the first detection unit 6114 detects a high-matching virtual camera position, which is a first virtual camera position having the highest degree of match between the actual shape information and the reference shape information, among M1×T1 first virtual camera positions P11 as the multiple first virtual camera positions P11, based on the calculation result by the first calculation unit 6113.

[0165] Here, for example, among the M1×T1 first virtual camera positions P11 as the multiple first virtual camera positions P11, the first virtual camera position P11 having the largest matching score calculated by the first calculation unit 6113 between information on the two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information may be detected as the high-match virtual camera position. Furthermore, when detecting the high-match virtual camera position, the first detection unit 6114 also detects the matching candidate position detected for the high-match virtual camera position by the first calculation unit 6113 as the target part candidate position.

[0166] Here, for example, it is assumed that one or more initial setting ranges related to the virtual camera position for the search target part defined in the setting range information 58 include the first initial setting range for the first position, and that it is at a timing when the search target part is positioned at the first position by the moving mechanism. At this timing, the search processing unit 61 can detect, by the first detection unit 6114, from among the multiple first virtual camera positions P11 corresponding to the first initial setting range, a high-matching virtual camera position that is the first virtual camera position P11 having the highest match between information about the two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information, based on the calculation result by the first calculation unit 6113. Here, the actual image capturing the search target part is an actual image captured by the camera CM at the timing when the search target part is positioned at the first position by the moving mechanism.

[0167] As described above, for example, if the search target part at the confirmation required timing is the first chuck and the first chuck is moved to the open position as the first position, 1A position, by the chuck moving mechanism 17, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the first chuck is moved to the open position by the chuck moving mechanism 17. For example, if the search target part at the confirmation required timing is the guard 23 and the guard 23 is moved to the lowered position as the first position, 1B position, by the guard moving mechanism 25, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the guard 23 is moved to the open position by the guard moving mechanism 25. For example, if the search target part at the confirmation required timing is the nozzle 33 and the nozzle 33 is moved to the standby position as the first position, 1C position, by the nozzle moving mechanism 35, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the nozzle 33 is moved to the standby position by the nozzle moving mechanism 35.

[0168] Here, for example, it is assumed that one or more initial setting ranges related to the virtual camera position for the search target part defined by the setting range information 58 include the second initial setting range for the second position, and that the timing is when the search target part is positioned at the second position by the moving mechanism. At this timing, the search processing unit 61 can detect, by the first detection unit 6114, from among the multiple first virtual camera positions P11 corresponding to the second initial setting range, a high-matching virtual camera position that is the first virtual camera position P11 having the highest match between information about the two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information, based on the calculation result by the first calculation unit 6113. Here, the actual image capturing the search target part is an actual image captured by the camera CM at the timing when the search target part is positioned at the second position by the moving mechanism.

[0169] As described above, for example, if the search target part at the confirmation required timing is the first chuck and the first chuck is moved to the closed position as the second position, 2A position, by the chuck moving mechanism 17, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the first chuck is moved to the closed position by the chuck moving mechanism 17. For example, if the search target part at the confirmation required timing is the guard 23 and the guard 23 is moved to the raised position as the second position, 2B position, by the guard moving mechanism 25, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the guard 23 is moved to the raised position by the guard moving mechanism 25. For example, if the search target part at the confirmation required timing is the nozzle 33 and the nozzle 33 is moved to the discharge position as the second position, 2C position, by the nozzle moving mechanism 35, the actual image capturing the search target part may be an actual image captured by the camera CM at the timing when the nozzle 33 is moved to the discharge position by the nozzle moving mechanism 35.

[0170] <3-2-2-2. Second search processing section> The second search processing unit 612 includes, as a plurality of functional processing units, for example, a matching target area setting unit 6121, a division surface generating unit 6122, a second shape information generating unit 6123, a second calculation unit 6124, and a second detection unit 6125. The matching target area setting unit 6121, the division surface generating unit 6122, the second shape information generating unit 6123, the second calculation unit 6124, and the second detection unit 6125 are realized, for example, by the CPU reading and executing the program Pg1 stored in the storage unit 45b in the calculation unit 45a. In the second search processing unit 612, for example, the division surface generating unit 6122, the second shape information generating unit 6123, the second calculation unit 6124, and the second detection unit 6125 can realize the secondary search process.

[0171] <<Matching target area setting unit 6121>> The matching target area setting unit 6121 sets an area (also referred to as a matching target area) to be used for matching (matching) processing in the second calculation unit 6124 in the edge image as actual shape information acquired by the contour extraction unit 592, based on the target part candidate position detected by the first detection unit 6114. This reduces the amount of calculation in the processing in the second calculation unit 6124, and the efficiency of the processing in the second calculation unit 6124 can be improved.

[0172] Fig. 26 is a diagram showing a specific example of setting the matching target area Re2 in the edge image Ir3 as actual shape information acquired by the contour extraction unit 592. As shown in Fig. 26, a region that includes the target part candidate position Pm1 detected by the first detection unit 6114 and has a size larger than the target part candidate position Pm1 is set as the matching target area Re2 in the edge image Ir3. In Fig. 26, the outer edge of the target part candidate position Pm1 is drawn with a thin two-dot chain line, and the outer edge of the matching target area Re2 is drawn with a thick dashed line. The size of the matching target area Re2 is set to be, for example, greater than 1 time and less than or equal to several times (for example, 2 times) the size of the target part candidate position Pm1 in both the vertical and horizontal directions.

[0173] <<Division plane generation unit 6122>> The split surface generating unit 6122 splits each of the T2 (T2 is a natural number of 2 or more) virtual surfaces St1, which include the high-matching virtual surface St1m that is a virtual surface on which the high-matching virtual camera position detected by the first detection unit 6114 is virtually set, among the M1 virtual surfaces St1 in each of the T1 surface aggregates As1, using the same rule. As a result, the split surface generating unit 6122 generates M2 (M2 is a natural number of 2 or more) virtual surfaces (also called virtual split surfaces) for each of the T2 virtual surfaces St1, thereby generating M2×T2 virtual split surfaces. Here, each of the T2 virtual surfaces St1 intersects with a straight line passing through the reference point Po and the high-matching virtual camera position P11m on the side of the reference point Po of the 3D model 3dm of the search target part that is closer to the high-matching virtual camera position P11m than the reference point Po of the 3D model 3dm of the search target part. The number T2 is, for example, the same as the number T1. The number T2 may be, for example, less than the number T1.

[0174] FIG. 27 is an image diagram showing an example of T2 virtual surfaces St1. In FIG. 27, a direction (also called a first direction) Dr11 from the reference point Po of the 3D model 3dm of the target part toward the high-matching virtual camera position P11m is shown by an arrow drawn with a thin line, and a straight line Ln11 passing through the reference point Po and the high-matching virtual camera position P11m is drawn with a thin two-dot chain line. In FIG. 27, a line passing through the reference point Po and passing through each of the outer edges of the T2 virtual surfaces St1 is drawn with a thin broken line. In FIG. 27, for convenience, three virtual surfaces St1 are drawn among the T2 virtual surfaces St1, and the illustration of the other virtual surfaces St1 is omitted. As shown in FIG. 27, each of the T2 virtual surfaces St1 intersects with a portion of the straight line Ln11 located on the side of the high-matching virtual camera position P11m from the reference point Po. Here, the T2 imaginary surfaces St1 have shapes that are similar to each other and are enlarged and reduced in radial directions with the reference point Po of the 3D model 3dm of the search target part as the center.

[0175] Here, the same rule for dividing the T2 virtual surfaces St1 may be a rule for dividing the T2 virtual surfaces St1 in the same shape as viewed from the reference point Po of the 3D model 3dm of the search target part. In other words, the same rule for dividing the T2 virtual surfaces St1 may be a rule for dividing each of the T2 virtual surfaces St1 with a line having a shape in a similar relationship that is enlarged and reduced in a radial direction centered on the reference point Po as viewed from the reference point Po of the 3D model 3dm of the search target part. In other words, the same rule for dividing the T2 virtual surfaces St1 may be a rule for dividing the T2 virtual surfaces St1 such that T2 divided virtual surfaces of the same shape are generated for each of the M2 virtual divided surfaces between the T2 virtual surfaces St1 as viewed from the reference point Po of the 3D model 3dm of the search target part. As a result, in the T2 virtual surfaces St1, there are T2 virtual dividing surfaces having shapes in a similar relationship that are mutually enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the target part for each of the M2 virtual dividing surfaces. If the same rule for dividing the T2 virtual surfaces St1 is, for example, a rule for dividing the dividing target surface into a plurality of surfaces by a plurality of line segments respectively connecting the center point of the dividing target surface (also called the dividing target surface) and all vertices of the dividing target surface, the dividing target surface can be easily divided. If each of the T2 virtual surfaces St1 is a triangular surface, the number M2 may be three.

[0176] FIG. 28 is a diagram showing a specific example of dividing a high-matching virtual surface St1m among T2 virtual surfaces St1 into a plurality of virtual split surfaces St2. FIG. 28 shows a specific example of a case where each of a plurality of virtual surfaces St1 constituting the surface aggregate As1 is a triangular surface as shown in FIG. 19. In FIG. 28, the high-matching virtual camera position P11m is shown by a white circle. As shown in FIG. 28, the split surface generating unit 6122 divides the high-matching virtual surface St1m into three virtual split surfaces St2 as M2 virtual split surfaces St2, for example, by three line segments connecting three vertices in the high-matching virtual surface St1m to the high-matching virtual camera position P11m. This makes it easy to divide the virtual surface St1. In this case, the split surface generating unit 6122 splits each of the T2 virtual surfaces St1 into three virtual split surfaces St2 by three line segments connecting the three vertices and the first virtual camera position P11, similar to the high matching virtual surface St1m shown in Fig. 28. As a result, each of the T2 virtual surfaces St1 is split into three triangular virtual split surfaces St2.

[0177] <<Second shape information generation unit 6123>> The second shape information generating unit 6123 generates reference shape information for each of the M2×T2 second virtual camera positions P12, assuming a case where the 3D model 3dm of the search target part is photographed from each of the M2×T2 second virtual camera positions P12 as the multiple second virtual camera positions P12, based on three-dimensional design information on the search target part stored in the storage unit 45b or the like. The M2×T2 second virtual camera positions P12 are M2×T2 virtual camera positions that are virtually set for each of the M2×T2 virtual dividing planes St2 generated by the dividing plane generating unit 6122. Here again, the reference shape information is information generated based on three-dimensional design information on the search target part, and is information on the two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from the second virtual camera position P12. The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane by a process such as rendering.

[0178] The second shape information generating unit 6123 virtually sets the second virtual camera position P12 at a predetermined position of the virtual dividing surface St2 in each virtual dividing surface St2. For example, the center position of the virtual dividing surface St2 is applied to the predetermined position. For example, the center of gravity of the virtual dividing surface St2 is applied to the center of the virtual dividing surface St2. For example, as shown in FIG. 28, the second virtual camera position P12 is set at a predetermined position of each of M2 virtual dividing surfaces St2 generated by dividing the high matching virtual surface St1m among the T2 virtual surfaces St1. In FIG. 28, the second virtual camera position P12 is indicated by a black circle. In this case, for example, the second virtual camera position P12 is virtually set at a predetermined position of the virtual dividing surface St2 in each of M2×T2 virtual dividing surfaces St2 generated by dividing each of the T2 virtual surfaces St1, similarly to the M2 virtual dividing surfaces St2 generated by dividing the high matching virtual surface St1m shown in FIG. 28.

[0179] When generating reference shape information for each second virtual camera position P12, the second shape information generating unit 6123 sets the shooting direction of the virtual camera that shoots the 3D model 3dm of the search target part from the second virtual camera position P12 to a direction from the second virtual camera position P12 toward the reference point Po of the 3D model 3dm of the search target part. In addition, at this time, the second shape information generating unit 6123 sets the roll angle γ that defines the attitude of the virtual camera that shoots the 3D model 3dm of the search target part from the second virtual camera position P12 to, for example, zero (0) degrees. Here, for each second virtual camera position P12, a reference image Iv1 is generated as reference shape information, for example, as shown in FIG. 11.

[0180] <<Second Calculation Unit 6124>> The second calculation unit 6124 calculates a numerical value indicating the degree of match between information about the two-dimensional shape of the object captured in the actual image capturing the part to be searched (actual shape information) and the reference shape information for each of the M2×T2 second virtual camera positions P12.

[0181] As described above, in the first embodiment, the second virtual camera position P12 is set on each of the multiple virtual splitting planes St2 generated by splitting the multiple virtual planes St1 including the high-matching virtual plane St1m, which is the virtual plane St1 on which the high-matching virtual camera position P11m detected by the first detection unit 6114 among the M1×T1 virtual planes St1 is set, and a numerical value indicating the degree of match between the actual shape information and the reference shape information is calculated for each second virtual camera position P12. Therefore, the multiple virtual planes St1 including the high-matching virtual plane St1m and the multiple virtual splitting planes St2 are not unrelated planes, and at least one increase in the number and area of ​​the multiple virtual splitting planes St2 can be reduced. This can reduce the amount of calculation required to recognize the orientation of the search target part captured in the actual image. As a result, the substrate processing apparatus 1 can efficiently recognize the orientation of the search target part.

[0182] Here, a specific example of the calculation process in the second calculation unit 6124 of a numerical value indicating the degree of matching between the actual shape information and the reference shape information for one second virtual camera position P12 will be described.

[0183] A specific example of the calculation process of the numerical value indicating the degree of agreement between the actual shape information and the reference shape information for one second virtual camera position P12 in the second calculation unit 6124 may be similar to a specific example of the calculation process of the numerical value indicating the degree of agreement between the actual shape information and the reference shape information for one first virtual camera position P11 in the above-mentioned first calculation unit 6113. However, the second calculation unit 6124 calculates a numerical value indicating the degree of agreement of the two-dimensional shape of the contour between, for example, the reference image as the reference shape information acquired by the second shape information generation unit 6123 and the matching target area Re2 ( FIG. 26 ) set in the edge image as the actual shape information by the matching target area setting unit 6121.

[0184] When calculating a numerical value indicating the degree of match between an edge image as actual shape information and a reference image as reference shape information for each second virtual camera position P12, the second calculation unit 6124 performs, for example, the following processes in the order listed: [Process 2a] a process of detecting the amount of deviation in the orientation of the two-dimensional shape of the contour corresponding to the deviation in roll angle γ between the edge image and the reference image; [Process 2b] a process of rotating the edge image in accordance with the deviation in roll angle γ; and [Process 2c] a process of detecting the position of the area where the degree of match between the rotated edge image and the reference image is greatest.

[0185] In the above process 2a, similarly to the above process 1a, the second calculation unit 6124 detects the deviation amount of the orientation of the two-dimensional shape of the contour, for example, by using the RIPOC method between the edge image as the actual shape information and the reference image as the reference shape information. The deviation amount of the orientation of the two-dimensional shape of the contour is the deviation amount of the contour in the rotation direction on the image. Here, the second calculation unit 6124 detects, for example, by the RIPOC method, a deviation amount γ2 in the rotation direction in which the contour can be most consistent between the comparison target area Re2 ( FIG. 26 ) set in the edge image as the actual shape information by the comparison target area setting unit 6121 and the reference image. This deviation amount γ2 corresponds to a roll angle γ at which the degree of consistency between the reference image as the reference shape information and the edge image as the actual shape information can be greater.

[0186] In the above process 2b, similarly to the above process 1b, the second calculation unit 6124 rotates, for example, the edge image as the actual shape information so as to correct the deviation amount γ2 detected in the above process 2a. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the edge image as the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information. Here, the second calculation unit 6124 rotates, for example, the edge image of the matching target area Re2 so as to correct the deviation amount γ2 detected in the above process 2a. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the matching target area Re2 constituting the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information.

[0187] In the above process 2c, similarly to the above process 1c, the second calculation unit 6124 performs template matching using a reference image as reference shape information, for example, on the rotated edge image related to the actual shape information generated in the above process 2b. Here, the second calculation unit 6124, for example, scans the reference image in the rotated edge image, and detects the position of an area where the degree of match (similarity) between each partial area in the rotated edge image and the reference image is the highest. Thereby, the position (matching candidate position) of an area where the degree of match between the two-dimensional shape of the contour of the 3D model 3dm of the search target part in the reference image is the highest can be detected in the edge image as actual shape information.

[0188] Here, the degree of matching is not particularly limited, but for example, the degree of matching of the reference image with respect to the partial area of ​​the edge image after rotation when each partial area of ​​the edge image after rotation is superimposed on the reference image is applied. This degree of matching (similarity) may be expressed using a score (matching score) indicating a known degree of matching (similarity) such as the sum of squares of pixel value differences, the sum of absolute values ​​of pixel value differences, normalized cross-correlation, or zero-mean normalized cross-correlation, as described above. In addition, here, when performing the above template matching, a numerical value indicating the degree of matching (similarity) when the degree of matching (similarity) between the partial area in the edge image after rotation and the reference image is maximized is calculated as a numerical value indicating the degree of matching between the edge image as actual shape information and the reference image as reference shape information for the second virtual camera position P12. In other words, for each second virtual camera position P12, the degree of matching when the degree of matching of the reference image with respect to the partial area in the edge image after rotation is maximized is regarded as the degree of matching between the edge image as actual shape information and the reference image as reference shape information. Here too, the numerical value indicating the degree of match may be, for example, a matching score indicating the degree of match.

[0189] <<Second detection unit 6125>> The second detection unit 6125 detects the virtual camera position among the high-match virtual camera position P11m detected by the first detection unit 6114 and the M2×T2 second virtual camera positions P12 described above that has the highest match between information about the two-dimensional shape of the object captured in the actual image capturing the part to be searched for (actual shape information) and the reference shape information.

[0190] Here, for example, the matching score calculated for the high-match virtual camera position P11m by the first calculation unit 6113 is compared with the matching score calculated for each of the M2×T2 second virtual camera positions P12 calculated by the second calculation unit 6124. Then, for example, from among the high-match virtual camera position P11m and the M2×T2 second virtual camera positions P12, a virtual camera position that has the largest matching score calculated between information about the two-dimensional shape of an object captured in an actual image capturing the search processing part (actual shape information) and the reference shape information is detected.

[0191] Here, the virtual camera position detected by the second detection unit 6125 is regarded as the virtual camera position detected first by the second detection unit 6125 in the search process for the search target part. In this specification, Xth (X is a natural number) means the Xth when performing search process for one search target part. In this first embodiment, the virtual camera position detected first by the second detection unit 6125 is, for example, a first reference virtual camera position described later. Then, in this first embodiment, the search processing unit 61 searches for a maximum matching virtual camera position based on the virtual camera position detected by the second detection unit 6125 by the second search processing unit 612.

[0192] <3-2-2-3. Repeating unit processing> For example, the search processing unit 61 executes a first unit process by the second search processing unit 612, and then executes one or more n-th unit processes (n is a natural number of 2 or more).

[0193] The unit process means a specific process in the case where two or more identical or similar specific processes are repeated. In the first embodiment, the first unit process and the n-th unit process are similar processes, and the n-th unit processes are substantially the same processes. Here, the first unit process means the first unit process in two or more unit processes. The variable n, which is a natural number of 2 or more in the n-th unit process, indicates which unit process the n-th unit process is. In other words, the n-th unit process means the n-th unit process in two or more unit processes. For example, if the variable n is 2, the n-th unit process means the second unit process, which is the second unit process. From another perspective, when one or more n-th unit processes are executed, the variable n, which is a natural number of 2 or more, is set to 2 as an initial value, and each time the n-th unit process is executed, 1 is added to the variable n, and the next n-th unit process is executed. And, executing the first unit process and then executing the nth unit process one or more times means, for example, executing the first unit process, the second unit process, etc. For example, executing the nth unit process once after executing the first unit process means executing the first unit process, the second unit process, etc. For example, executing the nth unit process once after executing the first unit process means executing the first unit process, the second unit process, and the third unit process, etc.

[0194] The first unit process is started, for example, in response to the virtual camera position being detected for the first time by the second detection unit 6125 when the search processing unit 61 performs the secondary search process on the search target part.

[0195] Here, the search processing unit 61 performs the following 1A process, the following 1B process, the following 1C process, and the following 1D process in order in the first unit process by the second search processing unit 612. Also, the search processing unit 61 performs the following n-th A process, the following n-th B process, the following n-th C process, and the following n-th D process in order in the n-th unit process by the second search processing unit 612.

[0196] <<1A Processing>> The 1A process is a process in which the split surface generating unit 6122 generates the first M3×T3 virtual split surfaces (M3 and T3 are each a natural number of 2 or more). In this process, the split surface generating unit 6122 splits each of the T3 virtual split surfaces (also referred to as the first T3 virtual split surfaces) generated by dividing each of the T2 virtual surfaces St1 using the same rule. As a result, the split surface generating unit 6122 generates the first M3×T3 virtual split surfaces by generating M3 virtual split surfaces (also referred to as the first M3 virtual split surfaces) for each of the first T3 virtual split surfaces. Here, the first T3 virtual split surfaces include a virtual split surface (also referred to as the first reference virtual split surface) that includes the first reference virtual camera position, which is the virtual camera position detected first by the second detection unit 6125, and have mutually different distances from the reference point Po of the 3D model 3dm of the search target part. Furthermore, the first T3 virtual dividing surfaces are a plurality of virtual dividing surfaces that intersect with a straight line passing through the reference point Po and the first reference virtual camera position on the side of the reference point Po of the 3D model 3dm of the search target part toward the first reference virtual camera position. The T3 surfaces are, for example, the same as the T2 surfaces. The T3 surfaces may be, for example, less than the T2 surfaces.

[0197] FIG. 29 is an image diagram showing an example of the first T3 virtual dividing surfaces St2. In FIG. 29, a direction (also called a second direction) Dr12 from the reference point Po of the 3D model 3dm of the search target part toward the first reference virtual camera position Ps1 is shown by an arrow drawn with a thin line, and a straight line Ln12 passing through the reference point Po and the first reference virtual camera position Ps1 is drawn with a thin two-dot chain line. In FIG. 29, a line passing through the reference point Po and passing through each of the outer edges of the first T3 virtual dividing surfaces St2 is drawn with a thin dashed line. In FIG. 29, for convenience, three virtual dividing surfaces St2 are drawn among the first T3 virtual dividing surfaces St2, and the illustration of the other virtual dividing surfaces St2 is omitted. As shown in FIG. 29, each of the first T3 virtual dividing surfaces St2 intersects with a portion of the straight line Ln12 located on the first reference virtual camera position Ps1 side from the reference point Po. Here, the first T3 virtual dividing surfaces St2 have shapes that are similar to each other and are enlarged and reduced in radial directions with the reference point Po of the 3D model 3dm of the search target part as the center.

[0198] Here, the same rule for dividing the first T3 virtual dividing surfaces St2 may be a rule for dividing the first T3 virtual dividing surfaces St2 in the same shape as viewed from the reference point Po of the 3D model 3dm of the part to be searched. In other words, the same rule for dividing the first T3 virtual dividing surfaces St2 may be a rule for dividing each of the first T3 virtual dividing surfaces St2 with a line having a shape of a similar relationship that is enlarged and reduced in a radial direction centered on the reference point Po as viewed from the reference point Po of the 3D model 3dm of the part to be searched. In other words, the same rule for dividing the first T3 virtual dividing surfaces St2 may be a rule for dividing the first T3 virtual dividing surfaces St2 such that T3 divided virtual surfaces of the same shape are generated for each of the M3 virtual dividing surfaces between the first T3 virtual dividing surfaces St2 as viewed from the reference point Po of the 3D model 3dm of the part to be searched. As a result, in the first T3 virtual dividing surfaces St2, there are T3 virtual dividing surfaces having shapes in a similar relationship that are mutually enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the search target part for each of the M3 virtual dividing surfaces. If the same rule for dividing the first T3 virtual dividing surfaces St2 is, for example, a rule for dividing the dividing target surface into a plurality of surfaces by a plurality of line segments respectively connecting the center point of the dividing target surface to be divided and all vertices of the dividing target surface, the dividing target surface can be easily divided. For example, M3 may be the same as M2. If each of the first T3 virtual dividing surfaces St2 is a triangular surface, M3 may be three.

[0199] In this 1A process, the method of generating the first M3×T3 virtual dividing planes differs depending on the virtual camera position (first reference virtual camera position) Ps1 first detected by the second detection unit 6125. More specifically, the method of generating the first M3×T3 virtual dividing planes differs between a case where the first reference virtual camera position Ps1 is any one of the above-mentioned M2×T2 second virtual camera positions P12 (also referred to as the first first case) and a case where the first reference virtual camera position Ps1 is the high-match virtual camera position P11m detected by the first detection unit 6114 (also referred to as the first second case).

[0200] <<<The first case>>> When the second detection unit 6125 detects any one of the above-mentioned M2×T2 second virtual camera positions P12 as the first reference virtual camera position Ps1, the degree of match between information about the two-dimensional shape of the object captured in the actual image capturing the part to be searched (actual shape information) and the reference shape information is greater for any one of the above-mentioned M2×T2 second virtual camera positions P12 than for the high-match virtual camera position P11m.

[0201] In other words, the degree of agreement between the actual shape information and the reference shape information for any of the M2×T2 second virtual camera positions P12 calculated by the second calculation unit 6124 in the second search process is greater than the degree of agreement between the actual shape information and the reference shape information for the high-match virtual camera position P11m calculated by the first calculation unit 6113 in the first search process.

[0202] Therefore, the division surface generating unit 6122 generates the first M3×T3 virtual division surfaces by using the second virtual camera position P12 that has the greatest degree of agreement between information about the two-dimensional shape of the object captured in the actual image capturing the search target part (actual shape information) and the reference shape information as the first reference virtual camera position that is the basis for dividing the next virtual surface, out of the above-mentioned M2×T2 second virtual camera positions P12. This makes it possible to search for a virtual camera position that has a greater degree of agreement between the reference shape information and the actual shape information in the secondary search process, for example.

[0203] Here, the split surface generating unit 6122 splits each of the first T3 virtual split surfaces St2, which include a virtual split surface (first reference virtual split surface) including the first reference virtual camera position Ps1 among the above-mentioned M2×T2 virtual split surfaces St2 and have different distances from the reference point Po of the 3D model 3dm of the search target part, using the same rule. As a result, the split surface generating unit 6122 generates the first M3 virtual split surfaces, which are M3 virtual split surfaces for each of the first T3 virtual split surfaces St2, thereby generating the first M3×T3 virtual split surfaces. The first T3 virtual split surfaces St2 are a plurality of virtual split surfaces St2 that intersect with the straight line Ln12 passing through the reference point Po and the first reference virtual camera position Ps1 on the side of the first reference virtual camera position Ps1 of the 3D model 3dm of the search target part.

[0204] Here, the same rule for dividing the first T3 virtual division surfaces St2 may be, for example, as described above, a rule for dividing the division target surface into a plurality of surfaces by a plurality of line segments each connecting the center point of the division target surface to all vertices of the division target surface. For example, if each of the first T3 virtual division surfaces St2 is a triangle, each of the first T3 virtual division surfaces St2 may be divided into three virtual division surfaces that are M3 virtual division surfaces by three line segments each connecting three vertices to the second virtual camera position P12. As a result, each of the first T3 virtual division surfaces St2 may be divided into three triangular virtual division surfaces. As a result, the first M3×T3 virtual division surfaces are generated as M3×T3 virtual division surfaces.

[0205] Here, for example, assume that the upper left second virtual camera position P12 of the three second virtual camera positions P12 in Fig. 28 is detected as the first reference virtual camera position Ps1. In this case, as shown in Fig. 28, of the three virtual splitting planes St2 generated by splitting the virtual plane St1, the virtual splitting plane St2 including the first reference virtual camera position Ps1 becomes the first reference virtual splitting plane Ss1.

[0206] FIG. 30 is a diagram showing a schematic diagram of a specific example of dividing the first reference virtual dividing surface Ss1 of the first T3 virtual dividing surfaces St2 into a plurality of virtual dividing surfaces St3 in the first first case. FIG. 30 shows a specific example of a case where each of the first T3 virtual dividing surfaces St2 is a triangular surface. In FIG. 30, the first reference virtual camera position Ps1 is shown by a white circle. As shown in FIG. 30, the dividing surface generating unit 6122 divides the first reference virtual dividing surface Ss1 into three virtual dividing surfaces St3 as the first M3 virtual dividing surfaces, for example, by three line segments connecting three vertices in the first reference virtual dividing surface Ss1 and the first reference virtual camera position Ps1. This makes it easy to divide the first reference virtual dividing surface Ss1. In this case, the split surface generating unit 6122 splits each of the first T3 virtual split surfaces St2 into three virtual split surfaces St3 by three line segments connecting the three vertices and the second virtual camera position P12, similar to the first reference virtual split surface Ss1 shown in Fig. 30. As a result, each of the first T3 virtual split surfaces St2 is split into three triangular virtual split surfaces St3 as the first M3 virtual split surfaces.

[0207] <<<The second case of the first one>>> When the second detection unit 6125 detects the high-match virtual camera position P11m as the first reference virtual camera position Ps1, the degree of match between information about the two-dimensional shape of the object captured in the actual image capturing the part to be searched (actual shape information) and the reference shape information is greater for the high-match virtual camera position P11m than for any of the M2 × T2 second virtual camera positions P12 described above.

[0208] In other words, the degree of correspondence between the actual shape information and the reference shape information for all of the M2×T2 second virtual camera positions P12 calculated by the second calculation unit 6124 in the secondary search process is smaller than the degree of correspondence between the actual shape information and the reference shape information for the high-match virtual camera position P11m calculated by the first calculation unit 6113 in the primary search process.

[0209] Therefore, the dividing surface generating unit 6122 generates the first M3×T3 virtual dividing surfaces St3 by using the high-matching virtual camera position P11m as the first reference virtual camera position Ps1 that is the reference for dividing the next virtual surface. This makes it possible to search for a virtual camera position that has a higher match between the reference shape information and the actual shape information in the secondary search process, for example.

[0210] Here, the division surface generating unit 6122 divides each of the first T3 virtual division surfaces, which include a virtual division surface (first reference virtual division surface) including the first reference virtual camera position Ps1 that is the same as the high-matching virtual camera position P11m and have mutually different distances from the reference point Po of the 3D model 3dm of the search target part, using the same rule. As a result, the division surface generating unit 6122 generates the first M3 virtual division surfaces, which are M3 virtual division surfaces for each of the first T3 virtual division surfaces, thereby generating the first M3×T3 virtual division surfaces. The first T3 virtual division surfaces are a plurality of virtual division surfaces that intersect with the straight line Ln12 passing through the reference point Po and the first reference virtual camera position Ps1 on the side of the first reference virtual camera position Ps1 of the 3D model 3dm of the search target part.

[0211] Here, the first reference virtual split surface is set according to a predetermined rule. For example, among M4 (M4 is a natural number equal to or greater than 2) virtual split surfaces generated by dividing the high-matching virtual surface St1m, the virtual split surface that includes the high-matching virtual camera position P11m as the first reference virtual camera position Ps1 is set as the first reference virtual split surface. Here, each of the M4 virtual split surfaces is a surface generated by dividing the virtual surface St1, and therefore is set as a re-set virtual split surface St2. For example, if the virtual surface St1 is a triangular surface, the predetermined rule for setting the first reference virtual split surface may be a rule that, among four triangular virtual split surfaces generated by dividing the high-matching virtual surface St1m by lines connecting the midpoints of each side, one virtual split surface that includes the high-matching virtual camera position P11m is set as the first reference virtual split surface Ss1. Here, the midpoint of each side may be, for example, a point slightly shifted from the midpoint of each side.

[0212] FIG. 31 is a diagram showing a schematic diagram of a specific example of setting the first reference virtual split surface Ss1 in the first second case. FIG. 31 shows a specific example of a case where each of the multiple virtual surfaces St1 is a triangular surface. In FIG. 31, the high matching virtual camera position P11m as the first reference virtual camera position Ps1 is shown with a white circle. For example, as shown in FIG. 31, the high matching virtual surface St1m is divided by a line connecting the midpoints of each side of the high matching virtual surface St1m, thereby generating four triangular virtual split surfaces St2, which are M4 in number. Then, among the four triangular virtual split surfaces St2, the virtual split surface St2 that includes the high matching virtual camera position P11m as the first reference virtual camera position Ps1 is set as the first reference virtual split surface Ss1.

[0213] Here, the first T3 virtual split surfaces including the first reference virtual split surface Ss1 may be T3 virtual split surfaces each including the first virtual camera position P11 among the M4×T3 virtual split surfaces generated by dividing each of the T3 virtual surfaces St1 among the above T2 virtual surfaces St1 into M4 virtual split surfaces according to the same rule as the high matching virtual surface St1m. Here again, each of the M4 virtual split surfaces in each of the T3 virtual surfaces St1 is a surface generated by dividing each of the T3 virtual surfaces St1, and therefore may be the reset virtual split surface St2.

[0214] Here, the first T3 virtual dividing planes St2 including the first reference virtual dividing plane Ss1 are part of the M4×T3 virtual dividing planes St2, and have shapes in a similar relationship of mutually enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the search target part. The T3 virtual planes St1 have shapes in a similar relationship of mutually enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the search target part. Each of the T3 virtual planes St1 is divided into M4 virtual dividing planes by the same rule as the high matching virtual plane St1m, which means that each of the T3 virtual planes St1 is divided by the same rule. This same rule may be a rule that divides the T3 virtual planes St1 in the same shape as seen from the reference point Po of the 3D model 3dm of the search target part. In other words, this same rule may be a rule that divides each of the T3 virtual surfaces St1 with a line having a shape in a similar relationship that is enlarged and reduced in a radial direction centered on the reference point Po when viewed from the reference point Po of the 3D model 3dm of the part to be searched for. In other words, this same rule may be a rule that divides the T3 virtual surfaces St1 such that, when viewed from the reference point Po of the 3D model 3dm of the part to be searched for, T3 divided virtual surfaces of the same shape are generated for each of the M4 virtual divided surfaces between the T3 virtual surfaces St1.

[0215] Here, for example, each of the T3 virtual surfaces St1 may be divided by a line connecting the midpoints of each side, similarly to the high-matching virtual surface St1m shown in FIG. 31. As a result, each of the T3 virtual surfaces St1 may be divided into four triangular virtual split surfaces St2. From another perspective, when viewed from the reference point Po of the 3D model 3dm of the target part, each of the T3 virtual surfaces St1 is divided by a line having a shape of a similar relationship in which they are mutually enlarged and reduced in a radial direction centered on the reference point Po, thereby generating 4×T3 virtual split surfaces St2. Then, the first T3 virtual split surfaces St2 including the first reference virtual split surface Ss1 are part of the 4×T3 virtual split surfaces St2, and have a shape of a similar relationship in which they are mutually enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the search target part.

[0216] Here, the same rule for dividing the first T3 virtual division surfaces St2 may be, for example, a rule for dividing the division target surface into a plurality of surfaces by a plurality of line segments each connecting the center point of the division target surface to all vertices of the division target surface, as described above. For example, if each of the first T3 virtual division surfaces St2 is a triangle, each of the first T3 virtual division surfaces St2 may be divided into three virtual division surfaces that are M3 virtual division surfaces by three line segments each connecting three vertices to the first virtual camera position P11. As a result, each of the first T3 virtual division surfaces St2 may be divided into three triangular virtual division surfaces. As a result, the first M3×T3 virtual division surfaces are generated as M3×T3 virtual division surfaces.

[0217] FIG. 32 is a diagram showing a schematic diagram of a specific example of dividing the first reference virtual dividing surface Ss1 of the first T3 virtual dividing surfaces St2 into a plurality of virtual dividing surfaces St3 in the first second case. FIG. 32 shows a specific example of a case where each of the first T3 virtual dividing surfaces St2 is a triangular surface. In FIG. 32, the high matching virtual camera position P11m as the first reference virtual camera position Ps1 is shown by a white circle. As shown in FIG. 32, the dividing surface generating unit 6122 divides the first reference virtual dividing surface Ss1 into three virtual dividing surfaces St3 as the first M3 virtual dividing surfaces, for example, by three line segments connecting the three vertices of the first reference virtual dividing surface Ss1 and the first reference virtual camera position Ps1. This makes it easy to divide the first reference virtual dividing surface Ss1. In this case, the split surface generating unit 6122 splits each of the first T3 virtual split surfaces St2 into three virtual split surfaces St3 by three line segments connecting the three vertices and the second virtual camera position P12, similar to the first reference virtual split surface Ss1 shown in Fig. 32. As a result, each of the first T3 virtual split surfaces St2 is split into three triangular virtual split surfaces St3 as the first M3 virtual split surfaces.

[0218] <<1B Processing>> The 1B process is a process in which the second shape information generating unit 6123 generates reference shape information for each of the first M3×T3 third virtual camera positions, assuming a case in which the 3D model 3dm of the search target part is photographed from each of the M3×T3 virtual camera positions (also referred to as the first M3×T3 third virtual camera positions) that are virtually set for each of the first M3×T3 virtual dividing planes St3 generated in the 1A process, based on three-dimensional design information related to the target part stored in the storage unit 45b or the like. Here, too, the reference shape information is information generated based on three-dimensional design information related to the search target part, and is information about the two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from the virtual camera position. The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane by a process such as rendering.

[0219] In each of the first M3×T3 virtual dividing surfaces St3, for example, a third virtual camera position P13 is virtually set at a predetermined position of the virtual dividing surface St3. For example, the position of the center of the virtual dividing surface St3 is applied to the predetermined position. For example, the center of gravity of the virtual dividing surface St3 is applied to the center of the virtual dividing surface St3. For example, as shown in FIG. 30 or FIG. 32, the third virtual camera position P13 is set on each virtual dividing surface St3. If each of the first M3×T3 virtual dividing surfaces St3 is a triangular surface, the center of the virtual dividing surface St3 may be, for example, the center of gravity of the triangle or the inner center. In FIG. 30 and FIG. 32, the third virtual camera position P13 is indicated by a black circle.

[0220] In the second shape information generating unit 6123, for example, for each of the multiple third virtual camera positions P13 shown in FIG. 30 or FIG. 32, reference shape information is generated about the two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from the third virtual camera position P13. At this time, the photographing direction of the virtual camera photographing the 3D model 3dm of the search target part from the third virtual camera position P13 is set to the direction from the third virtual camera position P13 toward the reference point Po of the 3D model 3dm of the search target part. The roll angle γ that defines the attitude of the virtual camera photographing the 3D model 3dm of the search target part from the third virtual camera position P13 is set to, for example, zero (0) degrees. Here, for each third virtual camera position P13, a reference image Iv1 is generated as reference shape information, for example, as shown in FIG. 11.

[0221] <<1C Processing>> The 1C process is a process in which the second calculation unit 6124 calculates a numerical value indicating the degree of match between information about the two-dimensional shape of the object captured in the actual image capturing the part to be searched for (actual shape information) and the reference shape information for each of the first M3 × T3 third virtual camera positions P13.

[0222] In the 1C process, the second calculation unit 6124 calculates a numerical value indicating the degree of similarity of the two-dimensional shapes of the contours, for example, between a reference image as reference shape information generated by the second shape information generation unit 6123 in the 1B process and a matching target area Re2 (Figure 26) set in the edge image as actual shape information by the matching target area setting unit 6121.

[0223] In the 1C process, when the second calculation unit 6124 calculates a numerical value indicating the degree of match between an edge image as actual shape information and a reference image as reference shape information for each third virtual camera position P13, it performs, for example, the following processes in the order listed: [Process 2a1] a process of detecting the amount of deviation in the orientation of the two-dimensional shape of the contour corresponding to the deviation in roll angle γ between the edge image and the reference image; [Process 2b1] a process of rotating the edge image in accordance with the deviation in roll angle γ; and [Process 2c1] a process of detecting the position of the area where the degree of match between the rotated edge image and the reference image is greatest.

[0224] In the above process 2a1, similarly to the above process 2a, the second calculation unit 6124 detects the deviation amount of the orientation of the two-dimensional shape of the contour, for example, by using the RIPOC method between the edge image as the actual shape information and the reference image as the reference shape information. The deviation amount of the orientation of the two-dimensional shape of the contour is the deviation amount of the contour in the rotation direction on the image. Here, the second calculation unit 6124 detects, for example, by the RIPOC method, the deviation amount γ21 in the rotation direction at which the contour coincidence degree is the largest between the matching target area Re2 ( FIG. 26 ) set in the edge image as the actual shape information by the matching target area setting unit 6121 and the reference image. This deviation amount γ21 corresponds to the roll angle γ at which the coincidence degree between the reference image as the reference shape information and the edge image as the actual shape information is the largest.

[0225] In the above process 2b1, similarly to the above process 2b, the second calculation unit 6124 rotates, for example, the edge image as the actual shape information so as to correct the deviation amount γ21 detected in the above process 2a1. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the edge image as the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information. Here, the second calculation unit 6124 rotates, for example, the edge image of the matching target area Re2 so as to correct the deviation amount γ21 detected in the above process 2a1. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the matching target area Re2 constituting the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information.

[0226] In the above process 2c1, similarly to the above process 2c, the second calculation unit 6124 performs template matching using a reference image as reference shape information, for example, on the rotated edge image related to the actual shape information generated in the above process 2b1. Here, the second calculation unit 6124, for example, scans the reference image in the rotated edge image, and detects the position of an area where the degree of match (similarity) between each partial area in the rotated edge image and the reference image is maximum. Thereby, the position (matching candidate position) of an area where the degree of match between the two-dimensional shape of the contour of the 3D model 3dm of the search target part in the reference image is maximum in the edge image as actual shape information can be detected. In addition, here, when performing the above template matching, a numerical value indicating the degree of match (similarity) when the degree of match (similarity) between the partial area in the rotated edge image and the reference image is maximum is calculated as a numerical value indicating the degree of match between the edge image as actual shape information and the reference image as reference shape information for the third virtual camera position P13. In other words, for each third virtual camera position P13, the degree of matching between the reference image and a partial region in the edge image after rotation is maximized, and this degree of matching is regarded as the degree of matching between the edge image as the actual shape information and the reference image as the reference shape information. Here, too, the numerical value indicating the degree of matching may be, for example, a matching score indicating the degree of matching.

[0227] <<1D processing>> The 1D process is a process in which the second detection unit 6125 detects a second reference virtual camera position among the first reference virtual camera position Ps1 and the first M3 × T3 third virtual camera positions P13, which is the virtual camera position that has the greatest degree of match between information about the two-dimensional shape of an object captured in an actual image capturing the part to be searched (actual shape information) and the reference shape information.

[0228] Here, for example, the matching score calculated for the first reference virtual camera position Ps1 and the matching scores calculated for each of the first M3×T3 third virtual camera positions P13 by the second calculation unit 6124 in the 1C process are compared. Then, for example, among the first reference virtual camera position Ps1 and the first M3×T3 third virtual camera positions P13, the virtual camera position having the largest matching score calculated between information about the two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information is detected as the second reference virtual camera position.

[0229] <<nAth Processing>> The n-th A process is a process in which the division surface generating unit 6122 generates the n-th M3×T3 virtual division surfaces. In this process, the division surface generating unit 6122 divides each of the T3 virtual division surfaces (also referred to as the n-th T3 virtual division surfaces) generated by dividing each of the n-1-th T3 virtual division surfaces, using the same rule. As a result, the division surface generating unit 6122 generates the n-th M3×T3 virtual division surfaces by generating M3 virtual division surfaces (also referred to as the n-th M3 virtual division surfaces) for each of the n-th T3 virtual division surfaces. Here, the n-th T3 virtual division surfaces include a virtual division surface (also referred to as the n-th reference virtual division surface) that includes the n-th reference virtual camera position, which is the virtual camera position detected n-th by the second detection unit 6125, and have mutually different distances from the reference point Po of the 3D model 3dm of the search target part. Furthermore, the nth T3 virtual dividing surfaces are a plurality of virtual dividing surfaces that intersect with a straight line passing through the reference point Po and the nth reference virtual camera position on the side of the nth reference virtual camera position of the reference point Po of the 3D model 3dm of the part to be searched.

[0230] Fig. 33 is an image diagram showing an example of the second T3 virtual dividing surfaces St3, which is a specific example of the n-th T3 virtual dividing surfaces St3. In Fig. 33, a direction (also called a third direction) Dr13 from the reference point Po of the 3D model 3dm of the search target part toward the second reference virtual camera position Ps2 is shown by an arrow drawn with a thin line, and a straight line Ln13 passing through the reference point Po and the second reference virtual camera position Ps2 is drawn with a thin two-dot chain line. In Fig. 33, a line passing through the reference point Po and passing through each of the outer edges of the second T3 virtual dividing surfaces St3 is drawn with a thin broken line. In Fig. 33, for convenience, three virtual dividing surfaces St3 are drawn among the second T3 virtual dividing surfaces St3, and the illustration of the other virtual dividing surfaces St3 is omitted. 33, each of the second T3 virtual dividing surfaces St3 intersects with a portion of the straight line Ln13 that is located on the second reference virtual camera position Ps2 side of the reference point Po. Here, the second T3 virtual dividing surfaces St3 have similar shapes that are mutually enlarged and reduced in radial directions with the reference point Po of the 3D model 3dm of the search target part as the center.

[0231] Here, the same rule regarding the division of the n-th set of T3 virtual division planes may be a rule that divides the n-th set of T3 virtual division planes in the same shape when viewed from the reference point Po of the 3D model 3dm of the component to be searched. In other words, the same rule regarding the division of the n-th set of T3 virtual division planes may be a rule that divides each of the n-th set of T3 virtual division planes with lines having a similar relationship of being mutually enlarged and reduced in the radial direction centered on the reference point Po when viewed from the reference point Po of the 3D model 3dm of the component to be searched. Further in other words, the same rule regarding the division of the n-th set of T3 virtual division planes may be a rule that divides the n-th set of T3 virtual division planes so that for each of the M3 virtual division planes among the n-th set of T3 virtual division planes, T3 division virtual planes of the same shape are generated when viewed from the reference point Po of the 3D model 3dm of the component to be searched. The same rule regarding the division of the n-th set of T3 virtual division planes may be, for example, a rule that divides the division target surface into a plurality of surfaces by a plurality of line segments connecting the center point of the division target surface to be divided and all the vertices of this division target surface. Thereby, the division of the division target surface can be easily performed.

[0232] In this n-th A process, the method for generating the n-th set of M3×T3 virtual division planes differs according to the virtual camera position (n-th reference virtual camera position) detected by the second detection unit 6125 at the n-th time. More specifically, when the n-th reference virtual camera position is any one of the third virtual camera positions among the (n - 1)-th set of M3×T3 third virtual camera positions (also referred to as the n-th first case), and when the n-th reference virtual camera position is the (n - 1)-th reference virtual camera position (also referred to as the n-th second case), the method for generating the n-th set of M3×T3 virtual division planes is different.

[0233] <<<The n-th first case>>> When the second detection unit 6125 detects any one of the above-mentioned n-1th M3 × T3 third virtual camera positions as the nth reference virtual camera position, there is a greater degree of match between information about the two-dimensional shape of the object captured in the actual image capturing the part to be searched (actual shape information) and the reference shape information for any one of the above-mentioned n-1th M3 × T3 third virtual camera positions than for the n-1th reference virtual camera position.

[0234] For example, a matching score indicating the degree of match between actual shape information and reference shape information for any of the n-1th M3 × T3 third virtual camera positions is greater than a matching score indicating the degree of match between actual shape information and reference shape information for the n-1th reference virtual camera position.

[0235] Therefore, the dividing surface generating unit 6122 generates the nth M3×T3 virtual dividing surfaces by using the third virtual camera position among the above-mentioned n-1th M3×T3 third virtual camera positions that has the highest degree of agreement between information on the two-dimensional shape of the object captured in the actual image capturing the search target part (actual shape information) and the reference shape information as the nth reference virtual camera position that is the basis for dividing the next virtual surface. This makes it possible to search for a virtual camera position that has an even higher degree of agreement between the reference shape information and the actual shape information in the secondary search process, for example.

[0236] Here, the division surface generating unit 6122 divides each of the n-1th M3×T3 virtual division surfaces described above, including the nth reference virtual division surface that is a virtual division surface that includes the nth reference virtual camera position and has different distances from the reference point Po of the 3D model 3dm of the search target part, using the same rule. As a result, the division surface generating unit 6122 generates the nth M3×T3 virtual division surfaces by generating the nth M3 virtual division surfaces that are M3 virtual division surfaces for each of the nth T3 virtual division surfaces. The nth T3 virtual division surfaces are a plurality of virtual division surfaces that intersect with a straight line passing through the reference point Po and the nth reference virtual camera position on the side of the nth reference virtual camera position relative to the reference point Po of the 3D model 3dm of the search target part.

[0237] Here, the same rule for dividing the n-th T3 virtual division surfaces may be, for example, a rule for dividing the division target surface into a plurality of surfaces by a plurality of line segments each connecting the center point of the division target surface to all vertices of the division target surface, as described above. For example, if each of the n-th T3 virtual division surfaces is a triangle, each of the n-th T3 virtual division surfaces may be divided into three virtual division surfaces that are the n-th M3 virtual division surfaces by three line segments each connecting three vertices to the n-1th third virtual camera position. As a result, each of the n-th T3 virtual division surfaces may be divided into three triangular virtual division surfaces. As a result, the n-th M3×T3 virtual division surfaces are generated as M3×T3 virtual division surfaces.

[0238] Here, for example, it is assumed that one of the three third virtual camera positions P13 in Fig. 30, which is the lower third virtual camera position P13, is detected as the second reference virtual camera position Ps2. In this case, as shown in Fig. 30, the virtual dividing surface St3 that includes the second reference virtual camera position Ps2 among the three virtual dividing surfaces St3 generated by dividing the virtual dividing surface St2 becomes the second reference virtual dividing surface Ss2.

[0239] FIG. 34 is a diagram schematically showing a specific example of dividing a second reference virtual division surface Ss2 among the second T3 virtual division surfaces St3 in the second first case. FIG. 34 shows a specific example in the case where each of the second T3 virtual division surfaces St3 is a triangular surface. In FIG. 34, the second reference virtual camera position Ps2 is indicated by a white circle. As shown in FIG. 34, for example, the division surface generation unit 6122 divides the second reference virtual division surface Ss2 into three virtual division surfaces St3a as the second M3 virtual division surfaces by three line segments connecting three vertices on the second reference virtual division surface Ss2 and the second reference virtual camera position Ps2. Thereby, the division of the second reference virtual division surface Ss2 can be easily performed. In this case, the division surface generation unit 6122 divides each of the second T3 virtual division surfaces St3 into three virtual division surfaces St3a by three line segments connecting three vertices and the third virtual camera position P13, respectively, in the same manner as the second reference virtual division surface Ss2 shown in FIG. 34. Thereby, each of the second T3 virtual division surfaces St3 is divided into three triangular virtual division surfaces St3a as the second M3 virtual division surfaces.

[0240] <<<the n-th second case>>> When the (n - 1)-th reference virtual camera position is detected as the n-th reference virtual camera position by the second detection unit 6125, the degree of coincidence between the information (actual shape information) about the two-dimensional shape of the object captured in the actual image capturing the search target component and the reference shape information is greater for the (n - 1)-th reference virtual camera position than for any of the above-described (n - 1)-th M3×T3 third virtual camera positions.

[0241] In other words, the degree of coincidence between the actual shape information and the reference shape information for all of the (n - 1)-th M3×T3 third virtual camera positions calculated by the second calculation unit 6124 in the (n - 1)-th unit process is smaller than the degree of coincidence between the actual shape information and the reference shape information for the (n - 1)-th reference virtual camera position calculated by the first calculation unit 6113 or the second calculation unit 6124.

[0242] Therefore, the division surface generating unit 6122 generates the nth M3×T3 virtual division surfaces by using the (n-1)th reference virtual camera position as the nth reference virtual camera position that is the basis for dividing the next virtual division surface. This makes it possible to search for a virtual camera position that further increases the degree of match between the reference shape information and the actual shape information in the secondary search process, for example.

[0243] Here, the division surface generating unit 6122 divides each of the nth T3 virtual division surfaces, which include a virtual division surface (nth reference virtual division surface) that includes the nth reference virtual camera position that is the same as the n-1th reference virtual camera position and have different distances from the reference point Po of the 3D model 3dm of the search target part, using the same rule. As a result, the division surface generating unit 6122 generates nth M3 virtual division surfaces that are M3 virtual division surfaces for each of the nth T3 virtual division surfaces, thereby generating nth M3 x T3 virtual division surfaces. The nth T3 virtual division surfaces are a plurality of virtual division surfaces that intersect with a straight line passing through the reference point Po and the nth reference virtual camera position on the side of the nth reference virtual camera position relative to the reference point Po of the 3D model 3dm of the search target part.

[0244] Here, the n-th reference virtual split surface is set according to a predetermined rule. For example, when the variable n is 2, among M4 virtual split surfaces generated by dividing the first reference virtual split surface Ss1 among the M2×T2 virtual split surfaces St2, a virtual split surface including the second reference virtual camera position Ps2, which is the same as the first reference virtual camera position Ps1, is set as the second reference virtual split surface Ss2. Here, each of the M4 virtual split surfaces is a surface generated by dividing the virtual split surface St2, and therefore is set as a reset virtual split surface St3. For example, if the virtual split surface St2 is a triangular surface, a predetermined rule for setting the second reference virtual split surface may be a rule that, among four triangular virtual split surfaces St3 generated by dividing the first reference virtual split surface Ss1 by lines connecting the midpoints of each side, one virtual split surface St3 including the second reference virtual camera position Ps2 is set as the second reference virtual split surface Ss2. Here, the midpoint of each side may be, for example, a point slightly shifted from the midpoint of each side.

[0245] Also, for example, when the variable n is 3 or more, among the M4 virtual division surfaces generated by dividing the n-1th reference virtual division surface among the n-2th M3×T3 virtual division surfaces, the virtual division surface that includes the n-th reference virtual camera position that is the same as the n-1th reference virtual camera position is set as the n-th reference virtual division surface. For example, if each of the n-2th M3×T3 virtual division surfaces is a triangular surface, a predetermined rule for setting the n-th reference virtual division surface may be a rule that, among four triangular virtual division surfaces generated by dividing the n-1th reference virtual division surface by lines connecting the midpoints of each side, the virtual division surface that includes the n-th reference virtual camera position is set as the n-th reference virtual division surface. Here, too, the midpoint of each side may be, for example, a point slightly shifted from the midpoint of each side.

[0246] FIG. 35 is a diagram showing a schematic diagram of a specific example of setting the second reference virtual split surface Ss2 in the second second case. FIG. 35 shows a specific example of a case where each of the multiple virtual split surfaces St2 is a triangular surface. In FIG. 35, the second reference virtual camera position Ps2 is shown by a white circle. For example, as shown in FIG. 35, the first reference virtual split surface Ss1 is divided by lines connecting the midpoints of each side of the first reference virtual split surface Ss1, thereby generating four triangular virtual split surfaces St3, M4 in number. Then, among the four triangular virtual split surfaces St3, the virtual split surface St3 that includes the second reference virtual camera position Ps2 is set as the second reference virtual split surface Ss2.

[0247] Here, the second T3 virtual split surfaces including the second reference virtual split surface Ss2 may be T3 virtual split surfaces St3 each including the second virtual camera position P12 among the M4×T3 virtual split surfaces generated by dividing each of the above-mentioned first T3 virtual split surfaces St2 according to the same rule as the first reference virtual split surface Ss1. Here again, each of the M4 virtual split surfaces in each of the T3 virtual split surfaces St2 is a surface generated by dividing each of the T3 virtual split surfaces St2, and therefore may be the reset virtual split surface St3.

[0248] Here, the second T3 virtual splitting surfaces St3 including the second reference virtual splitting surface Ss2 are part of the M4×T3 virtual splitting surfaces St3, and have shapes in a similar relationship of mutually enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the search target part. The first T3 virtual splitting surfaces St2 have shapes in a similar relationship of mutually enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the search target part. Each of the first T3 virtual splitting surfaces St2 is divided according to the same rule as the first reference virtual splitting surface Ss1, which means that each of the first T3 virtual splitting surfaces St2 is divided according to the same rule. This same rule may be a rule that divides the first T3 virtual splitting surfaces St2 in the same shape as seen from the reference point Po of the 3D model 3dm of the search target part. In other words, this same rule may be a rule that divides each of the first T3 virtual dividing surfaces St2 with a line having a shape that is in a similar relationship with each other and is enlarged and reduced in a radial direction centered on the reference point Po when viewed from the reference point Po of the 3D model 3dm of the part to be searched for. In other words, this same rule may be a rule that divides the first T3 virtual dividing surfaces St2 such that T3 divided virtual surfaces of the same shape are generated for each of the M4 virtual dividing surfaces between the first T3 virtual dividing surfaces St2 when viewed from the reference point Po of the 3D model 3dm of the part to be searched for.

[0249] Here, for example, each of the first T3 virtual dividing surfaces St2 may be divided by a line connecting the midpoints of each side, similarly to the first reference virtual dividing surface Ss1 shown in FIG. 35. As a result, each of the first T3 virtual dividing surfaces St2 may be divided into four triangular virtual dividing surfaces St3. From another perspective, when viewed from the reference point Po of the 3D model 3dm of the search target part, each of the first T3 virtual dividing surfaces St2 is divided by a line having a shape of a similar relationship in which the first and second virtual dividing surfaces St2 are enlarged and reduced in a radial direction centered on the reference point Po, thereby generating 4×T3 virtual dividing surfaces St3. Then, the second T3 virtual dividing surfaces St3 including the second reference virtual dividing surface Ss2 is a part of the 4×T3 virtual dividing surfaces St3, and has a shape of a similar relationship in which the first and second virtual dividing surfaces St2 are enlarged and reduced in a radial direction centered on the reference point Po of the 3D model 3dm of the search target part.

[0250] Here, the same rule for dividing the n-th T3 virtual dividing surfaces may be, for example, as described above, a rule for dividing the dividing surface into a plurality of surfaces by a plurality of line segments each connecting the center point of the dividing surface to be divided and all the vertices of the dividing surface. For example, when the variable n is 2, the same rule for dividing the second T3 virtual dividing surfaces St3 may be, for example, as described above, a rule for dividing the dividing surface into a plurality of surfaces by a plurality of line segments each connecting the center point of the dividing surface to be divided and all the vertices of the dividing surface. For example, if each of the second T3 virtual dividing surfaces St3 is a triangle, each of the second T3 virtual dividing surfaces St3 may be divided into three virtual dividing surfaces that are M3 virtual dividing surfaces by three line segments each connecting the center position (for example, the position of the center of gravity) as a predetermined position and the three vertices. As a result, each of the second T3 virtual dividing surfaces St3 may be divided into three triangular virtual dividing surfaces. As a result, second M3×T3 virtual divided surfaces are generated as M3×T3 virtual divided surfaces.

[0251] FIG. 36 is a diagram showing a schematic diagram of a specific example of dividing the second reference virtual dividing surface Ss2 of the second T3 virtual dividing surfaces St3 into a plurality of virtual dividing surfaces St3a in the second second case. FIG. 36 shows a specific example of a case where each of the second T3 virtual dividing surfaces St3 is a triangular surface. In FIG. 36, the second reference virtual camera position Ps2 is shown by a white circle. As shown in FIG. 36, the dividing surface generating unit 6122 divides the second reference virtual dividing surface Ss2 into three virtual dividing surfaces St3a as the second M3 virtual dividing surfaces, for example, by three line segments connecting three vertices in the second reference virtual dividing surface Ss2 and the second reference virtual camera position Ps2. This makes it easy to divide the second reference virtual dividing surface Ss2. In this case, the split surface generating unit 6122 splits each of the second T3 virtual split surfaces St3 into three virtual split surfaces St3a by three line segments connecting the three vertices and the second reference virtual camera position Ps2, similar to the second reference virtual split surface Ss2 shown in Fig. 36. As a result, each of the second T3 virtual split surfaces St3 is split into three triangular virtual split surfaces St3a as the second M3 virtual split surfaces.

[0252] <<nB Processing>> The nB process is a process in which the second shape information generating unit 6123 generates reference shape information for each of the nth M3×T3 third virtual camera positions (also called the nth M3×T3 third virtual camera positions) that are virtually set for each of the nth M3×T3 virtual dividing planes generated in the nA process, based on three-dimensional design information on the search target part stored in the storage unit 45b or the like, assuming that the 3D model 3dm of the search target part is photographed from each of the M3×T3 virtual camera positions (also called the nth M3×T3 third virtual camera positions). Here again, the reference shape information is information generated based on the three-dimensional design information on the search target part, and is information on the two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from the virtual camera position. The virtual image can be generated by projecting the 3D model 3dm onto a virtual plane by a process such as rendering, for example.

[0253] In each of the n-th M3×T3 virtual dividing surfaces, for example, a third virtual camera position (n-th third virtual camera position) is virtually set at a predetermined position of the virtual dividing surface. For example, the position of the center of the virtual dividing surface is applied to the predetermined position. For example, the center of gravity of the virtual dividing surface is applied to the center of the virtual dividing surface. For example, when the variable n is 2, as shown in FIG. 34 or FIG. 36, a third virtual camera position P13a is set to each virtual dividing surface St3a. If each of the n-th M3×T3 virtual dividing surfaces is a triangular surface, the center of the virtual dividing surface may be, for example, the center of gravity of the triangle or the inner center. In FIG. 34 and FIG. 36, the third virtual camera position P13a is indicated by a black circle.

[0254] In the second shape information generating unit 6123, for example, for each of the n-th M3×T3 third virtual camera positions, reference shape information is generated for the two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from the third virtual camera position. At this time, the photographing direction of the virtual camera photographing the 3D model 3dm of the search target part from each third virtual camera position is set to a direction from the third virtual camera position toward the reference point Po of the 3D model 3dm of the search target part. The roll angle γ that defines the attitude of the virtual camera photographing the 3D model 3dm of the search target part from the third virtual camera position is set to, for example, zero (0) degrees. Here, too, for each third virtual camera position, a reference image Iv1 is generated as reference shape information, for example, as shown in FIG. 11.

[0255] <<nC Processing>> The nth C process is a process in which the second calculation unit 6124 calculates a numerical value indicating the degree of match between information about the two-dimensional shape of the object captured in the actual image capturing the part to be searched for (actual shape information) and the reference shape information for each of the nth M3 × T3 third virtual camera positions.

[0256] In the nC processing, the second calculation unit 6124 calculates a numerical value indicating the degree of similarity of the two-dimensional shapes of the contours, for example, between a reference image as reference shape information acquired by the second shape information generation unit 6123 in the nB processing and a matching target area Re2 (Figure 26) set in the edge image as actual shape information by the matching target area setting unit 6121.

[0257] In the nC process, when the second calculation unit 6124 calculates a numerical value indicating the degree of match between an edge image as actual shape information and a reference image as reference shape information for each third virtual camera position, the second calculation unit 6124 performs, for example, the following processes in the order listed: [Process 2an] a process for detecting the amount of deviation in the orientation of the two-dimensional shape of the contour corresponding to the deviation in roll angle γ between the edge image and the reference image; [Process 2bn] a process for rotating the edge image in accordance with the deviation in roll angle γ; and [Process 2cn] a process for detecting the position of the area where the degree of match between the rotated edge image and the reference image is greatest.

[0258] In the above process 2an, similarly to the above process 2a1, the second calculation unit 6124 detects the deviation amount of the orientation of the two-dimensional shape of the contour, for example, by using the RIPOC method between the edge image as the actual shape information and the reference image as the reference shape information. The deviation amount of the orientation of the two-dimensional shape of the contour is the deviation amount of the contour in the rotation direction on the image. Here, the second calculation unit 6124 calculates, for example, by the RIPOC method, a deviation amount γ2n in the rotation direction that can maximize the degree of matching of the contour between the matching target area Re2 (FIG. 26) set in the edge image as the actual shape information by the matching target area setting unit 6121 and the reference image. This deviation amount γ2n corresponds to a roll angle γ that can maximize the degree of matching between the reference image as the reference shape information and the edge image as the actual shape information.

[0259] In the above process 2bn, similarly to the above process 2b1, the second calculation unit 6124 rotates, for example, the edge image as the actual shape information so as to correct the deviation amount γ2n detected in the above process 2an. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the edge image as the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information. Here, the second calculation unit 6124 rotates, for example, the edge image of the matching target area Re2 so as to correct the deviation amount γ2n detected in the above process 2an. As a result, a rotated edge image is generated, and the orientation is matched between the two-dimensional shape of the contour in the matching target area Re2 constituting the actual shape information and the two-dimensional shape of the contour in the reference image as the reference shape information.

[0260] In the above process 2cn, similarly to the above process 2c1, the second calculation unit 6124 performs template matching using a reference image as reference shape information, for example, on the rotated edge image related to the actual shape information generated in the above process 2bn. Here, the second calculation unit 6124, for example, scans the reference image in the rotated edge image, and detects the position of an area where the degree of match (similarity) between each partial area in the rotated edge image and the reference image is maximum. Thereby, the position (matching candidate position) of an area where the degree of match between the two-dimensional shape of the contour of the 3D model 3dm of the search target part in the reference image is maximum in the edge image as actual shape information can be detected. In addition, here, when performing the above template matching, a numerical value indicating the degree of match (similarity) when the degree of match (similarity) between the partial area in the rotated edge image and the reference image is maximum is calculated as a numerical value indicating the degree of match between the edge image as actual shape information and the reference image as reference shape information for the third virtual camera position. In other words, for each third virtual camera position, the degree of matching between the reference image and a partial region in the edge image after rotation is maximized, and this degree of matching is regarded as the degree of matching between the edge image as the actual shape information and the reference image as the reference shape information. Here again, the numerical value indicating the degree of matching may be, for example, a matching score indicating the degree of matching.

[0261] <<nD Processing>> The nD process is a process in which the second detection unit 6125 detects, among the nth reference virtual camera position and the nth M3 × T3 third virtual camera positions, the virtual camera position that has the greatest degree of match between information about the two-dimensional shape of an object captured in an actual image capturing the part to be searched for (actual shape information) and the reference shape information.

[0262] Here, for example, the matching score calculated for the nth reference virtual camera position and the matching scores calculated for each of the nth M3×T3 third virtual camera positions by the second calculation unit 6124 in the nC process are compared. Then, for example, from among the nth reference virtual camera position and the nth M3×T3 third virtual camera positions, the virtual camera position having the largest matching score calculated between information on the two-dimensional shape of an object captured in an actual image capturing the search target part (actual shape information) and the reference shape information is detected.

[0263] <<Effects of repeating unit processing>> As described above, when the unit process is repeatedly executed, a plurality of virtual division surfaces including a virtual division surface including the virtual camera position where the degree of coincidence between the actual shape information and the reference shape information is the highest and having different distances from the reference point Po of the 3D model 3dm of the search target part are divided by the same rule to generate a plurality of virtual division surfaces for setting the next virtual camera position. This makes it possible to reduce an increase in at least one of the number and area of ​​the virtual division surfaces after division, rather than a surface unrelated to the virtual division surfaces before and after division. As a result, the amount of calculation required to recognize the orientation of the search target part captured in the actual image can be reduced. Therefore, in the substrate processing apparatus 1, it is possible to efficiently recognize the orientation of the search target part.

[0264] <<End of nth unit processing one or more times>> The search processing unit 61 ends the execution of one or more n-th unit processes in response to, for example, the second detection unit 6125 detecting one of the reference virtual camera positions from the first reference virtual camera position Ps1 to the n-th reference virtual camera position a preset predetermined number of times (also referred to as the first predetermined number of times). The first predetermined number of times may be set to an arbitrary number of times, for example, two or more. In this case, if a virtual camera position at which the degree of agreement between the reference shape information and the actual shape information is greater is not detected even if the unit processes are repeated to a certain extent in the second search process, the repeated execution of the unit processes is ended. This can reduce the amount of calculation for recognizing the orientation of the search target part captured in the actual image. Therefore, in the substrate processing apparatus 1, the orientation of the search target part can be recognized efficiently.

[0265] Furthermore, for example, the search processing unit 61 may end one or more executions of the nth unit process in response to execution of the nth unit process a preset number of times (also referred to as a second predetermined number of times). The second predetermined number of times may be set to an arbitrary number of times, for example, equal to or greater than one. This can reduce the amount of calculation required to recognize the orientation of the search target part captured in the actual image. Therefore, in the substrate processing apparatus 1, the orientation of the search target part can be recognized efficiently.

[0266] Here, the search processing unit 61 detects, for example, the virtual camera position with the highest degree of matching between the actual shape information and the reference shape information detected by the second detection unit 6125 in the last n-th D process in one or more executed n-th unit processes as the maximum matching virtual camera position. In other words, for example, the virtual camera position detected by the second detection unit 6125 in the last n-th D process in one or more executed n-th unit processes is the maximum matching virtual camera position. Then, the search processing unit 61 can acquire, for example, the longitude α, longitude β, and distance D that define this maximum matching virtual camera position, and the roll angle γ corresponding to the shift amount (shift amount γ1, shift amount γ2, shift amount γ21, or shift amount γ2n) calculated by the first calculation unit 6113 or the second calculation unit 6124 for this maximum matching virtual camera position as the result of the search process. Thereby, the orientation of the search target part captured in the actual image can be recognized.

[0267] In addition, the search processing unit 61 may acquire, for example, for the maximum matching virtual camera position, as part of the result of the search process, a matching candidate position (also called a final matching position) that has the highest degree of matching (similarity) with the reference image as reference shape information within the edge image as actual shape information, detected by the first calculation unit 6113 or the second calculation unit 6124.

[0268] <4. Specific examples of processing in substrate processing equipment> Next, a specific example of the processing in the substrate processing apparatus 1 will be described with reference to FIG. 37 to FIG. 47. FIG. 37 is a flow chart showing a specific example of a schematic flow of the operation related to the processing of one or more substrates W in the substrate processing apparatus 1. FIG. 38 is a flow chart showing a specific example of a schematic flow of the operation related to the processing of detecting an abnormality in the substrate processing apparatus 1. FIG. 39 is a flow chart showing a specific example of the image processing flow in step S14 in FIG. 38. FIG. 40 is a flow chart showing a specific example of the processing flow in the search processing in step S15 in FIG. 38. FIG. 41 is a flow chart showing a specific example of the processing flow in the first search processing in step Sb1 in FIG. 40. FIGS. 42 to 44 are flow charts showing a specific example of the processing flow in the second search processing in step Sb2 in FIG. 40. FIG. 45 is a diagram showing an example of a state in which the target part is located at the initial position. Here, the initial position of the chuck 9 as the target part is the open position, the initial position of the guard 23 as the target part is the lowered position, and the initial position of the nozzle 33 as the target part is the standby position. Fig. 46 is a diagram showing an example of a state in which each chuck 9 is located at the closed position. Fig. 47 is a diagram showing an example of a state in which the nozzle 33 is located at the discharge position and the guard 23 is located at the raised position.

[0269] For example, in the substrate processing apparatus 1, when an operator operates the instruction unit 47 to instruct execution of a recipe in the recipe information 53, the operation control unit 51 controls the operation of each unit of the substrate processing apparatus 1 according to the recipe instructed to be executed. This allows the processing of each substrate W in the substrate processing apparatus 1 to proceed. At this time, the processing of steps S1 to S7 in FIG. 37 for the operation related to the processing of one or more substrates W in the substrate processing apparatus 1 is performed. Furthermore, when the processing of one or more substrates W in steps S1 to S7 in FIG. 37 is performed, the processing of steps S11 to S20 in FIG. 38 for the operation related to the processing of detecting an abnormality in the substrate processing apparatus 1 is performed in parallel with the processing of steps S1 to S7.

[0270] <4-1. Operations related to substrate processing in the substrate processing apparatus> <<Step S1>> In step S1, each target part moves to an initial position. Here, the operation control unit 51 controls the operations of the chuck moving mechanism 17, the guard moving mechanism 25, and the nozzle moving mechanism .

[0271] The operation control unit 51 controls the operation of the chuck moving mechanism 17 by, for example, a chuck operation command to move the chuck 9 as the target part to an open position as an initial position. Here, in a state where the substrate W is not placed on the lower surface support unit 11, the chuck 9 rotates around the rotation center PL2 in response to the chuck operation command, and the peripheral support unit 13 moves in a direction away from the rotation center PL1 of the spin chuck 3, so that the chuck 9 moves to the open position. At this time, the output signal of the first sensor Z1 is turned on. The operation control unit 51 recognizes that the chuck 9 has moved to the open position by the output signal of the first sensor Z1. In FIG. 45, the state where the chuck 9 is located at the open position is shown by a solid line, and the position of the outer edge of the substrate W when the substrate W is placed on the lower surface support unit 11 is shown by a thin two-dot chain line. Here, the peripheral support unit 13 of each chuck 9 moves to a position farther from the rotation center PL1 than the outer edge of the substrate W when the substrate W is placed on the lower surface support unit 11.

[0272] The operation control unit 51 controls the operation of the guard moving mechanism 25 by a guard operation command, for example, to move the guard 23, which is the target part, to a lowered position, which is an initial position. Here, the guard 23 moves to the lowered position. At this time, the output of the second sensor Z2 is turned on. The operation control unit 51 recognizes that the guard 23 has moved to the lowered position by the output signal of the second sensor Z2. In FIG. 45, the state in which the guard 23 is in the lowered position is shown by a solid line, and the state in which the guard 23 is in the raised position is shown by a thin two-dot chain line.

[0273] The operation control unit 51 controls the operation of the nozzle moving mechanism 35 by a nozzle operation command, for example, to move the nozzle 33, which is the target part, to a standby position, which is an initial position. Here, the nozzle 33 rotates about the rotation center PL3 and moves to the standby position off to the side of the guard 23. At this time, the output of the third sensor Z3 is turned on. The operation control unit 51 recognizes that the nozzle 33 has moved to the standby position by the output signal of the third sensor Z3. In FIG. 45, the state in which the nozzle 33 is located at the standby position is shown by a solid line, and the state in which the nozzle 33 is located at the discharge position is shown by a thin two-dot chain line.

[0274] Here, when the processes from step S1 to step S7 are started, if each target part is already positioned at its initial position, the process of step S1 may be omitted.

[0275] <<Step S2>> In step S2, the substrate W to be processed is carried into the substrate processing apparatus 1. The substrate W is carried into the substrate processing apparatus 1 by, for example, a transfer robot (not shown).

[0276] <<Step S3>> In step S3, the substrate W is processed according to the recipe. Specifically, first, the substrate W is placed on the multiple chucks 9. The operation control unit 51, for example, operates the chuck moving mechanism 17 in response to a chuck operation command to move each chuck 9 from the open position to the closed position. This state is, for example, as shown in FIG. 46. That is, with the substrate W placed on each chuck 9, the chucks 9 rotate about the rotation center PL2, and the peripheral support parts 13 move toward the rotation center PL1 of the spin chuck 3. As a result, the peripheral support parts 13 of the multiple chucks 9 come into contact with the outer diameter of the substrate W, thereby clamping the substrate W.

[0277] <<Step S4>> In step S4, the operation control unit 51 determines whether or not the processing of the substrate W has been completed. If the processing of the substrate W has been completed, the process proceeds to step S5. On the other hand, if the processing of the substrate W has not been completed, the process returns to step S3. For example, if the substrate W is simply placed on the multiple chucks 9 and clamped by the multiple chucks 9, the processing of the substrate W is not completed, and the process returns to step S3.

[0278] <<Step S3>> In step S3, the processing of the substrate W proceeds according to the recipe. Specifically, for example, the operation control unit 51 operates the guard moving mechanism 25 in response to a guard operation command, thereby moving the guard 23, which is located in the lowered position as shown in Fig. 45, to the raised position as shown in Fig. 47. At this point, the nozzle 33, which is located above the substrate W in Fig. 47, is located in the standby position.

[0279] <<Step S4>> In step S4, the operation control unit 51 determines whether or not the processing of the substrate W has been completed. If the processing of the substrate W has been completed, the process proceeds to step S5. On the other hand, if the processing of the substrate W has not been completed, the process returns to step S3. For example, if the substrate W is clamped by the multiple chucks 9 and the guard 23 has only moved to the raised position, the processing of the substrate W has not been completed, and the process returns to step S3.

[0280] <<Step S3>> In step S3, the processing of the substrate W is performed according to the recipe. Specifically, for example, the operation control unit 51 operates the nozzle moving mechanism 35 in response to a nozzle operation command to move the nozzle 33B of the nozzles 33A and 33B from the standby position shown in Fig. 45 to the discharge position shown in Fig. 47. The discharge position is, for example, a position on the rotation center PL1.

[0281] <<Step S4>> In step S4, the operation control unit 51 determines whether or not the processing of the substrate W has been completed. If the processing of the substrate W has been completed, the process proceeds to step S5. On the other hand, if the processing of the substrate W has not been completed, the process returns to step S3. For example, if the substrate W is merely clamped by the multiple chucks 9, the guard 23 is moved to the raised position, and the nozzle 33B is moved to the discharge position, the processing of the substrate W is not completed, and the process returns to step S3.

[0282] <<Steps S3 and S4>> After this, the process of step S3 and the process of step S4 are repeated. At this time, in step S3, the operation control unit 51 controls the operation of each unit in the substrate processing apparatus 1 according to the recipe instructed to be executed, and proceeds with the processing of the substrate W. Specifically, for example, the motor 7 is driven according to the recipe to rotate the spin chuck 3, and the nozzle 33B is supplied with a processing liquid onto the substrate W, in order, to process the substrate W by the operation of each unit according to the recipe instructed to be executed. Then, when the processing of the substrate W is completed, the process proceeds from step S4 to step S5.

[0283] <<Step S5>> In step S5, similarly to step S1, each target part is moved to its initial position. Here, the operation control unit 51 controls the operations of the chuck moving mechanism 17, the guard moving mechanism 25, and the nozzle moving mechanism .

[0284] <<Step S6>> In step S6, the substrate W placed on the multiple chucks 9 is carried out to the outside of the substrate processing apparatus 1. The substrate W is carried out to the outside of the substrate processing apparatus 1 by, for example, a transfer robot (not shown).

[0285] <<Step S7>> In step S7, the operation control section 51 determines whether or not there is a next substrate W to be processed according to the recipe instructed to be executed. If there is a next substrate W, the process proceeds to step S2. Therefore, as long as there is a next substrate W to be processed, the processes from step S2 to step S7 are repeated. Then, if there is no next substrate W to be processed in step S7, the operation relating to the processing of one or more substrates W in the substrate processing apparatus 1 shown in FIG. 37 is terminated.

[0286] <4-2. Operations Related to Processing for Detecting Abnormalities in Substrate Processing Apparatus> <<Step S11>> In step S11, the setting range information 58 is stored in the storage unit 45b. Here, for example, the operator operates the instruction unit 47, whereby the setting range information 58 is stored in the storage unit 45b. The setting range information 58 may be stored in the storage unit 45b in advance, for example. In this case, step S11 may be omitted. The process of storing the setting range information 58 in the storage unit 45b corresponds to the storage step of the present invention.

[0287] <<Step S12>> In step S12, the operation control unit 51 checks whether the target part is at the check-required timing. Specifically, the operation control unit 51 refers to the target part and its check-required timing in the parameter information 55. If the target part is at the check-required timing, the process proceeds to step S13, and if the target part is not at the check-required timing, the process proceeds to step S20.

[0288] More specifically, in step S12, for example, if it is time to position the chuck 9 as the target part in the open position, which is the 1A position, the process proceeds to step S13. In this case, the operation control unit 51 notifies the search processing unit 61 of information identifying the chuck 9 as the search target part, which is the target part in the confirmation timing, and information indicating the open position, which is the position where the chuck 9 should be in the confirmation timing. Then, the processes from step S13 to step S17 are performed for the chuck 9 in the confirmation timing.

[0289] Also, in step S12, if it is time to position the guard 23 as the target part in the lowered position, which is the 1B position, for example, the process proceeds to step S13. In this case, the operation control unit 51 notifies the search processing unit 61 of information identifying the guard 23 as the search target part, which is the target part in the confirmation timing, and information indicating the lowered position, which is the position where the guard 23 should be in the confirmation timing. Then, the processes from step S13 to step S17 are performed for the guard 23 in the confirmation timing.

[0290] Also, in step S12, for example, if it is time to position nozzle 33 as a target part in the standby position, which is position 1C, the process proceeds to step S13. In this case, operation control unit 51 notifies search processing unit 61 of information identifying nozzle 33 as a search target part, which is a target part in the confirmation timing, and information indicating the standby position, which is the position where nozzle 33 should be in the confirmation timing. Then, processes from step S13 to step S17 are performed for nozzle 33 in the confirmation timing.

[0291] In step S12, for example, if it is time to position the chuck 9 as the target part in the closed position, which is the 2A position, the process proceeds to step S13. In this case, the operation control unit 51 notifies the search processing unit 61 of information identifying the chuck 9 as the search target part, which is the target part in the confirmation timing, and information indicating the closed position, which is the position where the chuck 9 should be in the confirmation timing. Then, the processes from step S13 to step S17 are performed for the chuck 9 in the confirmation timing.

[0292] Also, in step S12, for example, if it is time to position the guard 23 as the target part in the raised position, which is the 2B position, the process proceeds to step S13. In this case, the operation control unit 51 notifies the search processing unit 61 of information identifying the guard 23 as the search target part, which is the target part in the confirmation timing, and information indicating the raised position, which is the position where the guard 23 should be at the confirmation timing. Then, the processes from step S13 to step S17 are performed for the guard 23 in the confirmation timing.

[0293] Also, in step S12, for example, if it is time to position the nozzle 33 as the target part at the discharge position, which is the 2C position, the process proceeds to step S13. In this case, the operation control unit 51 notifies the search processing unit 61 of information identifying the nozzle 33 as the search target part, which is the target part at the confirmation timing, and information indicating the discharge position, which is the position where the nozzle 33 should be at the confirmation timing. Then, the processes from step S13 to step S17 are performed for the nozzle 33 at the confirmation timing.

[0294] <<Step S13>> In step S13, the operation control unit 51 causes the camera CM to take a photograph. Here, an actual image capturing a search target part, which is any one of one or more target parts, is obtained by taking a photograph with the camera CM. This step S13 corresponds to the photographing step in the present invention.

[0295] More specifically, in step S13, for example, when it is time to position the chuck 9 as the target part at the open position, which is the 1A position, the operation control unit 51 operates the camera CM according to the timing when it is estimated that each chuck 9 has been moved to the open position by the chuck operation command. As a result, the camera CM captures an actual image including each chuck 9 by shooting.

[0296] Also, in step S13, for example, if it is time to place the guard 23 as the target part in the lowered position, which is the 1B position, the operation control unit 51 operates the camera CM according to the timing when it is estimated that the guard 23 has been moved to the lowered position by the guard operation command. As a result, the camera CM captures an actual image including the guard 23 by shooting.

[0297] Also, in this step S13, for example, when it is time to position the nozzle 33 as the target part at the standby position, which is the 1C position, the operation control unit 51 operates the camera CM according to the timing when it is estimated that the nozzle 33 has been moved to the standby position by the nozzle operation command. As a result, the camera CM captures an actual image including the nozzle 33 by shooting.

[0298] In step S13, for example, when it is time to position the chuck 9 as the target part at the closed position, which is the 2A position, the operation control unit 51 operates the camera CM according to the timing when it is estimated that each chuck 9 has been moved to the closed position by the chuck operation command. As a result, the camera CM acquires an actual image including each chuck 9 by photographing.

[0299] Also, in step S13, for example, if it is time to position the guard 23 as the target part in the raised position, which is the 2B position, the operation control unit 51 operates the camera CM according to the timing when it is estimated that the guard 23 has been moved to the raised position by the guard operation command. As a result, the camera CM captures an actual image including the guard 23 by shooting.

[0300] Also, in this step S13, for example, if it is time to position the nozzle 33 as the target part at the discharge position, which is the second C position, the operation control unit 51 operates the camera CM according to the timing when it is estimated that the nozzle 33 has been moved to the discharge position by the nozzle operation command. As a result, the camera CM captures an actual image including the nozzle 33 by shooting.

[0301] <<Step S14>> In step S14, the image processing unit 59 performs image processing on the actual image captured by the camera CM in step S13. In this step S14, the processes of steps Sa1 to Sa4 in FIG.

[0302] In step Sa1, the image processing unit 59 obtains, from the camera CM, an actual image capturing the search target part captured in the photographing in step S13.

[0303] In step Sa2, processing target area extraction section 591 of image processing section 59 sets a processing target area for the actual image capturing the search target parts. Here, a processing target area is set for each search target part.

[0304] In step Sa3, the processing target region extraction unit 591 of the image processing unit 59 extracts a portion relating to the processing target region from the actual image capturing the search target part as an image (actual processing target image).

[0305] In step Sa4, the contour extraction unit 592 of the image processing unit 59 performs a process of extracting contours for all parts captured in the processing target actual image. This allows an edge image to be acquired as information (actual shape information) about the two-dimensional shape of the object captured in the actual image capturing the search target part.

[0306] Here, for example, if the part to be searched is the chuck 9, an edge image is acquired as information (actual shape information) about the two-dimensional shape of an object including the chuck 9 captured in an actual image capturing the chuck 9. For example, if the part to be searched is the guard 23, an edge image is acquired as information (actual shape information) about the two-dimensional shape of an object including the guard 23 captured in an actual image capturing the guard 23. For example, if the part to be searched is the nozzle 33, an edge image is acquired as information (actual shape information) about the two-dimensional shape of an object including the nozzle 33 captured in an actual image capturing the nozzle 33.

[0307] <<Step S15>> In step S15, the search processing unit 61 performs a search process on the search target part. For example, if the search target part is the chuck 9, the search processing unit 61 performs a search process on the chuck 9. For example, if the search target part is the guard 23, the search processing unit 61 performs a search process on the guard 23. For example, if the search target part is the nozzle 33, the search processing unit 61 performs a search process on the nozzle 33.

[0308] Here, the search processing unit 61 executes a search process to search for a maximum matching virtual camera position, which is a virtual camera position having the highest matching degree between the reference shape information and the actual shape information, among the multiple virtual camera positions, based on multiple reference shape information and actual shape information. Here, the multiple reference shape information are information generated based on three-dimensional design information related to the search target part stored in the storage unit 45b or the like, and are information on the two-dimensional shape of the 3D model 3dm of the search target part in each of multiple virtual images that can be acquired by photographing the 3D model 3dm of the search target part from multiple virtual camera positions. For example, one piece of reference shape information is information generated based on three-dimensional design information related to one search target part, and is information on the two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from one virtual camera position. The actual shape information is information on the two-dimensional shape of an object in an actual image capturing the search target part obtained in step S13. More specifically, the actual shape information may be an edge image as the actual shape information acquired in step Sa4. This step S15 corresponds to the search step in which the calculation unit 45a executes the search process in the present invention.

[0309] In this step S15, the process of step Sb1 and the process of step Sb2 in FIG. 40 are performed for each target part in order. In step Sb1, the first search processing unit 611 performs the first search process. More specifically, in step Sb1, the processes of step Sb11 to step Sb14 in FIG. 41 are performed in order. In step Sb2, the second search processing unit 612 performs the second search process. More specifically, in step Sb2, the processes of step Sb21 to step Sb25 in FIG. 42, the processes of step Sb31 to step Sb34 in FIG. 43, and the processes of step Sb41 to step Sb48 in FIG. 44 are performed. Here, the processes of step Sb31 to step Sb34 in FIG. 43 correspond to the first unit process described above. The processes of step Sb42 to step Sb45 in FIG. 44 correspond to the n-th unit process described above.

[0310] <<<Step Sb1>>> <<<<Step Sb11>>>> In step Sb11, the density determination unit 6111 determines an area where the density of the component contour is low (low-density area) in the edge image as the actual shape information acquired in step Sa4. The determination result in the density determination unit 6111 is used in the processing in the first calculation unit 6113 and the second calculation unit 6124. For example, in the processing in the first calculation unit 6113 and the second calculation unit 6124, the low-density area in the edge image as the actual shape information is excluded from the calculation for calculating the numerical value indicating the degree of match, thereby reducing the amount of calculation required for the search processing.

[0311] <<<<Step Sb12>>>> In step Sb12, the first shape information acquisition unit 6112 acquires reference shape information generated for each of the first virtual camera positions P11, assuming a case where the 3D model 3dm of the search target part is photographed from each of the first virtual camera positions P11, based on the three-dimensional design information and the setting range information 58 related to the search target part stored in the storage unit 45b or the like. The first virtual camera positions P11 are located along a plurality of virtual spheres that surround the 3D model 3dm of the search target part with the reference point Po of the 3D model 3dm of the search target part as the center, and are a plurality of virtual camera positions corresponding to at least one initial setting range of one or more initial setting ranges for the search target part defined in the setting range information 58. The reference shape information is information generated based on the three-dimensional design information related to the search target part, and is information about the two-dimensional shape of the 3D model 3dm of the search target part in a virtual image that can be acquired by photographing the 3D model 3dm of the search target part from a virtual camera position. Specifically, the reference shape information is a reference image. This step Sb12 corresponds to the first shape information acquisition step of the present invention. Here, when the range of the position and orientation of the search target part is limited, the amount of calculation in the search process can be reduced by limiting the range in which the multiple first virtual camera positions P11 are set. This can reduce the amount of calculation for recognizing the orientation of the search target part captured in the actual image. As a result, the orientation of the search target part can be efficiently recognized in the substrate processing apparatus 1.

[0312] Here, for example, it is assumed that one or more initial setting ranges for the search target part defined in the setting range information 58 include the first initial setting range for the first position, and that the timing is such that the search target part is positioned at the first position by the moving mechanism. At this timing, the search processing unit 61 can acquire reference shape information generated for each of the multiple first virtual camera positions P11 corresponding to the first initial setting range, for example, by the first shape information acquisition unit 6112 based on the three-dimensional design information and setting range information 58 related to the search target part stored in the storage unit 45b or the like, assuming that the 3D model 3dm of the search target part is photographed from each of the multiple first virtual camera positions P11 corresponding to the first initial setting range. Thereby, for example, when the search target part moves between the first position and the second position, it is possible to limit the range in which the multiple first virtual camera positions P11 are set for the first position where the range in which the orientation of the search target part changes is more limited. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Here, the first initial setting range corresponds to at least one of the one or more initial setting ranges for the search target part described above. In other words, at least one of the one or more initial setting ranges for the search target part described above is the first initial setting range. The multiple virtual camera positions corresponding to the first initial setting range for the search target part may be multiple virtual camera positions that are virtually set within the range of the first initial setting range for the search target part.

[0313] For example, if the part to be searched is a first chuck, and the timing is for the chuck moving mechanism 17 to position the first chuck in an open position, which is the first position, namely, the firstA position, then the first initial setting range may be the firstA initial setting range for the open position of the first chuck.

[0314] For example, if the part to be searched is guard 23 and the timing is such that guard 23 is to be positioned in a lowered position, which is the first position, 1B position, by guard moving mechanism 25, the first initial setting range may be the 1B initial setting range for the lowered position of guard 23.

[0315] For example, if the part to be searched is nozzle 33, and the timing is such that nozzle 33 is to be positioned at a standby position, which is the first position, 1C position, by nozzle moving mechanism 35, the first initial setting range may be the 1C initial setting range for the standby position of nozzle 33.

[0316] Here, for example, it is assumed that one or more initial setting ranges for the search target part defined in the setting range information 58 include the second initial setting range for the second position, and that the timing is such that the search target part is positioned at the second position by the moving mechanism. At this timing, the search processing unit 61 can acquire reference shape information generated for each of the multiple first virtual camera positions P11 corresponding to the second initial setting range, for example, by the first shape information acquisition unit 6112 based on the three-dimensional design information and setting range information 58 related to the search target part stored in the storage unit 45b or the like, assuming that the 3D model 3dm of the search target part is photographed from each of the multiple first virtual camera positions P11 corresponding to the second initial setting range. Thereby, for example, when the search target part moves between the first position and the second position, it is possible to limit the range in which the multiple first virtual camera positions P11 are set for the second position where the range in which the orientation of the search target part changes is more limited. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced. Here, the second initial setting range corresponds to at least one of the one or more initial setting ranges for the search target part described above. In other words, at least one of the one or more initial setting ranges for the search target part described above is the second initial setting range. The multiple virtual camera positions corresponding to the second initial setting range for the search target part may be multiple virtual camera positions that are virtually set within the range of the second initial setting range for the search target part.

[0317] For example, if the part to be searched is a first chuck, and the timing is such that the chuck moving mechanism 17 is used to position the first chuck in a closed position, which is the second position, 2A position, the second initial setting range may be the 2A initial setting range for the closed position of the first chuck.

[0318] For example, if the part to be searched is guard 23 and the timing is to position guard 23 in an elevated position, which is the second position, 2B position, by guard moving mechanism 25, the second initial setting range may be the 2B initial setting range for the elevated position of guard 23.

[0319] For example, if the part to be searched is nozzle 33, and the timing is such that nozzle 33 is positioned at an ejection position, which is the second position, namely, the second C position, by nozzle moving mechanism 35, the second initial setting range may be the second C initial setting range for the ejection position of nozzle 33.

[0320] Here, for example, as shown in Fig. 4, if one or more initial setting ranges for the search target part each include a first numerical range which is a distance range, a second numerical range which is a latitude range, and a third numerical range which is a longitude range, the ranges for setting the multiple first virtual camera positions P11 can be limited when the ranges in which the angular relationship and the distance between the camera CM and the search target part change are limited. This can reduce the amount of calculation in the search process. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced.

[0321] 4, if the setting range information 58 includes information that specifies one or more initial setting ranges for the virtual camera position for each target part, the range in which the multiple first virtual camera positions P11 are set for each search target part as a target part that is in a confirmation timing state can be limited. This can reduce the amount of calculation in the search process. As a result, the amount of calculation for recognizing the orientation of the search target part captured in the actual image can be reduced.

[0322] More specifically, in step Sb12, the first shape information acquisition unit 6112 acquires reference shape information generated for each of the M1×T1 first virtual camera positions P11 (M1 and T1 are both natural numbers of 2 or more) assuming a case where a 3D model 3dm of the search target part is photographed from each of the M1×T1 first virtual camera positions P11, which are a plurality of first virtual camera positions P11, based on the three-dimensional design information and setting range information 58 related to the search target part stored in the storage unit 45b or the like. The M1×T1 first virtual camera positions P11 are a plurality of virtual camera positions that are virtually set for each of the M1 virtual surfaces St1 in each of the T1 surface aggregates As1 when T1 surface aggregates As1 having mutually different distances from the reference point Po of the 3D model 3dm of the search target part are virtually set. The T1 surface aggregates As1 each include a plurality of virtual surfaces St1 located along a virtual sphere surrounding the 3D model 3dm of the part to be searched for, centered at a reference point Po of the 3D model, and are a plurality of surface aggregates As1 corresponding to a first numerical range which is a distance range in at least one of the one or more initial setting ranges for the part to be searched for defined in the setting range information 58. The M1 virtual surfaces St1 in each of the T1 surface aggregates As1 are a plurality of virtual surfaces corresponding to a second numerical range which is a latitude range and a third numerical range which is a longitude range in at least one of the one or more initial setting ranges for the part to be searched for defined in the setting range information 58.

[0323] Here, for example, as shown in Figure 14, by setting the right-handed xyz coordinates with the reference point Po of the 3D model 3dm of the part to be searched defined in the three-dimensional design information of the part to be searched defined as the origin, as well as the angle (latitude) α in the direction of rotation about the x-axis, the angle (longitude) β in the direction of rotation about the z-axis, and the distance D from the origin, the position and orientation of each virtual surface St1, as well as each first virtual camera position P11, etc. can be defined.

[0324] Here, in each of the M1 virtual surfaces St1 in each of the T1 surface aggregates As1, for example, a first virtual camera position P11 is virtually set at a predetermined position of the virtual surface St1. For example, the position of the center of the virtual surface St1 is applied to the predetermined position. For example, the center of gravity of the virtual surface St1 is applied to the center of the virtual surface St1. The multiple surface aggregates As1 according to the first numerical range, which is a distance range in at least one of the one or more initial setting ranges for the search target part defined in the setting range information 58, may be, for example, multiple surface aggregates As1 in which the distance from the reference point Po of the 3D model 3dm of the search target part is included within the range of this first numerical range, or multiple surface aggregates As1 located along multiple virtual spherical surfaces in which the distance from the reference point Po of the 3D model 3dm of the search target part is included within the range of this first numerical range. Furthermore, the multiple virtual surfaces St1 corresponding to the second numerical range, which is a range of latitude, and the third numerical range, which is a range of longitude, in at least one initial setting range of one or more initial setting ranges for the search target part defined in the setting range information 58 may be multiple virtual surfaces St1 whose latitude is included within the range of this second numerical range and whose longitude is included within the range of this third numerical range. Furthermore, these multiple virtual surfaces St1 may be multiple virtual surfaces whose latitude of a predetermined position of each of the multiple virtual surfaces St1 is included within the range of this second numerical range and whose longitude of a predetermined position of each of the multiple virtual surfaces St1 is included within the range of this third numerical range. Each predetermined position of the virtual surface St1 may be, for example, a center such as a center of gravity of the virtual surface St1.

[0325] For example, each of the multiple virtual surfaces St1 constituting the surface assembly As1 may be a triangular surface, and the surface assembly As1 may be a polyhedron composed of multiple triangular surfaces. In other words, the surface assembly As1 may be a spherical polyhedron composed of multiple triangular surfaces. This makes it easy to set the surface assembly As1 including the multiple virtual surfaces St1. Here, the triangle may be an equilateral triangle. If the virtual surface St1 is a triangular surface, the center of the virtual surface may be the center of gravity of the triangle or the incenter.

[0326] In step Sb12, for example, the first shape information acquisition unit 6112 may set multiple first virtual camera positions P11 based on three-dimensional design information and setting range information 58 regarding the part to be searched stored in the memory unit 45b, etc., and generate reference shape information for each of the multiple first virtual camera positions P11, thereby acquiring the reference shape information generated for each of the multiple first virtual camera positions P11.

[0327] <<<<Step Sb13>>>> In step Sb13, the first calculation unit 6113 calculates a numerical value indicating the degree of agreement between the reference image as the reference shape information for each of the multiple first virtual camera positions P11 acquired in step Sb12 and the edge image as the information (actual shape information) on the two-dimensional shape of the object captured in the actual image acquired in step S13. The edge image as the actual shape information is the edge image as the actual shape information acquired in step Sa4. This step Sb13 corresponds to the first calculation step in the present invention.

[0328] More specifically, in step Sb13, the first calculator 6113 calculates a numerical value indicating the degree of agreement between an edge image as information (actual shape information) about the two-dimensional shape of the object captured in the actual image obtained in step S13 and a reference image as reference shape information obtained in step Sb12, for each of the M1×T1 first virtual camera positions P11 as the above-mentioned plurality of first virtual camera positions P11. The edge image as the actual shape information is the edge image as the actual shape information obtained in step Sa4.

[0329] Here, for example, when one or more initial setting ranges for the search target part defined in the setting range information 58 include a first initial setting range for the first position, at the timing when the search target part is positioned at the first position by the moving mechanism, the search processing unit 61 may calculate, by the first calculation unit 6113, a numerical value indicating the degree of match between an edge image as information on the two-dimensional shape of the object captured in the actual image obtained in step S13 (actual shape information) and a reference image as reference shape information acquired in step Sb12, for each of a plurality of first virtual camera positions P11 corresponding to the first initial setting range. Here, the actual image capturing the search target part may be an actual image obtained by shooting with the camera CM at the timing when the search target part is positioned at the first position by the moving mechanism.

[0330] For example, if the part to be searched for is a first chuck, and the timing is such that the first chuck is positioned in an open position, which is the first position, position 1A, by the chuck moving mechanism 17, the actual image capturing the part to be searched for may be an actual image obtained by photographing with the camera CM at the timing when the first chuck is positioned in the open position by the chuck moving mechanism 17.

[0331] For example, if the part to be searched for is the guard 23, and the guard 23 is to be moved to a lowered position, which is the first position, 1B position, by the guard moving mechanism 25, the actual image capturing the part to be searched for may be an actual image obtained by photographing with the camera CM at the time when the guard 23 is moved to the open position by the guard moving mechanism 25.

[0332] For example, if the part to be searched is nozzle 33, and the nozzle 33 is to be positioned at a standby position, which is the first position, 1C position, by the nozzle moving mechanism 35, the actual image capturing the part to be searched may be an actual image obtained by photographing with the camera CM at the timing when the nozzle 33 is to be positioned at the standby position by the nozzle moving mechanism 35.

[0333] Here, for example, when one or more initial setting ranges for the search target part defined in the setting range information 58 include a second initial setting range for the second position, at the timing when the search target part is positioned at the second position by the moving mechanism, the search processing unit 61 may calculate, by the first calculation unit 6113, a numerical value indicating the degree of match between an edge image as information on the two-dimensional shape of the object captured in the actual image obtained in step S13 (actual shape information) and a reference image as reference shape information acquired in step Sb12, for each of a plurality of first virtual camera positions P11 corresponding to the second initial setting range. Here, the actual image capturing the search target part may be an actual image obtained by shooting with the camera CM at the timing when the search target part is positioned at the second position by the moving mechanism.

[0334] For example, if the part to be searched for is the first chuck, and the timing is such that the first chuck is positioned in the closed position, which is the second position, 2A position, by the chuck moving mechanism 17, the actual image capturing the part to be searched for may be an actual image obtained by photographing with the camera CM at the timing when the first chuck is positioned in the closed position by the chuck moving mechanism 17.

[0335] For example, if the part to be searched for is the guard 23, and the guard 23 is to be positioned in an elevated position, which is the second position, 2B position, by the guard moving mechanism 25, the actual image capturing the part to be searched for may be an actual image obtained by photographi...

Claims

1. A substrate processing apparatus for processing a substrate, One or more target parts; one or more image capturing units configured to capture an actual image of a search target part, which is any one of the one or more target parts; a search processing unit that executes a search process to search for a maximum matching virtual camera position among the plurality of virtual camera positions that has the greatest matching degree between the reference shape information and the actual shape information, based on reference shape information about a two-dimensional shape of a three-dimensional model in each of a plurality of virtual images that can be obtained by photographing the three-dimensional model of the search target part from a plurality of virtual camera positions, the reference shape information being generated based on three-dimensional design information about the search target part, and actual shape information about a two-dimensional shape of an object in the actual image; a storage unit that stores setting range information that specifies one or more initial setting ranges related to a preset virtual camera position for each of at least one target part among the one or more target parts, The search processing unit is a first shape information acquisition unit that acquires the reference shape information generated for each of a plurality of first virtual camera positions based on the three-dimensional design information and the setting range information, assuming a case in which the three-dimensional model is photographed from each of a plurality of first virtual camera positions that are positioned along a plurality of virtual spheres that surround the three-dimensional model with a reference point of the three-dimensional model as a center and correspond to at least one initial setting range of the one or more initial setting ranges for the search target part defined by the setting range information; a first calculation unit that calculates a numerical value indicating a degree of matching between the actual shape information and the reference shape information for each of the first virtual camera positions; a first detection unit that detects, among the plurality of first virtual camera positions, a high-match virtual camera position having a highest match between the actual shape information and the reference shape information based on a calculation result by the first calculation unit, A substrate processing apparatus, wherein the search processing unit searches for the maximum degree of agreement virtual camera position based on the high degree of agreement virtual camera position detected by the first detection unit, or the high degree of agreement virtual camera position is the maximum degree of agreement virtual camera position.

2. The substrate processing apparatus according to claim 1 , Each of the one or more default ranges is A substrate processing apparatus comprising: a first numerical range which is a range of distance from the reference point; a second numerical range which is a range of latitude for a virtual sphere centered on the reference point and surrounding the three-dimensional model; and a third numerical range which is a range of longitude for the virtual sphere.

3. The substrate processing apparatus according to claim 2, The first shape information acquisition unit is Based on the three-dimensional design information and the setting range information, a number T1 (T1 is a natural number of 2 or more) of surface sets are virtually set, each of which includes a plurality of virtual surfaces positioned along a virtual sphere surrounding the three-dimensional model with the reference point as a center, and has different distances from the reference point corresponding to the first numerical range in the at least one initial setting range, and assuming a case in which the three-dimensional model is photographed from each of the M1×T1 first virtual camera positions, which are the plurality of first virtual camera positions virtually set for each of the M1 (M1 is a natural number of 2 or more) virtual surfaces in each of the T1 surface sets corresponding to each of the second numerical range and the third numerical range in the at least one initial setting range, the reference shape information is acquired, which is generated for each of the M1×T1 first virtual camera positions; The first calculation unit is Calculating a numerical value indicating a degree of agreement between the actual shape information and the reference shape information for each of the M1×T1 first virtual camera positions; The first detection unit is Detecting the high-matching virtual camera position, which is the first virtual camera position having the highest match between the actual shape information and the reference shape information, among the M1×T1 first virtual camera positions based on a calculation result by the first calculation unit; The search processing unit is a division surface generation unit that generates M2×T2 virtual divided surfaces by dividing, according to the same rule, T2 (T2 is a natural number of 2 or more) virtual surfaces, which include a high-matching virtual surface in which the high-matching virtual camera position is virtually set, intersect with a straight line passing through the reference point and the high-matching virtual camera position on the high-matching virtual camera position side of the reference point, and have mutually different distances from the reference point, among the M1 virtual surfaces in each of the T1 surface collections, to generate M2 (M2 is a natural number of 2 or more) virtual divided surfaces for each of the T2 virtual surfaces; a second shape information generation unit that generates the reference shape information for each of the M2×T2 second virtual camera positions, the M2×T2 second virtual camera positions being virtually set for each of the M2×T2 virtual divided surfaces based on the three-dimensional design information, assuming a case in which the three-dimensional model is photographed from each of the M2×T2 second virtual camera positions; a second calculation unit that calculates a numerical value indicating a degree of matching between the actual shape information and the reference shape information for each of the M2×T2 second virtual camera positions; a second detection unit that detects a virtual camera position having the highest degree of matching between the actual shape information and the reference shape information from among the high-matching virtual camera position and the M2×T2 second virtual camera positions, The substrate processing apparatus, wherein the search processing unit searches for the maximum degree of matching virtual camera position based on the virtual camera position detected by the second detection unit, or the virtual camera position detected by the second detection unit is the maximum degree of matching virtual camera position.

4. The substrate processing apparatus according to claim 3, The search processing unit is After executing a first unit process, execute one or more n-th unit processes (n is a natural number equal to or greater than 2) for the search target part; The search processing unit is In the first unit process, a first A process, a first B process, a first C process, and a first D process are performed in this order; The first A process is a process of generating first M3×T3 virtual division surfaces which are M3×T3 virtual division surfaces by dividing each of the T2 virtual surfaces by the division surface generation unit, the first T3 virtual division surfaces being T3 virtual division surfaces (T3 is a natural number of 2 or more) which include a virtual division surface including a first reference virtual camera position which is the virtual camera position first detected by the second detection unit, intersect with a straight line passing through the reference point and the first reference virtual camera position on the side of the first reference virtual camera position relative to the reference point, and have different distances from the reference point, by dividing each of the first T3 virtual division surfaces by the same rule to generate first M3 virtual division surfaces which are M3 (M3 is a natural number of 2 or more) virtual division surfaces for each of the first T3 virtual division surfaces; The first B process is a process of generating the reference shape information for each of the first M3×T3 third virtual camera positions by the second shape information generation unit, based on the three-dimensional design information, assuming a case in which the three-dimensional model is photographed from each of the first M3×T3 third virtual camera positions which are M3×T3 virtual camera positions virtually set for each of the first M3×T3 virtual divided surfaces; The first C process is a process of calculating, by the second calculation unit, a numerical value indicating a degree of matching between the actual shape information and the reference shape information for each of the first M×T third virtual camera positions; The first D process includes: a process of detecting, by the second detection unit, a second reference virtual camera position which is a virtual camera position having the highest degree of matching between the actual shape information and the reference shape information, among the first reference virtual camera position and the first M3×T3 third virtual camera positions; The search processing unit is In the nth unit process, an nth A process, an nth B process, an nth C process, and an nth D process are performed in this order; The nA treatment is a process of generating n-th M3×T3 virtual division surfaces which are M3×T3 virtual division surfaces by dividing, by the same rule, each of n-th T3 virtual division surfaces which are T3 virtual division surfaces including a virtual division surface including an n-th reference virtual camera position which is the virtual camera position detected n-th by the second detection unit, and which intersect with a straight line passing through the reference point and the n-th reference virtual camera position on the n-th reference virtual camera position side of the reference point and which have different distances from the reference point, for each of the n-th T3 virtual division surfaces, and generating n-th M3 virtual division surfaces which are M3 virtual division surfaces for each of the n-th T3 virtual division surfaces; The nB process is a process of generating the reference shape information for each of the nth M3×T3 third virtual camera positions by the second shape information generation unit, based on the three-dimensional design information, assuming a case in which the three-dimensional model is photographed from each of the nth M3×T3 third virtual camera positions which are M3×T3 virtual camera positions virtually set for each of the nth M3×T3 virtual dividing surfaces; The nC process is a process of calculating, by the second calculation unit, a numerical value indicating a degree of matching between the actual shape information and the reference shape information for each of the n-th M×T third virtual camera positions; The nD process is a process of detecting, by the second detection unit, a virtual camera position having the highest degree of matching between the actual shape information and the reference shape information, among the n-th reference virtual camera position and the n-th M3×T3 third virtual camera positions; the virtual camera position detected by the second detection unit in the last n-th D process in the one or more n-th unit processes is the maximum matching virtual camera position.

5. The substrate processing apparatus according to claim 4, The search processing unit is The substrate processing apparatus terminates execution of the one or more nth unit processes in response to the second detection unit detecting one of the reference virtual camera positions from the first reference virtual camera position to the nth reference virtual camera position a predetermined first predetermined number of times consecutively.

6. The substrate processing apparatus according to claim 4, The search processing unit is The substrate processing apparatus terminates the one or more executions of the nth unit process in response to the nth unit process being executed a second predetermined number of times.

7. A substrate processing apparatus according to any one of claims 3 to 6, The same rule is: A substrate processing apparatus comprising a rule for dividing a surface to be divided into a plurality of surfaces by a plurality of line segments respectively connecting a center point of the surface to be divided and all vertices of the surface to be divided.

8. A substrate processing apparatus according to any one of claims 3 to 6, Each of the plurality of virtual surfaces is It has a triangular surface, The surface collection includes: A substrate processing apparatus that is a polyhedron made up of multiple triangular faces.

9. The substrate processing apparatus according to claim 8, The division surface generation unit A substrate processing apparatus that divides the high-degree-of-match virtual surface into three virtual divided surfaces as the M2 virtual divided surfaces by three line segments respectively connecting three vertices on the high-degree-of-match virtual surface and the high-degree-of-match virtual camera position.

10. A substrate processing apparatus according to any one of claims 1 to 6, a moving mechanism for moving the search target part between a first position and a second position; The one or more default ranges are: a first default range for the first position; The search processing unit is At a timing when the movement mechanism positions the search target part at the first position, The first shape information acquisition unit acquires the reference shape information generated for each of the plurality of first virtual camera positions corresponding to the first initial setting range, assuming a case in which the three-dimensional model is photographed from each of the plurality of first virtual camera positions corresponding to the first initial setting range, which is the at least one initial setting range, based on the three-dimensional design information and the setting range information; the first calculation unit calculates, for each of the plurality of first virtual camera positions according to the first initial setting range, a numerical value indicating a degree of agreement between the actual shape information related to the actual image obtained by photographing with the one or more photographing units at a timing when the moving mechanism positions the search target part at the first position and the reference shape information; A substrate processing apparatus in which the first detection unit detects, based on the calculation result by the first calculation unit, the high-match virtual camera position among the multiple first virtual camera positions corresponding to the first initial setting range, which has the highest match between the actual shape information and the reference shape information.

11. The substrate processing apparatus according to claim 10, The one or more default ranges are: a second default range for the second position; The search processing unit is At a timing when the moving mechanism positions the search target part at the second position, The first shape information acquisition unit acquires the reference shape information generated for each of the plurality of first virtual camera positions corresponding to the second initial setting range, assuming a case in which the three-dimensional model is photographed from each of the plurality of first virtual camera positions corresponding to the second initial setting range, which is the at least one initial setting range, based on the three-dimensional design information and the setting range information; the first calculation unit calculates, for each of the plurality of first virtual camera positions corresponding to the second initial setting range, a numerical value indicating a degree of agreement between the actual shape information related to the actual image obtained by photographing with the one or more photographing units at a timing when the search target part is positioned at the second position by the moving mechanism, and the reference shape information; A substrate processing apparatus in which the first detection unit detects, based on the calculation result by the first calculation unit, the high-match virtual camera position among the multiple first virtual camera positions corresponding to the second initial setting range, which has the highest match between the actual shape information and the reference shape information.

12. The substrate processing apparatus according to claim 10, the substrate processing apparatus further comprising an abnormality detection unit that detects an abnormality in the search target part if the degree of matching between the reference shape information and the actual shape information for the maximum degree of matching virtual camera position detected by the search processing unit is smaller than a predetermined degree of matching set in advance.

13. The substrate processing apparatus according to claim 10, The one or more default ranges are: a second default range for the second position; The search processing unit is At a timing when the movement mechanism positions the search target part at the first position, the first shape information acquisition unit acquires the reference shape information generated for each of the plurality of first virtual camera positions corresponding to both the first initial setting range and the second initial setting range, based on the three-dimensional design information and the setting range information, assuming a case in which the three-dimensional model is photographed from each of the plurality of first virtual camera positions corresponding to both the first initial setting range and the second initial setting range, which are the at least one initial setting range; the first calculation unit calculates, for each of the plurality of first virtual camera positions corresponding to both the first initial setting range and the second initial setting range, a numerical value indicating a degree of agreement between the actual shape information related to the actual image obtained by photographing with the one or more photographing units at a timing when the search target part is positioned at the first position by the moving mechanism, and the reference shape information; the first detection unit detects, based on a calculation result by the first calculation unit, the high-match virtual camera position having the highest match between the actual shape information and the reference shape information among the plurality of first virtual camera positions corresponding to both the first initial set range and the second initial set range; The substrate processing apparatus includes: the substrate processing apparatus further comprising an abnormality detection unit that detects an abnormality in the search target part if the maximum matching virtual camera position is a position that belongs to the second initial setting range at the timing when the search target part is positioned at the first position by the moving mechanism.

14. A substrate processing apparatus according to any one of claims 1 to 6, the substrate processing apparatus further comprising an abnormality detection unit that detects an abnormality in the search target part by comparing actual information related to the posture of the search target part recognized based on the maximum matching virtual camera position with normal information related to the posture of the search target part based on the three-dimensional design information when the state of the search target part is normal.

15. A substrate processing apparatus according to any one of claims 1 to 6, The one or more target parts include two or more target parts; The at least one target part is the two or more target parts.

16. 1. A method for processing information in a substrate processing apparatus having one or more target components, comprising: a storage step of storing in a storage unit setting range information that defines one or more initial setting ranges related to a virtual camera position that is preset for each of at least one target part among the one or more target parts; a photographing step of photographing an actual image capturing a search target part, which is any one of the one or more target parts, by a photographing unit; a search step of executing a search process for searching for a maximum matching virtual camera position among the plurality of virtual camera positions, the maximum matching virtual camera position having the highest matching degree between the reference shape information and the actual shape information, based on reference shape information about a two-dimensional shape of a three-dimensional model in each of a plurality of virtual images that can be acquired by photographing the three-dimensional model of the search target part from a plurality of virtual camera positions, the reference shape information being generated by a calculation unit based on three-dimensional design information about the search target part, and actual shape information about a two-dimensional shape of an object in the actual image obtained in the photographing step; The searching step includes: a first shape information acquisition step of acquiring the reference shape information generated for each of a plurality of first virtual camera positions based on the three-dimensional design information and the setting range information, assuming a case in which the three-dimensional model is photographed from each of a plurality of first virtual camera positions that are positioned along a plurality of virtual spheres that surround the three-dimensional model with a reference point of the three-dimensional model as a center and correspond to at least one initial setting range of the one or more initial setting ranges for the search target part defined by the setting range information; a first calculation step of calculating a numerical value indicating a degree of agreement between the reference shape information and the actual shape information for each of the plurality of first virtual camera positions acquired by the first shape information acquisition step; a first detection step of detecting, from among the plurality of first virtual camera positions, a high-match virtual camera position having the highest match between the actual shape information and the reference shape information, based on a calculation result by the first calculation step; An information processing method in a substrate processing apparatus, wherein in the search step, the maximum degree of agreement virtual camera position is searched for based on the high degree of agreement virtual camera position detected in the first detection step, or the high degree of agreement virtual camera position is the maximum degree of agreement virtual camera position.

Images