Method and system for estimating normal vector of mold and fitting

Abstract

Provided are a method and system for estimating a normal vector of a jig and a normal vector of a mold. Based on depth data or coordinate data of a mounting tool such as a mounting bolt (82) of three points or a chuck of three points from an image, a normal vector of a head (60) is calculated and estimated using a coordinate or depth determining section (98) and a normal vector calculating section (99). Based on an image, at least three points or more of an extension portion (36A to 36D, 35A to 35D) are determined, the extension portion being a part of a fixed mold (31) or a movable mold (32, 34) or a part of a surrounding member and extending in a direction consistent with an opening and closing direction of the fixed mold (31) and the movable mold (32, 34). Based on depth data or coordinate data of the determined three points or more of the extension portion, a normal vector of the mold is calculated and estimated using the coordinate or depth determining section (98) and the normal vector calculating section (99).

Description

TECHNICAL FIELD

[0001] The present application relates to a method and a system for estimating a normal vector of a fitting installed in a state of an approach frame of a molded product taking-out machine and a normal vector of a mold installed in a state of a molding machine. BACKGROUND

[0002] In Japanese Patent Application Publication No. 2002-120175 (Patent Literature 1), an invention related to teaching of a molded product taking-out machine is disclosed. Conventionally, when a molded product is taken out from a mold of a molding machine, an operator observes a relationship between a taking-out head and the mold, and performs a setting operation of a teaching program which programs a path to the approach frame and a posture of the fitting while moving the taking-out head, so that the taking-out head (fitting) installed in the approach frame of the molded product taking-out machine does not collide with the mold.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2002-120175

[0006] In a case where the above operation is performed, a problem that the taking-out head and the mold collide and the mold is damaged occurs. This problem occurs because of the operator's awareness that the setting condition of the mold of the molding machine and the installation state of the structural member of the taking-out head are always constant. However, in reality, the installation state of the taking-out head is not always constant. In such a reality, for making or modifying the teaching program, a high degree of skill and observation ability of the operator is required. If the taking-out head is inclined with respect to the mold, there is a concern that, for example, the mold is contacted at the mold approach of the taking-out head and is damaged at the face engagement. Therefore, it is necessary to grasp the inclination of the taking-out head and the mold and to align the inclination at the mold approach and at the face engagement.

[0007] If automation and high precision of the teaching are considered, it is necessary to know the installation state of the mold and the fitting. The installation state of the mold and the fitting can be recognized by knowing a normal vector of the fitting installed in a state of an approach frame of a molded product taking-out machine and a normal vector of the mold installed in a state of a molding machine. SUMMARY

[0008] An object of the present application is to provide a method and a system for estimating a normal vector of a fitting installed in a state of an approach frame of a molded product taking-out machine and a normal vector of a mold installed in a state of a molding machine, respectively.

[0009] The present application is a method for estimating a normal vector of a fitting in a state where the fitting is installed in an approach frame of a molded product taking-out machine and a normal vector of a mold in a state where the mold is installed in a molding machine. In the present application, a vector perpendicular to a joint surface of a fixed mold and a movable mold of the mold in a state where the mold is installed in the molding machine is defined as the normal vector of the mold, and a vector extending in a direction along which the approach frame extends and being perpendicular to an imaginary surface parallel to the joint surface when the fitting is inserted between the fixed mold and the movable mold is defined as the normal vector of the fitting. In the present application, an intermediate mold interposed between a general fixed mold and a movable mold is also included in the movable mold. Further, an image including surrounding components around a joint surface of the fixed mold or a joint surface of the movable mold is captured using a photographing device capable of capturing an image including depth data or coordinate data of an object, at least three points or more of an extension part which is a part of the fixed mold or the movable mold or a part of the surrounding components and extends in a direction in which the fixed mold and the movable mold open and close are determined from the image, and a normal vector of the mold is estimated based on the depth data or the coordinate data of the three points or more of the extension part. In the present application, the "depth data" is data including coordinate data of a position of a point obtained from point group data obtained from a three-dimensional photographing device. That is, the "depth data" refers to information of an orientation (determined by an angle) and a distance r of an object observed from a photographing device (camera) when the object is observed from the photographing device. Further, the "coordinate data" refers to a coordinate P(x, y, z) of the object in an x, y, z orthogonal coordinate system with the photographing device (camera) as an origin. Further, the depth data can be converted into the coordinate data.

[0010] Further, in the present application, an image of three points or more of a fitting tool in which the fitting is installed in the approach frame or three points or more of a contact tool in contact with the molded product is captured using a photographing device capable of capturing an image including depth data or coordinate data of an object, the fitting tool or the contact tool extending in a direction orthogonal to the approach frame, and a normal vector of the fitting is estimated based on the depth data or the coordinate data of the three points of the fitting tool or the contact tool from the image. As the photographing device outputting the depth data and the coordinate data, a three-dimensional photographing device can be used. Further, as a method for estimating the normal vector based on the depth data or the coordinate data from three points or more of data, a known mathematical method can be used. In the present application, the normal vector of the fitting and the normal vector of the mold can be estimated by the photographing device taking image data including the depth data or the coordinate data obtained from the actually used mold and the fitting, and the normal vector closely related to the actual installation state of the mold and the fitting is estimated. Therefore, teaching automation, posture control of the fitting, and the like can be easily achieved.

[0011] The present application can also be determined as a method of estimating a normal vector of a mold in a state where the mold is installed in a molding machine. In this case, a vector perpendicular to the joint surface of the fixed mold and the movable mold of the mold in a state where the mold is installed in the molding machine is defined as the normal vector of the mold. Furthermore, an image including surrounding components around the joint surface of the fixed mold or the joint surface of the movable mold is captured using a photographing device capable of capturing an image including depth data or coordinate data of an object. Based on the image, an extension portion of at least three points or more, which is a part of the fixed mold or the movable mold or a part of the surrounding components and extends in a direction consistent with the opening and closing direction of the fixed mold and the movable mold, is determined, and based on the depth data or the coordinate data of the determined extension portion of three points or more, the normal vector of the mold is estimated.

[0012] Furthermore, the present application can also be determined as a method of estimating a normal vector of a fitting in a state where the fitting is installed in an entry frame of a molded product extraction machine. In this case, a vector extending in a direction along the entry frame and perpendicular to an imaginary plane parallel to the parting surface when the fitting is inserted between the fixed mold and the movable mold is defined as the normal vector of the fitting. Furthermore, an image including three points or more of a mounting tool or a contact tool of the fitting installed in the entry frame, which contact the fitting, extending in a direction orthogonal to the entry frame, is captured using a photographing device capable of capturing an image including depth data or coordinate data of an object. Based on the image, the depth data or the coordinate data of the three points of the mounting tool or the three points of the contact tool is obtained, and based on the depth data or the coordinate data, the normal vector of the fitting is estimated.

[0013] The extension portion can use a guide pin of the mold, an edge surface of the fixed mold or the movable mold, or a tie bar, etc.

[0014] The mounting tool is, for example, an edge surface of a mounting bolt or a mounting metal piece (mounting plate), and the contact tool is, for example, a suction cup. The photographing device is a three-dimensional photographing device configured to be capable of capturing an image of one joint surface of the movable mold and the fixed mold of the mold in an open state, and capable of capturing an image including an extraction surface of the fitting when the fitting is outside the mold.

[0015] Using the normal vector of the mold and the normal vector of the fitting estimated using the method of estimating a normal vector of the present application, the posture of the fitting when the fitting is inserted into the mold can be determined so that the extraction operation can be performed in a posture where the normal vector of the mold and the normal vector of the fitting are consistent.

[0016] Further, the normal vector of the accessory determined by the normal vector estimation method using the present application and the image of the photographing device can determine the normal vector of the accessory as a one-dimensional vector based on a three-dimensional size, thereby correctly measuring the maximum three-dimensional size of the accessory.

[0017] Further, the present application can be grasped as an estimation system of the normal vector of an accessory in a state where the accessory is installed in an entry frame of an ejection machine and the normal vector of a mold in a state where the mold is installed in a molding machine. In the system of the present application, a processor defines a vector perpendicular to the joint surface of the fixed mold and the movable mold of the mold installed in the molding machine as the normal vector of the mold, and defines a vector along the direction in which the entry frame extends and perpendicular to an imaginary surface parallel to the joint surface when the accessory is inserted between the fixed mold and the movable mold as the normal vector of the accessory. Also, the processor obtains depth data or coordinate data of an extension portion of at least three points or more determined from an image including surrounding parts of the fixed mold or the movable mold around the joint surface of the fixed mold or the movable mold photographed using a first photographing device capable of photographing an image including depth data or coordinate data of an object, and estimates the normal vector of the mold based on the depth data or the coordinate data. The extension portion is a part of the fixed mold or the movable mold or a part of the surrounding parts and extends in a direction consistent with the opening and closing direction of the fixed mold and the movable mold. Further, the processor obtains depth data or coordinate data of an installation tool of three points or a contact tool of three points determined from an image photographed using a second photographing device, the image including an installation tool of three points or more or a contact tool of three points or more contacting the molded product when the accessory is installed in the entry frame, the installation tool or the contact tool extending in a direction orthogonal to the entry frame, and estimates the normal vector of the accessory based on the depth data or the coordinate data. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a perspective view of a molded product manufacturing system including an orthogonal three-axis robot to which the normal vector estimation method of the present application can be applied, the orthogonal three-axis robot having an axis of movement in an X direction, a Y direction, and a Z direction.

[0019] Figure 2 is a left side view of the molded product manufacturing system.

[0020] Figure 3 is a front view of the molded product manufacturing system.

[0021] Figure 4is a plan view of a molded article manufacturing system.

[0022] Figure 5 is a block diagram showing the structure of a maximum dimension measuring system constructed in the control system of the molded article takeout machine for applying the normal vector estimation method of the present application.

[0023] Figure 6 is a diagram showing the maximum dimension of the takeout head.

[0024] Figure 7 (A) to (H) of FIG. 1 are image displays for explaining the maximum dimension measuring method.

[0025] Figure 8 (A) to (E) of FIG. 2 are image displays for explaining the maximum dimension measuring method.

[0026] Figure 9 is a block diagram showing the structure of a measuring system used in the case where the maximum dimension is measured using one three-dimensional imaging device as the imaging device.

[0027] Figure 10 (A) to (D) of FIG. 3 are diagrams used for explaining the case where the maximum dimensions in the X direction and the Z direction are measured using point group data.

[0028] Figure 11 is a diagram used for explaining the case where the maximum dimension in the Y direction is measured using point group data.

[0029] Figure 12 is a block diagram showing the structure of a normal vector estimation system used in the case where the normal vector estimation method of the present application is applied to the molded article manufacturing system.

[0030] Figure 13 is a diagram used for explaining an example of the method of estimating the normal vector of the takeout head.

[0031] Figure 14 (A) to (B) of FIG. 4 are diagrams used for explaining an example of the method of estimating the normal vector of the takeout head.

[0032] Figure 15 (A) of FIG. 5 is a diagram for explaining the depth data obtained by the three-dimensional imaging device, Figure 15 (B) of FIG. 5 is a diagram for explaining the case where three points of the imaging target are imaged by the three-dimensional imaging device.

[0033] Figure 16 is a flowchart showing the flow in the case where the normal vector is estimated by the operation from the vibration data.

[0034] Figure 17FIG. 1 is a diagram used to explain an example of a method of estimating a normal vector of a mold.

[0035] Figure 18 FIG. 1 is a diagram used to explain an example of a method of estimating a normal vector of a mold.

[0036] BRIEF DESCRIPTION OF DRAWINGS

[0037] 1: molding product manufacturing system;

[0038] 3: molding machine;

[0039] 31: fixed mold;

[0040] 32: intermediate mold;

[0041] 34: movable mold;

[0042] 5: molding product take-out machine;

[0043] 59B: lifting frame (entry frame);

[0044] 6: take-out head (fitting);

[0045] 71: imaging device system;

[0046] 72: image display device;

[0047] 73: screen;

[0048] 74: X-axis direction driving source;

[0049] 75: X-axis direction movement amount measurer;

[0050] 76: Y-axis direction driving source;

[0051] 77: Y-axis direction movement amount measurer;

[0052] 78: Z-axis direction driving source;

[0053] 79: Z-axis direction movement amount measurer;

[0054] C11: first imaging device;

[0055] C12: second imaging device;

[0056] 90: control device;

[0057] 91: teaching unit;

[0058] 92: data storage unit;

[0059] 93: usability determination unit;

[0060] 96: image control unit;

[0061] 97: operation section;

[0062] 98: coordinate or depth determination section;

[0063] 99: normal vector operation section. DETAILED DESCRIPTION

[0064] Hereinafter, an embodiment of the normal vector estimation method of the present application will be explained in detail with reference to the drawings. Figures 1 to 4 is a perspective view, a left side view, a front view, and a plan view of a shaped product manufacturing system 1 including an orthogonal three-axis robot to which the normal vector estimation method of the present application can be applied and to which a maximum dimension measurement method can be applied, the orthogonal three-axis robot having an axis of movement in an X direction, a Y direction, and a Z direction. In Figure 1 In the drawing, the X direction, the Y direction, and the Z direction used in the present embodiment are shown. The shaped product manufacturing system 1 is configured by combining a resin molding machine 3 and a shaped product takeout machine 5 as an orthogonal three-axis robot. The shaped product takeout machine 5 is a diagonal line slide type shaped product takeout machine, the base portion of which is supported by a fixed layer plate 30 of the resin molding machine 3.

[0065] A fixed mold 31 is fixed to the fixed layer plate 30 of the resin molding machine 3, and a movable mold 34 is fixed to a movable layer plate 33. Also, an intermediate mold 32 is regarded as a movable mold. Further, between the fixed layer plate 30 and the movable layer plate 33, four tie rods 35A to 35D that guide the movement of the movable layer plate 33 are arranged. The four tie rods 35A to 35D are arranged leaving equal intervals, and an imaginary center line that passes through the centers of the four tie rods 35A to 35D passes through the centers (nozzle centers) of the fixed mold 31 and the movable molds (32, 34). In addition, the intermediate mold 32 and the movable mold 34 are guided by guide pins 36A to 36D, respectively. The four guide pins 36A to 36D are also arranged leaving equal intervals, and an imaginary center line that passes through the centers of the four guide pins 36A to 36D also passes through the centers (nozzle centers) of the fixed mold 31 and the movable molds 34.

[0066] The molded article take-out machine 5 has a traverse shaft 53, a first traveling body 55, an extraction shaft 57, a runner elevating unit 58, and a molded article suction elevating unit 59. The traverse shaft 53 has a cantilever beam structure extending in the X-axis direction orthogonal to the length direction of the molding machine 3. The first traveling body 55 is supported by the traverse shaft 53 and advances and retreats in the X-axis direction along the traverse shaft 53 with an AC servo motor included in a servo mechanism as a driving source. The extraction shaft 57 is provided to the first traveling body 55 and extends in the Y-axis direction parallel to the length direction of the molding machine. The runner elevating unit 58 and the molded article suction elevating unit 59 are supported by the extraction shaft 57 so as to be movable in the Y direction with an AC servo motor included in a servo mechanism as a driving source. The runner elevating unit 58 has a structure in which a traveling body 58A supported so as to be movable by the extraction shaft 57 has an elevating frame 58B elevating in the Z direction. The traveling body 58A is driven by the AC servo motor and thereby moves in the Y direction. The elevating frame 58B elevates in the up-and-down direction (Z direction) by a driving source. The elevating frame 58B has a jig 58C as a fitting holding a discarded runner.

[0067] Further, the traveling body 59A included in the molded article suction elevating unit 59 moves in the Y direction on the extraction shaft 57 by being driven by the AC servo motor. The molded article suction elevating unit 59 has an elevating frame 58B elevating in the up-and-down direction (Z direction) by a driving source, a turnover unit 59C as a posture control device rotating about the axis of the elevating frame 59B, and a take-out head 60 provided to the turnover unit 59C. In the present embodiment, in the case of measuring the maximum dimension of the take-out head 60 as a fitting, or the like, the normal vector of the take-out head 60 is estimated.

[0068] In the present embodiment, in order to experimentally search for a preferable installation site, eight photographing devices C1 to C8 are installed at each part of the molded article take-out machine 5 and the molding machine 3, and on the table 7 placed in the lateral direction of the molding machine 3. As these photographing devices C1 to C8, three-dimensional cameras (three-dimensional photographing devices) are used. In the present embodiment, a photographing device from which a preferable image can be obtained can be selected from among these photographing devices C1 to C8, and thus a desired image can be obtained.

[0069] Figure 5 is a block diagram showing the structure of a maximum dimension measuring system constructed in the control system of the molded article take-out machine 5 in order to implement the method of measuring the three-dimensional shape dimension of a fitting. Further, Figure 6 is a view showing the maximum dimension of the take-out head 60. Figure 7 and Figure 8 is an image display for explaining the method of measuring the maximum dimension.

[0070] In the maximum dimension measurement method using the normal vector estimation method of the present embodiment, in a state where the extraction head 60 as a fitting is attached to the lifting frame 59B of the entry frame as the molding product extraction machine 5 which is an orthogonal three-axis robot, before the extraction head 60 starts the operation, based on the images of the extraction head 60 taken by at least one or more photographing devices (C1 to C8), the maximum dimensions of the extraction head 60 in the X direction, the Y direction, and the Z direction are measured. Although it also depends on the measurement method used, it is preferable to determine the three-dimensional position coordinates of the photographing device when photographed by at least one or more photographing devices, the three-dimensional position coordinates of the extraction head 60 (fitting) being photographed, and the photographing angle of the photographing device and the mounting posture of the extraction head 60 (fitting) to the lifting frame 59B (entry frame) so that the image obtained by the photographing device includes the information required for the measurement of the three-dimensional shape dimensions of the extraction head 60. The more coordinate information, photographing angles of the photographing device, mounting postures, and the like are provided in advance in this way, the easier the calculation when measuring the maximum dimensions based on the image becomes, and thus it is possible to reduce the correction calculation of the image data based on the difference in the mounting position and the mounting posture of the photographing device.

[0071] Therefore, in order to obtain the photographing angle information of the photographing device, the normal vector estimation method of the fitting of the present application can be used. That is, as long as the normal vector of the extraction head 60 is known, it is possible to make the angle of the normal vector coincide with the photographing angle of the photographing device.

[0072] In the case of measuring the maximum dimensions using the system shown in the block diagram of Figure 5 In order to measure the maximum dimensions by the operation of the operator, the photographing system 71 including one or more photographing devices uses, in the case of measuring the maximum dimensions using the system shown in the block diagram of FIG. 8, a first photographing device C11 for obtaining a first image of the extraction head 60 in a posture capable of measuring the maximum dimensions of the extraction head 60 in the X direction and the Z direction, a second photographing device C12 for obtaining a second image of the extraction head 60 in a posture capable of measuring the maximum dimensions of the extraction head 60 in the Y direction and the Z direction, and an image display device 72 provided with a screen 73 with an orthogonal coordinate axis indicator that displays the first image or the second image. Here, the screen 73 with the orthogonal coordinate axis indicator is a gage screen having two orthogonal axes (GZ-GX, GZ-GY).

[0073] In the present embodiment, the X-direction drive source 74 and the X-direction movement amount measurer 75 that move the lifting frame 59A in the X-direction, the Y-direction drive source 76 and the Y-direction movement amount measurer 77 that move the lifting frame 59A in the Y-direction, the Z-direction drive source 78 and the Z-direction movement amount measurer 79 that move the lifting frame 59A in the Z-direction are used. The operator performs the following operations using the operation section 97 including an operation switch or the like provided in the controller. In addition, in the following operations, the image display of the screen 73 of the image display device 72 is performed by the image control section 96 in the control device 90 constituted in the control section of the shaped product take-out machine 5. Furthermore, the calculation of the maximum dimension is performed by the dimension calculation section 95 based on the outputs of the X-direction movement amount measurer 75 to the Z-direction movement amount measurer 79. Also, according to the operation from the operation section 97, the operation command to the X-direction drive source 74 to the Z-direction drive source 78 is output from the drive control section 94. Further, the control device 90 is provided with the teaching section 91 that performs the operation of teaching to be described later, the data storage section 92 that stores the teaching data, and the usability determination section 93.

[0074] Specifically, as the first image device C11, the photographing device C1 or C2 shown in FIG. 1 can be used without changing the posture of the take-out head 60. With the first image device C11, the take-out head 60 as a fitting is photographed from the front. Furthermore, as the second image device C12, the photographing device C6 to C8 shown in FIG. 2 can be used without changing the posture of the take-out head 60. The photographing device C1 to C5 shown in FIG. 1 can be used without changing the posture of the take-out head 60. With the first image device C11, the take-out head 60 as a fitting is photographed from the front. The second image device C12 photographs the take-out head 60 from the side. The normal vector estimation method of the present application can be used when the first and second photographing devices C11 and C12 are disposed in front of or on the side of the take-out head 60. Figures 1 to 4 Figures 1 to 4 Figures 1 to 4

[0075] Furthermore, specifically, as Figure 7 ​​​As shown, the orientation of the orthogonal coordinate axes GZ and GX on the screen 73, which have orthogonal coordinate axis indicators G, is aligned with the X and Z directions in the first image from the first imaging device C11. This orientation is achieved while observing the image on the screen 73 and the image on the indicator G. Then, during the period when the Z-direction drive source 78, including a servo motor, moves the extraction head 60 in the Z-direction, the time from the outermost end of the extraction head 60 in one direction of the Z-direction in the first image, i.e., the suction cup 60A, crosses the reference line on the screen (in this example, GX) (assuming the measured value at this time is Zs) to the outermost end of the extraction head 60 in the other direction of the Z-direction, i.e., the suction cup 60B, crosses the reference line (in this example, GX) (assuming the measured value at this time is Ze) […]. Figure 7 (A) to Figure 7 [D], the distance measured by the Z-direction movement measuring device 79, including the encoder, etc. [refer to] Figure 6 [Zs-Ze] is used to calculate the maximum dimension in the Z direction.

[0076] Furthermore, during the period when the orthogonal coordinate axes GZ and GY on the screen 73 with the orthogonal coordinate axis indicator G extend in the same direction as the X and Z directions in the first image, the X-direction drive source 74 is driven to move the extraction head 60 in the X direction. During this period, from the outermost end of one direction of the X direction in the first image, i.e., the suction cup 60C, crossing the reference line on the screen (in this example, GZ) (let's assume the measured value at this time is Xs), to the outermost end of the other direction of the X direction, i.e., the suction cup 60D, crossing the reference line (GZ) (let's assume the measured value at this time is Xe), the distance measured by the X-direction movement measuring device [refer to...] Figure 6 [Xs-Xe], the maximum dimension in the X direction is determined.

[0077] Furthermore, such as Figure 8 As shown, during the period when the orthogonal coordinate axis GZ-GY on screen 73, with orthogonal coordinate axis indicator G, extends in the same direction as the Y and Z directions in the second image (side view), the distance measured by the Y-direction drive source 76 to move the extraction head 60 in the Y direction is as follows: from the outermost end 60E of one direction in the Y direction of the second image crossing the reference line on the screen (GZ in this example) (let's assume the measured value at this time is Ys) to the outermost end 60F of the other direction in the Y direction crossing the reference line (GZ) (let's assume the measured value at this time is Ye), the distance measured by the Y-direction movement measuring device [refer to...] Figure 6 [Ys-Ye], to determine the maximum dimension in the Y direction.

[0078] According to the maximum dimension measurement method, the maximum dimensions in the X direction, the Y direction, and the Z direction can be measured with a simple device and by a simple operation. With regard to the extraction head 60 mounted to the lifting frame 59B of the molded product extraction machine 5, the mounted parts are often replaced with parts different from the design specifications due to repair, the arrangement posture of the mounted parts is changed due to maintenance. For example, the extraction head 60 in the state of being mounted to the lifting frame 59B as an entry frame is configured to combine subsidiary parts including air tubes or wiring that supply power to the extraction head 60. Therefore, the positions and postures of these air tubes or wiring are sometimes changed every time the extraction head 60 is replaced. In addition, there are cases where the worker mounts an incorrect extraction head to the lifting frame 59B. Even in such a case, as long as the maximum dimensions in the X, Y, and Z directions of the extraction head 60 are measured in the state of being mounted to the lifting frame 59B, it is possible to determine the change in the shape of the extraction head 60 that is actually intended to be used before the extraction work actually using the extraction head 60 is started. As a result, it is possible to detect in advance the case where the extraction head 60 collides with a part or the like in the movement path, the case where an incorrect extraction head is mounted.

[0079] (Other measurement system)

[0080] Figure 9 is a block diagram showing the configuration of a measurement system used in a case where the maximum dimensions are measured using one three-dimensional camera C13 as a camera. In this measurement system, the maximum dimensions are measured using the three-dimensional camera C13 as a camera, which measures the surface of an object and outputs as point group data of a plurality of points having three-dimensional coordinates. A first point group data acquisition section 101 A acquires first point group data capable of measuring the maximum dimensions in the X direction and the maximum dimensions in the Z direction of the extraction head 60, from image data when the extraction head 60 is photographed from the front using the three-dimensional camera C13. In addition, a second point group data acquisition section 101 B acquires second point group data capable of measuring the maximum dimensions in the Y direction and the maximum dimensions in the Z direction of the extraction head 60, from image data when the extraction head 60 is photographed from the side using the three-dimensional camera C13. The normal vector estimation method of the present application can be used when the three-dimensional camera C13 is arranged in front of or on the side of the extraction head 60. In the case of using one three-dimensional camera, as long as the camera C1 of Figure 1 the camera C1 of the three-dimensional camera C13 is used to obtain the side image after the front image is photographed, using the turning unit 59C as a posture changing device to rotate the extraction head 60 by 90 degrees. Alternatively, of course, two three-dimensional cameras can be used to obtain both the front image and the side image.

[0081] The maximum size determination unit 100 in the control device 90' ​​determines the maximum size in the Z direction based on the coordinates of a point located at one of the outermost points in the Z direction and the coordinates of a point located at the other outermost point, according to the first set of data. Furthermore, it determines the maximum size in the X direction based on the coordinates of a point located at one of the outermost points in the X direction and the coordinates of a point located at the other outermost point, according to the first set of data. Further, it determines the maximum size in the Y direction based on the coordinates of a point located at one of the outermost points in the Y direction and the coordinates of a point located at the other outermost point, according to the second set of data.

[0082] Point group data obtained by a 3D imaging device is data that automatically measures the surface of an object and sets up a data file with the 3D coordinates of multiple points as point groups. That is, each point constituting a point group includes 3D coordinate information. Therefore, if a 3D imaging device... Figure 10 If the head is removed 60 as shown in (A) and photographed, the following can be obtained: Figure 10 The data consists of a set of points as shown in (B). Each point includes information about its three-dimensional coordinates. Therefore, as... Figure 10 As shown in (C) and (D), the maximum size determination unit 100 calculates the coordinates of the point with the largest and smallest X-direction coordinates among points with the same depth (same Y-coordinate value) in the X-direction from the point group data, and uses the difference between these coordinate values ​​as the maximum size Xm in the X-direction. Furthermore, it calculates the coordinates of the point with the largest and smallest Z-direction coordinates among points with the same depth (same Y-coordinate value) in the Z-direction from the point group data, and uses the difference between these coordinate values ​​as the maximum size Zm in the Z-direction. Next, as... Figure 11 As shown, from the point group data acquired by the second point group data acquisition unit 101B, the coordinate values ​​of the point with the largest and smallest Y-direction coordinates among points with the same depth (same X-coordinate value) in the Y-direction are obtained, and the difference between these coordinate values ​​is taken as the maximum dimension Ym in the Y-direction. In this way, the maximum three-dimensional dimensions Xm, Ym, and Zm of the extraction head 60 can be obtained from the point group data. After obtaining the maximum dimensions, they can be used in the same way as in the original embodiment.

[0083] Furthermore, image-based methods for determining the maximum size can utilize well-known measurement techniques, minimizing measurement errors caused by differences in operator skill, as illustrated in the example above. Alternatively, if image data is used, the maximum size can be automatically determined by comparing a standard image of the removed head (accessory) with the captured image, or by employing AI technology adept at image recognition. The result is automated measurement.

[0084] (Estimation of the normal vectors of components and molds)

[0085] Figure 12 is a block diagram showing a structure of a normal vector estimation system used when the normal vector estimation method of the present application is applied to the above-described molded product manufacturing system. In Figure 12 the block diagram, elements common to the elements of the block diagram of Figure 5 are labeled with the same reference numerals, and elements unnecessary for the estimation of the normal vector are not illustrated. In Figure 12 the block diagram, a coordinate or depth determination section 98 and a normal vector calculation section 99 are provided within the control device 90, and further, a posture control device driving section 59D that controls the turnover unit 59C as a posture control device based on the estimation result of the normal vector is illustrated. At least the coordinate or depth determination section 98 and the normal vector calculation section 99 within the control device 90 are each configured using a processor.

[0086] The normal vector estimation system estimates the normal vector of the extraction head 60 as a fitting member installed in the state of the lifting frame 59B as an entry frame of the molded product extraction machine 5 and the normal vector of the mold 31, 32, 34 installed in the state of the molding machine 3. In the present embodiment, a vector perpendicular to the joint surface of each of the fixed mold 31 and the movable mold (32, 34) of the mold installed in the state of the molding machine 3 becomes the normal vector of the mold. Further, a vector extending in the direction along which the lifting frame 59B as an entry frame extends and perpendicular to an imaginary surface parallel to the joint surface when the extraction head 60 as a fitting member is inserted between the fixed mold 31 and the movable mold (32, 34) becomes the normal vector of the extraction head 60 as a fitting member.

[0087] Furthermore, in the present embodiment, as shown in Figure 13 and Figure 14 , an image including a subject is captured using a photographing device C11 (three-dimensional photographing device) capable of capturing an image including depth data or coordinate data of the subject, the image including three or more points of the mounting bolt 82 as a mounting tool extending in a direction orthogonal to the lifting frame 59B as an entry frame (extraction direction) and mounting the extraction head (fitting member) 60 to the lifting frame 59B or three or more points of the suction pad 60A to 60D as a contact tool in contact with the molded product. As the first image device C11, the photographing device C1 or C2 shown in Figures 1 to 4 may be used. The first image device C11 captures the extraction head 60 as a fitting member from the front or the back. In the image data captured by the three-dimensional photographing device, depth data of the subject is included.

[0088] As shown in Figure 14As shown in (A), the four mounting bolts 82 visible on the back of the extraction head 60 serve as the mounting reference for the extraction head. Therefore, the imaginary surface PS1, formed by connecting the four mounting bolts 82 and orthogonal to the four mounting bolts 82, becomes a surface orthogonal to the normal vector of the extraction head 60. Furthermore, as... Figure 14 As shown in (B), the imaginary surface PS2, formed by connecting the centers of suction cups 60A to 60D and orthogonal to suction cups 60A to 60D, becomes a surface orthogonal to the normal vector of the extraction head 60. Then, based on these images, the normal vector of the extraction head 60 is calculated and estimated using the coordinate or depth determination unit 98 and the normal vector calculation unit 99, based on the depth data or coordinate data of the three mounting bolts 82 or the three suction cups. The coordinate or depth determination unit 98 can obtain the depth data or coordinate data of the four mounting bolts 82 or the four suction cups 60A to 60D based on the information obtained from the images captured by the three-dimensional imaging device used as the imaging device C11 (data stored in the data storage unit 92). In addition, it is obvious that these data correspond to the specifications of the three-dimensional imaging device used. In this embodiment, the mounting bolt 82 is used as an example of the mounting tool, but the mounting tool also includes a mounting plate 81. When mounting metal parts (mounting plates), the plurality of edge surfaces 81A can be regarded as mounting tools with more than three points.

[0089] The normal vector calculation unit 99, which includes a processor, performs calculations on the normal vector according to a calculation method for obtaining the normal vector from a plane in a known space. Figure 15 (A) is a graph used to illustrate the relationship between depth data and coordinate data. Figure 15 In (A), point O is the position of the 3D imaging device, and point P is, for example, a point on the front or back of the retrieved head. In this case, the so-called depth data refers to the orientation of the retrieved head as observed from the 3D imaging device when viewing point P of the retrieved head from the 3D imaging device (determined by angle θ and ...). (Determine) and distance r. The coordinate data of point P, x, y, z, can be represented as z = rcosθ.

[0090] Therefore, as Figure 15 As shown in (B), for example, when the coordinate data of three points P, Q, and R on the front of the head are set as P(x, y, z), Q(x, y, z), and R(x, y, z), the coordinate data x, y, and z of these three points can be obtained by using the depth data of points P, Q, and R obtained by the 3D imaging device. Placed into the above formula [ is obtained. Here, if A, B, C, D are real numbers in a plane in an xyz orthogonal coordinate space so as to be given by a plane equation of Ax + By + Cz + D = 0, a normal vector n of the front face of the head is n = (A, B, C), and thus if the coordinate data of P, Q, R are substituted into the plane equation and the simultaneous equations are solved, A, B, C can be obtained. Figure 16 The transformation and the operation are shown.

[0091] In addition, as another idea of obtaining the normal vector, if the base vectors of the x-axis, y-axis, and z-axis are i, j, and k, respectively, the normal vector a can be expressed by a = Ai + Bj + Ck. As the magnitude, the 1-unit normal vector n is obtained by n = (Ai + Bj + Ck) / (A 2 + B 2 + C 2 ) 1 / 2 Using this operation method, the normal vector of the imaginary plane PS1 or PS2 can be obtained by operation. In addition, other operation methods can be used, as a matter of course. The above operation is performed by the normal vector operation section 99.

[0092] In addition, in the present embodiment, in the case where the normal vector of the mold is obtained, an image including the surrounding members around the joint surface of the fixed mold 31 or the joint surface of the movable mold (32, 34) is captured using the second imaging device C12 (three-dimensional imaging device) capable of capturing an image including depth data or coordinate data. As the second imaging device C12, at least one of the imaging devices C2 to C8 capable of adjusting the angle of view as shown in FIG. 12 can be used. Furthermore, in the present embodiment, based on the image captured by the second imaging device including the three-dimensional imaging device, at least three points or more of the extension portions (refer to the fixed mold 31 or the movable mold 34 of the mold of FIG. 13) of the guide pins 36A to 36D, the edge surface 34A of the movable mold 34, the tie bars 35A to 35D, and the like, which are a part of the fixed mold 31 or the movable mold (32, 34) or a part of the surrounding members and extend in a direction coinciding with the opening / closing direction (extracting direction) of the fixed mold 31 and the movable mold (32, 34), are determined. The determination can be performed based on the image captured in the screen 73 of the image display device 72. As described above, in the image data captured by the three-dimensional imaging device and stored in the data storage section 92, the depth data and the coordinate data of the subject are also included, and thus the coordinate or depth determination section 98 determines the depth data or the coordinate data of the determined three or more points of the extension portions. Then, the normal vector operation section 99 estimates the normal vector of the mold by operation based on the depth data or the coordinate data determined by the coordinate or depth determination section 98. Figures 1 to 4 Figure 17 Figure 18 In addition, in the present embodiment, in the case where the normal vector of the mold is obtained, an image including the surrounding members around the joint surface of the fixed mold 31 or the joint surface of the movable mold (32, 34) is captured using the second imaging device C12 (three-dimensional imaging device) capable of capturing an image including depth data or coordinate data. As the second imaging device C12, at least one of the imaging devices C2 to C8 capable of adjusting the angle of view as shown in FIG. 12 can be used. Furthermore, in the present embodiment, based on the image captured by the second imaging device including the three-dimensional imaging device, at least three points or more of the extension portions (refer to the fixed mold 31 or the movable mold 34 of the mold of FIG. 13) of the guide pins 36A to 36D, the edge surface 34A of the movable mold 34, the tie bars 35A to 35D, and the like, which are a part of the fixed mold 31 or the movable mold (32, 34) or a part of the surrounding members and extend in a direction coinciding with the opening / closing direction (extracting direction) of the fixed mold 31 and the movable mold (32, 34), are determined. The determination can be performed based on the image captured in the screen 73 of the image display device 72. As described above, in the image data captured by the three-dimensional imaging device and stored in the data storage section 92, the depth data and the coordinate data of the subject are also included, and thus the coordinate or depth determination section 98 determines the depth data or the coordinate data of the determined three or more points of the extension portions. Then, the normal vector operation section 99 estimates the normal vector of the mold by operation based on the depth data or the coordinate data determined by the coordinate or depth determination section 98.​​

[0093] exist Figure 17 In the example, based on the image, depth or coordinate data of the four guide pins 36A to 36D or tie rods 35A to 35D is obtained. Based on this data, an imaginary surface PS3 or PS4 orthogonal to the four guide pins 36A to 36D or tie rods 35A to 35D is calculated. The guide pins 36A to 36D or tie rods 35A to 35D are extensions that do not change during mold installation. The mating surface of the mold installed using these guide pins is substantially parallel to the three-dimensional imaginary surface PS3 or PS4 calculated based on the coordinate or depth data of the guide pins 36A to 36D or tie rods 35A to 35D. Therefore, the normal vector of the mold can be calculated using the aforementioned formula.

[0094] exist Figure 18 In the example, Figure 1 A 3D imaging device is installed at the position shown for imaging device C5, allowing it to enter the mold and acquire 3D image data of four intersection points including the edge surface 34A of the movable mold 34 (the four edges surrounding the mating surface of the mold). Then, based on the coordinate or depth data of the four intersection points obtained with the image data, a 3D imaginary surface PS5 including these four intersection points is obtained. The mating surface of the mold is substantially parallel to the 3D imaginary surface PS5. The normal vector of the extraction direction can then be obtained from this data. In this example, since the normal vector is obtained based on information related to the part of the mold's mating surface, the normal vector that may change depending on the mold's mounting state can be accurately estimated.

[0095] (Usability, Teachability)

[0096] The method of measuring the maximum size and utilizing the normal vector as described above is arbitrary. In the above embodiment, before teaching is performed using the teaching unit 91, the usability determination unit 93 determines whether the use of the extraction head 60 in the extraction operation is suitable based on the three-dimensional shape and size information of the extraction head 60. If it is unusable, an alarm is output from the usability determination unit 93. That is, it can determine whether the extraction head 60 can enter the molded mold without colliding with the mold based on the maximum size. In addition, it can determine whether the molded product can be reliably extracted by the extraction head based on the degree of inconsistency between the normal vector of the extraction head and the normal vector of the mold. In other words, when teaching is performed, it is determined in advance that the extraction head 60 may collide with surrounding objects or that the molded product cannot be reliably extracted during its movement. If the extraction head 60 is not usable, an alarm is issued.

[0097] If it is a problem of the mounting posture of the extraction head 60, as long as based on the alarm, reinstallation of the extraction head 60 is performed, or by providing an instruction to the posture control device driving section 59D to cause the inversion unit 59C as a posture control device to act, thereby changing the posture of the extraction head 60 to an appropriate posture.

[0098] - Industrial applicability -

[0099] According to the present application, by the imaging device, using the image data including depth data or coordinate data obtained from the real-time mounted mold and the fitting, the normal vector of the fitting and the normal vector of the mold are estimated, thus the normal vector having a close relationship with the real-time mounting state of the mold and the fitting can be estimated. Therefore, the automation of teaching, the posture control of the fitting, etc. can be easily realized.

Claims

1. A normal vector estimation method of a normal vector of a fitting member in a state where the fitting member is attached to an approach frame of a molded product taking-out machine, and a normal vector of a mold in a state where the mold is attached to a molding machine, the normal vector estimation method being characterized by defining a vector perpendicular to a joint surface of each of a fixed mold and a movable mold of the mold in the state where the mold is attached to the molding machine as the normal vector of the mold, and defining a vector extending along a direction in which the approach frame extends and being perpendicular to an imaginary surface parallel to the joint surface when the fitting member is inserted between the fixed mold and the movable mold as the normal vector of the fitting member, capturing an image including a surrounding member around the joint surface of the fixed mold or the joint surface of the movable mold using a first photographing device capable of capturing an image including depth data or coordinate data of an object, determining an extension portion of at least three points or more which is a part of the fixed mold or the movable mold or a part of the surrounding member and extends in a direction coinciding with an opening and closing direction of the fixed mold and the movable mold based on the image, and estimating the normal vector of the mold based on depth data or coordinate data of the three points or more of the extension portion determined. capturing an image including a fitting tool of three points or more in which the fitting member is attached to the approach frame or a contact tool of three points or more in contact with a molded product using a second photographing device capable of capturing an image including depth data or coordinate data of an object, the fitting tool or the contact tool extending in a direction orthogonal to the approach frame, and estimating the normal vector of the fitting member based on the depth data or the coordinate data of the fitting tool of three points or more or the contact tool of three points or more obtained based on the image.

2. The normal vector estimation method according to claim 1, wherein the extension portion is a guide pin of the mold, an edge surface of the fixed mold or the movable mold, or a tie rod.

3. The normal vector estimation method according to claim 1, wherein the fitting tool is a fitting bolt or a fitting metal piece.

4. The normal vector estimation method according to claim 1, wherein the contact tool is a suction pad.

5. The normal vector estimation method according to claim 1, wherein the photographing device is a three-dimensional photographing device.

6. A fitting member posture determination method, characterized by determining a posture of the fitting member when the fitting member is inserted into the mold using the normal vector of the mold estimated by the normal vector estimation method of claim 1 and the normal vector of the fitting member so that a taking-out operation can be performed in a posture in which the normal vector of the mold and the normal vector of the fitting member coincide.

7. A normal vector estimation system of a normal vector of a fitting member in a state where the fitting member is attached to an approach frame of a molded product taking-out machine, and a normal vector of a mold in a state where the mold is attached to a molding machine, the normal vector estimation system being characterized by including a processor, A vector perpendicular to the joint surface of the fixed mold and the movable mold of the mold in a state where the mold is installed in the molding machine is defined as a normal vector of the mold, and a vector extending in a direction along which the entering frame extends and perpendicular to an imaginary surface parallel to the joint surface when the fitting is inserted between the fixed mold and the movable mold is defined as a normal vector of the fitting, The processor is configured to: obtain depth data or coordinate data of an extension portion of at least three points or more, the extension portion being determined from an image including surrounding components around the joint surface of the fixed mold or the joint surface of the movable mold, the surrounding components being determined from an image including depth data or coordinate data of the subject, the image being captured by a first imaging device capable of capturing an image including depth data or coordinate data of the subject, the extension portion being a part of the fixed mold or the movable mold or a part of the surrounding components and extending in a direction consistent with the opening and closing direction of the fixed mold and the movable mold, obtain depth data or coordinate data of a mounting tool of three points or a contact tool of three points, the mounting tool of three points or the contact tool of three points being determined from an image including the mounting tool of three points or the contact tool of three points of the fitting mounted to the entering frame, the mounting tool or the contact tool extending in a direction orthogonal to the entering frame, and the image being captured by a second imaging device capable of capturing an image including depth data or coordinate data of the subject.

Images