Control system, control method, machine learning method and already-learnt machine learning model

Abstract

To precisely make a model of an object to be controlled.SOLUTION: A control system includes: a guide mechanism that guides a payload to a predetermined position while the guide mechanism is bent and is advancing; an attitude actuator that changes the attitude of the guide mechanism; advancing and retracting actuator that causes the guide mechanism to move advance or to retract; and a control unit that controls the attitude actuator and the advancing and retracting actuator. The control unit controls each operation of each actuator using an already-learnt machine learning model which inputs the time-series of each 3D coordinate of each of multiple points on the guide mechanism, and outputs each time-series control information for each actuator, the machine learning model having undergone a machine learning with teacher data that is a combination of each time-series control information for each actuator obtained by operating each actuator so as to cause the guide mechanism to be bent and be advancing along a predetermined aiming route, and each time-series 3D coordinate of each of the multiple points on the guide mechanism.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a control system, a control method, a machine learning method, and a trained machine learning model. [Background technology]

[0002] Patent Document 1 describes a drive control device for an inspection tube, which includes a flexible tube (a control object) through which an inspection cable with a sensor at its tip can be inserted, a posture actuator capable of adjusting the posture of the tube, and an advance / retract actuator for moving the tube forward and backward. This inspection tube drive control device includes an inverse analysis unit that performs an analysis to move the tube along a predetermined route from a starting point to a destination point within the inspection object, and a forward analysis unit that acquires, based on the analysis results of the inverse analysis unit, the manipulated variables of the posture actuator and the advance / retract actuator when the tube is positioned at each location on the route. The inverse analysis unit also performs a simulation to move the tip of the tube from the starting point to the destination point while virtually bending multiple tube bodies on a three-dimensional analytical model so that the tube is forcibly moved while being constrained along the route. The forward analysis unit also acquires manipulated variables, which are the manipulated variables of the posture actuator and the advance / retract actuator as the tube moves in the simulation performed by the inverse analysis unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 090959 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the wire-driven tube structure exemplified in Patent Document 1 has the problem that, for example, the frictional force generated between the wire and the tube has different hysteresis in the direction of pulling and loosening the wire, making it difficult to model the controlled object.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a control system, a control method, a machine learning method, and a trained machine learning model that can accurately model a controlled object. [Means for solving the problem]

[0006] In order to solve the above problem, the control system of the present disclosure includes a guide mechanism that guides a predetermined payload while bending and advancing to a predetermined position, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that moves the guide mechanism forward and backward, and a control unit that controls the operation of the attitude actuator and the advance / retract actuator, wherein the control unit inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model that has been machine-learned using as training data a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of the three-dimensional coordinates of each of the plurality of points of the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path.

[0007] A control method according to the present disclosure is a control method for a control system including a guide mechanism that guides a predetermined payload while bending and advancing to a predetermined position, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that moves the guide mechanism forward and backward, and a control unit that controls the operation of the attitude actuator and the operation of the advance / retract actuator, wherein a machine learning model is input that inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and the operation of the attitude actuator and the advance / retract actuator is controlled using a trained machine learning model that has been machine-learned using as training data a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of the three-dimensional coordinates of each of the plurality of points of the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path.

[0008] The machine learning method according to the present disclosure is a machine learning method for a machine learning model used by a control unit in a control system including a guide mechanism that guides a predetermined payload while bending and advancing to a predetermined position, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that advances and retreats the guide mechanism, and a control unit that controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model, wherein the trained machine learning model inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and the machine learning model is trained using training data that is a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of the three-dimensional coordinates of each of the plurality of points in the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path.

[0009] The trained machine learning model according to the present disclosure is a trained machine learning model used by a control unit in a control system including a guide mechanism that guides a predetermined payload while bending and advancing to a predetermined position, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that moves the guide mechanism forward and backward, and a control unit that controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model, wherein the machine learning model inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and is trained by machine learning using as training data a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of the three-dimensional coordinates of each of the plurality of points in the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path. [Effects of the Invention]

[0010] According to the control system, control method, machine learning method, and trained machine learning model disclosed herein, it is possible to accurately model a controlled object. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example configuration of a control system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example configuration of a machine learning model according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a diagram illustrating an example of input data and output data of a machine learning model according to an embodiment of the present disclosure. [Figure 4] 1 is a schematic diagram illustrating a general configuration of a gas turbine having a turbine as an object to be inspected according to an embodiment of the present disclosure. [Figure 5] FIG. 1 is a schematic diagram illustrating a general configuration of a combustor and its surroundings according to an embodiment of the present disclosure. [Figure 6] 1A and 1B are schematic diagrams illustrating a general configuration for explaining a tube and an attitude actuator according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is an enlarged view of a main portion of a tube showing a tube body and a wire according to an embodiment of the present disclosure. [Figure 8] 10 is a cross-sectional view (VV cross-sectional view) of a main portion of a tube body showing a wire attachment position in the tube body at the distal end according to an embodiment of the present disclosure. FIG. [Figure 9] 6 is a cross-sectional view (VI-VI cross-sectional view) of a main portion of a tube body showing a wire attachment position in another tube body adjacent to the distal end tube body according to an embodiment of the present disclosure. FIG. [Figure 10] FIG. 2 is a cross-sectional view of a main part showing the structure of an attitude actuator according to an embodiment of the present disclosure. [Figure 11] FIG. 2 is a schematic diagram illustrating a route according to an embodiment of the present disclosure. [Figure 12] 1 is a flowchart illustrating an example of a machine learning method for a machine learning model according to an embodiment of the present disclosure. [Figure 13] FIG. 10 is a schematic diagram for explaining an example of generating teacher data according to an embodiment of the present disclosure. [Figure 14] FIG. 10 is a schematic diagram for explaining an example of generating teacher data according to an embodiment of the present disclosure. [Figure 15] FIG. 10 is a schematic diagram for explaining an example of generating teacher data according to an embodiment of the present disclosure. [Figure 16] FIG. 10 is a schematic diagram for explaining an example of generating teacher data according to an embodiment of the present disclosure. [Figure 17] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] A control system, a control method, a machine learning method, and a trained machine learning model according to embodiments of the present disclosure will be described below with reference to Figures 1 to 17. Note that the same or corresponding components in each figure are designated by the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0013] Fig. 1 is a diagram illustrating an example configuration of a control system according to an embodiment of the present disclosure. The control system 100 illustrated in Fig. 1 includes a guide mechanism 101, a payload 102, a follower 103, a guide tube 104, an attitude actuator 105, a forward / backward movement actuator 106, a platform 107, a monitor 108, and a control unit 109. Note that in Fig. 1, the up-down direction as viewed in the drawing is designated as the Y direction, the left-right direction as the Z direction, and the depth direction as the X direction.

[0014] The guide mechanism 101 is a joint mechanism for guiding a payload 102, such as an industrial endoscope cable, to an object OBJ, such as an inspection target in a narrow area inside a plant, while bending and moving forward (passing through). That is, the guide mechanism 101 is a guide that guides a predetermined payload 102 to a predetermined position while bending and moving forward. The guide mechanism 101 is, for example, a wire-driven articulated guide mechanism having a tubular shape.

[0015] The payload 102 is an internal component of the guide mechanism 101, and may be, for example, an industrial endoscope cable or a heat-resistant sheath for protecting the industrial endoscope in a high-temperature environment. Alternatively, the payload 102 may be a work tool, a tool for collecting foreign matter, or the like.

[0016] The following section 103 is a connecting section that connects the guide mechanism 101 and the attitude actuator 105. In order to insert the guide mechanism 101 into a narrow section at the back of the plant, the guide mechanism 101 and the following section 103 are guided by a guide pipe 104. The following section 103 also supports the reaction force of the guide mechanism 101.

[0017] The guide tube 104 is a tube that guides the guide mechanism 101 to the vicinity of the object OBJ. For example, if the inside of a turbine is to be inspected, the combustor is located between the inspection hatch and the inspection target. The combustor has a linear body with an outlet end that is bent several tens of degrees at the end, connecting the outlet end to the front of the turbine. Because the inside of the combustor is a long, linear cavity, if the guide mechanism 101 is placed inside as is, it may sag due to its own weight and not be able to reach the outlet end in a straight line. To address this issue, a pipe-shaped guide tube 104 is inserted and installed from the entrance of the inspection hatch, and the guide mechanism 101 is guided within it. If the space behind the inspection hatch is narrow, the guide tube 104 can be made into a detachable, split structure, and the tip of the guide tube 104 can be inserted to the outlet end of the combustor while connected.

[0018] The attitude actuator 105 is a drive mechanism for bending the guide mechanism 101. In other words, the attitude actuator 105 is an actuator that changes the attitude of the guide mechanism 101. When the guide mechanism 101 is driven by a wire, the attitude actuator 105 also includes, for example, an actuator that tendon-drives the wire, a sensor that detects the displacement and rotation angle of the actuator, and a load cell for monitoring the wire tension. In combination with the advance / retract actuator 106, the guide mechanism 101 is bent in synchronization with the propulsion of the advance / retract actuator 106 so that the guide mechanism 101 and the payload 102 can enter the interior while following a pre-planned path R.

[0019] The advance / retract actuator 106 and the base 107 are a propulsion mechanism that mounts the attitude actuator 105 on the advance / retract actuator 106, and moves the advance / retract actuator 106 forward and backward on the base 107 in the direction of arrow A1, thereby sending or withdrawing the attitude actuator 105, the following section 103 connected to it, and the guide mechanism 101 into or out of the plant. It is composed of, for example, a linear actuator. The advance / retract actuator 106 is an actuator that moves the guide mechanism 101 forward and backward.

[0020] The monitor 108 displays the output signal of a sensor, such as a camera, attached to the tip of the payload 102 .

[0021] The control unit 109 controls the operation of the attitude actuator 105 and the movement of the forward / backward actuator 106. Hereinafter, the attitude actuator 105 and the movement of the forward / backward actuator 106 are collectively referred to as actuators. The control unit 109 controls the position, speed, thrust, etc. of the actuators as a lower-level control system, and has a trained machine learning model (inverse kinematics model) 110 as a higher-level control system. The trained machine learning model 110 generates a target path R within the narrow passage and derives actuator commands for the guide mechanism 101 to follow the target path R. Note that although the target path R is illustrated in FIG. 1 as being distant from the guide mechanism 101, the target path R and the central axis of the guide mechanism 101 substantially coincide with each other. The guide mechanism 101 bends and moves from a state in which point PS1 on the guide mechanism 101 is located at a start point S1 corresponding to the tip of the guide tube 104, while points PS1, PS2, ... change their positions up, down, left, and right along the target path R, until point PS1 is located at a target point E1. Furthermore, the guide mechanism 101 bends and retreats in the opposite direction along the target path R, for example, after inspection of the object OBJ is completed.

[0022] The trained machine learning model 110 is a machine learning model that receives input of a time series of three-dimensional coordinates of multiple points (PS1, PS2, PS3, PS4, ...) in the guide mechanism 101 and outputs a time series of control information (q1, q2, q3, q4, ...) for the attitude actuator 105 and a time series of control information (p) for the advance / retract actuator 106. The trained machine learning model 110 is trained using training data that is a combination of the time series of control information (q1, q2, q3, q4, ...) for the attitude actuators, the time series of control information (p) for the advance / retract actuator 106, and the time series of three-dimensional coordinates of multiple points (PS1, PS2, PS3, PS4, ...) in the guide mechanism 101, which are acquired by operating the attitude actuator 105 and the advance / retract actuator 106 so that the guide mechanism 101 bends and advances along a predetermined target path R. A detailed example of the machine learning of the trained machine learning model 110 will be described later. The control information is a control input (control command) to the actuator, such as the amount of wire pulling (q1, q2, q3, q4, . . . ), the amount of advance and retreat of the propulsion part (p), and the like.

[0023] The control unit 109 controls the operation of the attitude actuator 105 and the operation of the advance / retract actuator 106 using the trained machine learning model 110. The control unit 109 creates a time series of three-dimensional coordinates of each point PS1, PS2, ... when the guide mechanism 101 is bent and advanced along a target path R (hereinafter also referred to as route R) using, for example, a three-dimensional design model of the inspection target, inputs the created time series data of the three-dimensional coordinates to the trained machine learning model 110, and acquires a time series of control information for the attitude actuator 105 and a time series of control information for the advance / retract actuator 106 as outputs of the trained machine learning model 110. The control unit 109 then controls the operation of the attitude actuator 105 and the operation of the advance / retract actuator 106 based on the acquired time series of control information for the attitude actuator 105 and the time series of control information for the advance / retract actuator 106. Note that some of these processes may be performed by a computer other than the computer constituting the control unit 109.

[0024] When the guide mechanism 101 has a structure that changes its posture by stretching multiple wires, it is desirable that the trained machine learning model 110 be configured, for example, as a neural network including a self-loop in which the output of an intermediate layer is input again to the intermediate layer. In this case, the trained machine learning model 110 is configured as a neural network that can handle time series, and can be a model that takes into account the influence of mechanical hysteresis, for example. Note that neural networks that can handle time series include recurrent neural networks (RNNs), which are neural networks with a recursive structure in the intermediate layer. RNNs also include configurations in which the intermediate layer includes a long short-term memory (LSTM) or a gated recurrent unit (GRU).

[0025] In this embodiment, a time series is a series of values ​​that continuously (that is, at predetermined intervals) represents a change over time, and time series data and time series are synonymous.

[0026] FIG. 2 is a diagram schematically illustrating an example configuration of a machine learning model (trained machine learning model 110) according to an embodiment of the present disclosure. The horizontal axis represents the time axis, and illustrates the correspondence between multidimensional variables X1, X2, X3, ..., Xn input to the input layer 201, the flow of data in the hidden layer 202, and multidimensional variables Y1, Y2, Y3, ..., Yn output from the output layer 203. In this embodiment, the variables X1, X2, X3, ..., Xn represent the time series of the three-dimensional coordinates of multiple points (PS1, PS2, PS3, PS4, ...) of the guide mechanism 101. The variables Y1, Y2, Y3, ..., Yn represent the time series of control information (q1, q2, q3, q4, ...) for the attitude actuator 105 and the time series of control information (p) for the advance / retreat actuator 106. In addition, in the intermediate layer 202, the input variables of the current step and the intermediate layer parameters of the immediately preceding step (weight matrix with time-series parameters and bias values) are input, and the output variables of the current step and the intermediate layer parameters of the current step are output.

[0027] FIG. 3 is a diagram illustrating an example of input data and output data of a machine learning model (trained machine learning model 110) according to an embodiment of the present disclosure. The input data to the trained machine learning model 110 is a time series (values ​​at times t1, t2, t3, t4, ...) of each three-dimensional coordinate (x, y, z) of each point (PS1, PS2, PS3, PS4, ...) of the guide mechanism 101. Note that each point (PS1, PS2, PS3, PS4, ...) is a predetermined point such as the tip or middle of a joint (connection portion between tube bodies) described below. The output data from the trained machine learning model 110 includes a time series (values ​​at times t1, t2, t3, t4, ...) of control information (or control input) values ​​q1, q2, q3, ... for the attitude actuator 105 and a time series (values ​​at times t1, t2, t3, t4, ...) of the control information (or control input) values ​​p for the advance / retreat actuator 106.

[0028] Next, with reference to Figures 4 to 16, an example will be described in which the object to be inspected by the control system 100 according to the present embodiment described with reference to Figure 1 etc. is a turbine of a power-generating GT (gas turbine) in an industrial plant. Figure 4 is a schematic diagram showing a general configuration of a gas turbine having a turbine that is an object to be inspected according to the embodiment of the present disclosure.

[0029] 1 correspond to the following respective components shown in FIGS. 4 to 16. The control system 100 shown in FIG. 1 corresponds to the inspection device 5. The guide mechanism 101 shown in FIG. 1 corresponds to the tube 62 (active unit 62A). The payload 102 shown in FIG. 1 corresponds to the inspection cable 61. The following unit 103 shown in FIG. 1 corresponds to the tube 62 (follower unit 62B). The guide tube 104 shown in FIG. 1 corresponds to the guiding jig 7. The attitude actuator 105 shown in FIG. 1 corresponds to the attitude actuator 65. The advance / retract actuator 106 shown in FIG. 1 corresponds to the advance / retract actuator 67 (advance / retract drive unit 671). The mount 107 shown in FIG. 1 corresponds to the guide rail 672. The monitor 108 shown in FIG. 1 corresponds to the camera image monitor 91. The control unit 109 shown in FIG. 1 corresponds to the drive control device 8.

[0030] As shown in FIG. 4, the gas turbine 1 includes a compressor 2 that generates high-pressure air, a turbine 3 that is driven by combustion gas, and a plurality of combustors 4 that mix fuel with the high-pressure air and burn the fuel to generate combustion gas and supply the combustion gas to the turbine 3.

[0031] The turbine 3 has a turbine rotor 31 that rotates about the axis O1, and a turbine casing 35 that covers the turbine rotor 31 from the outer periphery. The turbine rotor 31 has a columnar shape that extends along the axis O1. A plurality of turbine rotor blade stages 32 are provided on the outer periphery of the turbine rotor 31 and are arranged at intervals in the direction of the axis O1, which is the direction in which the axis O1 extends. Each turbine rotor blade stage 32 has a plurality of rotor blades 33 that are arranged on the outer periphery of the turbine rotor 31 and are arranged at intervals in the circumferential direction centered on the axis O1.

[0032] The turbine casing 35 has a cylindrical shape centered on the axis O1. A plurality of turbine stator vane stages 36 are provided on the inner peripheral surface of the turbine casing 35, arranged at intervals in the direction of the axis O1. These turbine stator vane stages 36 are provided upstream of each turbine rotor blade stage 32 in a one-to-one correspondence with the turbine rotor blade stages 32. As a result, the turbine stator vane stages 36 and the turbine rotor blade stages 32 are arranged alternately in the direction of the axis O1. Each turbine stator vane stage 36 has a plurality of stator vanes 37 arranged side by side at intervals in the circumferential direction on the inner peripheral surface of the turbine casing 35. In this embodiment, the rotor blades 33 and stator vanes 37 arranged most upstream inside the turbine casing 35 are referred to as first-stage rotor blades 331 and first-stage stator vanes 371.

[0033] The combustor 4 is provided at a connection portion between the compressor 2 and the turbine casing 35. A plurality of combustors 4 are provided at intervals in the circumferential direction around the axis O1. The combustor 4 of this embodiment has a hollow cylindrical combustion liner 41.

[0034] Inside the combustion liner 41, a mixture of high-pressure air compressed by the compressor 2 and fuel gas is combusted to generate combustion gas. The combustion liner 41 is a cylindrical member. In a portion including the upstream end of the combustion liner 41, its imaginary central axis O4 extends at an angle so as to approach the axis O1 as it moves from the compressor 2 toward the turbine 3. The downstream end (outlet) of the combustion liner 41 is connected to the inlet of the turbine 3. In the combustion liner 41, the portion including the downstream end is bent so as to follow the direction of the axis O1 relative to the portion including the upstream end.

[0035] The inspection device 5 of this embodiment is a non-destructive inspection device that visually checks the inside of the turbine 3 without opening the turbine casing 35. The inspection device 5 uses an inspection cable 61 having a sensor 612 attached to its tip, as shown in Fig. 5. The inside of the turbine 3 is the space inside the turbine casing 35 through which combustion gas flows, and is the space in which the rotor blades 33 and the stator blades 37 are arranged.

[0036] The inspection device 5 is a device that can check the inside of the turbine 3 from the outside. The inspection device 5 is fixed to the combustor 4, and thereby makes it possible to visually check narrow sections and bent sections inside the turbine 3 that are difficult to see through the inside of the combustor 4. The inspection device 5 of this embodiment includes an inspection cable 61, an inspection tube 6, a guide jig 7, a drive control device 8, and a camera image monitor 91.

[0037] The inspection cable 61 has a highly flexible cable main body 611 and a sensor 612 that is provided at the tip of the cable main body 611 and can inspect the inside of the turbine 3. The cable main body 611 can be bent in any direction that intersects with the cable extension direction, which is the direction in which the cable main body 611 extends, by an operator operating an operation unit (not shown). The cable main body 611 is a separate member from the tube 62 and is detachably fixed to the tube 62. An actuator (not shown) for moving the cable is provided in the cable main body 611 so that the cable main body 611 can be driven independently of the inspection tube 62.

[0038] The sensor 612 is fixed to the tip of the cable main body 611. The sensor 612 and the cable main body 611 are built into the tube 62. The sensor 612 in this embodiment is a camera capable of capturing images of the inside of the turbine 3. Captured data such as videos and images captured by the sensor 612 is sent to the camera image monitor 91 via a cable extending from the end (rear end) of the cable main body 611 on which the sensor 612 is not provided. As the inspection cable 61 in this embodiment, for example, a borescope (industrial endoscope) is used for observing and inspecting deep areas that cannot be directly seen with the naked eye.

[0039] The camera image monitor 91 displays the image captured by the sensor 612. Image information captured by the sensor 612 is input to the camera image monitor 91 through the inspection cable 61. The image information input to the camera image monitor 91 is sent to the drive control device 8.

[0040] The inspection cable 61 may be any cable that has a bendable structure, and may be, for example, a snake-like robot having a multi-joint structure in which a plurality of highly flexible members are connected.

[0041] Furthermore, the sensor 612 is not limited to being a camera as in this embodiment. For example, the sensor 612 in this embodiment may be a sensor 612 having a dimension measurement function (for example, three-dimensional phase measurement) or a sensor 612 capable of measuring temperature or the presence or absence of scratches.

[0042] The inspection tube 6 includes a tube 62 , a posture actuator 65 , and a forward / backward movement actuator 67 .

[0043] As shown in FIG. 6 , the tube 62 has a hollow portion formed therein through which the inspection cable 61 can be inserted. The tube 62 is flexible. The tube 62 has a multi-joint structure that allows it to be bent at multiple locations. Therefore, the tube 62 can be bent in any direction intersecting the tube extension direction, which is the direction in which the tube 62 extends. Note that each joint portion of the tube 62 preferably has a structure that is easy to bend, but is difficult to twist and compress. The outer diameter of the tube 62 is large enough to allow it to be inserted into the combustor 4 and into narrow spaces between the rotor and stator blades of the turbine 3. A cable main body 611 is detachably attached to the tube 62. The tube 62 of this embodiment is configured by connecting multiple tube main bodies 63.

[0044] The multiple tube bodies 63 are arranged side by side in the extending direction of the tube bodies 63 and are connected to each other. The tube bodies 63 are deformable from an initial state in which they extend linearly along their central axis to a deformed state in which they are bent. As shown in FIG. 7 , the tube bodies 63 have a tubular portion 631 with both ends open and flange portions 632 protruding radially outward from the outer peripheral surfaces of both ends of the tubular portion 631. The tubular portion 631 has a cylindrical shape through which the inspection cable 61 can be inserted. The tubular portion 631 has, for example, multiple slits (not shown) formed therein, allowing it to be bent in any direction. The flange portions 632 are annular and integrally formed with the tubular portion 631.

[0045] As shown in Fig. 6, the attitude actuator 65 is capable of adjusting the attitude of the tube 62. Here, the attitude of the tube 62 refers to the position and direction of the tip of the tube 62 on an imaginary plane intersecting the tube extension direction. The attitude actuator 65 of this embodiment is fixed to the rear end of the tube 62. As shown in Fig. 10, the attitude actuator 65 has a plurality of wires 651, a housing unit 652, a pulley 653, a wire driver 654, and a wire load detector 655.

[0046] As shown in FIG. 7 , a plurality of wires 651 (for example, three in this embodiment) are provided for one tube body 63. The tips of the wires 651 are fixed to a flange portion 632 located on the tip side of the tube body 63. As shown in FIG. 8 , the wires 651 are fixed to one flange portion 632 so as to be spaced apart from one another so as to be out of phase with each other (for example, 120 degrees). Furthermore, the wires 651 are arranged with a phase shift for each adjacent tube body 63. Therefore, as shown in FIG. 9 , the fixing position of the wire 651 in one tube body 63 located on the tip side is shifted by, for example, 60 degrees from that of another tube body 63 adjacent to the rear end. Therefore, a wire insertion hole 633 is formed in the flange portion 632 of the rear end tube body 63 to insert the wire 651 fixed to a tube body 63 located further tip than the tube body 63. Therefore, the tube body 63 located closest to the rear end has more wire insertion holes 633 formed therein.

[0047] 10, the housing 652 is fixed to the rear end of the tube 62. The housing 652 houses one end of the wire 651 inside. The housing 652 has a housing through-hole 652A formed therein, through which the cable main body 611 protruding from the rear end of the tube 62 can be inserted. The housing through-hole 652A is formed to penetrate the housing 652.

[0048] Pulley 653 is rotatably attached inside housing 652. Pulley 653 reverses the direction in which wire 651 extends inside housing 652. A pulley 653 is provided for each wire 651. In other words, one pulley 653 is provided for each wire 651. A plurality of pulleys 653 are provided spaced apart from each other to surround housing through-hole 652A.

[0049] The wire driving unit 654 is fixed inside the housing 652. A wire driving unit 654 is provided for each wire 651. That is, one wire driving unit 654 is provided for each wire 651. The wire driving unit 654 is connected to the rear end of the wire 651, which is the end of the wire 651 that is not fixed to the tube main body 63, via a wire load detection unit 655. The wire driving unit 654 is capable of moving the wire 651 forward and backward relative to the pulley 653. As the wire driving unit 654, for example, an electric slider, an electric cylinder, or a ball screw is used.

[0050] The wire load detection unit 655 is disposed between the rear end of the wire 651 and the wire driving unit 654. The wire load detection unit 655 measures the load (wire tensile force) acting on the wire 651 and sends the measurement result to the wire driving unit 654. If the sent measurement result is equal to or greater than a value determined to be excessive (e.g., a value that may damage the wire 651), the wire driving unit 654 is driven to loosen the wire 651. If the sent measurement result is equal to or less than a value determined to be insufficient (e.g., a value at which the wire 651 is deemed to be bent), the wire driving unit 654 is driven to tension the wire 651 to a degree that does not loosen it. The wire load detection unit 655 may be, for example, a load cell that can directly measure the load. Alternatively, the load may be measured indirectly based on the motor current value in the wire driving unit 654.

[0051] Furthermore, the attitude actuator 65 drives some of the multiple tube bodies 63 that are located near the tip. The number of tube bodies 63 driven by the attitude actuator 65 may be one or more. As shown in FIG. 6 , the tube 62 of this embodiment is divided into an active part 62A that is driven by the attitude actuator 65 and a driven part 62B that is not driven (deformed or moved) by the attitude actuator 65.

[0052] In the active portion 62A, a wire 651 is fixed to the flange portion 632 of each tube body 63. The active portion 62A is a region of a predetermined length from the tip of the tube 62. Here, the predetermined length is a length that can reach a desired inspection range.

[0053] The driven part 62B moves following the movement of the active part 62A. In the driven part 62B, the wire 651 is not fixed to the flange part 632 of each tube main body 63. The driven part 62B is the area from the rear end of the tube 62 to the active part 62A. In this embodiment, the driven part 62B is the area sandwiched between the housing part 652 and the active part 62A.

[0054] 5, the advance / retract actuator 67 is capable of advancing and retracting the tube 62. Here, advancing and retracting the tube 62 means moving the tube 62 in the tube extension direction. The advance / retract actuator 67 of this embodiment is capable of moving the housing part 652 to which the tube 62 is fixed. The advance / retract actuator 67 has a guide rail 672 and an advance / retract drive part 671.

[0055] The guide rail 672 can be fixed to the upstream end of the combustor 4 via a guide jig 7. In this embodiment, the guide rail 672, when fixed to the combustor 4, extends parallel to an imaginary central axis O4 of a portion including the upstream end of the combustion liner 41.

[0056] The advancing / retracting drive unit 671 moves on a guide rail 672. A housing unit 652 is fixed to the advancing / retracting drive unit 671. The advancing / retracting drive unit 671 is, for example, an electric slider. When the advancing / retracting drive unit 671 moves on the guide rail 672 so as to approach the connection position with the combustor 4, the tube 62 is inserted deep inside (downstream side) the turbine 3. Conversely, as shown in FIG. 5 , when the advancing / retracting drive unit 671 moves on the guide rail 672 so as to move away from the connection position with the combustor 4, the tube 62 is moved from deep inside the turbine 3 to near the inlet of the turbine 3 (upstream side).

[0057] The guide jig 7 guides the tube 62 from the outside of the gas turbine 1 to the inside of the turbine 3. The guide jig 7 of the present embodiment is inserted into the combustion liner 41 from the upstream side of the combustor 4, thereby guiding the tip of the tube 62 from the outside of the combustor 4 to the outlet of the combustor 4 (near the upstream side of the first stage stator vane 371 inside the turbine 3).

[0058] As shown in FIG. 5 , the drive control device 8 is capable of sending signals to the attitude actuator 65 and the advance / retract actuator 67 to control the movement of the tube 62. At this time, the drive control device 8 controls the operation of the attitude actuator 65 and the advance / retract actuator 67 using a trained machine learning model 110, similar to the control unit 109 shown in FIG. 1 . The drive control device 8 first determines a route R based on three-dimensional shape data of the interior of the turbine 3 previously stored in a predetermined storage area. As shown in FIG. 11 , the route R is a path from a start point P1, where the tip of the tube 62 is initially positioned inside the turbine 3, to a target point P2 to be inspected (checked) in the inspection device 5 of this embodiment. The start point P1 is the point where the tip of the tube 62 is located before the start of inspection. In this embodiment, the start point P1 is the connection between the inlet of the turbine 3 and the outlet of the combustor 4, near the upstream end of the first-stage stator vane 371. The target point P2 is an arbitrary inspection position inside the turbine 3, such as the stator vane 37 or rotor blade 33 to be inspected.

[0059] Next, based on the information about the route R, the drive control device 8 uses the trained machine learning model 110 to acquire time series of control information for the attitude actuator 65 and the advance / retract actuator 67 for moving the tube 62 from the start point P1 to the target point P2 along the route R. Then, the drive control device 8 controls the operation of the attitude actuator 65 and the advance / retract actuator 67 based on the acquired time series of control information for the attitude actuator 65 and the advance / retract actuator 67.

[0060] In addition, when creating teacher data for machine learning of the trained machine learning model 110, the drive control device 8 is connected to an external computer such as a personal computer (not shown) for creating the teacher data, and controls the operation of the attitude actuator 65 and the movement actuator 67 in accordance with instructions from the external computer.

[0061] Next, a method for generating training data used for machine learning of the trained machine learning model 110 and a method for machine learning will be described with reference to Figs. 12 to 16. Fig. 12 is a flowchart showing an example of a machine learning method for a machine learning model (trained machine learning model 110) according to an embodiment of the present disclosure. Figs. 13 to 16 are schematic diagrams for explaining an example of generating training data according to an embodiment of the present disclosure.

[0062] The process shown in FIG. 12 is executed by using a computer such as a personal computer, using an application for the process shown in FIG. 12, etc. In this case, the tube 62 has nine joints, and the posture of each joint is controlled by three wires 651. Therefore, the control information for the posture actuator 65 includes information indicating the wire pull amounts qi (i = 1 to 27) of 9 × 3 = 27 wires 651. Furthermore, the control information for the forward / backward movement actuator 67 includes information indicating the thrust amount p. In the process shown in FIG. 12, training data is created using a mockup 3m of a turbine 3 as shown in FIG. 13. Note that the configurations in FIG. 13 with the letter "m" added to the end correspond to the configurations with the corresponding reference numbers in FIG. 11. In the example shown in FIG. 13, the tube 62 is not a mockup but an actual device.

[0063] The process shown in FIG. 12 is started in response to an instruction from an operator. In step S11, each joint (tube body 63) is bent and advanced by minute amounts along the target path R from the start point P1 to the target point P2 by inching (inching) the attitude actuator 65 and the forward / backward actuator 67, and the wire pull amount qi (i = 1 to 27) and the thrust amount p are acquired at multiple key poses while the tube 62 passes through the wing. Here, the key pose may be, for example, a posture in which the posture of the tube 62 matches (or nearly matches) the target path R, for example, by visual inspection, the gap between the wing is greater than a predetermined amount, and the tube 62 has been advanced a certain distance from the previous key pose. However, the present invention is not limited to this example, and the posture may be, for example, a posture in which the tube 62 has been advanced a certain distance from the previous key pose and the gap between the wing is greater than a predetermined amount. Also, the wire pulling amount qi (i=1 to 27) when the tip of the tube 62 is deviated up, down, left, or right from the target path R at a plurality of specific thrust amounts (for example, at each position indicated by the dashed circle SP in FIG. 13) while bending and advancing from the start point P1 to the target point P2 is also obtained. Note that inching (inching) means causing a moving part to perform a very slight movement such as a linear movement such as a parallel movement or a reciprocating movement, or a rotational movement by operating, for example, a switch, button, lever, etc.

[0064] 14 schematically shows an example of the relationship between the position indicated by a circle SP corresponding to target point P2, the position indicated by the previous circle SP, each key pose position, and each position shifted up, down, left, or right from the target path R at the position indicated by the circle SP, with the tip position of the tube 62 as the reference. At the position indicated by the previous circle SP, first, the wire pulling amount qi (i = 1 to 27) and the thrust amount p of the key pose are acquired at position P11. Next, the tip position of the tube 62 is shifted to each of positions P12, P13, P14, and P15 by inching the posture actuator 65, and the wire pulling amount qi (i = 1 to 27) is acquired at each position. Next, the tip position of the tube 62 is returned to position P11 manually or automatically, and then bent and advanced to positions P16, P17, P18, P19, and P20 (target point P2) by inching, and the wire pulling amount qi (i = 1 to 27) and thrust amount p of the key pose are obtained at each position. Next, the tip position of the tube 62 is shifted to each of positions P21, P22, P23, and P24 by inching the posture actuator 65, and the wire pulling amount qi (i = 1 to 27) is obtained at each position.

[0065] Next, in step S12, one or more values ​​(interpolated values ​​of qi and p) of the wire pull amount qi (i = 1 to 27) for each key pose and the thrust amount p between the key poses are calculated by linear interpolation, and time-related information is added to create a time series, generating a time series of the wire pull amount qi (i = 1 to 27) and the thrust amount p. Adding time-related information to each value means, for example, assigning time information that results in a predetermined time interval for each predetermined amount of change in the thrust amount p. In this case, it is easy to maintain the thrust amount at a constant value by controlling each actuator based on the time information added based on the time series generated in the processing of step S12.

[0066] In step S12, for example, the wire pulling amounts qi (i = 1 to 27) and the thrust amounts p at positions P31, P32, P33, P34, and P35, which are indicated by black diamonds in Fig. 15, are calculated by interpolation based on the wire pulling amounts qi (i = 1 to 27) and thrust amounts p acquired at positions P11, P16, P17, P18, P19, and P20, and time information is added to each value at positions P11, P16, P17, P18, P19, and P20, and positions P31, P32, P33, P34, and P35. In this case, a time series of the wire pulling amounts qi (i = 1 to 27) and the thrust amounts p corresponding to positions P11, P31, P16, P32, P17, P33, P18, P34, P19, P35, and P20 is generated.

[0067] Next, in step S13, another time series of the wire pulling amount qi (i = 1 to 27) and the thrust amount p is generated based on the generated time series of the wire pulling amount qi (i = 1 to 27) and the thrust amount p, and the wire pulling amount qi (i = 1 to 27) when the tip is shifted up, down, left, and right at specific multiple thrust amounts.

[0068] In step S13, for example, a time series of the wire pulling amount qi (i = 1 to 27) and the thrust amount p corresponding to positions P12, P41, P42, P43, P44, P45, P46, P47, P48, P49, and P21 shown in FIG. 16 is generated based on the time series of the wire pulling amount qi (i = 1 to 27) and the thrust amount p corresponding to positions P11, P31, P16, P32, P17, P33, P18, P34, P19, P35, and P20, the difference between the wire pulling amount qi (i = 1 to 27) at positions P12 and P11, the difference between the wire pulling amount qi (i = 1 to 27) at positions P20 and P21, etc.

[0069] Next, in step S14, the tube 62 (guide mechanism 101) is bent and advanced based on the generated time series of the wire-pulling amounts qi (i = 1 to 27) and the thrust amounts p (the time series generated in step S12) and another time series (the time series generated in step S13), and a time series of the three-dimensional coordinates of each of multiple points on the tube 62 (guide mechanism 101) is acquired. In step S14, for example, as shown in FIG. 14, the tube 62 is removed from a 3 m turbine mockup, and a 7 m guide jig mockup is used, and the attitude actuator 65 and the advance / retract actuator 67 are controlled based on the generated time series of the wire-pulling amounts qi (i = 1 to 27) and the thrust amounts p to bend and advance the tube 62. At this time, motion capture markers Mkt and Mkm are attached to the tube 62, for example, at the tip and middle of each of nine joints, and a video including each marker is captured using multiple motion capture cameras (not shown) to acquire a time series of the three-dimensional coordinates of each marker. In the example shown in FIG. 14, for example, the start point P1 (the coordinate system of the base of the joint) can be set as the reference coordinate of the three-dimensional coordinate system.

[0070] Next, in step S15, the posture actuator 65 and the forward / backward actuator 67 are operated to bend and advance the tube 62 (guide mechanism 101) along a predetermined target path R, and multiple sets of combinations of time series of the wire pulling amount qi (i = 1 to 27) and the propulsion amount p obtained by these are used as training data to machine-learn a machine learning model and generate a trained machine learning model 110.

[0071] As described above, according to this embodiment, a learning model is created by creating multiple teaching paths that are tilted forward, backward, left, and right relative to a reference path that follows the target path, and command values ​​are generated, thereby reducing the effort required to create teaching paths.

[0072] Furthermore, according to this embodiment, since it is possible to accommodate fine corrections and swinging of the route, the workload of teaching a plurality of target routes can be reduced.

[0073] Furthermore, in this embodiment, the target position can be taught by inching the actuator rather than by touching the arm to move it, making it easy to manually operate the wire pulling amount to approach the target position.

[0074] (Action and effect) In the control system, control method, machine learning method, and trained machine learning model configured as described above, the control system 100 (inspection device 5) includes a guide mechanism (guide mechanism 101, tube 62) that guides a predetermined payload (payload 102, inspection cable 61) while bending and advancing to a predetermined position (target point P2), attitude actuators (105, 65) that change the attitude of the guide mechanism, advance / retract actuators (106, 67) that move the guide mechanism forward and backward, and a control unit (control unit 109, drive control device 8) that controls the operation of the attitude actuator and the advance / retract actuator. The control unit then inputs a time series of three-dimensional coordinates of each of the multiple points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model 110 that has been machine-learned using as training data a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of three-dimensional coordinates of each of the multiple points in the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path R.

[0075] According to this configuration, the operation of the attitude actuator and the advance / retract actuator is controlled using a trained machine learning model 110 that has been machine-learned using training data that is a combination of a time series of control information for the attitude actuator, a time series of control information for the advance / retract actuator, and a time series of three-dimensional coordinates of each of multiple points on the guide mechanism, which are acquired by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path R. Therefore, the control target (guide mechanism 101, tube 62) can be modeled with high accuracy.

[0076] (Other embodiments) The above describes in detail the embodiments of the present disclosure with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present disclosure.

[0077] <Computer Configuration> FIG. 17 is a schematic block diagram showing the configuration of a computer according to at least this embodiment. The computer 900 includes a processor 901 , a main memory 902 , a storage 903 , and an interface 904 . The above-described control unit 109 and drive control device 8 are implemented in a computer 900. The operations of the above-described processing units are stored in the form of a program in a storage 903. The processor 901 reads the program from the storage 903, loads it into a main memory 902, and executes the above-described processing in accordance with the program. The processor 901 also allocates storage areas in the main memory 902 corresponding to the above-described storage units in accordance with the program.

[0078] The program may be for realizing some of the functions to be performed by the computer 900. For example, the program may be combined with other programs already stored in storage or other programs implemented in other devices to perform the functions. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0079] Examples of storage 903 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. Storage 903 may be an internal medium directly connected to the bus of computer 900, or an external medium connected to computer 900 via interface 904 or a communication line. Furthermore, when this program is distributed to computer 900 via a communication line, computer 900 that receives the program may load the program into main memory 902 and execute the above-described processing. In at least one embodiment, storage 903 is a non-transitory tangible storage medium.

[0080] <Additional Notes> The aspect of this embodiment can be understood, for example, as follows.

[0081] (1) A control system according to a first aspect includes a guide mechanism that guides a predetermined payload to a predetermined position while bending and advancing, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that advances and retreats the guide mechanism, and a controller that controls the operation of the attitude actuator and the advance / retract actuator, wherein the controller inputs a time series of three-dimensional coordinates of each of multiple points on the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model that uses combinations of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of three-dimensional coordinates of each of the multiple points on the guide mechanism obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path as training data. This aspect and the following aspects enable accurate modeling of a controlled object.

[0082] (2) A control system according to a second aspect is the control system of (1), in which the guide mechanism has a structure that changes its posture by stretching multiple wires, and the machine learning model is configured with a neural network including a self-loop that inputs the output of an intermediate layer back into the intermediate layer. According to this aspect, it is possible to accurately model the controlled object even when mechanical hysteresis occurs.

[0083] (3) A control system according to a third aspect is the control system according to (2), in which the intermediate layer includes a long short-term memory. This aspect can avoid the problems of gradient vanishing (or gradient explosion) and accuracy degradation due to repeated learning, and can model the controlled object with higher accuracy.

[0084] (4) A fourth aspect of the control system is the control system of any one of (1) to (3), wherein the teacher data is acquired by: a first step of acquiring discrete values ​​of control information for the attitude actuator and the advance / retraction actuator acquired when the attitude actuator and the advance / retraction actuator are inched so that the guide mechanism bends and advances along the target path; a second step of generating a time series of control information for the attitude actuator and the advance / retraction actuator by interpolating the discrete values ​​of control information for the attitude actuator and the discrete values ​​of control information for the advance / retraction actuator and adding time-related information; and a third step of controlling the attitude actuator and the advance / retraction actuator based on the time series of control information for the attitude actuator and the time series of control information for the advance / retraction actuator to acquire a time series of three-dimensional coordinates of each of the multiple points of the guide mechanism. This aspect allows efficient generation of teacher data.

[0085] (5) A control system according to a fifth aspect is the control system of (4), wherein in the first step, discrete values ​​of control information for the attitude actuator are further acquired when the attitude actuator is moved so as to deviate the guide mechanism from the target path at some of the multiple stop positions of the advance / retract actuator during the inching; in the second step, another time series of control information for the attitude actuator and another time series of control information for the advance / retract actuator are further generated based on the time series of control information for the attitude actuator and the time series of control information for the advance / retract actuator generated by bending and advancing the guide mechanism along the target path, and the discrete values ​​of control information for the attitude actuator acquired when the attitude actuator is moved so as to deviate the guide mechanism from the target path; and in the third step, the attitude actuator and the advance / retract actuator are controlled based on the other time series of control information for the attitude actuator and the other time series of control information for the advance / retract actuator to further acquire another time series of three-dimensional coordinates of each of the multiple points of the guide mechanism. According to this aspect, training data can be generated more efficiently.

[0086] (6) A control method according to a sixth aspect is a control method for a control system including a guide mechanism that guides a predetermined payload while bending and advancing to a predetermined position, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that moves the guide mechanism forward and backward, and a control unit that controls the operation of the attitude actuator and the operation of the advance / retract actuator, wherein a machine learning model is input that inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and the control method controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model that has been trained using as training data a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of the three-dimensional coordinates of each of the plurality of points of the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path.

[0087] (7) A seventh aspect of the machine learning method is a machine learning method for a machine learning model used by a control unit in a control system including a guide mechanism that guides a predetermined payload while bending and advancing to a predetermined position, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that moves the guide mechanism forward and backward, and a control unit that controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model, wherein the trained machine learning model inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and the machine learning model is trained using training data that is a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of three-dimensional coordinates of each of the plurality of points in the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path.

[0088] (8) A trained machine learning model according to an eighth aspect is a trained machine learning model used by a control unit in a control system including a guide mechanism that guides a predetermined payload while bending and advancing to a predetermined position, an attitude actuator that changes the attitude of the guide mechanism, an advance / retract actuator that moves the guide mechanism forward and backward, and a control unit that controls the operation of the attitude actuator and the advance / retract actuator using a trained machine learning model, wherein the machine learning model inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and is trained by machine learning using as training data a combination of the time series of control information for the attitude actuator, the time series of control information for the advance / retract actuator, and the time series of three-dimensional coordinates of each of the plurality of points in the guide mechanism, which are obtained by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path. [Explanation of symbols]

[0089] 100...Control system 101...Guide mechanism 102...Payload 103...following section 104...Guide tube 105... Posture actuator 106...Advance and retreat actuator 107... stand 108...Monitor 109...Control unit 110...Trained machine learning model 5...Inspection equipment 61...Inspection cable 611...Cable body 612...Sensor 6...Test tube 62...Tube 62A...active part 62B…Driver part 65... Posture actuator 67...Advance and retreat actuator 7...Guide jig 8...Drive control device 91...Camera image monitor 651...Wire

Claims

1. a guide mechanism that guides a predetermined payload to a predetermined position while bending and advancing; an attitude actuator that changes the attitude of the guide mechanism; an actuator for moving the guide mechanism forward and backward; a control unit that controls the operation of the attitude actuator and the operation of the advance / retreat actuator; Equipped with The control unit a machine learning model that inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retreat actuator, a trained machine learning model that is machine-learned using training data that is a combination of a time series of control information for the attitude actuator, a time series of control information for the advance / retract actuator, and a time series of three-dimensional coordinates of each of the multiple points of the guide mechanism, the time series of control information for the advance / retract actuator being acquired by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path, Controlling the operation of the attitude actuator and the movement of the forward / backward actuator Control system.

2. the guide mechanism has a structure that changes its posture by stretching a plurality of wires, The machine learning model is composed of a neural network that includes a self-loop in which the output of the intermediate layer is input again to the intermediate layer. The control system of claim 1 .

3. The intermediate layer includes long short-term memory The control system of claim 2 .

4. The teacher data is a first step of acquiring discrete values ​​of control information for the attitude actuator and discrete values ​​of control information for the advance / retract actuator, the discrete values ​​being acquired when the attitude actuator and the advance / retract actuator are inched so that the guide mechanism bends and advances along the target path; a second step of generating a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator by interpolating each discrete value of control information for the attitude actuator and each discrete value of control information for the advance / retract actuator and adding information related to time; a third step of controlling the attitude actuator and the advance / retract actuator based on a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator, and acquiring a time series of three-dimensional coordinates of each of the plurality of points of the guide mechanism; Obtained by The control system of claim 3 .

5. In the first step, discrete values ​​of control information for the attitude actuator when the attitude actuator is moved so as to deviate the guide mechanism from the target path at some of a plurality of stop positions of the advance / retreat actuator during the inching movement are further acquired, in the second step, based on a time series of control information for the attitude actuator and a time series of control information for the advance / retract actuator generated by bending and advancing the guide mechanism along the target path, and discrete values ​​of the control information for the attitude actuator acquired when the attitude actuator is moved so as to deviate the guide mechanism from the target path, another time series of control information for the attitude actuator and another time series of control information for the advance / retract actuator are further generated; In the third step, the attitude actuator and the advance / retract actuator are controlled based on another time series of control information for the attitude actuator and another time series of control information for the advance / retract actuator, and another time series of three-dimensional coordinates of each of the plurality of points of the guide mechanism is further acquired. The control system of claim 4.

6. a guide mechanism that guides a predetermined payload to a predetermined position while bending and advancing; an attitude actuator that changes the attitude of the guide mechanism; an actuator for moving the guide mechanism forward and backward; a control unit that controls the operation of the attitude actuator and the operation of the advance / retreat actuator; A control method for a control system comprising: a machine learning model that inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retreat actuator, a trained machine learning model that is machine-learned using training data that is a combination of a time series of control information for the attitude actuator, a time series of control information for the advance / retract actuator, and a time series of three-dimensional coordinates of each of the multiple points of the guide mechanism, the time series of control information for the advance / retract actuator being acquired by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path, Controlling the operation of the attitude actuator and the movement of the forward / backward actuator Control method.

7. a guide mechanism that guides a predetermined payload to a predetermined position while bending and advancing; an attitude actuator that changes the attitude of the guide mechanism; an actuator for moving the guide mechanism forward and backward; a control unit that controls the operation of the attitude actuator and the operation of the advance / retreat actuator using a trained machine learning model; A machine learning method for the machine learning model used by the control unit in a control system comprising: The trained machine learning model inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism, and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retreat actuator, The machine learning model is trained by machine learning using as training data a combination of a time series of control information for the attitude actuator, a time series of control information for the advance / retract actuator, and a time series of three-dimensional coordinates of each of the multiple points of the guide mechanism, which are acquired by operating the attitude actuator and the advance / retract actuator so that the guide mechanism bends and advances along a predetermined target path. Machine learning methods.

8. a guide mechanism that guides a predetermined payload to a predetermined position while bending and advancing; an attitude actuator that changes the attitude of the guide mechanism; an actuator for moving the guide mechanism forward and backward; a control unit that controls the operation of the attitude actuator and the operation of the advance / retreat actuator using a trained machine learning model; The trained machine learning model used by the control unit in a control system comprising: the machine learning model inputs a time series of three-dimensional coordinates of each of a plurality of points in the guide mechanism, and outputs a time series of control information for the attitude actuator and a time series of control information for the advance / retreat actuator, The posture actuator and the forward / backward actuator are operated so that the guide mechanism bends and advances along a predetermined target path, and a combination of a time series of control information for the posture actuator, a time series of control information for the forward / backward actuator, and a time series of three-dimensional coordinates of each of the multiple points of the guide mechanism is used as training data for machine learning. A trained machine learning model.

Images