Control device, learning device, control system, control method, and program for scarfing robots.

Abstract

Automation is achieved by teaching scarfing robots in a simulation environment. [Solution] A control device that controls a scarfing robot that performs scarfing on an object and is connected to the scarfing robot includes: an input unit that receives operations from a user; a first simulation unit that constructs a simulation environment that simulates the operation of the scarfing robot based on the operations; a teaching unit that performs teaching in the simulation environment that the scarfing robot operates in a real environment when the operations are input; and a second simulation unit that, when scarfing is performed on the object based on the operations, simulates the state of defect removal in the object that reflects the processing results of scarfing in the simulation environment.

Description

Technical Field

[0005] ,

[0001] The present invention relates to a control device, a learning device, a control system, a control method, and a program for a scarfing robot.

Background Art

[0002] Efforts are being made to promote the Sustainable Development Goals (2030 Agenda for Sustainable Development, adopted at the United Nations Summit on September 25, 2015 (Heisei 27), hereinafter referred to as "SDGs"). Specifically, in Goal "9", technologies are required to build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation.

[0003] Conventionally, there is a so-called scarfing process for removing defective portions from an object such as metal. A robot that performs such a work process is known.

[0004] For example, a scarfing device includes a carriage that travels in a direction perpendicular to the rolling line, a swivel table that swivels on the carriage, a vertical arm, and a manipulator including a horizontal arm, and a master device that operates the manipulator. Then, the manipulator and the like are operated from a manipulator operation room installed between the rolling mill operation room and the rolling mill. In this way, technologies for increasing the degree of freedom of the manipulator, accelerating the scarfing process, optimizing the position of the operation room, and improving safety are known (see, for example, Patent Document 1, etc.).

[0005] Furthermore, methods for removing surface defects from materials are known. Specifically, first, the surface of the material to be treated is treated by penetrant testing. Next, an image sensor detects the defect pattern that appears on the treated surface. In this detection, the detected signal is processed as an image, and defect information relating to the location, shape, and depth of the defect is generated based on the image. Based on this defect information, the defect is removed. Techniques for automating the defect removal process in this manner are known (see, for example, Patent Document 2).

[0006] In addition, automated grinding is also known. Specifically, a grinding robot is equipped with a hand unit fitted with a defect detector, a grinder, and a grinding wheel contact degree detector. Furthermore, a drive control device drives the grinding robot based on an image processing device that detects the location of defects and a processing device that processes the signals. In this way, even if the state, location, and wear of the grinding wheel of the object change, a technology is known that aims to reduce labor, improve safety, and prevent health hazards in grinding work by combining detection, grinding, holding, and moving mechanisms (see, for example, Patent Document 3). [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 59-92165 [Patent Document 2] Japanese Patent Application Publication No. 1-157770 [Patent Document 3] Japanese Patent Publication No. 62-63059 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] Conventional technologies have limitations in enabling automation through teaching in a simulation environment and ensuring quality for scarfing robots.

[0009] The present invention aims to enable automation and quality assurance for scarfing robots through teaching in a simulation environment. [Means for solving the problem]

[0010] To solve the above problems, a control device that controls a scarfing robot performing scarfing on an object and is connected to the scarfing robot, according to one aspect of the present invention, An input section for receiving user input, A first simulation unit constructs a simulation environment for performing a simulation of operating the scarfing robot based on the above operation, In the aforementioned simulation environment, when the aforementioned operation is input, a teaching unit performs teaching so that the scarfing robot operates in the actual environment, When scarfing is performed on the object based on the above operation, a second simulation unit simulates the defect removal state in the object, reflecting the processing results by scarfing, in the simulation environment. It is characterized by being equipped with [the following features]. [Effects of the Invention]

[0011] According to the present invention, automation and quality assurance can be achieved for scarfing robots through teaching in a simulation environment. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example of a control system for a scarfing robot. [Figure 2] This figure shows an example of control device hardware. [Figure 3] This figure shows an example of a master-slave configuration environment. [Figure 4] This figure shows an example of a simulation environment. [Figure 5] This figure shows an example of a completed teaching session. [Figure 6] It is a diagram showing an example of a round nozzle. [Figure 7] It is a diagram showing an example of performing scarfing. [Figure 8] It is a diagram showing an example of a defect. [Figure 9] It is a diagram showing an example of the processing setting of scarfing. [Figure 10] It is a diagram showing an example of the height setting of scarfing. [Figure 11] It is a diagram showing an example of the angle setting of scarfing. [Figure 12] It is a diagram showing an example of the depth output. [Figure 13] It is a diagram showing an example of the simulation of scarfing marks. [Figure 14] It is a diagram showing an example of the simulation output of scarfing. [Figure 15] It is a diagram (part 1) showing an example of the setting of a virtual area. [Figure 16] It is a diagram (part 2) showing an example of the setting of a virtual area. [Figure 17] It is a diagram showing an example of the restriction by a virtual area. [Figure 18] It is a diagram showing an example of the first setting of a virtual permission area. [Figure 19] It is a diagram showing an example of the second setting of a virtual permission area. [Figure 20] It is a diagram showing an example of preprocessing. [Figure 21] It is a diagram showing an example of execution processing. [Figure 22] It is a diagram showing an example of the execution of execution processing and simulation. [Figure 23] It is a diagram showing an example of the overall processing of AI learning and execution. [Figure 24] It is a network configuration diagram showing an example of the configuration of AI. [Figure 25] It is a diagram showing an example of the overall processing of the first embodiment. [Figure 26] It is a diagram showing an example of the overall processing of the second embodiment. [Figure 27]This figure shows an example of the functional configuration of the first embodiment. [Figure 28] This figure shows an example of the functional configuration of the second embodiment. [Modes for carrying out the invention]

[0013] [First Embodiment] The following examples will be explained with reference to the attached drawings. In the following explanation, the reference numerals in the drawings refer to the same elements. Furthermore, the embodiments are not limited to the following examples, and embodiments may include elements other than those shown in the drawings.

[0014] [About scarving and scarving robots] Scarfing (also called "scarfing" or "scaling") uses a gas containing multiple types of gases. The gas contains oxygen. Specifically, scarfing uses a gas mixture of oxygen and flammable gases such as hydrogen, acetylene (propane), and LPG (Liquefied Petroleum Gas). More than two types of gases may be mixed with oxygen. The gas is used in scarfing under high pressure. In other words, scarfing is a process that removes defects from an object by blowing a gas under high pressure onto it. Hereinafter, the state after the scarfing process has been performed will be referred to as the "removed state."

[0015] Furthermore, the removal process allows for processing such as melting, gouging, cutting, or fillet cutting of the target object.

[0016] The scarfing simulation results, i.e., the removal state, show, for example, the "removal area," "depth," "removal amount," and "shape" of the object that is removed by the scarfing process.

[0017] The area removed refers to the area removed on a plane, assuming that the scarfing process is applied to a plane of the object.

[0018] The "depth" of removal refers to the distance from the plane to the bottom of the area being removed, if the scarfing process is applied to a plane of the object.

[0019] The removal amount refers to the quantity of material removed during the scarfing process (e.g., expressed in units of volume or weight).

[0020] "Shape" refers to the shape of the area that was processed as a result of the scarfing process. For example, "shape" can be shown on the screen as a 3D model or similar.

[0021] Furthermore, in the following explanation, a robot that performs scarfing will be referred to as a "scarfing robot." A scarfing robot may be a robot that performs scarfing exclusively, or it may be a robot that performs other tasks in addition to scarfing.

[0022] Hereinafter, the object subjected to scarfing will be referred to as the "object." The object is, for example, metal. Specifically, the object is in the form of a slab (also called a "flat plate" or "semi-finished product," etc.). For example, if the object is steel, in a continuous casting process, the molten steel produced in the converter is cooled and processed into a plate. Then, the steel is cut after it has cooled and solidified to become a slab. For example, scarfing is a process performed on such a slab. However, the object is not limited to a slab; it may be in various shapes such as a billet. In other words, the object may be a so-called square bar or a flat bar, and its shape is not restricted.

[0023] A defect refers to, for example, a part of the material that has become iron oxide. Alternatively, a defect refers to a part of the material that has a large amount of impurities mixed in, or a part that has scratches (including cracks, etc.).

[0024] For example, scarfing is a process performed in the manufacturing process of an object. Therefore, scarfing is performed as part of the manufacturing process or near manufacturing equipment, and may be carried out in high-temperature environments.

[0025] Therefore, it is desirable for scarfing robots to be heat-resistant (the temperature they can withstand varies depending on the metals they handle). In other words, general industrial robots that are not heat-resistant may not be able to be installed in scarfing environments because they cannot operate in high-temperature environments.

[0026] Heat resistance can be achieved by covering the object with a heat-resistant jacket, for example. For instance, if the object contains steel, its melting point is 1580°C. Therefore, when installing a scarfing robot as part of a manufacturing process, it is desirable that the scarfing robot be able to operate even in environments close to such high temperatures. Hence, it is desirable that the scarfing robot has heat-resistant properties (including cases where a part of the scarfing robot, or the scarfing robot, temporarily acquires heat-resistant properties).

[0027] Parameters that affect the quality of scarfing include the speed at which the nozzle is moved, the angle between the nozzle and the object, the distance between the nozzle and the object, preheating conditions (such as the temperature at which the object was heated during pretreatment before the scarfing process), the mixing ratio of flammable gases and oxygen in the gas, the gas pressure, and the nozzle structure.

[0028] For example, scarfing uses three types of oxygen: high-pressure oxygen (hereinafter referred to as "high-pressure oxygen"), oxygen at a lower pressure than high-pressure oxygen (hereinafter referred to as "low-pressure oxygen"), and LPG. The following explanation will use an example with these three types, but scarfing may be performed using different types, numbers of types, and procedures other than those described below.

[0029] Furthermore, high-pressure oxygen only needs to be at a higher pressure than low-pressure oxygen; the exact pressure is not specified. In other words, the pressures of high-pressure and low-pressure oxygen are set appropriately depending on the object being tested.

[0030] In the first step, low-pressure oxygen and LPG are mixed, and a pilot light is ignited. In the second step, after the pilot light has been ignited in the first step, high-pressure oxygen is further mixed into the two-component gas mixture. Next, the third step adjusts the flame produced in the second step. Specifically, the third step adjusts the mixing ratio of low-pressure oxygen and LPG (or the output amount of each per unit time of low-pressure oxygen and LPG). For example, the third step is performed by observing the size of the flame produced in the second step. However, it is not limited to observing the size of the flame; adjustments may also be made based on flow meter readings, etc.

[0031] Note that in the third step, you may adjust only one of the two options. For example, in the third step, you may leave the LPG fully open and adjust only the low-pressure oxygen.

[0032] In the fourth step, the pressure of the hyperbaric oxygen is adjusted based on the flame that was adjusted in the third step.

[0033] As described above, scarfing is performed using the flame produced through the first to fourth steps. Therefore, if the mixing ratio of low-pressure oxygen and LPG adjusted in the third step, and the pressure of high-pressure oxygen adjusted in the fourth step can be set as parameters, the scarfing process and simulation will be easier to perform.

[0034] Note that parameters of types other than those listed above may also be set.

[0035] Additionally, it would be good to have a cover that is designed to protect against dirt and stains.

[0036] As described above, it is desirable for scarfing robots to be heat-resistant. Therefore, it is desirable for scarfing robots to be equipped with environmentally resistant materials such as heat-resistant jackets or covers. Environmentally resistant materials may be added later or be removable.

[0037] The objects can have various shapes. Therefore, in order to perform scarfing, the scarfing robot changes its posture by translating and rotating in accordance with the shape of the object. Specifically, the scarfing robot has degrees of freedom (DOF) that allow the nozzle to be translated in the left-right direction, translated in the up-down direction, translated in the depth direction (including diagonal movement by combining translation of two or more axes), rotate on the Roll axis, rotate on the Pitch axis, and rotate on the Yaw axis. In other words, the scarfing robot is a mechanism with at least 6DOF.

[0038] [Example System Configuration] Figure 1 shows an example of a control system for a scarfing robot. For example, the control system for a scarfing robot (hereinafter referred to as "control system 100") includes a scarfing robot 11, a control panel 12, and a control device 10, etc.

[0039] The control system 100 may have a PLC 13 (Programmable Logic Controller) or the like, either internally or externally. Furthermore, the control system 100 may be connected to external systems such as a higher-level system 14.

[0040] The scarfing robot 11 is the robot controlled by the control panel 12 and the control device 10. There may be multiple scarfing robots 11. For simplicity, the following explanation will use a single scarfing robot 11 as an example.

[0041] Furthermore, the control system 100 may also control robots other than the scarfing robot 11. However, the robots controlled by the control system 100 will operate in harsh environments such as the surrounding environment of the manufacturing process. Therefore, they will be robots that can withstand the surrounding environment, such as heat resistance.

[0042] In addition, the control system 100 may have other devices externally or internally. Furthermore, each of the above devices may be composed of multiple devices. Moreover, each of the above devices may be, for example, an integrated scarfing robot 11 and control panel 12.

[0043] In the control system 100, for example, when a user 101 inputs an operation to the control device 10, the scarfing robot 11 operates based on that operation. In other words, in the control system 100, the scarfing robot 11 is the slave and the control device 10 is the master, forming a master-slave configuration.

[0044] However, the scarfing robot 11 does not necessarily need user 101 to operate (including partial operation, not the entire operation). For example, the scarfing robot 11 may operate based on a program or the like, even without user 101's input.

[0045] [Example of hardware configuration of control device 10] Figure 2 shows an example of the control device hardware. For example, the control device 10 has a hardware configuration that includes a CPU (Central Processing Unit, hereinafter referred to as "CPU10H1"), a storage device 10H2, an input device 10H3, an output device 10H4, and a communication device 10H5. In other words, the control device 10 is an information processing device such as a PC (Personal Computer) or a server. Note that the control device 10 may consist of multiple information processing devices.

[0046] The CPU 10H1 is an arithmetic unit and control unit. Therefore, the CPU 10H1 performs calculations in cooperation with the memory device 10H2 and other components to execute processing and control.

[0047] The storage device 10H2 is a memory device, etc. Therefore, the storage device 10H2 stores data, etc. The storage device 10H2 may also have an auxiliary storage device.

[0048] The input device 10H3 is, for example, a keyboard or a mouse. Thus, the input device 10H3 is a device that receives data from an external device or from human input.

[0049] The output device 10H4 is, for example, a display. Thus, the output device 10H4 is a device that outputs data to an external device or displays information to a person.

[0050] The communication device 10H5 is a device that transmits and receives data to and from external devices via wired or wireless communication.

[0051] The control device 10 may further include sensors, computing devices, control devices, input devices, output devices, memory devices, communication devices, or auxiliary devices, either internally or externally.

[0052] [Example of a master-slave configuration environment] Figure 3 shows an example of a master-slave configuration environment. The following explanation will use a metal 102 as the target object. Note that in the following system configuration example, the PLC 13 and other components will be omitted.

[0053] In metal 102, the plane on which scarfing is performed is defined as the "XY plane." Therefore, with respect to metal 102, the left-right direction (i.e., the horizontal direction) is defined as the "X-axis direction." Conversely, the up-down direction (i.e., the vertical direction) is defined as the "Y-axis direction." And the depth direction is defined as the "Z-axis direction." Furthermore, in the following example, scarfing is basically performed in the direction of the Z-axis.

[0054] The scarfing robot 11 has nozzles 15. However, there may be two or more nozzles 15.

[0055] Hereinafter, the environment in which the scarfing robot 11 and the metal 102 actually exist as hardware, rather than in a virtual environment, will be referred to as the "real environment 201". Therefore, in the real environment 201, the scarfing robot 11 and the metal 102 are actually placed, and the scarfing robot 11 actually operates.

[0056] In the actual environment 201, the control device 10 is the master and the scarfing robot 11 is the slave. Furthermore, it is desirable that the master and slave are separated by distance, meaning the scarfing robot 11 is operated remotely by commands input to the control device 10.

[0057] In a real-world environment 201, the scarfing robot 11 is installed around metal 102. Therefore, the scarfing robot 11 is often subjected to harsh environments such as high temperatures that are unsuitable for humans. For this reason, it is desirable that the control device 10, i.e., the user 101, be able to remotely control the scarfing robot 11 from a location away from it.

[0058] For example, in the control device 10, the user 101 inputs an operation to instruct the scarfing robot 11 to perform a translation, rotation, or scarfing operation. Based on such an input, the scarfing robot 11 performs a scarfing operation and other operations, as well as translation and other movements.

[0059] [First simulation and example simulation environment] Figure 4 shows an example of a simulation environment. For example, the control device 10 virtualizes the scarfing robot 11 in the real environment 201 to construct the simulation environment 202.

[0060] The simulation environment 202 is a virtual space that virtualizes the surrounding environment and a virtual model that performs the same operations as the scarfing robot 11 in the actual environment 201. Note that the virtual model and virtual space may differ from those in the actual environment 201.

[0061] Hereinafter, the virtual model of the scarfing robot 11 in the actual environment 201 will be referred to as "virtual robot 21". Similarly, the virtual model of the metal 102 in the actual environment 201 will be referred to as "virtual metal 112". Note that the simulation environment 202 may also include other devices, etc., that are virtualized in addition to the virtual robot 21 and virtual metal 112.

[0062] Furthermore, in the simulation environment 202, virtual models such as the virtual robot 21, which are displayed on the simulation environment 202 and operate based on user input, are sometimes referred to as "virtual operating entities."

[0063] Once the simulation environment 202 is established, just like in the real environment 201, when an operation to operate the scarfing robot 11 is input to the control device 10, the virtual robot 21 and other components will operate in the simulation environment 202 based on that operation. Therefore, even without the real environment 201, the user 101 can input an operation to operate the virtual robot 21 in the simulation environment 202 and confirm what kind of actions are performed based on that operation.

[0064] The state in which the simulation environment 202 is constructed is equivalent to the state in which the scarfing robot 11 is connected, even if the equipment of the actual environment 201, in particular the scarfing robot 11, is not present (however, even if the scarfing robot 11 actually exists and is connected to the control device 10 via a network, etc., it is sufficient if the power is not turned on or it is not accepting commands).

[0065] In simulation environment 202, teaching (also known as "robot teaching," "instruction," or "teaching operation") is performed.

[0066] Teaching is the process of inputting in advance how the scarfing robot 11 should operate in the real environment 201. For example, teaching is performed by operating a virtual robot 21 in the simulation environment 202. In other words, teaching is the process of reproducing how the scarfing robot 11 should operate in the real environment 201 under the simulation environment 202.

[0067] Figure 5 shows an example of completed teaching. Hereafter, it is assumed that "teaching data 103" is generated when teaching is performed. That is, if teaching data 103 is available, the scarfing robot 11 will perform the actions indicated by the teaching data 103 in the actual environment 201.

[0068] Therefore, by using the teaching data 103, the scarfing robot 11 can reproduce the same behavior in the real environment 201 as it did in the simulation environment 202.

[0069] The teaching data 103 (including cases where only a portion is generated or modified) may be generated in a format such as code input. For example, the teaching data 103 shows the position, velocity, acceleration, angle (such as changes in posture due to rotation) and processing details of the scarfing robot 11 in the actual environment 201 in chronological order.

[0070] Therefore, the teaching data 103 may be a collection of data indicating coordinate positions, etc. However, the format of the teaching data 103 is not limited as long as it can instruct the scarfing robot 11 on what to do. In addition, the teaching data 103 may be converted in format and optimized using conversion software, etc.

[0071] Therefore, in the simulation environment 202, the operation of the scarfing robot 11 in the actual environment 201 is considered, and teaching data 103 is generated. Next, once the teaching data 103 is generated, the scarfing robot 11 can operate without any operation by the user 101. For example, when performing similar tasks repeatedly, the user 101 does not need to repeatedly input information for each task.

[0072] Furthermore, in the actual environment 201, the scarfing robot 11 does not need to be connected to the control device 10 if teaching data 103 is available. In other words, after teaching, the scarfing robot 11 can operate unmanned and automatically in the actual environment 201.

[0073] However, even if the scarfing robot 11 is operating based on the teaching data 103, if the scarfing robot 11 receives an operation from the control device 10, it may be configured to prioritize and execute instructions from the control device 10.

[0074] [Example of nozzle 15] The nozzle 15 used for scarfing has, for example, the following configuration. However, the nozzle 15 may have a configuration other than that shown below.

[0075] Figure 6 shows an example of a round nozzle. Specifically, Figure 6(A) is a perspective view of a round nozzle, while Figure 6(B) is a front view of the round nozzle. Note that the figures are from Nippon Steel Corporation. I quote from Technical Report No. 364, "Automatic Slab Check Scarfing Device: Auto Defect-Detector with Scarfer for Slabs," published in 1997, authored by Hayato Kanayama, Kenji Mori, Jiro Matsuo, Katsumasa Konno, Mitsuru Sakakibara, and Shinichiro Sasamori.

[0076] The circular nozzle has a configuration in which, for example, multiple oxygen holes 151 for releasing oxygen and LPG holes 152 for releasing LPG are arranged in a circular pattern on the XY plane.

[0077] Furthermore, the round nozzle has a scarf oxygen pore 153 that releases scarf oxygen at the central position where the oxygen pores 151 and LPG pores 152 are located. In the round nozzle, the scarf oxygen pore 153 is round in shape.

[0078] In the nozzle 15 described above, the number, positional relationship, ratio, and diameter of the oxygen holes 151 and LPG holes 152 are not limited to those described above. Furthermore, the oxygen holes 151 and LPG holes 152 may also emit gases other than oxygen and LPG. Similarly, the shape, diameter, and position of the scarf oxygen holes 153 are not restricted.

[0079] The oxygen pores 151 and LPG pores 152 are holes that release gas to create a preheating flame for preheating the object. The preheating flame generated by the gas heats the object (sometimes only in certain areas). When the scarf oxygen released from the scarf oxygen pores 153 is blown onto the heated area, the defective area is removed.

[0080] [Example of scarfing] Figure 7 shows an example of scarfing. Hereafter, the object is assumed to be a plate-shaped metal 102 with its length in the Z-axis direction. Note that the nozzle 15 may not be in the orientation, size, shape, or installation position shown in the figure.

[0081] For example, while performing scarfing, the scarfing robot 11 moves in the Z-axis direction, that is, along the longitudinal direction of the metal 102. And, if the nozzle 15 and other degrees of freedom are not moved, the scarfing processing position moves in the Z-axis direction due to the translation of the scarfing robot 11.

[0082] In this way, the scarfing robot 11 can perform scarfing in a linear manner in the Z-axis direction.

[0083] However, the scarfing is not limited to the above in terms of position, range, and depth. For example, the scarfing robot 11 may perform scarfing other than that described above by combining translation and rotation. Furthermore, the scarfing robot 11 may perform scarfing other than that described above by changing the conditions and posture of the nozzle 15.

[0084] Specifically, the scarfing robot 11 may perform translational movements in two dimensions along the plane of the metal 102, i.e., the XZ plane. Alternatively, the scarfing robot 11 may be installed in a suspended ceiling configuration, for example.

[0085] Furthermore, in scarfing, it is desirable that the sparks generated by the process be captured by a camera. Therefore, it is desirable that the scarfing robot 11 be equipped with a camera or other imaging device capable of capturing images of the area being processed by scarfing. It is also desirable that image data capturing the processing situation, including sparks, is generated.

[0086] Sparks can occur when the flow of molten metal is obstructed, causing some of the metal to scatter into the air and react with high-pressure oxygen, etc., through oxidation. Therefore, sparks often emit a bright light. Furthermore, the luminescence of a spark is very short, lasting approximately 100 to 500 milliseconds.

[0087] Scarfing allows the quality of the process to be confirmed by observing the sparks produced. Therefore, if sparks are captured, the user 101 can confirm them even when operating remotely.

[0088] Furthermore, areas where sparks and scarfing are processed are often brighter, i.e., have higher luminance, compared to other areas. Therefore, a camera with a high dynamic range is desirable. In addition, since sparks often travel at high speeds, a camera capable of shooting at a high frame rate of 60fps or higher is desirable. Alternatively, high-luminance subjects such as sparks may be photographed and then processed. Another method for photographing sparks is to place a filter in front of the camera.

[0089] Figure 8 shows an example of a defect. The following explanation will use an example of a defect in a plate-shaped metal 102. Specifically, if a defect is found in the metal 102, a "defect region 301" is set for the area of ​​the defect. For example, an area with a "flaw" on the surface of the metal 102 is set as a defect region 301.

[0090] The defect area 301 may be set by visual inspection or by detection using a sensor such as a camera. For example, the defect area 301 may be set based on an image of the metal 102 taken in the actual environment 201, or on data of the metal 102.

[0091] Once the defect area 301 is defined, for example, the location, range, and depth of the defect are determined. Note that there may be other settings for the defect area 301.

[0092] Furthermore, defects may occur on surfaces other than the flat surface of the metal 102. For example, defects may occur on the sides or edges of the metal 102.

[0093] Figure 9 shows an example of scarfing processing settings. For example, the following scarfing processing settings are applied to the defect region 301 shown in Figure 8.

[0094] For example, the area where scarfing will be performed is set with "arrow 302". As in this example, arrow 302 is set so that the defective area 301 is sufficiently subject to scarfing processing.

[0095] Arrow 302 indicates the direction in which it will move on the plane (XZ plane) of the metal 102. Furthermore, the starting point of arrow 302 becomes the starting point for the scarfing process. Therefore, the setting of arrow 302 determines the position, range, number of times, and direction of the scarfing process.

[0096] Furthermore, settings such as the position, range, number of times, and direction of scarfing may be configured using a GUI (Graphical User Interface) other than arrow 302.

[0097] Figure 10 shows an example of scarfing height settings. For example, it is desirable that "Height H" can be set as a scarfing setting item, as shown below.

[0098] Height H is the distance between the nozzle 15 performing scarfing and the surface of the metal 102 (in the diagram, this is the distance on the Y-axis. Since this distance is the shortest distance between the nozzle 15 and the surface of the metal 102, the point of reference will differ depending on the orientation of the nozzle 15 and the metal 102). For example, Height H can be set numerically, such as "〇 millimeters," or it can be set by selecting from a predetermined number of levels (for example, three levels such as "far," "intermediate," or "close").

[0099] The height H may be set, for example, for each defect region 301, or for each scarfing process (each arrow 302 in Figure 11). Alternatively, the height H may be set to be constant (fixed, i.e., common to the processes set by multiple arrows 302) in the metal 102.

[0100] Being able to set the height H allows for more accurate scarfing results in simulations.

[0101] Figure 11 shows an example of setting the scarfing angle. For example, it is desirable that the "angle θ" can be set as a setting item for scarfing, as shown below.

[0102] The angle θ is the relative angle between the nozzle 15 performing scarfing and the metal 102 (in the figure, the metal 102 is parallel to the XY plane, and the angle θ is set on the vertical axis. Depending on the orientation of the nozzle 15 and the metal 102, the angle referred to will differ). For example, the angle θ may be set numerically, such as "X degrees," or it may be set by selecting from a predetermined number of stages (for example, three stages such as "large incline," "medium," or "small incline").

[0103] The angle θ may be set, for example, for each defect region 301, or for each scarfing process (each arrow 302 in Figure 11). Alternatively, the angle θ may be set to be constant (fixed, i.e., common to the processes set by multiple arrows 302) in the metal 102.

[0104] Being able to set the angle θ allows for more accurate scarfing results in simulations.

[0105] Additionally, scarfing may be configured to process while translating. The translation speed may be synchronized with the operation or set using a velocity coefficient, etc. Furthermore, the translation may be configured in a way that allows setting a start point and an end point.

[0106] [Simulation Example] Figure 12 shows an example of depth output. For example, depth can be represented by four levels (from "shallow" to "deep") using color. However, depth may be divided into fewer than four levels, or into five or more levels. Furthermore, depth is not limited to color; it may also be represented by hatching or other methods.

[0107] "Deep" refers to a removal state where a large amount of material is removed by scarfing. On the other hand, "shallow" refers to a removal state where a small amount of material is removed by scarfing.

[0108] The depth is determined by parameters such as the speed at which the nozzle 15 is moved, the angle between the nozzle 15 and the object, the distance between the nozzle 15 and the object, the preheating conditions, the mixing ratio of flammable gas and oxygen in the gas, and the nozzle structure. Therefore, by performing a simulation based on these conditions, it is possible to accurately simulate how deep the process will be by scarfing.

[0109] Figure 13 shows an example of a scarfing trace simulation. For example, if the settings are as shown in Figure 9, the scarfing process result (hereinafter, the area after the scarfing process is referred to as "scarfing trace 303") is simulated as follows.

[0110] The scarfing marks 303 are simulated, for example, by color-coding their depth as shown in Figure 12.

[0111] Thus, the simulation reproduces the location, extent, and depth of the scarfing marks 303. However, the simulation may also simulate other aspects.

[0112] Figure 14 shows an example of the output of a scarfing simulation. For example, it is desirable that the simulation output a magnified view of the area where the scarfing process is being performed.

[0113] In the simulation, it is desirable that the smoke and sparks generated by scarfing are hidden or made transparent. In the real environment 201, when scarfing is performed, smoke is generated, making it difficult to see with the naked eye. On the other hand, in the simulation environment 202, the smoke is output as hidden. In this way, in the simulation environment 202, even when processing is in progress, the view is not obstructed by smoke, making it easy to see the areas where scarfing is being performed.

[0114] Furthermore, the sparks generated during the scarfing process are very bright. Therefore, viewing these sparks in a real environment (201) can be harmful to eye health. On the other hand, in the simulation environment (202), the sparks are either hidden or output at a light intensity that does not harm eye health. Thus, in the simulation environment (202), even when processing is in progress, there is no strong light, making the areas where scarfing is being performed easier to see.

[0115] Furthermore, cross-sectional views and similar formats are difficult to view in the actual environment 201. Therefore, it is desirable that the simulation output can be displayed from viewpoints that are difficult to view in the actual environment 201, such as cross-sectional views.

[0116] [Example of virtual area] Figure 15 is a diagram (part 1) showing an example of setting up a virtual domain. The following explanation will use an example where the nozzle 15 (hereinafter, the trajectory of the nozzle 15's movement is referred to as the "nozzle trajectory") moves along the trajectory shown by the dotted line. For example, user 101 can view the simulation results on an output screen like the one shown, or perform operations related to the simulation.

[0117] Figure 15 shows an example of an operation screen for scarfing. As shown in the figure, the operation for the scarfing simulation involves inputting the defect area to be processed, its position (the start and end points of the process), distance, angle, and velocity. These input results are shown below as "movement arrows 16". Therefore, the above input items may be entered numerically as coordinates, or they may be entered in a format such as specifying the position using an input device.

[0118] As described above, when an instruction to perform scarfing is input, arrow 302, i.e., the simulation of the scarfing to be performed, is executed.

[0119] The starting point of the movement arrow 16 indicates the position where the operation was input. The length of the movement arrow 16 indicates the force, velocity, or acceleration applied to the nozzle 15. Furthermore, the ending point of the movement arrow 16 indicates the indicated direction.

[0120] For example, the virtual areas are configured as follows: Virtual Area 1 V1, Virtual Area 2 V2, and so on.

[0121] A virtual region is an area set up in the simulation environment 202 and does not exist in the real environment 201. Entry of virtual operating objects into the virtual region is restricted. Therefore, in this example, since the first virtual region V1 and the second virtual region V2 are set up, even if the movement arrow 16 is directed towards the first virtual region V1 and the second virtual region V2 (in the figure, the X-axis direction), it will be ineffective. Consequently, the nozzle 15 cannot enter the areas where the first virtual region V1 and the second virtual region V2 are set up, and no processing is performed.

[0122] Thus, once a virtual region is created, it becomes possible to perform a parallel translation along any object (in this example, the surface of the first virtual region V1), a so-called "following motion." Note that this is not limited to parallel translation; it could also be done by keeping the height (the position of the nozzle 15 in the Y-axis direction) constant, for example.

[0123] Furthermore, inputs that cause noise during user 101's operation, such as camera shake, can be canceled out.

[0124] Furthermore, during operation, the control device 10 may receive feedback when it is determined that it is in contact with the surface of the first virtual region V1. For example, if the input device 10H3 is equipped with an actuator, tactile feedback may be provided indicating that it is in contact with the surface of the first virtual region V1.

[0125] Figure 16 is a diagram (part 2) showing an example of setting up a virtual area. Compared to Figure 15, the X-axis position of the movement arrow 16 in Figure 16 is different from that in Figure 15. That is, Figure 16 is an input that performs scarfing separately from Figure 15, for example, after the scarfing that is performed based on the operation entered in Figure 15. Specifically, Figure 16 is an example of giving an instruction to perform scarfing in the Z-axis direction, similar to Figure 15. On the other hand, in Figure 16, the X-axis position is different from that of Figure 15; that is, the movement arrow 16 entered in Figure 16 is entered parallel to that of Figure 15.

[0126] In this case, the first virtual region V1 and the second virtual region V2 are moved to match the position where scarfing is performed, i.e., the position of the movement arrow 16. Therefore, in Figure 16, the positions of the first virtual region V1 and the second virtual region V2 are different from those in Figure 15.

[0127] In Figure 16, as in Figure 15, if the positions of the first virtual region V1 and the second virtual region V2 can be changed when performing a "copy operation," as shown in the change from Figure 15 to Figure 16, it is possible to handle cases where the "copy operation" is performed multiple times.

[0128] Figure 17 shows an example of a limitation imposed by a virtual domain. For example, let's explain using an example where a second virtual domain V2 is set. Next, let's assume that the virtual operating object is moved in parallel to the second virtual domain V2 in the first movement direction DR1.

[0129] For such operations, the virtual operating object is not restricted from translation by the second virtual region V2 until it reaches a position where it contacts the surface of the second virtual region V2 (hereinafter, the position where the virtual operating object contacts the surface of the second virtual region V2 is referred to as the "contact point TP"). It then moves in the first movement direction DR1.

[0130] Next, when the virtual moving body moves in parallel to the contact point TP, that is, when the virtual moving body comes into contact with the surface of the second virtual region V2, the virtual moving body then moves in parallel in the second movement direction DR2 along the surface of the second virtual region V2. Hereafter, the corrected direction will be referred to as the "corrected direction".

[0131] In this example, the correction direction is the second movement direction DR2. That is, initially it was the first movement direction DR1, but after correction, it becomes the correction direction of the second movement direction DR2.

[0132] In this example, the first movement direction DR1 includes the directional component of the second movement direction DR2 as its second directional component. Therefore, based on the shape of the second virtual region V2 and the position it is set at, the virtual moving body is corrected so that the translation of the first movement direction DR1 is translated parallel to the second movement direction DR2 from the contact point TP, while limiting the first directional component. In this way, the virtual region can be set to various shapes and positions.

[0133] [Example of virtual permitted area] Figure 18 shows a first example of setting up a virtual permission area. For example, for the metal 102 shown in Figure 8, an area (hereinafter referred to as the "virtual permission area") may be set up for the parts where processing is permitted, as follows.

[0134] The first virtual permission area V10 is configured in the same way as the first virtual area V1 and the second virtual area V2. However, the difference is that while the first virtual area V1 and the second virtual area V2 restrict nozzle 15 from entering the configured area, the virtual permission area restricts nozzle 15 from entering areas other than the configured area. Therefore, the configured area of ​​the virtual permission area, such as the first virtual permission area V10, is where scarfing processing is possible.

[0135] Furthermore, the virtual permission area, like the virtual area, is set up in the simulation environment 202 and does not exist in the actual environment 201.

[0136] For example, the first virtual permission area V10 is set to have a predetermined "thickness" relative to the defect area 301. Hereinafter, "thickness" refers to the space created between the metal 102 and the first virtual permission area V10. The direction of the "thickness" can be set and any direction is acceptable.

[0137] Specifically, the first virtual permission area V10 is set to be larger than the defect area 301 by a set value of "thickness" in any of the X-axis, Y-axis, or Z-axis directions. In other words, the first virtual permission area V10 is set to cover the defect area 301. However, the area in which the nozzle 15 can enter may be set to be wider than the area in which the first virtual permission area V10 is set. The virtual permission area may have a so-called "margin" in relation to the width of the area to be set.

[0138] The "thickness," that is, how much larger the virtual permitted area should be than the defective area 301, can be set in advance, for example.

[0139] When the first virtual permission area V10 is set for the defective area 301, operations that can be performed on the defective area 301 are permitted, while operations on other areas are restricted. In this way, setting a virtual permission area prevents erroneous operations that would cause areas other than the defective area 301 to be processed incorrectly. Furthermore, if the defective area 301 is smaller in area than a normal area, i.e., an area that does not require processing, setting a virtual permission area requires setting a smaller area than setting a virtual area, thus reducing the amount of operations required.

[0140] Figure 19 shows a second example of the virtual permission area configuration. Compared to Figure 18, Figure 19 differs in that the shape of the virtual permission area is the same as that of the second virtual permission area V11. The following explanation will focus on the differences, omitting redundant explanations.

[0141] As shown in Figure 19, the virtual authorization area does not need to have equal areas in the X, Y, and Z directions. For example, a virtual authorization area that is frustoconical in shape, like the second virtual authorization area V11, is easy to use. Specifically, the second virtual authorization area V11 has a shape in which the circular area expands upward in the Y direction. Therefore, the second virtual authorization area V11 is wider at the top and the area of ​​the XZ plane narrows downward, i.e., as it approaches the defect area.

[0142] Furthermore, the second virtual permission area V11 is not limited to a frustum of a cone; for example, it may have steps. With the above configuration, a simulation environment 202 can be constructed that simulates the operation of the scarfing robot 11 in the actual environment 201. Then, when the movement of the nozzle 15, etc., can be input in the simulation environment 202, the scarfing robot 11 can be taught in the simulation environment 202.

[0143] Therefore, once teaching is performed in the simulation environment 202 and teaching data 103 is generated, the scarfing robot 11 can be automated.

[0144] [Second Embodiment] The second embodiment will be described below. Components similar to those in the first embodiment will be denoted by the same reference numerals and their descriptions will be omitted.

[0145] [About AI (Artificial Intelligence)] The AI ​​used in this embodiment learns based on training data through "preprocessing." Hereinafter, the AI ​​that is the target of the learning stage, i.e., "preprocessing," will be referred to as "learning model A1." As learning progresses through "preprocessing," learning model A1 becomes "trained model A2." Hereinafter, the execution stage in which output processing is performed using trained model A2 will be referred to as "execution processing."

[0146] However, even with a pre-trained model A2, training may be performed after pre-processing, similar to the training model A1. Therefore, the AI ​​may, for example, perform "execution processing" and then perform "pre-processing" on the data used in the "execution processing" and add it to the training. Also, the term "AI" is sometimes used to refer to both the training model A1 and the pre-trained model A2 without distinction.

[0147] "Preprocessing" is performed before "execution." However, "preprocessing," i.e., additional training of the trained model A2, may be performed immediately after "execution." The following explanation will describe an example where "preprocessing" and "execution" are performed separately.

[0148] [Example of pre-processing] Figure 20 shows an example of preprocessing. For example, preprocessing is performed by the control device 10. However, preprocessing may be performed by devices other than the control device 10. For example, preprocessing may be performed by multiple information processing devices.

[0149] Learning model A1 learns by inputting the following learning data D1. In other words, learning model A1 performs what is known as "supervised learning."

[0150] The training data D1 has a data structure that includes, for example, object information D11, defect information D12, processing content information D13, and correct answer data D20.

[0151] Object information D11 is data that indicates the characteristics or properties of the object. Specifically, object information D11 may include material, dimensions, manufacturing date, or quality grade. For example, object information D11 may be in the form of text, CAD (Computer-Aided Design) drawings, or images of the object.

[0152] Defect information D12 is data about defects occurring in the object. Specifically, defect information D12 includes the type, dimensions, grade, or location of the defect in the object.

[0153] For example, if defect information D12 is to show the type, dimensions, grade, and location numerically, it may be in a data format such as text. Alternatively, defect information D12 may be data indicating the location of the defect in an image, or data resulting from defect detection by a sensor, etc.

[0154] Processing information D13 is data that indicates the details of the scarfing process performed for a defect. For example, processing information D13 is prepared separately for each defect.

[0155] However, if the processing content information D13 involves processing common to multiple defects (including cases where some parts are common), the information may be standardized. Specifically, the processing content information D13 consists of processing conditions such as the starting point position where processing begins, the ending point position where processing ends, the height H at which processing is performed, the angle of the nozzle 15 performing the processing, or the speed at which the nozzle 15 is moved. For example, the processing content information D13 may be in data format such as text or a collection of coordinate values.

[0156] The correct answer data D20 is data that shows the pass / fail result of the process, obtained by scarfing the combination of the object information D11, defect information D12, and processing content information D13 described above. In other words, the correct answer data D20 is data that shows the evaluation result of the processing quality, obtained by processing under the processing conditions etc. indicated by the processing content information D13.

[0157] Ideally, the correct answer data D20 should include image data of the sparks generated during scarving. Having such image data allows the AI ​​to learn about the processing quality that can be assessed from the sparks.

[0158] Note that the correct answer data D20 may consist of one data point per object, or it may be divided into several sets, such as one set for each defect.

[0159] For example, the correct answer data D20 is data entered by the evaluator, such as "Pass" or "Fail" (these can be replaced with "Yes / No" or a binary value, etc.). Note that the correct answer data D20 may also include reasons, etc., in addition to "Pass" or "Fail". Furthermore, the correct answer data D20 may not be limited to a two-tiered system like "Pass" or "Fail", but may also be an evaluation score within a pre-set range, such as from "0" to "100".

[0160] Therefore, the correct answer data D20 is in the form of text or numerical data indicating "Pass" or "Fail," etc.

[0161] Furthermore, the training data D1 may also contain information such as the settings of the nozzle 15, the specifications of the nozzle 15, or the environment in which the processing is performed (temperature, humidity, etc.). In addition, big data D4 may be used for preprocessing. For example, big data D4 may be publicly available information on the internet or a set of data acquired by sensors, etc.

[0162] The training data D1 may include text data, image data, audio data, video data, sensor data, gesture data, or a combination of these. In other words, the AI ​​may use multiple forms of data, a so-called multimodal approach.

[0163] In the preprocessing stage, the training data D1 is used to associate "correct answers" with combinations of input objects, defects, and processing content. Hereinafter, combinations of objects, defects, and processing content for which the "correct answer" is known will be referred to as the "first processing condition." The data showing the correct answer will be referred to as the correct answer data D20.

[0164] The following explanation will use a pair of examples where each piece of information constituting the training data D1 and each piece of ground truth data D20 are used. However, each piece of information constituting the training data D1 and each piece of ground truth data D20 may consist of multiple data points.

[0165] Furthermore, each piece of information constituting the training data D1, and the ground truth data D20, may be preprocessed during the preprocessing stage. For example, if each data set has multiple input formats, it is desirable to preprocess it by normalizing or standardizing it to the same expression. If the data is normalized and the numerical ranges are unified, the AI ​​can learn with greater accuracy.

[0166] Furthermore, if an expansion process that increases the training data D1 is performed as a preprocessing step, the amount of data that the learning model A1 uses for training can be increased. In this way, increasing the training data D1 allows the AI ​​to learn with greater accuracy.

[0167] Alternatively, preprocessing could involve separating the object into different material types based on object information D11 for learning. By dividing the object into categories based on material and other factors, the AI ​​can learn with greater accuracy.

[0168] Furthermore, if images are included, preprocessing may involve filtering the images.

[0169] Thus, in the training data D1, the first processing condition, whose correct answer is known, is associated with the correct answer for the first processing condition as shown in the correct answer data D20. Furthermore, training data D1 is data where the first processing condition and the correct answer are paired. Therefore, the preprocessing is the process of training the learning model A1 to learn the correlation between the first processing condition and the correct answer.

[0170] Once the above preprocessing is performed, the learning model A1 is trained and a pre-trained model A2 is generated. In the execution process below, the pre-trained model A2 generated in the preprocessing is used.

[0171] [Example of execution process] Figure 21 shows an example of execution processing. For example, the execution processing is performed by the control device 10. However, the execution processing may be performed by devices other than the control device 10. For example, the execution processing may be performed by multiple information processing devices.

[0172] In contrast to pre-processing, the difference is that in pre-processing, the correct data D20 is associated with the processing conditions, which are a combination of object information D11, defect information D12, and processing content information D13, whereas in execution processing, there is no correct data D20.

[0173] The unknown object D21 is, for example, information with a similar structure to the object information D11.

[0174] The unknown defect information D22 is, for example, information with a similar structure to the defect information D12.

[0175] The unknown processing content information D23 is, for example, information with a similar structure to the processing content information D13.

[0176] Input data D2 is data that includes, for example, an unknown object D21, unknown defect information D22, and unknown processing content information D23.

[0177] The trained model A2 is the state in which the learning model A1 has been trained through preprocessing. In other words, when preprocessing is performed on the learning model A1, the trained model A2 is generated. Thus, the trained model A2, which has been trained using training data D1 and big data D4, etc. as training data, is what is known as "generative AI".

[0178] When the trained model A2 receives input data D2, it uses the input of input data D2 as a trigger to generate output data D3.

[0179] Input data D2 represents a processing condition whose "correct answer" is unknown; in other words, the "correct answer" for the processing condition is unknown at the time of input. Hereafter, the processing condition whose "correct answer" is unknown as indicated by input data D2 will be referred to as the "second processing condition." Therefore, the data representing the second processing condition is input data D2.

[0180] The training data D1 is data where the "correct answer" is known, in order to associate the correct answer with the first processing condition, whereas the input data D2 is data where the "correct answer" is unknown. Specifically, the training data D1 includes the correct answer data D20, while the input data D2 does not. Therefore, the relationship between the first processing condition and the correct answer data D20 is known for the training data D1.

[0181] On the other hand, the input data D2 does not contain the ground truth data D20, and the "ground truth" for input data D2 is unknown. The trained model A2 then generates output data D3 for input data D2 based on the correlation between the training data D1 and the ground truth data D20, which were learned in the preprocessing stage.

[0182] The output data D3 is, for example, teaching data 103. Therefore, by generating output data D3 and inputting it to the scarfing robot 11 in the real environment 201, the scarfing robot 11 can be operated.

[0183] Alternatively, output data D3 is data that shows, for example, how to operate the scarfing robot 11 in the simulation environment 202. In other words, output data D3 is data that constructs an environment for simulating the operation of the scarfing robot 11 in the simulation environment 202. For example, the simulation is performed as follows.

[0184] Figure 22 shows an example of execution processing and simulation. For example, when input data D2 is input, the trained model A2 constructs a simulation environment 202 for the input data D2 and simulates what actions the virtual robot 21 will perform. When such simulation results are output, user 101 can check what kind of scarfing processing is performed in the simulation environment 202.

[0185] Furthermore, user 101 may be able to modify the simulation results. Since the so-called "initial settings" are generated by the AI, teaching can be started from a state where some teaching has already been completed, rather than starting from scratch, thus reducing the amount of teaching work.

[0186] As described above, when the simulation results are checked or corrected in the simulation environment 202, teaching data 103 is generated. Next, when the teaching data 103 is input to the scarfing robot 11, the scarfing robot 11 can be operated. In other words, teaching can be performed on the scarfing robot 11.

[0187] Figure 23 shows an example of the overall process of AI learning and execution. As shown above, the relationship between the pre-processing and execution processes is the same as the relationship between the generation and use of the learning model A1.

[0188] Furthermore, the pre-processing and execution processes do not necessarily have to be performed in a consecutive order as illustrated in the diagram. Therefore, it is not essential that the preparation period by pre-processing and the subsequent execution period be consecutive. Consequently, the execution process may be performed after a certain amount of time has elapsed since the pre-processing, provided that the trained model A2 has been created. Also, once the trained model A2 has been generated, the execution process may be performed by reusing the trained model A2.

[0189] In the learning process and the execution process, the input data, i.e., the learning data D1 and the input data D2, are different. Also, the AI ​​is initially the learning model A1 during the learning phase, but after a certain amount of learning has progressed, it becomes the trained model A2.

[0190] The first processing condition shown by the training data D1 and the second processing condition shown by the input data D2 are both the same scarfing processing conditions. In other words, the processing conditions are various settings and conditions for causing the scarfing robot 11 to perform the scarfing process.

[0191] For example, the processing conditions are entered by user 101 in text format or the like. Furthermore, the results of user 101's operations on the simulation environment 202, such as how the scarfing robot 11 is translated, rotated, or processed, may also become part of the processing conditions.

[0192] The execution process may be partially replaced by processing using tables, etc. Thus, the preprocessing in processing that uses tables, etc. (so-called rule-based processing) is the process of preparing to input tables (also called look-up tables (LUTs), etc.) or mathematical formulas, etc.

[0193] [Example configuration of trained model A1 and pre-trained model A2] Figure 24 is a network configuration diagram showing an example of an AI configuration. The learning model A1 and the trained model A2 are AIs with the configuration shown in network configuration 300 below, for example.

[0194] In the following explanation, the learning model A1 and the trained model A2 will be described using an example where they are implemented on the cloud. However, some or all of the learning model A1 and the trained model A2 may be implemented on the control device 10, etc.

[0195] The network configuration 300 is, for example, a configuration having an input layer L1, an intermediate layer L2 (also called a "hidden layer," etc.), and an output layer L3, etc.

[0196] The input layer L1 is the layer into which data is input.

[0197] The hidden layer L2 transforms the data input to the input layer L1 based on weights (e.g., coefficients used for multiplication) and biases (e.g., adding constants). The results processed in the hidden layer L2 are then transmitted to the output layer L3.

[0198] The output layer L3 is the layer that outputs the output content, etc.

[0199] Through learning, the weight coefficients (for example, the coefficients for input characters or images are changed during learning) and the parameters that are changed during learning are optimized. Note that the network configuration 300 is not limited to the configuration shown in the figure. In other words, the AI ​​may be implemented using other machine learning methods.

[0200] For example, the AI ​​may be configured to perform preprocessing such as dimensionality reduction (for instance, transforming a relationship with three or more dimensions into a relationship that can be obtained through simplified calculations of three dimensions or less) using unsupervised machine learning. Ideally, the relationship between input and output should be processed using simple calculations such as linear equations. Such calculations can reduce computational costs.

[0201] Furthermore, the AI ​​may undergo processes to mitigate overfitting (also known as "overfitting" or "over-adjustment"), such as dropout. Other preprocessing steps, such as dimensionality reduction and normalization, may also be performed.

[0202] AI may have a network structure such as a CNN (Convolutional Neural Network). Alternatively, the network structure may also have configurations such as an RNN (Recurrent Neural Network) or LSTM (Long Short-Term Memory). In other words, AI may have a network structure other than deep learning.

[0203] Furthermore, the AI ​​may have a configuration that includes hyperparameters. That is, the AI ​​may be configured such that some settings are made by user 101 or the like. In addition, the AI ​​may specify the features to be learned, or user 101 may set some or all of the features to be learned.

[0204] Furthermore, the learning model A1 and the trained model A2 may utilize other machine learning methods. For example, the learning model A1 and the trained model A2 may undergo preprocessing such as normalization using unsupervised models. Moreover, the learning method may be reinforcement learning (a method in which the AI ​​is made to make choices and given evaluations (rewards) for those choices, and the learning method aims to increase the evaluation).

[0205] During training, data augmentation may be performed. That is, to increase the amount of training data D1 used to train the learning model A1, preprocessing may be performed to augment one experimental dataset or similar data into multiple training datasets D1. Increasing the amount of training data D1 in this way allows for further training of the learning model A1.

[0206] Furthermore, the learning model A1 and the trained model A2 may be configured to perform transfer learning or fine tuning. In other words, since the execution environment of the control device 10 often differs from device to device, the settings may differ for each device to match the execution environment. For example, the basic configuration of the AI ​​is learned on a separate information processing device. After that, each information processing device may undergo additional learning or configuration to optimize it for its respective execution environment.

[0207] As described above, AI may be applied to the present invention. For example, a trained model is generated by training a learning model with processes that are executed repeatedly. The training data used for training includes data that indicates the content of the process as the correct answer, and also includes data that indicates the target of the process. A learning model that performs deep learning is trained using such training data.

[0208] Using the trained model generated in this way, when data indicating processing conditions for which the correct answer is unknown is input, the trained model can control the scarfing robot 11 based on the correlations learned from the training data. In this way, AI can be used to realize teaching and other functions. AI may also be applied to various areas such as image recognition or input assistance.

[0209] [Overall processing example] Figure 25 shows an example of the overall processing in the first embodiment. Note that the overall processing is not limited to what is shown below, and other processing may be added.

[0210] In step S01, the control device 10 constructs a simulation environment. For example, the simulation environment 202 is constructed as shown in Figure 4. In constructing the simulation environment 202, parameters such as defect reflection, metal, end effector, robot, obstacles, virtual region, nozzle 15, etc., or scarfing parameters (for example, parameters that affect the quality of scarfing, i.e., the speed at which the nozzle is moved, the angle between the nozzle and the object, the distance between the nozzle and the object, preheating conditions, the mixing ratio of flammable gas and oxygen contained in the gas, and the nozzle structure, etc.) may be set. The simulation environment 202 is constructed with these settings reflected.

[0211] In step S02, the control device 10 performs teaching on the scarfing robot 11. For example, the control device 10 accepts input from the user 101 to the scarfing robot 11, such as translation, rotation, or execution of various processes. Based on such input, teaching data 103 is generated.

[0212] In step S03, the control device 10 simulates the removal state, etc. For example, the simulation result outputs metal 102 with defects removed after scarfing. Various settings such as the simulation time are set in advance. Other options include setting the preheating time, a timeline, simulation using 3D video, or a demonstration operation in the actual environment 201.

[0213] Furthermore, the system may input operations to modify the simulation results. If such modifications are made, the simulation results will be output again, reflecting the modifications.

[0214] Through the overall processing described above, once teaching data 103 is generated, the scarfing robot 11 can be operated in the actual environment 201 based on the teaching data 103. The user 101 can pre-verify how the robot will operate based on the teaching data 103 through simulation.

[0215] Once the simulation environment 202 is established, user 101 can teach the scarfing robot 11 in the simulation environment 202. Therefore, user 101 can perform teaching remotely, or without operating the scarfing robot 11 in the actual environment 201.

[0216] Figure 26 shows an example of the overall processing in the second embodiment. For example, when the "pre-processing" and "execution processing" are executed consecutively, the overall processing is as follows. However, the "pre-processing" and "execution processing" do not have to be executed consecutively.

[0217] "Preprocessing" includes, for example, steps S21 and S22 as follows.

[0218] Step S21 is when the control device 10 receives the learning data D1.

[0219] In step S22, the control device 10 trains the learning model A1 based on the learning data D1.

[0220] For example, steps S21 and S22 are executed as shown in Figure 21. Therefore, when steps S21 and S22 are executed, a trained model A2 is generated. Using the trained model A2 generated by this preprocessing, the "execution process" is performed as follows.

[0221] The "execution process" is, for example, steps S23 to S25 as follows:

[0222] In step S23, the control device 10 receives input data D2.

[0223] In step S24, the control device 10 generates output data D3 using the trained model A2.

[0224] In step S25, the control device 10 causes the scarfing robot 11 to perform scarfing based on the output data D3, or performs a simulation on the simulation environment 202.

[0225] [Example of Functional Configuration] Figure 27 shows an example of the functional configuration of the first embodiment. For example, the control system 100 has a functional configuration that includes an input unit 100F1, a first simulation unit 100F2, a teaching unit 100F3, and a second simulation unit 100F4, etc. However, the control system 100 may have other functions as well.

[0226] The input unit 100F1 performs an input procedure to receive operations from the user 101. For example, the input unit 100F1 can be implemented using an input device 10H3 or the like.

[0227] The first simulation unit 100F2 performs the first simulation procedure to construct the simulation environment 202. For example, the first simulation unit 100F2 is implemented using a CPU 10H1 or the like.

[0228] The teaching unit 100F3 performs a teaching procedure in the simulation environment 202 to generate teaching data 103, etc., that allows the scarfing robot 11 to operate in the real environment, based on the input operations. For example, the teaching unit 100F3 is implemented by a CPU 10H1 or the like.

[0229] The second simulation unit 100F4 performs a second simulation procedure in the simulation environment 202, simulating the removal state and other conditions based on the operation. For example, the second simulation unit 100F4 is implemented by a CPU 10H1 or the like.

[0230] With the above configuration, once the simulation environment 202 is established, user 101 can perform teaching on the scarfing robot 11 in the simulation environment 202.

[0231] Figure 28 shows an example of the functional configuration of the second embodiment. For example, the control system 100 includes a learning device 30 and a control device 10. Hereinafter, we will assume that the learning device 30 performs "preprocessing" to train a learning model A1 and generate a trained model A2. Therefore, the control device 10 is configured to use the trained model A2 generated by the learning device 30. However, the learning device 30 and the control device 10 may have the same information processing device configuration.

[0232] The learning device 30 has a functional configuration that includes a learning data input unit 100F21 and a learning unit 100F22. For example, the learning device 30 is an information processing device, etc., with a hardware configuration similar to that of the control device 10, that is, the hardware configuration shown in Figure 2.

[0233] The learning data input unit 100F21 performs a learning data input procedure to input learning data D1. For example, the learning data input unit 100F21 can be implemented using an input device 10H3 or the like.

[0234] The learning unit 100F22 performs a learning procedure to train the learning model A1 using the learning data D1 and generate the trained model A2. For example, the learning unit 100F22 can be implemented using a CPU 10H1 or the like.

[0235] The control device 10 has a functional configuration that includes an input data input unit 100F23 and a teaching unit 100F3.

[0236] The input data input unit 100F23 performs an input data input procedure to input input data D2. For example, the input data input unit 100F23 can be implemented by an input device 10H3 or the like.

[0237] When input data D2 is received, the teaching unit 100F3 performs a teaching procedure to teach the scarfing robot 11 to operate in the real environment 201 or the simulation environment 202 based on the learned model A2. For example, the teaching unit 100F3 is implemented by a CPU 10H1 or the like.

[0238] With the above configuration, once the simulation environment 202 is established, user 101 can teach the scarfing robot 11 in the simulation environment 202. Therefore, by teaching the scarfing robot 11 remotely, the automation of the scarfing robot 11 can be achieved.

[0239] Furthermore, using AI makes it possible to efficiently simulate the operation of the scarfing robot 11 in the simulation environment 202, or to teach the operation of the scarfing robot 11.

[0240] [Calibration example] The simulation environment 202 and the actual environment 201 may differ in terms of the nozzle 15, target object, obstacles, or surrounding environment. Therefore, it is desirable to perform calibration to match the simulation environment 202 and the actual environment 201 and correct the teaching data 103, etc.

[0241] In scarfing, the initial position and dimensions of objects often differ each time. Therefore, it is desirable to calibrate each object so that its position and dimensions are reflected in the simulation environment 202.

[0242] The position and dimensions of the object are captured by cameras, measuring instruments, drones, etc., or measured by sensors such as touch sensors, and this data is used for calibration.

[0243] For example, the dimensions of the object, or mechanical positional deviations such as the orientation of the nozzle 15, may differ between the simulation environment 202 and the actual environment 201. These differences are detected, for example, by sensors. Specifically, if dimensions are measured by sensors, the dimensions of the object, etc., are corrected in the teaching data 103 based on the measurement results. Furthermore, it is desirable that related processes (for example, the range of scarfing processing, or the range of setting the virtual area) are also corrected in conjunction with this correction.

[0244] Once this calibration is performed, the differences between the simulation environment 202 and the real environment 201 are adjusted, enabling highly accurate teaching.

[0245] [Variations of input devices] For teaching and other purposes, it is desirable that the input device 10H3 be hardware that has positional information and orientation (height H, angle θ, etc.). Specifically, the input device 10H3 may be a combination of a touch panel and a pen-shaped device that the user 101 holds in their hand and moves in three dimensions.

[0246] For scarfing, an input device 10H3 that allows for easy specification of a two-dimensional position is desirable. Therefore, if a point (which will be a coordinate) can be input via a touch panel to indicate the position or the target of processing, operation will be easier.

[0247] Then, when the orientation of the input device 10H3 changes in three dimensions due to the operation of the user 101, the scarf-wing robot 11 performs a synchronized translation or rotation in the real environment 201 or the simulation environment 202. The input device 10H3 measures its position and angle in three dimensions in real time using a position sensor (for example, a gyroscope).

[0248] Furthermore, the input device 10H3 can be activated or deactivated (i.e., switched ON / OFF) by the user 101 pressing a switch.

[0249] Thus, it is desirable that the input device 10H3 has hardware that indicates a three-dimensional position and a switch that indicates the switching of processing. With such hardware, the user 101 can operate it intuitively, improving usability.

[0250] Furthermore, the input device 10H3 is not limited to the above, and it is desirable that a game controller or touch panel be applicable. Thus, it is desirable that the control device 10 be able to support multiple types of input devices 10H3.

[0251] Scarfing teaching involves actions not commonly performed in other work processes. For example, it frequently requires special actions such as maintaining a constant distance from the object while performing scarfing, or performing parallel movement by sliding the object sideways while maintaining the distance. On the other hand, the speed or angle may be changed depending on the processing position. An input device 10H3 that facilitates the operation of instructing such actions is desirable.

[0252] [Other embodiments] This embodiment may also take the following forms.

[0253] [About the metaverse] The virtual space can also be what is known as the metaverse. The term "metaverse" is a combination of "meta" (transcendence) and "universe" (cosmos, world). The metaverse refers to a three-dimensional virtual space on a computer network that can accommodate many participants and allow them to act freely within it.

[0254] In the metaverse, multiple people can participate using avatars, for example. Within the metaverse space, transactions or processing may also occur within a space that utilizes three-dimensional image processing.

[0255] Transactions on the metaverse often utilize arbitrary tokens. These transactions can encompass a wide variety of activities, such as online games, virtual concerts, or e-commerce.

[0256] Furthermore, the metaverse can sometimes be realized using "XR" technologies such as AR (Augmented Reality) or VR (Virtual Reality). In addition, the metaverse can also be realized using technologies such as 3DCG, high-speed communication technology, AI, and blockchain.

[0257] Furthermore, each device does not necessarily have to be a single device. In other words, each device may be a combination of multiple devices.

[0258] The present invention may be implemented by a process for realizing the control method exemplified above, or by a program (including firmware and things equivalent to a program; hereinafter simply referred to as "program") that performs a process equivalent to the process described above.

[0259] In other words, the present invention may be implemented by a program written in a programming language or the like, which issues commands to a computer to obtain a predetermined result. The program may also be configured so that a part of the processing is executed by hardware such as an IC (integrated circuit).

[0260] A program causes the computer to perform the above-mentioned processes by having its arithmetic unit, control unit, and memory device work together. In other words, a program is loaded into main memory, issues commands to the arithmetic unit to perform calculations, and operates the computer.

[0261] Furthermore, the program may be provided on a computer-readable recording medium or via telecommunication lines such as a network.

[0262] The present invention may be implemented in a system composed of a plurality of devices. That is, an information processing system using a plurality of computers may execute the above-described processing in a redundant, parallel, distributed, or a combination thereof. Therefore, the present invention may be implemented in a device other than the above-described hardware configuration and a system other than the above-described device.

[0263] [Contribution to SDGs] The present invention realizes a technique for teaching a surfing robot in a simulation environment. Thereby, it provides a technique for realizing the construction of a resilient infrastructure aimed at Goal "9", promoting inclusive and sustainable industrialization, and promoting innovation, and contributes to the SDGs.

[0264] Note that the present invention is not limited to each of the embodiments exemplified above. Therefore, the present invention can be added or modified in components without departing from the technical gist. Thus, all technical matters included in the technical idea described in the claims are the subject of the present invention. The embodiments exemplified above are suitable specific examples in implementation. And those skilled in the art can realize various modified examples from the disclosed content, and such modified examples are included in the technical scope described in the claims. [Description of Reference Numerals]

[0265] 10: Control device 10H1: CPU 10H2: Storage device 10H3: Input device 10H4: Output device 10H5: Communication device 11: Surfing robot 12: Control panel 14: Upper system 15: Nozzle 16: Movement arrow 21: Virtual robot 30: Learning device 100: Control system 100F1: Input section 100F2: First Simulation Department 100F21: Learning data input section 100F22: Learning Department 100F23: Input data input section 100F3: Teaching Department 100F4: Second Simulation Department 101: User 102: Metal 103: Teaching Data 112: Virtual Metal 151: Oxygen pore 152 :LPG hole 153: Scarf oxygen pores 154: Air vent 155: Slit groove 161: 1st direction component 162: 2nd direction component 201: Real-world environment 202: Simulation Environment 300: Network Configuration 301: Defect area 302: Arrow 303: Scarfing marks A1: Learning Model A2: Pre-trained model D1: Training data D11: Object Information D12: Defect Information D13: Processing details information D2: Input data D20: Correct data D21: Unknown object D22: Unknown defect information D23: Information on unknown processing details D3: Output data D4: Big Data DR1: 1st movement direction DR2: 2nd movement direction H: Height L1: Input layer L2: Middle layer L3: Output layer P1: First passing point P2: Second Passage Point P3: Third Passage Point P4: 4th Passage Point P5: 5th Passage Point P6: 6th Passage Point P7: 7th Passage Point TP: Contact Point V1: The First Imaginary Realm V2: The Second Provisional Realm V3: The Third Virtual Realm V4: The Fourth Virtual Realm θ: angle

Claims

1. A control device that controls a scarfing robot that performs scarfing on an object, and connects to the scarfing robot, An input section for receiving user input, A first simulation unit constructs a simulation environment for performing a simulation of operating the scarfing robot based on the above operation, In the aforementioned simulation environment, when the aforementioned operation is input, a teaching unit performs teaching so that the scarfing robot operates in the actual environment, When scarfing is performed on the object based on the above operation, a second simulation unit simulates the defect removal state in the object, reflecting the processing results by scarfing, in the simulation environment. A control device for a scarfing robot equipped with [a specific feature / feature].

2. The aforementioned object is It is a metal, The scarfing robot is The nozzle has a mechanism for blowing gas onto the object, Heat resistant, The scarfing robot is located remotely from the control device. A control device for a scarfing robot according to claim 1.

3. The first simulation unit is, The scarfing robot is operated by a virtual operating body shown in the simulation environment in accordance with the operation. A virtual region is set in the simulation environment that restricts the movement of the virtual moving object, including translation or rotation. When the virtual domain is set in the simulation environment and it is determined that the virtual entity is located in the virtual domain, When the operation instructs a translation that includes a first directional component which is a directional component in the direction that restricts the virtual operating body within the virtual region, and a second directional component which is a directional component in the direction that aligns the virtual operating body with the surface of the virtual region, The direction in which the virtual moving body is translated is corrected based on the surface of the virtual region, and in the simulation environment, the virtual moving body is translated in the corrected direction. The aforementioned virtual area is A nozzle for heating the object and blowing oxygen into it, the object itself, or an obstacle other than the nozzle, and the objects being arranged so as not to interfere with each other. A control device for a scarfing robot according to claim 1.

4. The first simulation unit is, The scarfing robot is operated by a virtual operating body shown in the simulation environment in accordance with the operation. A virtual permission area is set in the simulation environment that allows operations including translation or rotation of the virtual moving object. A control device for a scarfing robot according to claim 1.

5. Based on the speed at which the nozzle is moved, the angle between the nozzle and the object, the distance between the nozzle and the object, the preheating conditions, the mixing ratio of the flammable gas and low-pressure oxygen contained in the gas used for scarfing, the pressure of the high-pressure oxygen, which is at a higher pressure than the low-pressure oxygen, that is further mixed into the gas, and the scarfing parameters including the nozzle structure, The aforementioned removal state is, The removal area, the depth of removal, the amount of removal, and the shape of removal are simulated and output. A control device for a scarfing robot according to claim 1.

6. The system detects the difference between the simulation environment and the actual environment, and performs calibration to correct the teaching data generated by the teaching unit based on the difference. In the aforementioned real-world environment, the scarfing robot detects the removal state in the object where the defect has been removed, and this is used as the difference. A control device for a scarfing robot according to claim 1.

7. The first simulation unit is, A virtual working body including the scarfing robot and a virtual model including the object are generated, and the simulation environment is constructed by placing the virtual model in a virtual space. When an instruction is given to the input unit to operate the scarfing robot, the virtual operating body in the simulation environment performs a translation or rotation operation. The aforementioned teaching unit is, Based on the results of operating the virtual entity in the aforementioned simulation environment, teaching data is generated. Using the aforementioned teaching data, the scarfing robot operates in the actual environment based on the teaching data, and the simulation environment reproduces the simulation results. A control device for a scarfing robot according to claim 1.

8. The aforementioned defect is, The object is made of metal, These are areas in the object that have formed iron oxide or the like, areas in the object that have impurities mixed in, or areas in the object that are damaged. A control device for a scarfing robot according to claim 1.

9. A learning device connected to a scarfing robot that performs scarfing on an object, A learning data input unit inputs learning data that includes object information indicating the characteristics or properties of the object, defect information relating to defects occurring in the object, processing conditions including the content of scarfing processing performed on the defects, and learning data including evaluation results for processing under the processing conditions as correct data. A learning unit that trains a learning model using the aforementioned training data and generates a trained model that learns the correlation between the processing conditions and the evaluation results. A learning device equipped with the following features.

10. The aforementioned correct data is, The image data includes images of sparks generated when scarfing is performed on the object in a real environment. The learning device according to claim 9.

11. A control device that controls a scarfing robot that performs scarfing on an object, and connects to the scarfing robot, An input data input unit inputs input data including object information that describes the characteristics or properties of the object, defect information regarding defects occurring in the object, and processing conditions including the content of scarfing processing to be performed on the defects. When the aforementioned input data is received, a teaching unit trains a learning model using object information indicating the characteristics or properties of the object, defect information regarding defects occurring in the object, processing conditions including the content of scarfing processing to be performed on the defects, and learning data including evaluation results for processing under the processing conditions as ground truth data. Based on the trained model, which has learned the correlation between the processing conditions and the evaluation results, a teaching unit performs teaching to operate the scarfing robot in a real environment. A control device for a scarfing robot equipped with [a specific feature / feature].

12. A control system comprising a control device that controls a scarfing robot that performs scarfing on an object and is connected to the scarfing robot, and a learning device connected to the control device, The learning device is A learning data input unit inputs learning data that includes object information indicating the characteristics or properties of the object, defect information relating to defects occurring in the object, and a first processing condition including the content of scarfing processing to be performed on the defects, and learning data including evaluation results for processing under the first processing condition as correct answer data. A learning unit that trains a learning model using the aforementioned training data and generates a trained model that learns the correlation between the first processing conditions and the evaluation results. Equipped with, The control device is An input data input unit inputs input data including object information that describes the characteristics or properties of the object, defect information relating to defects occurring in the object, and a second processing condition that includes the content of scarfing processing to be performed on the defects. When the aforementioned input data is received, a teaching unit performs teaching to operate the scarfing robot in a real environment or a simulation environment based on the previously learned model. Equipped with Control system.

13. A control system having a control device that controls a scarfing robot that performs scarfing on an object and is connected to the scarfing robot, The control device is An input section for receiving user input, A first simulation unit constructs a simulation environment for performing a simulation of operating the scarfing robot based on the above operation, In the aforementioned simulation environment, when the aforementioned operation is input, a teaching unit performs teaching so that the scarfing robot operates in the actual environment, When scarfing is performed on the object based on the above operation, a second simulation unit simulates the defect removal state in the object, reflecting the processing results by scarfing, in the simulation environment. A control system equipped with the following features.

14. A control method performed by a control device that controls a scarfing robot that performs scarfing on an object and is connected to the scarfing robot, Input procedure for receiving user input, A first simulation procedure for constructing a simulation environment for performing a simulation of operating the scarfing robot based on the above operation, In the aforementioned simulation environment, when the aforementioned operation is input, a teaching procedure is performed to teach the scarfing robot to operate in the actual environment, When scarfing is performed on the object based on the above operation, a second simulation procedure is performed in the simulation environment to simulate the defect removal state in the object that reflects the processing results of scarfing. A control method including

15. A program for causing a computer to execute the control method described in claim 14.

Images