Anomaly detection method, anomaly handling method, information processing device, welding system, and program

Abstract

To allow for determining abnormality of feature points on image data even under en environment where disturbance occurs.SOLUTION: An abnormality determination method for determining abnormality of information of one or a plurality of feature points extracted from image data includes a calculation step of calculating geometric volume data derived from the information of the one or plurality of feature points, an abnormality detection step of, with respect to time series data constituted of the geometric volume data, using one or plural pieces of predetermined abnormality detection means corresponding to reasons of abnormality to detect abnormality, and a determination step of determining the occurrence of abnormality in the time series data on the basis of detection results of the one or plural pieces of abnormality detection means.SELECTED DRAWING: Figure 12

Description

Technical Field

[0001] The present invention relates to an abnormality determination method, a processing method at the time of abnormality, an information processing apparatus, a welding system, and a program.

Background Art

[0002] In recent years, in the production sites of various industries, attempts have been made to improve productivity and quality by adopting visual sensors and utilizing image recognition technology for the image data obtained by the visual sensors. In image recognition, the features of the captured image data are analyzed by various methods, feature points or feature quantities are extracted from the image, and the image data is identified based on these.

[0003] As an example of a field where productivity has been improved by automation using this image recognition technology, there is the welding field. In such a field, for example, Patent Document 1 can be cited. In Patent Document 1, in a groove extending in the horizontal direction formed between two weldment members arranged in the vertical direction, when the welding progress direction is the forward direction, an arc welding is performed while alternately weaving a welding torch in the forward-downward direction and the rear-upward direction, a camera that photographs the arc and the molten pool generated in the groove by the arc welding, a detection unit that detects the position of the tip of the molten pool in the camera image photographed by the camera, and a determination unit that determines a correction amount of the welding speed based on the distance when the distance between the arc and the tip of the molten pool is within a predetermined range. A system is disclosed that enables automatic welding in the case of single-sided welding and a lateral posture.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In various production environments, disturbances can occur not only in the welding applications disclosed in Patent Document 1, but also in other fields. Examples of disturbances in welding include misalignment of the workpiece, reduction in gas flow rate, magnetic blow, unstable wire feeding, unstable current supply, oil adhesion to the workpiece, rust on the workpiece (hereinafter also referred to as oxide film), and spatter adhesion to the workpiece. As mentioned above, image recognition technology extracts feature points from image data, but taking welding, where disturbances are significant, as an example, the oxide film or spatter adhesion on the workpiece may obscure the subject or part of the subject, making it impossible to recognize feature points, or the feature points may change abruptly.

[0006] Furthermore, disturbances such as a decrease in gas flow rate or magnetic blowing can cause arc instability and molten pool fluctuations, resulting in situations where feature points change abruptly. In this way, if the feature points themselves cannot be recognized due to disturbances, or if the subject changes abruptly and the feature points are misrecognized, it may adversely affect the identification of image data based on feature points. In other words, if it is not possible to distinguish whether the feature points on the image data are unrecognizable or misrecognized, and the image data is identified, it will affect subsequent processing steps. It should be noted that Patent Document 1 does not take such disturbances into consideration at all. Therefore, in the method of Patent Document 1, disturbances may adversely affect the identification of image data, which in turn may affect the amount of correction for the welding speed and make automation difficult.

[0007] Therefore, even if disturbances occur, there is a need for anomaly detection technology that can determine whether feature points on image data are unrecognizable or misrecognized, and for processing the detected anomalies in a way that does not affect the identification of the image data.

[0008] Therefore, one objective of the present invention is to provide an anomaly detection method that can determine anomalies in feature points on image data even in environments where disturbances occur. Furthermore, one objective is to provide a method for handling anomalies determined by the anomaly detection method in a manner that does not affect the identification of image data. Furthermore, one objective is to provide a control device, a welding system, and a program capable of executing the anomaly detection method. [Means for solving the problem]

[0009] To solve the above problems, the present invention has the following configuration. That is, an anomaly detection method for determining anomalies in the information of one or more feature points extracted from image data, A calculation step for calculating geometric quantity data derived from the information of one or more feature points, An anomaly detection step in which anomalies are detected in time series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A determination step of determining the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, It holds.

[0010] Furthermore, the present invention has the following configuration: a processing method for handling abnormalities according to a determination result obtained by an abnormality determination method for determining abnormalities in the information of one or more feature points extracted from image data, The system includes a processing step that, for time-series data, removes data from the period in which an anomaly occurred, corrects it using a predetermined value, or corrects it using the value immediately before the anomaly occurred. The aforementioned abnormality determination method is: A calculation step for calculating geometric quantity data derived from the information of one or more feature points, An anomaly detection step in which anomalies are detected in time series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A determination step of determining the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, It holds.

[0011] Furthermore, the present invention has the following configuration: a processing method for handling abnormalities according to a determination result obtained by an abnormality determination method for determining abnormalities in the information of one or more feature points extracted from image data, An analysis process to analyze patterns of anomalies in time-series data, A step of correcting the setting conditions for the alarm or abnormality reason based on the analysis results in the aforementioned analysis step, It has, The aforementioned abnormality determination method is: A calculation step for calculating geometric quantity data derived from the information of one or more feature points, An anomaly detection step in which anomalies are detected in time series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A determination step of determining the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, It holds.

[0012] Furthermore, another embodiment of the present invention has the following configuration: an information processing device for determining anomalies in the information of one or more feature points extracted from image data, A calculation unit that calculates geometric quantity data derived from the information of one or more feature points, An anomaly detection unit detects anomalies in time-series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A determination unit that determines the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, It holds.

[0013] Also, as another aspect of the present invention, it has the following configuration. That is, a welding system configured to include an information processing device that determines an abnormality in information on one or more feature points extracted from image data, The information processing device, a calculation unit that calculates geometric quantity data derived from the information on the one or more feature points, an abnormality detection unit that detects an abnormality in the time-series data composed of the geometric quantity data using one or more abnormality detection means determined in advance corresponding to the reason for the abnormality, a determination unit that determines the occurrence of an abnormality in the time-series data based on the detection result by the one or more abnormality detection means, and has.

[0014] Also, as another aspect of the present invention, it has the following configuration. That is, a program, causes a computer to, a calculation step of calculating geometric quantity data derived from information on one or more feature points extracted from image data, an abnormality detection step of detecting an abnormality in the time-series data composed of the geometric quantity data using one or more abnormality detection means determined in advance corresponding to the reason for the abnormality, 1] a determination step of determining the occurrence of an abnormality in the time-series data based on the detection result by the one or more abnormality detection means, and execute.

Advantages of the Invention

[0015] According to the present invention, it is possible to determine an abnormality in feature points on image data even in an environment where disturbances occur. Furthermore, based on the determined abnormality, it is possible to perform processing so as not to affect the identification of image data.

Brief Description of the Drawings

[0016] [Figure 1] A schematic diagram showing an example of the system configuration according to an embodiment of the present invention. [Figure 2]A perspective view illustrating the arrangement of a vision sensor according to one embodiment of the present invention. [Figure 3] An example diagram showing an example of image data acquired by a visual sensor according to one embodiment of the present invention. [Figure 4] A block diagram showing an example configuration of a robot control device according to one embodiment of the present invention. [Figure 5] A block diagram showing an example configuration of a data processing device related to one embodiment of the present invention. [Figure 6] A conceptual diagram illustrating the learning process related to one embodiment of the present invention. [Figure 7] A schematic diagram illustrating an example of a screen used in teaching operations according to one embodiment of the present invention. [Figure 8] A schematic diagram illustrating an example of a screen used in teaching operations according to one embodiment of the present invention. [Figure 9] An example diagram illustrating the analysis results of a welding image relating to one embodiment of the present invention. [Figure 10] An example diagram illustrating the analysis results of a welding image relating to one embodiment of the present invention. [Figure 11] An example diagram illustrating the analysis results of a welding image relating to one embodiment of the present invention. [Figure 12] Flowchart of the abnormality detection process related to one embodiment of the present invention. [Figure 13] A graph illustrating an example of time-series data related to one embodiment of a civil engineering invention. [Figure 14] A graph showing an example of a judgment result related to one embodiment of cash. [Figure 15] A graph showing an example of a judgment result related to one embodiment of cash. [Figure 16] Sequence diagram of robot control processing related to one embodiment of the present invention. [Modes for carrying out the invention]

[0017] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings and other materials. It should be noted that the embodiments described below are merely one embodiment for illustrating the present invention and are not intended to be interpreted as limiting the present invention, nor are all configurations described in each embodiment necessarily essential for solving the problems of the present invention.

[0018] In this embodiment, the invention is most effective in the welding field, where there are many disturbance factors and the effects of disturbances are significant. Therefore, the explanation will focus on welding applications, but the field to which the invention applies is not particularly limited. For example, cutting and additive manufacturing fields are also included. Furthermore, while this embodiment shows an example using a 6-axis robot system, the number of axes of the robot is not limited, and it may also be a portable welding robot, a welding device with a drive unit such as a trolley, or a semi-automatic welding system. In addition, in each drawing, the same components are given the same reference numeral to indicate their correspondence.

[0019] As described later, in this embodiment, a learning device is used to extract feature points of an arbitrary object from image data. In the following description of the learning device, "learning" or "machine learning" refers to generating a "trained model" by performing learning using learning data and an arbitrary learning algorithm. The trained model is updated as learning progresses using multiple learning data, and its output changes even with the same input. Therefore, the trained model is not limited to any particular point in time. Here, the model used in learning is referred to as the "learning model," and the learning model that has undergone a certain degree of learning is referred to as the "trained model." Furthermore, specific examples of "learning data" will be described later, but its composition may be changed depending on the learning algorithm used. In addition, the learning data may include training data used for learning itself, validation data used to validate the trained model, and test data used to test the trained model. In the following description, when comprehensively referring to data related to learning, it is referred to as "learning data," and when referring to the data used when performing learning itself, it is referred to as "training data." It should be noted that this is not intended to clearly classify the training data into training data, validation data, and test data; for example, depending on the training, validation, and testing methods, all of the training data may also serve as training data.

[0020] [Welding system configuration] Figure 1 shows an example of the configuration of a welding system 1 according to this embodiment. The welding system 1 shown in Figure 1 includes a welding robot 10, a robot control device 20, a power supply device 30, a vision sensor 40, and a data processing device 50. As described above, when applying the features of the present invention to a portable welding robot, a welding device having a drive unit such as a trolley, or semi-automatic welding, further configurations may be included in accordance with their respective configurations.

[0021] The welding robot 10 shown in Figure 1 is composed of a 6-axis articulated robot, and a welding torch 11 for GMAW is attached to its tip. GMAW includes, for example, MIG (Metal Inert Gas) welding and MAG (Metal Active Gas) welding, and in this embodiment, MAG welding will be used as an example. Furthermore, the welding robot 10 is not limited to a 6-axis articulated robot; for example, a portable small robot may be used. Examples of portable small robots include orthogonal robots with 3 or fewer axes.

[0022] The welding torch 11 is supplied with welding wire 13 from the wire feeder 12. The welding wire 13 is fed from the tip of the welding torch 11 toward the welding area. The power supply unit 30 supplies power to the welding wire 13. This power applies an arc voltage between the welding wire 13 and the workpiece W, generating an arc. In this embodiment, single-sided welding is assumed, and the workpiece W is made of steel plates joined together, with a backing material placed on the back side, i.e., the side opposite to the welding surface. The power supply unit 30 is equipped with a current sensor (not shown) for detecting the welding current flowing from the welding wire 13 to the workpiece W during welding, and a voltage sensor (not shown) for detecting the arc voltage between the welding wire 13 and the workpiece W.

[0023] The power supply unit 30 has a processing unit and a memory unit (not shown). The processing unit is composed of, for example, a CPU (Central Processing Unit). The memory unit is composed of, for example, volatile or non-volatile memory such as an HDD (Hard Disk Drive), ROM (Read Only Memory), or RAM (Random Access Memory). The processing unit controls the power applied to the welding wire 13 by executing a computer program for power control stored in the memory unit. The power supply unit 30 is also connected to the wire feeder 12, and the processing unit controls the feeding speed and amount of the welding wire 13. The composition and type of welding wire 13 are selected according to the object to be welded.

[0024] The visual sensor 40 is composed of, for example, a CCD (Charge Coupled Device) camera. The placement of the visual sensor 40 is not particularly limited; it may be directly attached to the welding robot 10, or it may be fixed in a specific location in the surrounding area as a surveillance camera. When the visual sensor 40 is directly attached to the welding robot 10, the visual sensor 40 moves in accordance with the operation of the welding robot 10 to photograph the area around the tip of the welding torch 11. The number of cameras that make up the visual sensor 40 may be multiple. For example, the visual sensor 40 may be composed of multiple cameras with different functions and installation locations.

[0025] Furthermore, the direction in which the visual sensor 40 captures images is not particularly limited. For example, if the direction in which welding is progressing is forward, the sensor may be positioned to capture the front side, or it may be positioned to capture the side or rear side. Therefore, the shooting range of the visual sensor 40 may be determined as appropriate. In order to suppress interference from the welding torch 11, it is preferable to capture from the front side, and in this embodiment, the image is captured from the front side. The captured images are transmitted to the data processing device 50 and used by the data processing device 50. At this time, the data processing device 50 may, for example, select arbitrary images from the captured images at predetermined intervals and use them for processing described later. The method of selection and selection settings here may be switched according to, for example, the configuration and function of the visual sensor 40 and the performance of the data processing device 50.

[0026] In this embodiment, a visual sensor 40 is directly attached to and fixed to the welding robot 10. A moving image is captured as a welding image, such that the shooting range includes at least the workpiece W, the welding wire 13, and the arc as objects (targets) to be included in the image data. Various shooting settings related to the welding image may be predetermined or switched according to the operating conditions of the welding system 1. Examples of shooting settings include frame rate, number of pixels in the image, resolution, and shutter speed.

[0027] Each component of the welding system 1 is connected via various wired and wireless communication methods. The communication method is not limited to one; multiple methods may be combined for connection.

[0028] Figure 2 is a perspective view illustrating the placement of the visual sensor 40. In this embodiment, the workpiece W is a butt joint. The workpiece W consists of two metal plates butted together with a groove in between. A ceramic backing material 14 is attached to the back side of the two butted metal plates. A metal backing material may be used on the back side, or there may be no backing material at all. Therefore, the material of the backing material is not particularly limited and may vary depending on the material of the workpiece W, etc. In a butt joint, arc welding is performed in one direction along the groove. Hereinafter, the direction in which the welding progresses will be referred to as the "welding direction". In Figure 2, the direction in which the welding progresses is indicated by an arrow. For this reason, the welding torch 11 is located behind the visual sensor 40.

[0029] In this embodiment, the workpiece W is positioned horizontally so that the surface to be welded faces vertically upward. Therefore, the welding robot 10 welds the workpiece W from above. As shown in Figure 2, the visual sensor 40 may be positioned diagonally above the welding position of the workpiece W. Note that Figure 2 shows an example of downward welding as the welding position, but it is not limited to this. For example, in the case of other positions such as sideways welding, the orientation and position of the workpiece W and welding torch 11 are changed as appropriate.

[0030] Figure 3 shows an example of image data captured by the visual sensor 40. In the two-dimensional coordinate system shown in Figure 3, the X-axis indicates the welding direction, and the Y-axis indicates the direction perpendicular to the X-axis. As shown in Figure 3, the shooting range of the visual sensor 40 includes the welding position of the workpiece W, and it captures images of the welding position during arc welding. This image data includes the molten pool, welding wire 13, and arc. In this embodiment, the visual sensor 40 can capture still images of, for example, 1024 × 768 pixels in succession. In other words, the visual sensor 40 can capture welding images as moving images. The resolution of still images that can be captured by the visual sensor 40 is not particularly limited. For example, if the visual sensor 40 is composed of multiple cameras, each of the multiple cameras may acquire welding images with different resolutions. Also, before inputting into the trained model described later, preprocessing such as cutting out arbitrary feature regions from the captured welding images may be performed for the purpose of reducing processing time. The arbitrary feature region may be a fixed-size range arranged so that a predetermined area is centered. The size of the arbitrary feature region may also be changed according to the welding conditions.

[0031] [Robot control system configuration] Figure 4 shows an example configuration of a robot control device 20 that controls the operation of a welding robot 10. The robot control device 20 consists of a CPU 201 that controls the entire device, a memory 202 that stores data, an operation panel 203 that includes multiple switches, a teaching pendant 204 used for teaching operations, a robot connection unit 205, and a communication unit 206. The memory 202 is composed of volatile or non-volatile storage devices such as ROM, RAM, and HDD. The memory 202 stores a control program 202A used to control the welding robot 10. The CPU 201 controls various operations of the welding robot 10 by executing the control program 202A.

[0032] Instructions for the robot control device 20 can be input using the operation panel 203 and the teaching pendant 204, with the teaching pendant 204 being primarily used. The teaching pendant 204 is connected to the main body of the robot control device 20 via the communication unit 206. The operator can input a teaching program using the teaching pendant 204. The robot control device 20 controls the welding robot 10 according to the teaching program input from the teaching pendant 204. The teaching program can also be automatically created using, for example, a computer (not shown) based on CAD (Computer-Aided Design) information. The actions defined in the teaching program are not particularly limited and may vary depending on the specifications of the welding robot 10 and the welding method.

[0033] The drive circuit of the welding robot 10 is connected to the robot connection unit 205. The CPU 201 outputs control signals based on the control program 202A to the drive circuit (not shown) of the welding robot 10 via the robot connection unit 205. The communication unit 206 is configured to include a communication module for wired or wireless communication. The communication unit 206 is used for data and signal communication with the power supply unit 30, data processing unit 50, teaching pendant 204, etc. The communication method and standard used in the communication unit 206 are not particularly limited, and multiple methods may be combined, or they may differ for each connected device. From the power supply unit 30, for example, the current value of the welding current detected by a current sensor (not shown) and the voltage value of the arc voltage detected by a voltage sensor (not shown) are provided to the CPU 201 via the communication unit 206.

[0034] The robot control device 20 controls the movement speed and protrusion direction of the welding torch 11 by controlling each axis of the welding robot 10. Furthermore, when performing a weaving motion, the robot control device 20 also controls the weaving motion of the welding robot 10 according to the set period, amplitude, and welding speed. Weaving motion refers to the alternating oscillation of the welding torch 11 in a direction intersecting the welding direction. Along with the weaving motion, the robot control device 20 performs welding line tracing control. Welding line tracing control is the operation of controlling the left and right position of the welding torch 11 with respect to the direction of travel so that a bead is formed along the welding line. The robot control device 20 also controls the feeding speed of the welding wire 13 by controlling the wire feeder 12 via the power supply device 30.

[0035] [Data Processing Unit Configuration] Figure 5 is an explanatory diagram illustrating an example configuration of a data processing device 50. The data processing device 50 is composed of, for example, a computer. The computer is composed of a main unit 510, an input unit 520, and a display unit 530. The main unit 510 is composed of a CPU 511, a GPU (Graphical Processing Unit) 512, a ROM 513, a RAM 514, a non-volatile storage device 515, an input / output interface 516, a video output interface 517, a communication interface 518, and a calculation unit 519. The CPU 511, GPU 512, ROM 513, RAM 514, non-volatile storage device 515, input / output interface 516, video output interface 517, communication interface 518, and calculation unit 519 are connected to each other so as to be able to communicate with each other by buses or signal lines.

[0036] The non-volatile storage device 515 stores a learning program 515A that performs deep learning using predetermined training data, a trained model 515B generated through the execution of the learning program 515A, an information generation program 515C that generates welding information related to welding using the trained model 515B, and image data 515D. In addition, the non-volatile storage device 515 also has an operating system and application programs installed.

[0037] The data processing unit 50 implements various functions through the execution of programs by the CPU 511 and GPU 512. In this embodiment, the data processing unit 50 implements the function of generating a trained model by machine learning and the function of performing various processes during actual welding using the trained model. The details of these functions will be described later. The data processing unit 50 may be divided into two parts to match the function of generating a trained model and the function of performing control processing based on information output from the trained model during actual welding. From the standpoint of versatility, it is more preferable to configure the data processing unit 50 by dividing it into multiple devices according to each function. The GPU 512 is used as the arithmetic unit when executing the learning program 515A and the information generation program 515C. The ROM 513 stores the BIOS (Basic Input Output System) and the like that are executed by the CPU 511. The RAM 514 is used as the working area for programs read from the non-volatile storage device 515.

[0038] The input / output interface 516 is connected to the input unit 520, which consists of a keyboard, mouse, etc. The vision sensor 40 is also connected to the input / output interface 516. Image data output from the vision sensor 40 is provided to the CPU 511 and GPU 512 via the input / output interface 516. The communication interface 518 is a communication module for wired or wireless communication. The video output interface 517 is connected to the display unit 530, which consists of, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and outputs a video signal to the display unit 530 according to the video data provided by the CPU 511. The calculation unit 519 works in cooperation with the CPU 511 and GPU 512 to perform various processes such as calculating geometric quantity data and anomaly detection processing according to this embodiment. Details of the processing of the calculation unit 519 will be described later.

[0039] [Generating a Learning Model] The following describes the feature points extracted from image data and the trained model used to extract these feature points in this embodiment. Figure 6 is a diagram conceptually illustrating the process of generating the trained model 515B through the training process. The trained model 515B in this embodiment is composed of a convolutional neural network and includes multiple convolutional layers and multiple pooling layers. Note that the configuration of the convolutional neural network is not limited to the above, and the number of layers and configuration may be different.

[0040] The trained model 515B takes image data output from the visual sensor 40 as input and outputs feature points related to various welding information appearing in the image data. In this embodiment, the image data input to the trained model 515B includes, as shown in Figure 3, at least the molten pool, welding wire 13 (see Figure 1), and arc as objects (targets), and feature points obtained from each of these objects, or from between multiple objects, are extracted. Based on the extracted feature points, welding information such as arc stability, deposition amount, arc tracking condition, or penetration depth can be obtained in real time. This image data may also be referred to as a welding image below.

[0041] In this embodiment, the feature points related to welding information include the tip of the welding wire 13 (wire tip), the center point of the arc (arc center), the positions of the left and right (or top and bottom) ends of the molten pool, and the positions of the left and right (or top and bottom) ends of the molten pool. The input of feature points to be used as training data is performed by the operator specifying a specific position on the welding image according to the instructions on the operation screen that supports the teaching work. Therefore, the training data is composed of pairs of the welding image and the coordinate information of the feature points specified by the operator. In the learning process, the feature points output from the learning model are compared with the feature points included in the training data, and the parameters are adjusted by feeding back the error. Learning progresses by repeating this process.

[0042] Figure 7 illustrates an example of a screen used for teaching. The welding image shown in Figure 7 includes the molten pool 15, welding wire 13, and arc 16. In Figure 7, the molten pool 15 is shown with shading.

[0043] Figure 8 is an explanatory diagram showing a specific example of a welding image obtained by welding, and an example of welding information within that welding image. For the sake of clarity, the positions corresponding to the coordinates indicated by the feature points are drawn on the welding image. As mentioned above, the image has coordinates and is a coordinate plane consisting of two axes, the X and Y axes.

[0044] In this embodiment, the visual sensor 40 is installed so that the weld line direction and the X-axis direction are parallel; therefore, in this embodiment, the X-axis direction may be referred to as the weld line direction. Also, the Y-axis direction is perpendicular to the X-axis, or in other words, it is the groove width direction which is perpendicular to the weld line; therefore, the Y-axis direction may be referred to as the groove width direction.

[0045] In this embodiment, the operator is taught the following feature points: the coordinate position of the arc center (ArcX, ArcY), the coordinate position of the wire tip (WireX, WireY), the coordinate position of the left end of the molten pool (Pool_Lead_Lx, Pool_Lead_Ly), the coordinate position of the right end of the molten pool (Pool_Lead_Rx, Pool_Lead_Ry), the coordinate position of the left edge of the molten pool (Pool_Ly), and the coordinate position of the right edge of the molten pool (Pool_Ry). Feature point input is performed by the operator indicating a specific position on the screen. The coordinates that define the boundary between the welding wire 13 and the arc are an example of the wire tip position coordinates. The left end of the molten pool, the right end of the molten pool, the left edge of the molten pool, and the right edge of the molten pool are examples of feature points related to the behavior of the molten pool 15. For example, if the feature points of the left edge and the right edge of the molten pool are known, the width of the molten pool 15 can be calculated as a geometric quantity. Note that Figure 7 uses downward welding as an example of the welding position. Therefore, depending on the welding position and the orientation of the image, "left edge" and "right edge" may be replaced with "top edge" and "bottom edge," etc.

[0046] [Time-series data] In this embodiment, the difference in the X direction between one of the predetermined molten pool tip positions and the wire tip position (hereinafter referred to as "LeadX") is calculated, and sampling is performed at predetermined intervals. In addition, the difference in the Y direction between the left and right molten pool tips (hereinafter referred to as "LeadW") is calculated, and sampling is performed at predetermined intervals. This sampled data will also be referred to as time-series data below. The number of samples per unit time is not particularly limited, but in this embodiment, it is similar to the frame rate of a video. In this embodiment, calculated values ​​such as LeadX and LeadW that constitute the time-series data will also be referred to as geometric quantity data.

[0047] The analysis will be explained in detail based on the welding image in Figure 9. Using the feature points related to the welding information output from the trained model, the coordinate positions of the left end of the molten pool (Pool_Lead_Lx, Pool_Lead_Ly), the right end of the molten pool (Pool_Lead_Rx, Pool_Lead_Ry), and the wire tip (WireX, WireY) are used, with LeadX representing the right end of the molten pool. In this case, the calculation unit 519 calculates LeadX as the difference in the X-axis direction between the coordinate position of the right end of the molten pool and the wire tip ("Pool_Lead_Rx" - "WireX"). The calculation unit 519 also calculates LeadW as the difference in the Y-axis direction between the coordinate position of the left end of the molten pool and the right end of the molten pool ("Pool_Lead_Ry" - "Pool_Lead_Ly"). The distance calculated may be in pixels or converted to any unit such as "mm" or "cm". In the example shown in Figure 9, LeadX is detected as 5.2 mm and LeadW as 5.0 mm.

[0048] By organizing the geometric quantity data, LeadX and LeadW, in a time series, the normality of the weld can be determined. As will be described later, if it is determined to be unsteady, the welding speed is corrected and automatic control is performed to return to a steady state.

[0049] In this embodiment, LeadX and LeadW were calculated as examples of geometric quantity data, and the welding accuracy was determined from the time-series data. However, the invention is not limited to these geometric quantity data. For example, the geometric quantity data constituting the time-series data may be (1) the coordinate positions of one or more feature points obtained from a single object, (2) the distance between any multiple feature points, or (3) the area formed by any multiple feature points. Alternatively, the geometric quantity data constituting the time-series data may be (4) the distance between feature points of different objects obtained from different objects.

[0050] (1) More specifically, in the case of the coordinate positions of one or more feature points, if a single object is an arc, geometric quantity data of ArcX, which is the X-axis coordinate position of the arc center, and ArcY, which is the Y-axis coordinate position, can be calculated, and the stability of the arc can be determined from the time series data. (2) The distance between any multiple feature points can be given as geometric quantity data, such as LeadW obtained from the molten pool object. (3) In the case of an area formed by any multiple feature points, for example, the area of ​​the molten pool can be used as geometric quantity data, and this can be used to evaluate the amount of weld. (4) The distance between the feature points of each object can be given as geometric quantity data such as LeadX obtained from different objects such as the welding wire and the molten pool, the difference between the wire tip position and the molten pool center position, and the distance between the groove and the molten pool front. Time series data consisting of geometric quantity data consisting of the distance between the wire tip position and the molten pool center position can be applied to weld line tracing, and time series data consisting of geometric quantity data consisting of the distance between the groove and the molten pool front can be applied to penetration determination.

[0051] Figures 10 and 11 are diagrams illustrating the analysis results of welding images according to this embodiment. Figure 10 shows an example where there are no abnormalities, and Figure 11 shows an example where there are abnormalities. In Figure 10, image 1000 shows an image of the area around the molten pool. Image 1010 shows the results of the geometric quantity data calculation process. In image 1010, as a result of the above processing, feature point 1011 indicating the tip of the welding torch 11, feature point 1012 indicating the top of the molten pool, and feature point 1013 indicating the bottom of the molten pool are identified. Furthermore, LeadX and LeadW are derived based on these feature points. Parameter 1014 shows the calculated values ​​of LeadX and LeadW, where LeadX = 4.0 [mm] and LeadW = 6.6 [mm] are calculated.

[0052] In Figure 11, Image 1100 shows an image of the area around the molten pool. Here, Image 1100 includes an object 1101 that corresponds to an oxide film (equivalent to rust on the workpiece, etc.). Image 1110 shows the results of the geometric quantity data calculation process. In Image 1110, as a result of the above process, feature point 1112 indicating the tip of the welding torch 11 and feature point 1113 indicating the area below the tip of the molten pool are identified. However, due to object 1111 corresponding to the oxide film, feature points corresponding to the area above the tip of the molten pool cannot be detected. Therefore, LeadX and LeadW are not derived. Consequently, the parameter 1114 does not display the values ​​of LeadX and LeadW. In such cases, it should be treated as if some kind of abnormality has occurred.

[0053] [Method for detecting abnormalities and methods for handling abnormalities] Taking welding as an example, potential disturbances include misalignment of the workpiece, decreased gas flow rate, magnetic blow, unstable wire feeding, unstable current supply, oil adhesion to the workpiece, oxide film, and spatter adhesion to the workpiece. For example, oxide film and spatter adhesion on the workpiece can obscure the object or part of the object, as shown in Figure 11, making it impossible to recognize feature points, or causing feature points to change abruptly. Therefore, geometric quantity data calculated based on feature points under such conditions may be flawed. Using such data could negatively impact the entire automatic control system.

[0054] In this embodiment, geometric quantity data is calculated, and anomaly detection is performed based on time series data composed of geometric quantity data, and locations corresponding to anomalies in the time series data are identified. Then, corrections to welding conditions, etc., are made based on the time series data other than the locations determined to be anomaly, thereby realizing automatic control that is robust to disturbances. The anomaly detection method according to this embodiment will be described in detail below.

[0055] The anomaly detection method will be explained using Figure 12. Figure 12 is a flowchart showing the process from extracting feature points from a welding image using the trained model 515B, performing anomaly detection, and outputting a correction signal for automatic control considering the nature of the anomaly.

[0056] The following processes are realized by each processing unit of the data processing device 50 reading and executing various programs stored in the non-volatile memory device 515, etc. This processing flow also starts simultaneously with the start of welding, and at the same time, image capture by the visual sensor 40 also begins.

[0057] In S1201, the data processing device 50 receives welding images captured from the vision sensor 40 while automatic welding is being performed.

[0058] In S1202, the data processing device 50 performs image processing as a preprocessing step for the welding image received in S1201. Examples of image processing include reducing the image size or converting it to a grayscale image. Further processing may be performed, or some processing may be omitted depending on the processing load.

[0059] In S1203, the data processing device 50 inputs the welding image preprocessed in S1202 into the trained model described above and obtains feature points as welding information that are output as a result. In this embodiment, the coordinates of the arc center (ArcX, ArcY), the wire tip (WireX, WireY), the left end of the molten pool (Pool_Lead_Lx, Pool_Lead_Ly), the right end of the molten pool (Pool_Lead_Rx, Pool_Lead_Ry), the left edge of the molten pool (Pool_Ly), and the right edge of the molten pool (Pool_Ry) may be output.

[0060] In S1204, the data processing device 50 calculates geometric quantity data based on the feature points output by the trained model. In this embodiment, the geometric quantity data "LeadX" calculated as the difference in the X-axis direction between the coordinate position of the right end of the molten pool and the coordinate position of the wire tip ("Pool_Lead_Rx" - "WireX") and the geometric quantity data "LeadW" calculated as the difference in the Y-axis direction between the coordinate position of the left end of the molten pool and the coordinate position of the right end of the molten pool are used.

[0061] In S1205, the data processing device 50 performs anomaly detection using time-series data composed of geometric quantity data "LeadX" and time-series data composed of geometric quantity data "LeadW". The means for anomaly detection can be set in advance. There are many types of disturbances, and there are various reasons for misrecognition or failure to recognize feature points. To address this problem, the inventors have identified multiple reasons for the occurrence of anomalies in feature points in advance and have provided multiple detection means corresponding to each reason. Then, the anomaly is determined using the multiple detection means corresponding to the reason for the occurrence of the anomaly. As a result, it has been found that anomalies in feature points caused by any disturbance can be determined with high accuracy.

[0062] In this embodiment, as a means for detecting "abnormal reasons for feature point recognition failure," (1) a means for detecting an anomaly when the detection rate within a predetermined interval (time) is below a threshold is used. In this detection rate, as shown in Figure 11, if geometric quantity data cannot be calculated when a welding image is input, it is counted as a welding image that has not been detected. Furthermore, as a means for detecting "abnormal reasons for feature point misrecognition," (2) a means for detecting an anomaly when it is outside the range defined by a predetermined outlier identification method, and (3) a means for detecting an anomaly when the feature point positions differ (are far apart) by more than a predetermined threshold between adjacent image data frames are used. Note that the configuration may employ at least one of these means. (2) A predetermined outlier identification method is, for example, a method that detects an anomaly when the Hampel identifier falls outside the applicable range of the 3σ method. Note that the above multiple anomaly detection means are just examples, and other means may be used. Therefore, the combination of anomaly detection means is not limited to the above. Furthermore, from the viewpoint of accuracy in anomaly detection, it is preferable to employ multiple detection means, and it is even more preferable to employ a combination of two or more of the above-mentioned detection means (1) to (3).

[0063] In S1206, the data processing device 50 sets an anomaly graph based on the anomaly information detected by each anomaly detection means in S1205. In this embodiment, the value of the anomaly flag is set to ON for intervals determined to be anomaly by at least one of the multiple anomaly detection means, and an ON signal is output. When the anomaly flag is ON, it indicates that an anomaly has occurred in the welding image corresponding to that timing.

[0064] In S1207, the data processing device 50 performs a process to remove data from the ON signal interval of the abnormal flag. Note that the removal of data from the abnormal interval is just one example, and interpolation processing may be performed, such as replacing the data from the ON signal interval of the abnormal flag with a predetermined value or the median value within a predetermined range. Alternatively, when performing interpolation processing, the value removed may be interpolated with the value immediately preceding the ON signal interval of the abnormal flag, or the value to be interpolated may be switched depending on the length of the ON signal interval. Furthermore, in order to reduce minute noise in the data after the data has been removed, the data processing device 50 applies a smoothing filter such as a moving average filter to perform smoothing processing. By performing this smoothing processing, the accuracy of automatic control is further improved. Note that the smoothing processing may be omitted depending on the result of the data removal. In addition, the content of the filtering process using the predetermined filter is not particularly limited and may differ depending on the content of the data removal process and interpolation process.

[0065] In S1208, the data processing device 50 calculates a correction signal based on the time-series data from which abnormal intervals were removed in S1207. The calculation method here is not particularly limited, but for example, a correction signal indicating a correction amount related to welding speed, welding voltage, etc., may be calculated based on a predetermined rule.

[0066] In S1209, the data processing device 50 outputs a correction signal calculated based on the disturbance determination information to the robot control device 20.

[0067] In S1210, the data processing device 50 determines whether or not it has received a stop command from the robot control device 20. The stop command here refers to a welding stop command transmitted from the robot control device 20. If no stop command has been received (NO in S1210), the data processing device 50 returns to S1201. On the other hand, if a stop command has been received (YES in S1210), this processing flow is terminated.

[0068] The use of the abnormal flag is not limited to the data removal described above. The abnormal flag pattern may be analyzed to predict the reason for the abnormality or the type of disturbance based on the duration and frequency of the abnormal flag's ON or OFF signals. Furthermore, based on the analysis results, an alarm (error issuance) may be issued, or various setting conditions may be corrected. For example, if the number of times the abnormal flag is ON signals exceeds a predetermined threshold within a predetermined period, it may be determined that large spatter particles are occurring frequently, and an alarm may be issued to stop welding. In addition, if it is determined that large spatter particles are occurring frequently, control measures such as increasing the arc voltage setting may be taken to stabilize the arc.

[0069] [Example of abnormality detection] Below, specific examples of abnormality detection according to this embodiment will be explained using Figures 13 to 15.

[0070] Figure 13 shows the time series data of the geometric quantity data LeadW, which is used as input data in the anomaly detection process. Note that the time series data at this point is raw data that has not been edited or otherwise processed. In Figure 13, the horizontal axis represents time [s] and the vertical axis represents the value of LeadW [mm]. Figure 14 shows the data after the range of data determined to be anomaly has been removed as a result of the anomaly detection process described above. In Figure 14, the horizontal axis represents time [s] and the vertical axis represents the value of LeadW [mm]. Figure 15 shows the time series data of the anomaly flag value in the anomaly detection process described above. In Figure 15, the horizontal axis represents time [s] and the vertical axis represents the value of the anomaly flag (0 or 1). A value of 1 for the anomaly flag corresponds to the ON signal described above. In Figures 13 to 15, the time on the horizontal axis corresponds to the time.

[0071] According to Figures 13 to 15, in the range where the abnormal flag value in Figure 15 is 1, the values ​​of the time series data shown in Figure 13 are removed, and the data in the entire range is smoothed, resulting in the time series data shown in Figure 14.

[0072] [Welding control method] The welding operation of welding system 1 during actual welding will be described below. When performing arc welding, the operator starts the robot control device 20, the power supply device 30, and the data processing device 50. The robot control device 20 controls the movement of the welding robot 10, and the welding robot 10 performs welding. The data processing device 50 receives the welding image captured by the vision sensor 40 and sequentially outputs characteristic points related to the arc welding. In this embodiment, the welding information is defined as the position of the wire tip, the arc center, the right and left ends of the molten pool tip, and the right and left ends of the molten pool.

[0073] Figure 16 is a flowchart illustrating the processing operations of the robot control device 20 and the power supply device 30. When the operator starts arc welding, they operate the teaching pendant 204 provided by the robot control device 20 to input the teaching program to be applied and various setting values ​​to the robot control device 20. The teaching program here is defined as a pre-programmed program that has been taught the movements of the welding robot 10 and the welding start and welding end instructions. In this embodiment, the processing sequence in Figure 16 and the processing flow in Figure 12 are executed simultaneously.

[0074] In S1601, the robot control device 20 receives the teaching program and various setting value instructions.

[0075] In S1602, the robot control device 20 moves the robot to a predetermined welding start position after the teaching program has started and commands the power supply unit 30 to start welding (arc on). Note that sensing may be performed before the welding program starts to correct the settings of the teaching program.

[0076] In step S1603, the power supply unit 30 receives a command from the robot control device 20 to start welding.

[0077] In step S1604, the power supply unit 30 controls the built-in power supply circuit (not shown) to supply power and start welding. As a result, a voltage is applied between the welding wire 13 (see Figure 1) and the workpiece W (see Figure 1), and an arc is generated at the welding start position.

[0078] In S1605, the robot control device 20 transmits a control signal to the power supply unit 30 or the welding robot 10 to perform welding control. The welding control includes, for example, automatic welding control (S1620), weaving motion control (S1621), and welding line tracing control (S1622). In automatic welding control, the data processing device 50 automatically moves the welding torch 11 in the welding direction and transmits a correction signal to the welding robot 10 or power supply unit 30 to control at least one of the welding speed, welding current, or arc voltage, and the welding robot 10 or power supply unit 30 performs welding according to the correction signal. From the viewpoint of ease of control, it is preferable to include welding speed control in automatic welding control, but in this embodiment only welding speed control is performed.

[0079] In S1606, the robot control device 20 determines whether or not it is necessary to stop welding. For example, it may determine that it is necessary to stop welding if it receives a welding stop instruction from the operator, detects the welding end position by the teaching program, or detects a welding abnormality. If it is not necessary to stop welding (NO in S1606), the robot control device 20 proceeds to S1607. On the other hand, if it is necessary to stop welding (YES in S1606), the robot control device 20 proceeds to S1608.

[0080] In S1607, the robot control device 20 receives welding information from the data processing device 50. The welding information received here is a correction signal calculated based on data obtained by the data processing device 50 using a trained model to output feature points, and then performing anomaly detection and anomaly removal processing on time-series data based on these feature points. Details of the generation of the welding information received in this process are as described in Figure 12 above. After that, the process returns to S1605, and the robot control device 20 repeats the processing using the received welding information.

[0081] At S1608, the robot control device 20 stops the welding control.

[0082] In S1609, the robot control device 20 commands the power supply device 30 to stop welding. The welding is stopped by cutting off the supply of welding power.

[0083] In S1610, the robot control device 20 commands the data processing device 50 to stop generating welding information.

[0084] In step S1611, the power supply unit 30 receives a command to stop welding from the robot control device 20.

[0085] In step S1612, the power supply unit 30 controls the power supply circuit using a CPU (not shown) to stop welding. This terminates the operation of the robot control device 20 and the power supply unit 30.

[0086] As described above, this embodiment makes it possible to determine anomalies in feature points on image data even in environments where disturbances occur. Furthermore, it becomes possible to process the detected anomalies in a way that does not affect the identification of the image data.

[0087] <Other Embodiments> In the above embodiment, as shown in Figure 6, feature points are acquired using a trained model, geometric quantity data is calculated on the calculation unit 519 side, and anomaly detection processing is performed from the time series data of the geometric quantity data. However, the system is not limited to this configuration, and for example, a configuration in which a trained model is generated by performing training processing so that the trained model can also perform anomaly detection may be used. In this case, the training data may be configured to include label information indicating the anomaly detection result in the training processing. The label information here may be an anomaly flag or a classification indicating the cause of the anomaly. With this configuration, the trained model can identify the location where anomalies occurred and the cause of the anomaly in a series of welding images by outputting label information for the input welding image. The calculation unit 519 may be configured to output a correction signal based on the label information, which is the anomaly detection result output from the trained model. Alternatively, a trained model capable of outputting the above-mentioned label information may be generated by performing training processing different from that of the trained model described in the first embodiment. In this case, multiple trained models may be used to output welding information and label information, respectively.

[0088] In the present invention, the functions of one or more embodiments described above can be realized by supplying a program or application to a system or device using a network or storage medium, and one or more processors in the computer of that system or device reading and executing the program.

[0089] Alternatively, it may be implemented by a circuit that performs one or more functions. Examples of circuits that perform one or more functions include ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays).

[0090] As described above, the following matters are disclosed in this specification: (1) An anomaly detection method for determining anomalies in the information of one or more feature points extracted from image data, A calculation step for calculating geometric quantity data derived from the information of one or more feature points, An anomaly detection step in which anomalies are detected in time series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A determination step of determining the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, An abnormality detection method characterized by having the following: This configuration makes it possible to detect anomalies in feature points on image data even in environments where disturbances occur.

[0091] (2) The one or more anomaly detection means, A means for determining whether the detection rate of feature points in one or more image data within a predetermined time is below a threshold, with the failure to recognize feature points from the aforementioned image data being the reason for the anomaly, A means for determining whether a misrecognition of a feature point from the aforementioned image data is outside the range defined by a predetermined outlier identification method, A means for determining whether the positions of corresponding feature points recognized in adjacent image data differ by more than a threshold, based on the misrecognition of feature points from the aforementioned image data as the reason for the anomaly, The abnormality determination method according to (1), characterized in that it includes at least one of the above. This configuration makes it possible to perform anomaly detection in response to various reasons for anomalies caused by disturbances contained in image data.

[0092] (3) The image data includes at least one of the following as an object: molten pool, arc, or welding wire. The anomaly detection method according to (1) or (2), characterized in that the aforementioned feature points are extracted from the object. This configuration makes it possible to perform abnormality detection on objects involved in welding.

[0093] (4) The anomaly detection method according to (3), characterized in that the geometric quantity data is at least one of the coordinates of one or more feature points obtained from one object, the distance between multiple feature points, the area formed by multiple feature points obtained from one object, and the distance between feature points in different objects. This configuration makes it possible to perform anomaly detection using geometric quantitative data that captures the characteristics of each object identified from image data.

[0094] (5) The method further includes an acquisition step of acquiring information of one or more feature points from the image data using a trained model, The anomaly detection method according to any one of (1) to (4), characterized in that the trained model is generated by performing a training process using training data which associates image data with information on feature points obtained from said image data, so as to take image data as input and output information on feature points corresponding to said image data. This configuration makes it possible to obtain feature points from image data using a pre-trained model.

[0095] (6) In the determination step, a trained model is used to obtain label information regarding anomalies contained in the image data from the image data, In the determination step, the occurrence of an anomaly in the time series data is determined based on the label information. The anomaly detection method according to any one of (1) to (4), characterized in that the trained model is generated by performing a training process using training data which associates image data, information on feature points obtained from the image data, and label information regarding anomalies contained in the image data, so as to take the image data as input and output label information regarding anomalies contained in the image data. This configuration makes it possible to obtain information about anomalies in image data using a pre-trained model.

[0096] (7) A processing method for performing an abnormality in accordance with the determination result obtained by any of the abnormality determination methods described in (1) to (6), An anomaly handling method characterized by having a processing step of removing data from the time series data for the period in which the anomaly occurred, correcting it using a predetermined value, or correcting it with the value immediately before the anomaly occurred. This configuration makes it possible to process the image data based on the detected anomaly in a way that does not affect the identification of the image.

[0097] (8) The processing method according to (7), further comprising a filtering step of performing a filtering process using a predetermined filter on the time-series data processed in the processing step. This configuration makes it possible to suppress the impact of processing time-series data based on the detected anomalies.

[0098] (9) A processing method for performing an abnormality in accordance with the determination result obtained by any of the abnormality determination methods described in (1) to (6), An analysis step for analyzing the patterns of anomalies in the aforementioned time-series data, A step of correcting the setting conditions for the alarm or abnormality reason based on the analysis results in the aforementioned analysis step, A method for handling abnormal situations, characterized by having the following features. This configuration allows for subsequent control based on the detected anomaly.

[0099] (10) An information processing device for determining anomalies in the information of one or more feature points extracted from image data, A calculation unit that calculates geometric quantity data derived from the information of one or more feature points, An anomaly detection unit detects anomalies in time-series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A determination unit that determines the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, An information processing device characterized by having the following features. This configuration makes it possible to detect anomalies in feature points on image data even in environments where disturbances occur.

[0100] (11) A welding system comprising the information processing device described in (10). This configuration makes it possible to provide a welding system that can identify anomalies contained in image data and perform welding control according to those anomalies.

[0101] (12) To the computer, A calculation process for calculating geometric quantity data derived from information of one or more feature points extracted from image data, An anomaly detection step in which anomalies are detected in time series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A determination step of determining the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, A program to execute. This configuration makes it possible to detect anomalies in feature points on image data even in environments where disturbances occur. [Explanation of Symbols]

[0102] 1. Welding System 10 Welding robots 11 Welding Torch 12 Wire feeder 13 Welding wire 14. Backing material 20 Robot control devices 201 CPU 202 memory 202A Control Program 203 Control Panel 204 Instructional Pendant 205 Robot connection section 206 Communications Department 30 Power supply 40 Vision Sensors 50 Data Processing Devices 510 Main Unit 511 CPU 512 GPU 513 ROM 514 RAM 515 Non-volatile memory 515A Learning Program 515B Pre-trained Model 515C Information Generation Program 515D Image Data 516 Input / Output Interfaces 517 Video output interface 518 Communication Interface 519 Calculation Unit 520 Input section 530 Display section Double job

Claims

1. An anomaly detection method for determining anomalies in the information of multiple feature points extracted from image data, A calculation process for calculating geometric quantity data derived from the information of the aforementioned multiple feature points, An anomaly detection step in which anomalies are detected in time series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, The system includes a determination step of determining the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, The aforementioned image data includes a melting pool as an object, The aforementioned feature points are the position of the wire tip extracted from the object, the position to the left of the tip of the molten pool with respect to the welding line direction, and the position to the right of the tip of the molten pool. The aforementioned geometric quantity data is a width calculated from the distances between multiple feature points obtained from the object of the melting pool. An abnormality determination method characterized in that the width is calculated from the difference in the welding line direction between the position of either the left or right tip of the molten pool and the position of the wire tip, and from the difference in the groove width direction between the position of the left tip and the right tip of the molten pool.

2. The one or more anomaly detection means described above are: A means for determining whether the detection rate of feature points in one or more image data within a predetermined time is below a threshold, with the failure to recognize feature points from the image data being the reason for the anomaly, A means for determining whether a misrecognition of a feature point from the aforementioned image data is outside the range defined by a predetermined outlier identification method, A means for determining whether the positions of corresponding feature points recognized in adjacent image data differ by more than a threshold, based on the misrecognition of feature points from the aforementioned image data as the reason for the anomaly, The abnormality determination method according to claim 1, characterized in that it includes at least one of the following.

3. The process further includes an acquisition step of obtaining information on the multiple feature points from the image data using a trained model. The anomaly detection method according to claim 1 or 2, characterized in that the trained model is generated by performing a training process using training data which associates image data with feature point information obtained from said image data, so as to take image data as input and output feature point information corresponding to said image data.

4. In the aforementioned determination step, a trained model is used to obtain label information regarding anomalies contained in the image data from the image data, In the determination step, the occurrence of an anomaly in the time series data is determined based on the label information. The anomaly detection method according to claim 1 or 2, characterized in that the trained model is generated by performing a training process using training data which associates image data, information on feature points obtained from the image data, and label information regarding anomalies contained in the image data, so as to take image data as input and output label information regarding anomalies contained in the image data.

5. A processing method for performing an abnormality in accordance with the determination result obtained by the abnormality determination method described in any one of claims 1 to 4, An anomaly handling method characterized by having a processing step of removing data from the time series data for the period in which the anomaly occurred, correcting it using a predetermined value, or correcting it with the value immediately before the anomaly occurred.

6. The processing method according to claim 5, further comprising a filtering step of performing a filtering process using a predetermined filter on the time-series data processed in the processing step.

7. A processing method for performing an abnormality in accordance with the determination result obtained by the abnormality determination method described in any one of claims 1 to 4, An analysis step for analyzing the patterns of anomalies in the aforementioned time-series data, A step of correcting the setting conditions for the alarm or abnormality reason based on the analysis results in the aforementioned analysis step, A method for handling abnormal situations, characterized by having the following features.

8. An information processing device for determining anomalies in the information of multiple feature points extracted from image data, A calculation unit that calculates geometric quantity data derived from the information of the aforementioned multiple feature points, An anomaly detection unit detects anomalies in time-series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, The system includes a determination unit that determines the occurrence of an anomaly in the time-series data based on the detection results from the one or more anomaly detection means, The aforementioned image data includes a melting pool as an object, The aforementioned feature points are the position of the wire tip extracted from the object, the position to the left of the tip of the molten pool with respect to the welding line direction, and the position to the right of the tip of the molten pool. The aforementioned geometric quantity data is a width calculated from the distances between multiple feature points obtained from the object of the melting pool. The information processing device is characterized in that the width is calculated from the difference in the welding line direction between the position of either the left or right tip of the molten pool and the position of the wire tip, and from the difference in the groove width direction between the position of the left tip and the right tip of the molten pool.

9. A welding system comprising the information processing device described in claim 8.

10. On the computer, A calculation process for calculating geometric quantity data derived from information of multiple feature points extracted from image data, An anomaly detection step in which anomalies are detected in time series data composed of the aforementioned geometric quantity data using one or more predetermined anomaly detection means corresponding to the reason for the anomaly, A program for executing a determination step of determining the occurrence of an anomaly in the time-series data based on the detection results of the one or more anomaly detection means, The aforementioned image data includes a melting pool as an object, The aforementioned feature points are the position of the wire tip extracted from the object, the position to the left of the tip of the molten pool with respect to the welding line direction, and the position to the right of the tip of the molten pool. The aforementioned geometric quantity data is a width calculated from the distances between multiple feature points obtained from the object of the melting pool. The program calculates the width from the difference in the welding line direction between the position of either the left or right tip of the molten pool and the position of the wire tip, and from the difference in the groove width direction between the position of the left tip and the right tip of the molten pool.

Images