Apparatus or Method for Determining Success or Fail of Robot Task Based on Sensor Data

Abstract

According to one embodiment of the present invention, a device for determining whether a robot's task is successful is proposed, and the device may be configured to include: a transceiver configured to receive sensor data related to the robot's task; a memory configured to store a model for determining whether the robot's task is successful; and a processor configured to input the received sensor data into the model, compare output data output from the model with the input sensor data, and determine whether the robot's task is successful based on the comparison result.

Description

Technology Field

[0001] The present invention relates to an apparatus or method for determining the success or failure of a robot's task based on sensor data, and more specifically, to a technology capable of determining the success or failure of a specific task of a robot using sensor data obtainable from a robot. Background Technology

[0002] Collaborative robots refer to robots designed to work alongside humans. Unlike conventional industrial robots, they are characterized by a design that allows them to collaborate directly with workers without safety fences. Collaborative robots support repetitive or precision tasks, reducing the burden on workers and increasing productivity.

[0003] For this purpose, the productivity of collaborative robots is affected by whether the task is performed correctly, and to this end, a system is required for the robot to monitor the success of the task.

[0004] Conventionally, sensors were used to detect the state or control of a robot, but this method incurs costs due to the need to add sensors and has the disadvantage of requiring them to be connected to a controller. Additionally, it has the disadvantage of requiring additional space to attach sensors to the robot.

[0005] Therefore, there is a need for technology that can determine the success of a robot's task using sensor values ​​without the addition of extra sensors. The problem to be solved

[0006] The present invention proposes an apparatus or method for determining the success or failure of a task of a collaborative robot based on sensor data.

[0007] The problems of the present invention are not limited to those described above. Other problems not described above will be understood by a person skilled in the art from the description of the present invention below. means of solving the problem

[0008] According to one embodiment of the present invention, a device for determining whether a robot's task is successful is proposed, and the device may be configured to include: a transceiver configured to receive sensor data related to the robot's task; a memory configured to store a model for determining whether the robot's task is successful; and a processor configured to input the received sensor data into the model, compare output data output from the model with the input sensor data, and determine whether the robot's task is successful based on the comparison result.

[0009] Additionally or alternatively, the processor may be configured to include a data collection and processing unit configured to collect sensor data for training the model and process the collected sensor data; and a learning unit configured to input sensor data corresponding to the success of the robot's task into the model and to train the model in a direction in which the difference between the output data output from the model and the input sensor data is reduced.

[0010] Additionally, or alternatively, the learning of the above model may be configured to be performed independently for each type of task of the robot.

[0011] Additionally or alternatively, the data collection and processing unit may be configured to assign sequence values ​​according to the time series of the collected sensor data to the collected sensor data.

[0012] Additionally or alternatively, the processor may be configured to include a decision unit configured to determine whether the robot's task is successful based on the input sensor data of the model and the output data of the model for the input sensor data.

[0013] Additionally or alternatively, the judgment unit is configured to set a threshold to compare with the difference between the input sensor data of the model and the output data of the model corresponding to the input sensor data, and the threshold may be set between the average value of the difference between a plurality of first sensor data corresponding to the success of the robot's task and the second output data obtained by inputting the first sensor data into the model, and the average value of the difference between a plurality of second sensor data corresponding to the failure of the robot's task and the second output data obtained by inputting the second sensor data into the model.

[0014] Additionally or alternatively, the number of the first sensor data may be greater than the number of the second sensor data.

[0015] Additionally or alternatively, sensor data related to the operation of the robot may include at least one of the joint angle, angular velocity, current value, torque value, and residual torque value of each axis of the robot.

[0016] Additionally or alternatively, sensor data related to the operation of the robot may include at least one of the robot's attitude information and external force.

[0017] According to another embodiment of the present invention, a method for determining whether a robot's task is successful is proposed, and the method may be configured to include: receiving sensor data related to the task of the robot; inputting the received sensor data into a model for determining whether the robot's task is successful; comparing output data output from the model with the input sensor data; and determining whether the robot's task is successful based on the comparison result.

[0018] The means of solution of the present invention described above are part of the embodiments of the present invention. Various means of solution other than the means of solution of the problem described above may be derived and understood based on the detailed description of the present invention to be explained below. Effects of the invention

[0019] The present invention has the following effects.

[0020] The present invention can determine whether the work result of a robot is successful through sensor data of a collaborative robot without the need for additional sensors to monitor whether the work result of the collaborative robot is successful.

[0021] The effects of the present invention are not limited to those described above. Other effects not described above may be understood by a person skilled in the art from the description of the present invention below. Brief explanation of the drawing

[0022] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present invention, provide embodiments of the present invention and explain the contents of the present invention together with the detailed description. FIG. 1 illustrates the success and failure of a robot's work according to one embodiment of the present invention. FIGS. 2 and 3 illustrate block diagrams of a device for determining whether a robot's operation is successful according to the present invention. FIG. 4 illustrates the input / output process of model learning for determining whether a robot's task is successful according to the present invention. FIG. 5 illustrates a block diagram of a device for determining whether a robot's operation is successful according to the present invention. Figure 6 shows the input and output data of the model according to the success and failure of the robot's work according to the present invention. Figure 7 shows the distribution of error values ​​according to task success and task failure in a model for determining whether a task of a robot according to the present invention is successful. FIG. 8 illustrates a flowchart of a method for determining whether a robot's operation is successful according to the present invention. Specific details for implementing the invention

[0023] Embodiments of the present invention will be described below with reference to the attached drawings.

[0024] The embodiments described below are intended to aid in understanding the invention and are therefore not limited to the embodiments described below. Additionally, in the accompanying drawings, certain components may be depicted in an exaggerated or reduced manner to aid in understanding the invention. The invention is not limited to the form depicted in the accompanying drawings.

[0025] Throughout the specification, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0027] FIG. 1 illustrates the success and failure of a robot's work according to one embodiment of the present invention.

[0028] Figure 1 (a) shows a state in which a robot (1) has grasped and lifted an object (2), which indicates a state in which the robot has successfully completed a task under control.

[0029] Figure 1 (b) shows a state in which the robot (1) has grasped an object (2) but has not lifted it properly, which indicates a state in which the robot has not successfully completed the task according to the control.

[0030] In order to detect the state of Fig. 1 (a) or Fig. 1 (b), conventionally, a separate detection sensor, such as a contact sensor for gripping detection, was attached to the robot, and the success of gripping or lifting could be confirmed through the sensing value of the detection sensor.

[0031] In this specification, we propose a method for determining whether the success or failure of a task of a robot (1) can be determined from sensor data related to the task of the robot (1).

[0033] FIGS. 2 and 3 illustrate block diagrams of a device for determining whether a robot's work is successful according to the present invention. FIG. 2 shows a block diagram of a device (10) configured to determine the success of a robot's work, and FIG. 3 shows a block diagram of a processor (101) of the device (10).

[0034] Referring to FIG. 2, a device (10) configured to determine whether the robot's work is successful may include a memory (100), a processor (101), and a transceiver (102).

[0035] The memory (100) may be configured to store a model for determining the success of a robot's task. The model may be configured to be learned through reinforcement learning (deep learning) or machine learning. The transceiver (102) may be configured to receive sensor data related to the robot's task. The processor (101) may be configured to input the received sensor data into the model and compare the output data from the model with the input sensor data. Additionally, the processor (101) may be configured to determine whether the robot's task is successful based on the comparison result.

[0037] Referring to FIG. 3, the processor (101) may include a data collection and processing unit (1010), a learning unit (1020), and a task success determination unit (1030).

[0038] The data collection and processing unit (1010) may be configured to collect or process sensor data related to the robot's operation for a model to determine whether the robot's operation is successful. The collected sensor data may include at least one of the joint angle, angular velocity, current value, torque value, and residual torque value of each axis of the robot.

[0039] Joint angle represents the absolute angle of rotation of each joint. It is a key value that determines the robot's current attitude and position. Angular velocity refers to the rate at which each joint changes (amount of rotation per unit time). Current value indicates the amount of current consumed by the motors of each joint; more current flows as the load increases, and abnormal current may be detected if a failure occurs. Torque value refers to the force (rotational force) generated at each joint. Residual torque value represents the difference between the desired torque and the actual torque generated.

[0040] In addition, the sensor data collected above may include at least one of the attitude information of the robot's end-part and external forces. For the robot to perform a specific movement (task), attitude, speed, and force must be precisely controlled. The attitude information of the robot's end-part indicates the position and orientation of the robot's TCP (tool center point) in three-dimensional space, and the TCP typically refers to the point where the robot's hand tip performs the task. The attitude of the robot's end-part is expressed by six values ​​and consists of position (X, Y, Z) and orientation (Rx, Ry, Rz). Orientation is typically expressed by roll, pitch, and yaw values. External forces represent external forces applied to the collaborative robot.

[0041] Preferably, the sensor data collected for learning may include sensor data when the robot's operation is successful.

[0042] Additionally, the data collection and processing unit (1010) may be configured to assign a sequence value to each sensor data of the sensor data collected in a time series. Thus, the collected or processed sensor data may include information on the flow of time.

[0043] For example, if the robot is a 6-axis robot, sensor data provided for each axis and sensor data related to the robot's TCP are collected over time. Therefore, the individual axis and TCP-related sensor data collected over time are organized into two dimensions, and by collecting and processing this for multiple (6) axes, the collected sensor data can be organized into three-dimensional data.

[0044] The above model may be configured to learn using sensor data of the robot as input. For example, the above model may include an autoencoder model.

[0045] A learner (1020) may be configured to train the above model. More specifically, the learner (1020) may be configured to input collected sensor data into the above model and to train the above model in a direction in which the difference between the output data output from the above model and the input sensor data is reduced.

[0046] FIG. 4 illustrates the input / output process of model learning for determining whether a task of a robot according to the present invention is successful. The input (x) corresponds to sensor data of the robot, and the learner (1020) may be configured to encode it and convert it into low-dimensional data. Then, the learner (1020) may be configured to decode the low-dimensional data again to obtain restored data, i.e., reconstructed input data (x'). The model may be set to learn in a direction where there is no difference between the input (x) and the reconstructed input data (x'), i.e., the restored data or output data, i.e., in a direction where the difference decreases. To this end, the learner (1020) may include an encoder (1021) and a decoder (1022).

[0047] When learning is repeated using input sensor data corresponding to a task success, the decoder can be configured to acquire restored data, i.e., output data, that is, data that is approximate or similar to the input sensor data at the time of the robot's task success.

[0048] The task success determination unit (1030) may be configured to determine whether the task of the robot is successful based on the input sensor data of the model and the output data corresponding to the input sensor data.

[0050] Meanwhile, the data collection, processing, or learning described above can be performed independently for each robot task. For example, a model can be configured to determine whether the robot's "gripping" is successful; to this end, sensor data indicating successful gripping can be collected or processed and configured to be used as input to the model.

[0052] FIG. 5 illustrates a block diagram of a device for determining whether a robot's operation is successful according to the present invention.

[0053] The task success determination unit (1030) may be configured to include an input sensor data acquirer (1031), a model output data acquirer (1032), an error calculator (1033), and a threshold setter (1034).

[0054] The input sensor data acquirer (1031) may be configured to acquire sensor data of the robot. This is used as an input value (i.e., input sensor data) of the model.

[0055] The model output data acquirer (1032) may be configured to acquire output data for input sensor data. As shown in FIG. 4, the input sensor data is encoded and converted into low-dimensional data, then decoded and restored to the reconstructed input sensor data, and the restored data corresponds to the output data.

[0056] The error calculator (1033) may be configured to calculate the difference between the input sensor data and the output data, i.e., the error. The calculated error is used to determine whether the robot's operation is successful.

[0057] The threshold setter (1034) may be configured to set a threshold to compare with the calculated error. The threshold may be determined or set between the average value of the difference between a plurality of input sensor data corresponding to the success of the robot's task and the corresponding output data, and the average value of the difference between a plurality of input sensor data corresponding to the failure of the robot's task and the corresponding output data.

[0059] Figure 6 shows the input and output data of the model according to the success and failure of the robot's work according to the present invention.

[0060] Figure 6(a) shows the input sensor data and the corresponding output data when the robot succeeds in a task. The two data tend to match.

[0061] Figure 6(b) shows input sensor data and corresponding output data when a robot task fails. The two data tend to show a somewhat pronounced difference.

[0062] By utilizing the above characteristics, the device of the present invention can be configured to determine the success or failure of a robot's operation using sensor data of the robot.

[0064] Figure 7 shows the distribution of error values ​​according to task success and task failure in a model for determining whether a task of a robot according to the present invention is successful.

[0065] Figure 7(a) shows the distribution of errors when the robot succeeds in a task. It shows the result where the most cases occurred at an error value slightly smaller than 0.02.

[0066] Figure 7(b) shows the distribution of errors when the robot fails a task. It shows the results where the most cases occurred around an error value of 0.45.

[0067] Therefore, a threshold can be set using the average error value when the robot's task is successful and the average error value when the robot's task fails, and if the calculated error is smaller than the set threshold, the robot's task is determined to be successful; if the calculated error is greater than or equal to the set threshold, the robot's task is determined to be a failure.

[0068] On the other hand, ideally, robot task failures occur less frequently than task successes, and in practice, this is the case. Therefore, the number of sensor data points during a task success may be greater than or higher than the number of sensor data points during a task failure.

[0070] FIG. 8 illustrates a flowchart of a method for determining whether a robot's operation is successful according to the present invention. The method illustrated in FIG. 8 can be performed by a device (10) configured to determine whether a robot's operation is successful. Hereinafter, the “device (10)” is described as performing the method of FIG. 8.

[0071] The device (10) can be configured to acquire sensor data related to the robot's work (S810).

[0072] The device (10) can be configured to acquire reconstructed data, i.e., restored data, corresponding to the sensor data acquired using a learned model.

[0073] The device (10) may be configured to compare the acquired sensor data with the acquired reconstructed data. To this end, the device (10) may be configured to calculate the difference, i.e., the error, between the acquired sensor data and the acquired reconstructed data. Then, the device (10) may be configured to compare the calculated error with a set threshold.

[0074] The device (10) may be configured to determine the operation of the robot as a failure if the calculated error is greater than or equal to the set threshold (S830).

[0075] The device (10) may be configured to determine the robot's operation as successful if the calculated error is less than the set threshold (S840).

[0076] Thus, according to the present invention, the success or failure of a robot's task can be determined by using the robot's sensor data and a learned model.

[0078] In the above specification, it has been described that a device for determining the success or failure of a collaborative robot's operation, or each component included therein, performs control; however, "device," "system," and the components belonging thereto are merely names, and the scope of rights is not subordinate to them.

[0079] That is, the proposed technology may be implemented under names other than device, processor, or controller, and furthermore, the methods or methods described above may be implemented by software or code readable by a computer or other machine, device, etc., for determining the success or failure of a collaborative robot's operation.

[0080] In addition, as another aspect of the present invention, the operation of the proposed technology described above may also be provided as code that can be implemented, practiced, or executed by a "computer" (a comprehensive concept including a system on chip (SoC) or a (micro)processor, etc.), or as a computer-readable storage medium or computer program product that stores or contains said code. The scope of the present invention may be extended to said code or as a computer-readable storage medium or computer program product that stores or contains said code.

[0081] The detailed description of the preferred embodiments of the present invention disclosed as described above is provided so that a person skilled in the art can implement and practice the present invention.

[0082] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention as described in the following claims.

[0083] Accordingly, the present invention is not intended to be limited to the embodiments shown herein, but to be given the broadest scope consistent with the principles and novel features disclosed herein.

Claims

Claim 1 A device for determining whether a robot's task is successful, comprising: a transceiver configured to receive sensor data related to the robot's task; a memory configured to store a model for determining whether the robot's task is successful; and a processor configured to input the received sensor data into the model, compare output data output from the model with the input sensor data, and determine whether the robot's task is successful based on the comparison result. Claim 2 The device according to claim 1, wherein the processor comprises: a data collection and processing unit configured to collect sensor data for training the model and process the collected sensor data; and a learning unit configured to input sensor data corresponding to the success of the robot's task into the model and to train the model in a direction in which the difference between the output data output from the model and the input sensor data is reduced. Claim 3 A device according to paragraph 2, wherein the learning of the above model is configured to be performed independently for each type of task of the above robot. Claim 4 In paragraph 2, the data collection and processing unit is configured to assign a sequence value according to the time series of the collected sensor data to the collected sensor data. Claim 5 A device according to claim 1, wherein the processor comprises a determination unit configured to determine whether the robot's task is successful based on input sensor data of the model and output data of the model for the input sensor data. Claim 6 In claim 5, the above-mentioned judgment unit is configured to set a threshold value to be compared with the difference between the input sensor data of the model and the output data of the model corresponding to the input sensor data, wherein the threshold value is set between the average value of the difference between a plurality of first sensor data corresponding to the success of the robot's work and the second output data obtained by inputting the first sensor data into the model, and the average value of the difference between a plurality of second sensor data corresponding to the failure of the robot's work and the second output data obtained by inputting the second sensor data into the model. Claim 7 In paragraph 6, the device, wherein the number of the first sensor data is greater than the number of the second sensor data. Claim 8 A device according to claim 1, wherein sensor data related to the operation of the robot includes at least one of the joint angle, angular velocity, current value, torque value, and residual torque value of each axis of the robot. Claim 9 A device according to claim 1, wherein sensor data related to the operation of the robot includes at least one of attitude information of the robot's end portion and an external force. Claim 10 A method for determining whether a robot's task is successful, comprising: receiving sensor data related to the task of the robot; inputting the received sensor data into a model for determining whether the robot's task is successful; comparing output data output from the model with the input sensor data; and determining whether the robot's task is successful according to the comparison result.

Images