The Method and System That Provide Learning Status Information of An Artificial Neural Network-based Control Model for A Vision-based Automated Robot

Abstract

The present invention relates to a method and system for providing learning state information of an artificial neural network-based control model of a vision-based automation robot, wherein a contribution matrix representing the substantial contribution of each patch constituting an input image is generated using attention weight information obtained from a first model based on a transformer algorithm and gradient information of the loss value of the control model, and heatmap information generated based thereon is superimposed on the input image to visualize the image area that has substantially contributed to the action decision of the automation robot, thereby supporting intuitive identification of the image area that has substantially contributed to the action decision of the automation robot.

Description

Technology Field

[0001] The present invention relates to a method and system for providing learning state information of an artificial neural network-based control model of a vision-based automation robot, wherein the method supports intuitively identifying image regions that have substantially contributed to the action decision of the automation robot by visualizing heatmap information representing the substantial contribution of each patch constituting the input image using attention weight information obtained from a first model based on a transformer algorithm and gradient information of the loss value of the control model, by overlaying the input image with heatmap information. Background Technology

[0003] With the recent rapid advancement of artificial intelligence (AI) technology, "vision-based robot control" technology, which directly generates robot control commands from visual information acquired through vision sensors such as cameras, is garnering attention. In particular, as the Transformer architecture—which revolutionized the field of natural language processing—has been successfully applied to the field of computer vision, active attempts are being made to utilize it as a model for robot perception and control.

[0004] Conventional robot control methods include a control method for an underwater robot equipped with a multi-degree-of-freedom robotic arm, as described in Korean Registered Patent No. 10-2652885. This technology utilizes propeller simulation neural networks and actuator simulation neural networks to predict thrust or torque, which are difficult to measure directly by sensors, and to precisely control the robot based on these predictions. In this way, using artificial neural networks offers the advantage of effectively solving non-linear control problems that are difficult to address through mathematical modeling alone.

[0005] However, as with the aforementioned conventional technology, deep learning-based control models have so-called 'black box' characteristics, where the complex computational processes between input and output are opaque. Unlike simple image classification tasks, in the field of robot control that interacts with a physical environment, 'explainability' and 'interpretability' regarding why a robot decided on such behavior are essential for verifying safety and ensuring reliability.

[0006] Therefore, there is a need to develop technology capable of deriving learning state information by visualizing the attention map inside the Transformer model to identify which area of ​​the image the model is focusing on. Prior art literature

[0008] Republic of Korea Registered Patent No. 10-2652885 (March 26, 2024) The problem to be solved

[0009] The present invention aims to provide a method and system for providing learning state information of an artificial neural network-based control model of a vision-based automation robot, wherein the method generates a contribution matrix representing the substantial contribution of each patch constituting an input image using attention weight information obtained from a first model based on a transformer algorithm and gradient information of the loss value of the control model, and supports intuitively identifying image regions that have substantially contributed to the action decision of the automation robot by visualizing the heatmap information generated based thereon by superimposing it on the input image. means of solving the problem

[0011] In order to solve the above problems, one embodiment of the present invention provides a method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot performed in a computing system comprising one or more processors and one or more memories, wherein the control model is based on a transformer algorithm and includes a first model that outputs an embedding from an input image; an information acquisition step of acquiring attention weight information in a matrix form derived from query information and key information in each of one or more layers of the first model for the input image; a first information calculation step of calculating gradient information in a matrix form representing the rate of change of the loss value of the control model with respect to the attention weight information; a second information calculation step of calculating heatmap information representing the importance of each region of the input image based on the attention weight information and the gradient information; and a learning information generation step of generating learning state information in an image form by superimposing the heatmap information and the input image.

[0012] In one embodiment of the present invention, the information acquisition step further comprises: a token generation step of dividing the input image into a plurality of patches having a preset size and embedding them to generate a plurality of patch tokens; and a sequence configuration step of combining a classification token for learning the entire context at the front of the plurality of patch tokens to form an input token sequence of the first model; wherein the attention weight information and the gradient information may be characterized as being a matrix calculated based on the input token sequence.

[0013] In one embodiment of the present invention, the second information calculation step may include a matrix generation step of generating a contribution matrix representing the contribution of each of the plurality of patches to the loss value by performing element-wise multiplication of the attention weight information and the gradient information.

[0014] In one embodiment of the present invention, the second information calculation step may further include: a matrix correction step for generating a corrected contribution matrix reflecting residual connections by summing an identity matrix to the contribution matrix; and a matrix accumulation step for generating a cumulative contribution matrix that accumulates information flow between layers by sequentially multiplying the corrected contribution matrices generated for each layer from the input layer to the output layer of the first model.

[0015] In one embodiment of the present invention, the second information output step may further include: a column vector extraction step for extracting a column vector corresponding to the classification token from the cumulative contribution matrix; and a heatmap information generation step for generating the heatmap information by rearranging the component values ​​of the extracted column vector into a two-dimensional form corresponding to a preset patch array of the input image.

[0016] In one embodiment of the present invention, the control model may include: a first model that receives input information including the input image and outputs an embedding including feature information of the input image; and a second model that receives the output value of the first model and non-image information and outputs a predicted behavior of the automation robot.

[0017] In one embodiment of the present invention, the loss value of the control model is calculated from the difference between the predicted action value of the automation robot output by the second model and the predefined correct action value, and the first information calculation step can calculate the gradient information by partially differentiating the loss value with respect to the attention weight information of the first model.

[0018] In one embodiment of the present invention, the non-image information may be characterized by including at least one of the pose of the automation robot, the state of the actuator, and a control command from a user.

[0019] In one embodiment of the present invention, the predicted action may be characterized as a continuous numerical value including at least one of the angle value of a joint constituting an automated robot, the open / closed state of a gripper, and the three-dimensional coordinate value of an end effector.

[0020] In one embodiment of the present invention, the first model has a multi-head attention structure, and the attention weight information and the gradient information each may be characterized by using the average of values ​​calculated from a plurality of heads.

[0021] In order to solve the above problems, one embodiment of the present invention provides a computing system that includes one or more processors and one or more memories and performs a method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot, wherein the control model is based on a transformer algorithm and includes a first model that outputs an embedding from an input image; an information acquisition unit that acquires attention weight information in the form of a matrix derived from query information and key information in each of one or more layers of the first model for the input image; a first information calculation unit that calculates gradient information in the form of a matrix representing the rate of change of the loss value of the control model for the attention weight information; a second information calculation unit that calculates heatmap information representing the importance of each region of the input image based on the attention weight information and the gradient information; and a learning information generation unit that generates learning state information in the form of an image by superimposing the heatmap information and the input image. Effects of the invention

[0023] In one embodiment of the present invention, by applying gradient information regarding loss values ​​as weights to the attention weight information of a transformer-based control model to calculate a contribution matrix, the model can clearly distinguish between the area it simply observes and the area that contributed to the actual action decision, thereby providing visualization information with a clear causal relationship.

[0024] In one embodiment of the present invention, by applying a method of summing the identity matrix to the contribution matrix and accumulating it layer by layer, the flow of information preserved through residual connections of the transformer structure can be tracked without omission, thereby enabling the effect of preventing the heatmap information from being lost or diluted even when passing through deep layers.

[0025] In one embodiment of the present invention, by performing backpropagation based on the robot's behavioral error (Loss) predicted by the second model, it is possible to achieve the effect of presenting a behavior-oriented basis for judgment regarding the specific control actions of the robot, rather than simple image classification.

[0026] In one embodiment of the present invention, by generating learning state information using only computational parameters within the control model, it is possible to achieve the effect of implementing an explainable artificial intelligence efficiently and economically without the need to construct separate correct answer data for explanation or provide a separate explanation generation module.

[0027] In one embodiment of the present invention, by intuitively identifying whether the robot has referenced background noise or interfering factors during the decision-making process through the generated heatmap information, it is possible to quickly debug the cause of malfunction in the control model and significantly improve the operational safety and reliability of the robot automation system. Brief explanation of the drawing

[0029] FIG. 1 schematically illustrates the overall flow of a method for providing learning state information of an artificial neural network control model of a vision-based automation robot according to one embodiment of the present invention. FIG. 2 schematically illustrates a method for providing learning state information according to an embodiment of the present invention and the internal configuration of a computing system that performs the same. FIG. 3 schematically illustrates the process of generating tokens by dividing an input image into a plurality of patches according to an embodiment of the present invention. FIG. 4 schematically illustrates the process of forming an input sequence by combining classification tokens according to one embodiment of the present invention. FIG. 5 schematically illustrates the process of obtaining attention weight information using query information and key information in a first model according to an embodiment of the present invention. FIG. 6 schematically illustrates a method of averaging and utilizing a plurality of head values ​​in a multi-head attention structure according to one embodiment of the present invention. FIG. 7 schematically illustrates the concept of applying different weights to each head group according to another embodiment of the present invention. FIG. 8 schematically illustrates the process of calculating gradient information according to one embodiment of the present invention. FIG. 9 schematically illustrates the process of generating a contribution matrix according to one embodiment of the present invention. FIG. 10 schematically illustrates the process of calculating a cumulative contribution matrix according to one embodiment of the present invention. FIG. 11 schematically illustrates the process of generating heatmap information according to one embodiment of the present invention. FIG. 12 schematically illustrates the combined structure of the first model and the second model and the backpropagation process of the behavior-centered loss value according to one embodiment of the present invention. FIG. 13 illustrates, in an exemplary manner, the configuration of non-image information input to a control model according to one embodiment of the present invention. FIG. 14 illustrates, in an exemplary manner, the configuration of a predicted behavior output by a control model according to one embodiment of the present invention. FIG. 15 illustrates, in an exemplary manner, the internal configuration of a computing device according to one embodiment of the present invention. Specific details for implementing the invention

[0030] Hereinafter, various embodiments and / or aspects are disclosed with reference to the drawings. For illustrative purposes, numerous specific details are disclosed in the following description to aid in a general understanding of one or more aspects. However, it will also be recognized by those skilled in the art that these aspects may be practiced without such specific details. The following description and the accompanying drawings describe specific exemplary aspects of one or more aspects in detail. However, these aspects are exemplary, and some of the various methods in the principles of the various aspects may be used, and the description is intended to include all such aspects and their equivalents.

[0032] In addition, various aspects and features will be presented by a system that may include multiple devices, components and / or modules, etc. It should also be understood and recognized that various systems may include additional devices, components and / or modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in relation to the drawings.

[0033] Terms such as “embodiment,” “example,” “aspect,” “example,” etc. as used herein may not be interpreted as implying that any aspect or design described is superior or advantageous to other aspects or designs. Terms used below, such as “part,” “component,” “module,” “system,” “interface,” etc., generally refer to computer-related entities and may refer, for example, to hardware, a combination of hardware and software, or software.

[0034] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that the relevant feature and / or component is present, but not to exclude the presence or addition of one or more other features, components and / or groups thereof.

[0035] Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0036] Furthermore, in the embodiments of the present invention, all terms used herein, including technical or scientific terms, unless otherwise defined, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the embodiments of the present invention.

[0038] FIG. 1 schematically illustrates the overall flow of a method for providing learning state information of an artificial neural network control model of a vision-based automation robot according to one embodiment of the present invention.

[0040] Conventional deep neural network-based robot control technology has a 'black box' characteristic in which the computational process between input and output is opaque, making it difficult to determine why the robot decided on such an action. Figure 1 briefly illustrates the overall system of the present invention for solving this problem, and the method for providing learning state information according to the present invention is performed by a computing system (1000).

[0041] The above computing system (1000) uses an artificial neural network-based control model for controlling an automated robot to process an input image, which is visual information of the automated robot (hereinafter referred to as the robot), and to control the predicted behavior of the robot. The input image shown at the top of FIG. 1 corresponds to an image taken of the robot's work environment (e.g., an environment where the gripper of the robot arm attempts to pick up a box).

[0042] The present invention backpropagates the loss value generated during the process of the control model calculating the predicted action to find learning state information as an image region that the control model substantially referenced for making an action decision. As shown on the right side of FIG. 1, the cloud-shaped region displayed over a specific object (e.g., a box) within the input image is a heatmap produced through the backpropagation process, which visualizes the region that the control model most intensively monitored to perform the corresponding action.

[0044] Meanwhile, in one embodiment of the present invention, the computing system (1000) may be an embedded system mounted on the robot body, or is not limited thereto to a remote server connected to the robot via wireless communication. For example, to reduce the computational load of the robot, the robot may perform image acquisition and control signal reception, and the process of computation of the control model and heat map generation may have a distributed processing structure in which the process is performed on a remote server equipped with a high-performance GPU.

[0046] FIG. 2 schematically illustrates a method for providing learning state information according to one embodiment of the present invention and the internal configuration of a computing system (1000) that performs the same.

[0048] As illustrated in FIG. 2, a method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot, performed in a computing system (1000) comprising one or more processors and one or more memories, wherein the control model is based on a transformer algorithm and includes a first model that outputs an embedding from an input image, and for the input image, an information acquisition step (S100) for acquiring attention weight information in a matrix form derived from query information and key information in each of one or more layers of the first model; a first information calculation step (S200) for calculating gradient information in a matrix form representing the rate of change of the loss value of the control model with respect to the attention weight information; and a second information calculation step (S300) for calculating heatmap information representing the importance of each region of the input image based on the attention weight information and the gradient information. and a learning information generation step (S400) that generates learning state information in the form of an image by overlapping the heatmap information and the input image.

[0049] Additionally, a computing system (1000) comprising one or more processors and one or more memories, and performing a method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot, wherein the control model is based on a transformer algorithm and includes a first model that outputs an embedding from an input image, and for the input image, an information acquisition unit (1100) that acquires attention weight information in the form of a matrix derived from query information and key information in each of one or more layers of the first model; a first information calculation unit (1200) that calculates gradient information in the form of a matrix representing the rate of change of the loss value of the control model with respect to the attention weight information; and a second information calculation unit (1300) that calculates heatmap information representing the importance of each region of the input image based on the attention weight information and the gradient information. and a learning information generation unit (1400) that generates learning state information in the form of an image by overlapping the heatmap information and the input image.

[0051] A computing system (1000) according to one embodiment of the present invention may be configured to include an information acquisition unit (1100), a first information output unit (1200), a second information output unit (1300), and a learning information generation unit (1400) as logical or physical components for performing the method for providing learning state information of the present invention. Additionally, through each of these components, an information acquisition step (S100), a first information output step (S200), a second information output step (S300), and a learning information generation step (S400), which are time-series processes, are performed.

[0053] Specifically, the information acquisition unit (1100) performs an information acquisition step (S100). In the information acquisition step (S100), an input image acquired through the robot's visual sensor, etc., is input into the first model of the control model. The information acquisition unit (1100) acquires attention weight information in the form of a matrix representing the association between patches by using query information and key information calculated within each layer of the first model. At this time, the input image is divided into multiple patch tokens and processed in a combined form with classification tokens for learning the entire context.

[0054] Preferably, the method for providing learning state information of the present invention may further include a preprocessing step prior to the information acquisition step (S100) for removing noise from an input image or normalizing brightness to be robust to changes in lighting.

[0056] The first information output unit (1200) performs the first information output step (S200). In the first information output step (S200), the rate of change for the attention weight information is calculated based on the loss value (Loss), which is the error between the predicted action and the correct action, which is the final output of the control model. More specifically, the first information output unit (1200) calculates matrix-type gradient information by taking partial derivatives of the loss value with respect to the attention weight information through a backpropagation algorithm.

[0057] The second information output unit (1300) performs the second information output step (S300). The second information output step (S300) is a core computational process of the present invention, and calculates heatmap information representing the importance of each region of the input image based on the attention weight information and the gradient information. The second information output unit (1300) does not stop at simply combining the two pieces of information, but applies a rollout technique that considers the residual connection of the transformer to accurately calculate the actual contribution accumulated through the deep layer.

[0059] Finally, the learning information generation unit (1400) performs the learning information generation step (S400). In the learning information generation step (S400), the generated heatmap information is overlaid with the original input image to generate visualized learning state information. At this time, the generated learning state information image is provided so that the user can intuitively understand the basis of the robot's judgment by highlighting the area that the robot intensively referenced for action decisions (e.g., the cloud-shaped highlight area shown on the right side of FIG. 1) with identifiable colors or shades.

[0061] The reason for visualizing the gaze area in this manner is to verify the reliability of the artificial neural network control model. Specifically, even if the robot performs the correct action (e.g., grasping a cup), the control model may have determined the action by accidentally referencing background noise (e.g., a stain on a desk) instead of the cup. Such 'accidental correct answers' become a critical risk factor that can cause malfunctions even with slight changes in the environment.

[0062] Accordingly, through the learning state information provided by the learning information generation unit (1400) of the present invention, the user can immediately determine whether the robot has accurately recognized and judged the correct object or is incorrectly focusing on an irrelevant area. This serves as a key judgment indicator for analyzing the cause of failure of the control model and verifying safety.

[0064] FIG. 3 schematically illustrates a process of generating tokens by dividing an input image into a plurality of patches according to an embodiment of the present invention. FIG. 4 also schematically illustrates a process of constructing an input sequence by combining classification tokens according to an embodiment of the present invention.

[0066] As illustrated in FIGS. 3 and 4, the information acquisition step (S100) further includes: a token generation step of dividing the input image into a plurality of patches having a preset size and embedding them to generate a plurality of patch tokens; and a sequence configuration step of combining a classification token for learning the entire context with the front end of the plurality of patch tokens to form an input token sequence of the first model.

[0068] Specifically, as in one embodiment illustrated in FIG. 3, the information acquisition unit (1100) divides an acquired input image of size 224*224 pixels into a predetermined grid unit to generate a plurality of patches (P). In this embodiment, a 14*14 grid structure is applied to divide the input image into a total of 196 patches. At this time, each divided patch (P) corresponds to an image fragment having a resolution of 16*16 pixels.

[0069] An embodiment illustrated in FIG. 4 is an embodiment that follows the embodiment illustrated in FIG. 3. As illustrated in FIG. 4, each divided patch is converted into a vector form that can be computed by a control model (preferably, a first model). Specifically, each patch (P) of size 16*16 undergoes a Flatten process to be converted into a 256-dimensional 1-dimensional vector, and then passes through a Linear Projection layer to be mapped to token vectors (F2 to F197) having a preset embedding dimension (e.g., 512 dimensions).

[0071] In the present invention, a classification token (F1) is combined at the very beginning of the generated token vectors to contain the entire context information that serves as a reference point for learning.

[0072] Therefore, the data sequence finally input to the first model is composed of a total of 197 512-dimensional vectors arranged in a sequence, such as [F1 (classification token), F2 (patch token 1), F3 (patch token 2), ..., F197 (patch token 196)].

[0074] Preferably, the first model may further perform a process of summing or combining positional embedding vectors element by element with each of the patch tokens to preserve spatial coordinate information where each patch token is located within the original input image.

[0076] Meanwhile, in the aforementioned FIG. 4 and FIG. 11 to be described later, a total of 197 tokens (F1 to F197) are shown as actual implementation examples, but in the following FIGs. 5 to 10, the number of tokens is reduced to 4 (F1 to F4) for convenience of explanation and readability of the drawings and is shown in a simplified form.

[0078] FIG. 5 schematically illustrates the process of obtaining attention weight information using query information and key information in a first model according to an embodiment of the present invention.

[0080] As illustrated in FIG. 5, a method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot performed in a computing system (1000) comprising one or more processors and one or more memories comprises: an information acquisition step (S100) of acquiring attention weight information in the form of a matrix derived from query information and key information in each of one or more layers of the first model for the input image.

[0081] In addition, the attention weight information and the gradient information are characterized as being matrices calculated based on the input token sequence.

[0083] Specifically, the information acquisition unit (1100) inputs the previously configured input token sequence (F1, F2, F3, ..., F197) into the transformer layer of the first model and performs operations. The first model can generate a Query (Q), Key (K), and Value (V) vector for each token by multiplying each input token vector by a learned weight matrix. The Query (Q) indicates what the current token wants to find in other tokens, and the Key (K) serves as an index of the information that each token can provide.

[0084] In the present invention, attention weight information (A L ) is calculated through the dot-product of the query (Q) and the key (K). As shown in the formula in Fig. 5, the attention weight information follows the Scaled Dot-Product Attention method and can be calculated by the following [Equation 1].

[0086] [Mathematical Formula 1]

[0087]

[0088] Preferably, Q is the query matrix, K T is the transpose of the key matrix, d k corresponds to the number of dimensions of the key vector, and is a scaling factor used to increase learning stability by adjusting the scale of the inner product value.

[0090] The attention weight information produced as a result of the above operation takes the form of an N*N square matrix corresponding to the number of tokens (4 in the case of Fig. 5). Due to Softmax, each component value in the matrix (e.g., 0.40, 0.55, etc.) is normalized to a scalar value between 0 and 1, which represents the level of attention indicating how importantly a specific token references another token.

[0091] For example, in the matrix of Fig. 5, the value 0.40 in row 1, column 1 means that the first token (F1) refers to itself (F1) with a weight of 40%, and the value 0.30 in row 1, column 2 means that the first token (F1) refers to the second token (F2) with a weight of 30%.

[0093] FIG. 6 schematically illustrates a method of averaging and utilizing a plurality of head values ​​in a multi-head attention structure according to one embodiment of the present invention.

[0095] As illustrated in FIG. 6, the first model has a multi-head attention structure, and the attention weight information and the gradient information are each characterized by using the average of values ​​calculated from a plurality of heads.

[0097] Specifically, as an embodiment illustrated in FIG. 6, the first model may have a multi-head attention structure in which multiple (e.g., 4) heads perform operations in parallel, rather than a single attention. This is intended to extract rich feature information by analyzing the input image from various perspectives. For example, Head 1 may perform shape-centered operations, and Head 2 may perform texture-centered operations.

[0098] The information acquisition unit (1100) integrates the attention weight information (attention map) calculated from each of the plurality of heads formed in each layer to finally generate one representative attention weight information. As a basic embodiment, the attention map (A) of each head as shown in [Equation 2] below h You can use a method of taking a simple average of ).

[0100] [Mathematical Formula 2]

[0101]

[0102] Preferably, H is the total number of heads, A h corresponds to the attention map of the h-th head.

[0103] In addition, the gradient information corresponding to the above attention weight information may also use the average value of the gradient values ​​calculated for each head, or a single gradient map calculated by taking the partial derivative of the loss value (Loss) based on the integrated attention weight information.

[0105] Meanwhile, in one embodiment illustrated in FIG. 6, the number of heads is exemplified as 4, but this is merely to aid in understanding the invention and may be changed differently depending on the size of the control model or the resolution of the input image.

[0107] FIG. 7 schematically illustrates the concept of applying different weights to each head group according to another embodiment of the present invention.

[0109] As illustrated in FIG. 7, the plurality of heads are classified into a plurality of head groups trained to detect preset visual attributes, and each of the attention weight information and the gradient information is characterized as a weighted average value calculated by assigning different weights to each of the plurality of head groups according to the type of predicted behavior derived by the second model.

[0111] Specifically, the present invention does not stop at averaging all heads equally, but can actively adjust the importance of visual information according to the nature of the task to be performed by the robot.

[0112] To this end, the plurality of heads may be classified into a plurality of head groups specialized to detect preset visual attributes. For example, they may be divided into shape / position-centered head groups that primarily learn the contours or positional information of an object, and color / texture-centered head groups that primarily learn the color or surface texture of an object.

[0114] In this embodiment, the information acquisition unit (1100) dynamically changes the weight to be applied to each head group according to the type of predicted behavior predicted by the second model.

[0115] For example, when a robot grasps a cup handle, accurately determining the object's shape and location is essential for a precise grip. Therefore, high weights are assigned to the 'Shape-Centric' and 'Location-Centric' head groups, while lower weights are assigned to the relatively less important 'Color-Centric' and 'Texture-Centric' head groups for summation.

[0116] In another embodiment, when a robot selects a red apple, color information can be a more decisive clue than shape in object classification tasks. Therefore, the highest weight is assigned to the 'color-centric' head group to clearly reflect in the heatmap whether the robot focused on color information when making a decision.

[0118] By adopting this 'behavior-tailored weighted average' method, the present invention can selectively highlight and visualize the visual evidence most closely related to the robot's behavioral intention, which can provide the effect of offering significantly higher explanatory power compared to a simple average method.

[0120] Meanwhile, the aforementioned weights may use predefined fixed values, but can also be automatically adjusted through a pre-configured gating network.

[0122] FIG. 8 schematically illustrates the process of calculating gradient information according to one embodiment of the present invention.

[0124] As illustrated in FIG. 8, a method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot performed in a computing system (1000) comprising one or more processors and one or more memories comprises: a first information calculation step (S200) for calculating gradient information in the form of a matrix representing the rate of change of the loss value of the control model with respect to the attention weight information.

[0125] In addition, the first information calculation step (S200) calculates the gradient information by partially differentiating the loss value with respect to the attention weight information of the first model.

[0127] Specifically, the first information output unit (1200) calculates gradient information based on the attention weight information of the first model, based on the loss value, which is the error between the predicted action and the correct action output by the control model.

[0128] As illustrated in FIG. 8(a), the gradient information (G L ) is the loss value (Loss) as shown in [Equation 3] below, the attention weight (A L It is a matrix obtained by taking partial derivatives with respect to ).

[0130] [Mathematical Formula 3]

[0131]

[0132] Preferably, the calculated gradient matrix (G L ) is the attention weight matrix (A L It has the same size as ). As in one embodiment illustrated in FIG. 8(b), a specific component (G) of the gradient matrix i,j ) means “the sensitivity of the behavior (Attention) of the i-th token (Query) referencing the j-th token (Key) to the final result, the loss value (Loss).”

[0133] For example, the gradient component (G) of row 2 and column 4 2,4 ) is an indicator representing “how much the robot’s action error (Loss) changes when the proportion of patch token 2 (F2) referencing patch token 4 (F4) is slightly changed.” If the value of the corresponding component is a large negative (-), it means that the error decreases as the reference relationship is strengthened; therefore, that area is interpreted as having made an essential contribution to the robot’s correct action decision. Conversely, if the value is close to 0, it means that the reference was meaningless information (e.g., background) for the action decision.

[0135] Therefore, the present invention can generate visualization information with clear causal relationships by selecting 'places that were actually helpful' through gradient information, rather than simply deriving 'places that were frequently viewed'.

[0137] FIG. 9 schematically illustrates the process of generating a contribution matrix according to one embodiment of the present invention.

[0139] As illustrated in FIG. 9, the second information calculation step (S300) includes a matrix generation step that generates a contribution matrix representing the contribution of each of the plurality of patches to the loss value by performing element-wise multiplication of the attention weight information and the gradient information.

[0141] Specifically, the second information output unit (1300) combines the previously calculated attention weight information and gradient information to generate a contribution matrix representing the actual contribution relationship between each patch.

[0142] Preferably, in this specification, the 'Column' component of the attention weight information and contribution matrix may be defined as representing the destination token to which the information reaches, and the 'Row' component may be defined as representing the source token of the information.

[0143] The above contribution matrix (E L ) is attention weight information (A as shown in [Equation 4] below. L Each component of ) and gradient information (G L It is produced through an element-wise multiplication operation that multiplies each corresponding component of ) one-to-one.

[0145] [Mathematical Formula 4]

[0146]

[0147] As a technical feature of the present invention, the technical significance of this process lies in filtering only 'valid grounds' from 'simple attention'. As illustrated in the example at the bottom of FIG. 9, the component of row 2, column 4 (E 2,4 ) is 'Token 2 (F2) references Token 4 (F4) (A 2,4 ) and the referencing behavior contributed to the reduction of the loss value (G 2,4 It indicates the 'degree of'.

[0148] Even if the robot looks at a specific area a lot (high A), if that area is a background meaningless to the result, the gradient (G) converges to 0 and the contribution (E) becomes low. Conversely, when the robot looks at a specific area a lot (high A), if that area was decisive in getting the correct answer (high absolute value negative G), the contribution (E) will have a high value (large absolute value).

[0150] Therefore, semantically, the above contribution matrix (E L ) can be described as data that quantitatively extracts the amount of information that contributed to increasing the accuracy of the prediction behavior among the flow of information generated in the corresponding layer (L) of the first model.

[0152] Meanwhile, the second information output step (S300) may further include a threshold filtering step in which components having an absolute value less than a preset threshold in the contribution matrix are replaced with 0. This is to remove noise components that have a minor impact on the robot's behavior decision, thereby improving the visibility of the heatmap information and further highlighting the key contribution area.

[0154] FIG. 10 schematically illustrates the process of calculating a cumulative contribution matrix according to one embodiment of the present invention.

[0156] As illustrated in FIG. 10, the second information output step (S300) further includes: a matrix correction step for generating a corrected contribution matrix that reflects residual connections by summing the identity matrix to the contribution matrix; and a matrix accumulation step for sequentially multiplying the corrected contribution matrices generated for each layer from the input layer to the output layer of the first model to generate a cumulative contribution matrix that accumulates the information flow between layers.

[0158] Specifically, the second information output unit (1300) uses the previously calculated contribution matrix to perform an operation that integrates the residual connection, which is a structural feature of the transformer, and the information flow of the multilayer.

[0159] First, as illustrated in FIG. 10(a), the second information output unit (1300) has a contribution matrix (E) of each layer (L). L The correction contribution matrix (K) is obtained by summing the identity matrix (I) to ). L Creates ).

[0160] Adding the identity matrix (I) here is intended to mathematically preserve the skip connection path in the Transformer model, where input information is passed directly to the next layer without undergoing attention operations. If this process were omitted, the information from the original input image would be diluted as one moves to deeper layers, potentially degrading the accuracy of the heatmap.

[0162] Next, as illustrated in FIG. 10(b), the second information output unit (1300) includes an input layer (K) included in the first model. 1 From ) to the output layer (K 4 Correction contribution matrices (K) generated sequentially up to ) 1 , K 2 , K 3 , K 4The matrix is ​​multiplied and accumulated. Through this rollout process, the computing system (1000) can globally track how the information of the input image is transformed and transmitted as it passes through numerous layers. The finally calculated cumulative contribution matrix is ​​like a final report card indicating which pixel of the input image the result judged at the deepest part of the model originated from.

[0164] FIG. 11 schematically illustrates the process of generating heatmap information according to one embodiment of the present invention.

[0166] As illustrated in FIG. 11, the second information output step (S300) further includes: a column vector extraction step for extracting a column vector corresponding to the classification token from the cumulative contribution matrix; and a heatmap information generation step for generating the heatmap information by rearranging the component values ​​of the extracted column vector into a two-dimensional form corresponding to a preset patch array of the input image.

[0168] Specifically, the learning information generation unit (1400) receives the cumulative contribution matrix produced through the preceding rollout process. The cumulative contribution matrix is ​​data that aggregates all information flows from the input end to the output end of the first model.

[0169] The first step of this stage is 'extraction of contribution vectors'. The first column of the cumulative contribution matrix is ​​a set of contribution scores indicating how much weight the classification token (F1), representing the entire context, and the remaining patch tokens (F2 to F197) contributed to the information flow in making a final judgment. Accordingly, the learning information generation unit (1400) extracts this first column and then separates only the remaining 196 components (contribution values ​​of each patch for the entire context) excluding the value of the classification token itself (component (1,1) of the matrix) to construct a contribution vector.

[0170] Meanwhile, the numbers '0' to '196' shown in Fig. 11 are examples of the identification numbers (Index) of each patch token.

[0172] The second process is 'spatial rearrangement'. The one-dimensional contribution vector consisting of the 196 components mentioned above is rearranged into a 14x14 two-dimensional grid in correspondence with the spatial location information in which the original input image was divided into pre-set grid units. At this point, the data is in the initial heatmap state where the importance of each patch area is numerically mapped.

[0173] The final process is 'normalization and overlay'. As an example, the calculated heatmap values ​​may be normalized to values ​​between 0 and 1 (or between 0 and 255) to aid visual perception, and high-importance areas may be mapped in red or with high brightness, while low-importance areas may be mapped in blue or transparent. Finally, the heatmap information containing the heatmap is semi-transparently overlaid on the original input image and provided as learning state information that allows the user to intuitively identify the object and location where the robot's gaze is focused.

[0175] FIG. 12 schematically illustrates the combined structure of the first model and the second model and the backpropagation process of the behavior-centered loss value according to one embodiment of the present invention.

[0177] As illustrated in FIG. 12, the control model includes: a first model that receives input information including the input image and outputs an embedding including feature information of the input image; and a second model that receives the output value of the first model and non-image information and outputs a predicted behavior of the automation robot.

[0178] In addition, the loss value of the above control model is calculated from the difference between the predicted action value of the automation robot output by the second model and the predefined correct action value.

[0180] Specifically, the control model includes a structure in which a first model that functionally processes visual information and a second model that determines behavior are connected in series. First, the first model processes the input image in patch units to output an embedding vector that encapsulates the visual features of the input image. Preferably, the first model follows the Vision Transformer (ViT) structure described above, and attention weight information may be generated internally during this process.

[0182] Next, the second model receives the embedding (visual information) output from the first model and non-image information obtained from a separate sensor or input device. The second model fuses and analyzes these two types of heterogeneous data, and then calculates and outputs the final predicted action value that the robot must perform.

[0184] The feature of the present invention lies in the learning and backpropagation mechanism. As shown at the bottom of FIG. 12, the computing system (1000) calculates a loss value by calculating the error between the predicted behavior output by the second model and the predefined correct behavior (Ground Truth). Subsequently, the loss value is propagated in the reverse direction as indicated by the arrow. That is, the loss value passes through the second model and is backpropagated to the first model, and in this process, gradient information, heatmap information, and learning state information that identify the visual cause of the robot performing the action incorrectly (or correctly) are calculated within the first model.

[0186] Through this structure, the present invention can achieve the effect of performing behavior-centered visualization that specifically identifies the causal relationship regarding 'why the robot performed such an action' on the visual image, rather than simply explaining 'what the input image is'.

[0188] FIG. 13 illustrates, in an exemplary manner, the configuration of non-image information input to a control model according to one embodiment of the present invention. FIG. 14 also illustrates, in an exemplary manner, the configuration of a predicted behavior output by a control model according to one embodiment of the present invention.

[0190] As illustrated in FIG. 13, the non-image information is characterized by including at least one of the pose of the automation robot, the actuator state, and a user's control command.

[0191] In addition, as illustrated in FIG. 14, the predicted behavior is characterized by being a continuous numeric value including at least one of the angle value of a joint constituting the automation robot, the open / closed state of a gripper, and the three-dimensional coordinate value of an end effector.

[0193] As in one embodiment illustrated in FIG. 13, non-image information may include pose information, state information, and control command information.

[0194] Pose information may include angle values ​​currently taken by each joint of the robot, spatial position and orientation data of the links, etc.

[0195] State information may include the status of the end effector, which is the robot's end device (e.g., the open / closed state of the gripper), the torque load of the actuator, or the battery status. Control command information includes language commands such as "grab the cup" or "move to the right" entered by the user, or target coordinate values.

[0196] These non-image information are converted into vector form, combined with the embedding vector (image) of the first model, and then used as input to the second model. Through this, the control model can make a comprehensive judgment such as, “My arm is currently bent at 30 degrees (pose), and since the user has told me to grab the cup (command), I should extend my arm further toward the cup visible in front of me (image).”

[0198] In addition, as in one embodiment illustrated in FIG. 14, the predicted behavior data finally output by the second model includes specific numerical values ​​for physically controlling the robot, and the predicted behavior may include at least one of a joint angle, a gripper opening / closing, and end effector coordinates.

[0199] The computing system (1000) according to the present invention can calculate a loss value by calculating the difference between the predicted action values ​​and the correct action that the actual robot must perform, and based on this, perform the backpropagation and heat map generation process described above. That is, when an error occurs such as “the joint angle should have been turned 5 degrees more but was turned less,” it visually identifies which part of the input image was misread.

[0200] Through the learning state information in the form of images, the user can intuitively grasp the causal relationship that “this angle error occurred because, although the robot should have accurately aimed at the cup handle, it actually referenced the surrounding shadow area more strongly.”

[0202] This can maximize the performance of automated robots by providing crucial clues for setting the direction of control model retraining or correcting errors in the dataset.

[0204] FIG. 15 illustrates the internal configuration of a computing device (11000) according to one embodiment of the present invention.

[0206] The computing system (1000) mentioned in the description of FIG. 1 may include components of the computing device (11000) illustrated in FIG. 15, which will be described later.

[0208] As illustrated in FIG. 15, the computing device (11000) may include at least one processor (11100), memory (11200), peripheral interface (11300), input / output subsystem (I / O subsystem) (11400), power circuit (11500), and communication circuit (11600).

[0210] Specifically, the memory (11200) may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. The memory (11200) may include software modules, instruction sets, or various other data required for the operation of the computing device (11000).

[0211] At this time, access to the memory (11200) from other components, such as the processor (11100) or the peripheral device interface (11300), can be controlled by the processor (11100). The processor (11100) may be composed of a single or multiple units and may include processors in the form of GPUs and TPUs to improve computational processing speed.

[0212] The above peripheral device interface (11300) can connect input and / or output peripheral devices of the computing device (11000) to the processor (11100) and the memory (11200). The processor (11100) can perform various functions for the computing device (11000) and process data by executing a software module or instruction set stored in the memory (11200).

[0213] The input / output subsystem (11400) may connect various input / output peripheral devices to the peripheral device interface (11300). For example, the input / output subsystem (11400) may include a controller for connecting peripheral devices such as a monitor, keyboard, mouse, printer, or, if necessary, a touchscreen or sensor to the peripheral device interface (11300). According to another aspect, the input / output peripheral devices may be connected to the peripheral device interface (11300) without passing through the input / output subsystem (11400).

[0214] The power circuit (11500) may supply power to all or part of the components of the terminal. For example, the power circuit (11500) may include one or more power sources such as a power management system, a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for power generation, management, and distribution.

[0215] The communication circuit (11600) may enable communication with another computing device using at least one external port. Alternatively, as described above, the communication circuit (11600) may enable communication with another computing device by including an RF circuit and transmitting and receiving an RF signal, also known as an electromagnetic signal, as needed.

[0217] The embodiment of FIG. 15 is merely an example of the computing device (11000), and the computing device (11000) may have some components shown in FIG. 15 omitted, additional components not shown in FIG. 15 added, or a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may include a touchscreen or sensors in addition to the components shown in FIG. 15, and the communication circuit (1160) may include a circuit for RF communication of various communication methods (Wi-Fi, 3G, LTE, 5G, 6G, Bluetooth, NFC, Zigbee, etc.). The components that can be included in the computing device (11000) may be implemented as hardware, software, or a combination of both hardware and software, including one or more integrated circuits specialized for signal processing or applications.

[0218] Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. An application to which the present invention is applied may be installed on a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file upon a request from the user terminal.

[0220] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0221] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be standardized and stored or executed in a standardized manner on a networked computing device. Software and data may be stored on one or more computer-readable recording media.

[0222] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0224] In one embodiment of the present invention, by applying gradient information regarding loss values ​​as weights to the attention weight information of a transformer-based control model to calculate a contribution matrix, the model can clearly distinguish between the area it simply observes and the area that contributed to the actual action decision, thereby providing visualization information with a clear causal relationship.

[0225] In one embodiment of the present invention, by applying a method of summing the identity matrix to the contribution matrix and accumulating it layer by layer, the flow of information preserved through residual connections of the transformer structure can be tracked without omission, thereby enabling the effect of preventing the heatmap information from being lost or diluted even when passing through deep layers.

[0226] In one embodiment of the present invention, by performing backpropagation based on the robot's behavioral error (Loss) predicted by the second model, it is possible to achieve the effect of presenting a behavior-oriented basis for judgment regarding the specific control actions of the robot, rather than simple image classification.

[0227] In one embodiment of the present invention, by generating learning state information using only computational parameters within the control model, it is possible to achieve the effect of implementing an explainable artificial intelligence efficiently and economically without the need to construct separate correct answer data for explanation or provide a separate explanation generation module.

[0228] In one embodiment of the present invention, by intuitively identifying whether the robot has referenced background noise or interfering factors during the decision-making process through the generated heatmap information, it is possible to quickly debug the cause of malfunction in the control model and significantly improve the operational safety and reliability of the robot automation system.

[0230] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents to the claims below are also within the scope of the claims.

Claims

Claim 1 A method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot performed in a computing system comprising one or more processors and one or more memories, wherein the control model is based on a transformer algorithm and comprises: a first model that receives input information including an input image and outputs an embedding including feature information of the input image; and a second model that fuses two types of heterogeneous data, which are the output value of the first model and non-image information, to output a predicted behavior of the automation robot; and for the input image, an information acquisition step of acquiring attention weight information in a matrix form derived from query information and key information in each of one or more layers of the first model; a first information calculation step of calculating gradient information in a matrix form representing the rate of change of the loss value of the control model with respect to the attention weight information; and a second information calculation step of calculating heatmap information representing the importance of each region of the input image based on the attention weight information and the gradient information. A method for providing learning state information, comprising: a learning information generation step of generating image-shaped learning state information by superimposing the heatmap information and the input image; wherein the loss value of the control model is calculated from the difference between the predicted action value of the automation robot output by the second model and a predefined correct action value, and the first information calculation step calculates the gradient information by taking the partial derivative of the loss value backpropagated to the first model through the second model with respect to the attention weight information of the first model. Claim 2 A method for providing learning state information according to claim 1, wherein the information acquisition step further comprises: a token generation step of dividing the input image into a plurality of patches having a preset size and embedding them to generate a plurality of patch tokens; and a sequence configuration step of combining a classification token for learning the entire context with the preceding portion of the plurality of patch tokens to form an input token sequence of the first model; wherein the attention weight information and the gradient information are a matrix calculated based on the input token sequence. Claim 3 A method for providing learning state information according to claim 2, wherein the second information generation step comprises a matrix generation step of generating a contribution matrix representing the contribution of each of the plurality of patches to the loss value by performing element-wise multiplication of the attention weight information and the gradient information. Claim 4 A method for providing learning state information according to claim 3, wherein the second information output step further comprises: a matrix correction step of generating a corrected contribution matrix reflecting residual connections by summing an identity matrix to the contribution matrix; and a matrix accumulation step of sequentially matrix-multiplying the corrected contribution matrices generated for each layer from the input layer to the output layer of the first model to generate a cumulative contribution matrix that accumulates the information flow between layers. Claim 5 A method for providing learning state information according to claim 4, wherein the second information output step further comprises: a column vector extraction step for extracting a column vector corresponding to the classification token from the cumulative contribution matrix; and a heatmap information generation step for generating the heatmap information by rearranging the component values ​​of the extracted column vector into a two-dimensional form corresponding to a preset patch array of the input image. Claim 6 delete Claim 7 delete Claim 8 A method for providing learning state information according to claim 1, wherein the non-image information comprises at least one of the pose of an automated robot, the state of an actuator, and a control command of a user. Claim 9 A method for providing learning state information according to claim 1, wherein the predicted action is a continuous numeric value comprising at least one of the angle value of a joint constituting an automated robot, the open / closed state of a gripper, and the three-dimensional coordinate value of an end effector. Claim 10 A method for providing learning state information according to claim 1, wherein the first model has a multi-head attention structure, and the attention weight information and the gradient information each utilize the average of values ​​calculated from a plurality of heads. Claim 11 A computing system comprising one or more processors and one or more memories, and performing a method for providing learning state information of an artificial neural network-based control model of a vision-based automation robot, wherein the control model is based on a transformer algorithm and includes: a first model that receives input information including an input image and outputs an embedding including feature information of the input image; and a second model that fuses two types of heterogeneous data, which are the output value of the first model and non-image information, to output a predicted behavior of the automation robot; and for the input image, an information acquisition unit that acquires attention weight information in a matrix form derived from query information and key information in each of one or more layers of the first model; a first information calculation unit that calculates gradient information in a matrix form representing the rate of change of the loss value of the control model with respect to the attention weight information; and a second information calculation unit that calculates heatmap information representing the importance of each region of the input image based on the attention weight information and the gradient information. A computing system comprising: a learning information generation unit that generates learning state information in the form of an image by superimposing the heatmap information and the input image; wherein the loss value of the control model is calculated from the difference between the predicted action value of the automation robot output by the second model and a predefined correct action value, and the first information calculation unit calculates the gradient information by taking the partial derivative of the loss value backpropagated to the first model through the second model with respect to the attention weight information of the first model.

Images