Method and computing device for visualizing artificial intelligence model

Abstract

According to one embodiment disclosed herein, disclosed is a method performed by a user terminal for providing a user interface (UI) for visualizing an artificial intelligence model. The method may comprise the steps of: receiving an artificial intelligence model to be visualized; rendering, on the user interface, a topology in a graph format in which an abstracted structure of the artificial intelligence model is expressed visually, wherein the graph format includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes; displaying a topology change option object on the user interface when a user selection input selecting one area on the topology of the user interface is obtained, wherein the topology change option object has a correlation with the selected area; and on the basis of a user change input requesting a change in the topology change option object, re-rendering the topology on the user interface so that the change is reflected in the topology.

Description

Method and computing device for visualizing artificial intelligence models

[0001] The present disclosure relates to artificial intelligence technology, and more specifically to a method and apparatus for visualizing an artificial intelligence model.

[0002] Driven by advancements in deep learning technology, artificial intelligence models are being utilized in various fields. In particular, AI models demonstrate exceptional performance in solving complex problems, such as computer vision, natural language processing (NLP), and speech recognition. AI models find optimal solutions to specific problems through training on large-scale data, and their performance and accuracy continue to improve.

[0003] Artificial intelligence models are structured in the form of computational graphs, where numerous operations are performed sequentially or in parallel. To represent the series of processes involved in performing complex operations on input data and deriving a final result, a computational graph represents each operation through operators, and these operators can be interconnected according to the flow of data.

[0004] The internal structure of an artificial intelligence model consists of a combination of complex operators. For example, a Convolutional Neural Network (CNN) can be composed of a combination of convolution operators, pooling operators, and activation function operators. The operators constituting an AI model process data and, depending on the connections between them, can pass the processing results to the next operator. The types of these operators and their connections can determine the overall operation of the AI ​​model.

[0005] As processor processing speeds increase, the scale of AI models grows and the number of operators increases exponentially, making it extremely difficult to grasp the overall computational graph structure. These large-scale AI models contain billions of parameters. Training and performing inference on such models requires a vast amount of computing resources. The use of computing resources can lead to high operational costs in cloud environments and becomes a major constraint that makes model deployment and utilization difficult in resource-constrained environments, such as edge devices.

[0006] Throughout the development lifecycle of an artificial intelligence model, developers may continuously generate new versions to improve the performance of existing models. This process may involve various modifications, such as structural changes like adding or removing specific operators, the application of pruning or quantization for model lightweighting, or fine-tuning through hyperparameter changes. Consequently, multiple model versions are derived, making it a critical task to clearly identify and manage the structural and functional differences between them.

[0007] Existing technologies related to artificial intelligence models primarily provide functions that display performance metrics of the AI ​​model (e.g., accuracy, loss rate), visualize activation maps of specific operators or layers, or provide a schematic representation of the overall structure of the AI ​​model.

[0008] KR10-2021-0023401A presents a neural network computation method.

[0009] The present disclosure is devised in response to the aforementioned background technology and is intended to efficiently provide graph visualization of various artificial intelligence models.

[0010] The present disclosure aims to enhance the user experience through a user interface for visualizing artificial intelligence models.

[0011] The technical problems of the present disclosure are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below.

[0012] According to one embodiment of the present disclosure, a method for visualizing an artificial intelligence model executed by a computing device (e.g., a server) is disclosed. The method may include: receiving an artificial intelligence model from a user terminal; generating an intermediate representation structure by abstracting the artificial intelligence model; converting the intermediate representation structure into structured text data for the visualization of the artificial intelligence model using attribute values ​​included in the intermediate representation structure; and transmitting the structured text data to the user terminal so that the user terminal renders a graph-format topology for the artificial intelligence model.

[0013] In one embodiment, the step of converting the intermediate representation structure into the structured text data may include the step of querying attribute values ​​of the intermediate representation structure to extract the input of the artificial intelligence model, the output of the artificial intelligence model, nodes included in the artificial intelligence model, connection information between the nodes, and metadata included in the artificial intelligence model from the intermediate representation structure, and the step of generating the structured text data by arranging the extracted information according to a predefined text data structure.

[0014] In one embodiment, the step of generating the intermediate representation structure may include the step of generating the intermediate representation structure by abstracting the artificial intelligence model to include a node independent of the framework and a tensor representing the output of the node.

[0015] In one embodiment, the step of converting the intermediate representation structure into the structured text data may include the step of generating the structured text data by changing the tensors included in the intermediate representation structure into edges connecting the nodes.

[0016] In one embodiment, the intermediate representation structure may include nodes independent of the framework, a first tensor representing parameters of the artificial intelligence model, and a second tensor representing activation values ​​of the nodes. The step of converting the intermediate representation structure into the structured text data may include generating edges connecting the nodes using the second tensor and the nodes, and converting the intermediate representation structure into the structured text data using the first tensor, the nodes, and the edges.

[0017] In one embodiment, the structured text data may have a text structure in which a node representing a basic operation unit, a module representing a set of nodes, an edge connecting the nodes, a tensor representing a parameter, and metadata representing quantization-related information are each represented as a pair of key and value.

[0018] In one embodiment, the step of converting the intermediate representation structure into the structured text data may include: a step of extracting node identification information, input tensor identification information, output tensor identification information, and operator type information from each of the plurality of nodes by traversing each of the plurality of nodes included in the intermediate representation structure in the direction from input to output of the artificial intelligence model; and a step of generating the structured text data for each of the plurality of nodes by using the extracted information as a value and mapping it to a key of the structured text data.

[0019] In one embodiment, as the first schema of the structured text data is changed to a second schema for graph rendering, the structured text data is reconstructed to be represented as nodes and edges connecting the nodes, and the topology of the graph format for the artificial intelligence model can be rendered based on the reconstructed structured text data.

[0020] In one embodiment, the step of generating the intermediate representation structure may include the step of generating a first intermediate representation structure by abstracting the artificial intelligence model, and the step of generating a second intermediate representation structure by modifying some nodes of the first intermediate representation structure. The step of converting the intermediate representation structure into the structured text data may include: the step of obtaining node performance change values ​​for the modified nodes within the second intermediate representation structure by performing inference using the first intermediate representation structure and inference using the second intermediate representation structure; and the step of generating the structured text data including the node performance change values ​​from the first intermediate representation structure or the second intermediate representation structure.

[0021] In one embodiment, the method may further include: in response to receiving a change request for the artificial intelligence model from the user terminal, identifying the target of the change request within the artificial intelligence model using information included in the change request among a node identification number, a node name, a node operator type, an edge identification number, and an edge name, and changing the intermediate representation structure of the artificial intelligence model according to the change request; changing the structured text data based on the changed intermediate representation structure; and transmitting the changed structured text data to the user terminal.

[0022] In one embodiment, a method for visualizing an artificial intelligence model is disclosed, which is performed by a computing device (e.g., a user terminal). The method may include: receiving structured text data corresponding to an artificial intelligence model from a server; extracting nodes and edges from the structured text data; reconstructing the structured text data by changing a first schema of the structured text data into a second schema for graph rendering using the nodes and edges; and rendering a graph-format topology for the artificial intelligence model using the reconstructed structured text data.

[0023] In one embodiment, the method may further include: a step of re-rendering a modified topology corresponding to a modification request without waiting for a response related to the modification request from the server in response to receiving a user input including a modification request for the rendered topology; and a step of rolling back the modified topology to a topology prior to the modification in response to receiving a modification failure response from the server.

[0024] In one embodiment, the method may further include: a step of re-rendering a modified topology corresponding to a modification request without waiting for a response related to the modification request from the server in response to receiving a user input including a modification request for the rendered topology; and a step of, when a modification success response is received from the server, integrating metadata and tensors included in the modified structured text into at least one of the nodes and edges of the re-rendered modified topology using the modified structured text data included in the modification success response.

[0025] In one embodiment, the method may further include the step of receiving a user input including a request to change the rendered topology, transmitting change information related to the change request to the server, and re-rendering the changed topology corresponding to the change request without waiting for a response related to the change request from the server. As the change information related to the change request is transmitted to the server, a changed artificial intelligence model corresponding to the re-rendered topology may be generated.

[0026] In one embodiment, the step of reconstructing the structured text data may include the step of converting the first schema, which includes nodes, edges, tensors, and metadata arranged separately, into the second schema represented by the nodes and edges by integrating the tensors and metadata in the structured text data into at least one of the nodes and edges.

[0027] In one embodiment, the step of receiving the structured text data may include receiving the first structured text data corresponding to the first artificial intelligence model and the second structured text data corresponding to the second artificial intelligence model.

[0028] In one embodiment, the step of rendering the topology in graph form may include: identifying differential nodes representing the difference between the first topology of the first artificial intelligence model and the second topology of the second artificial intelligence model using identification information of nodes included in each of the first structured text data and the second structured text data; and re-rendering at least one of the first topology and the second topology using the differential nodes. Predefined visual attributes may be applied to the differential nodes so that the differential nodes can be re-rendered visually distinguished from common nodes existing in both the first topology and the second topology.

[0029] In one embodiment, the step of rendering the graph-formatted topology for the artificial intelligence model may include determining a display color corresponding to the node based on a node performance value or a node performance change value inherent in the node within the topology, and rendering the node within the topology with the determined display color.

[0030] In one embodiment, the display color corresponding to the node may be determined such that the saturation or luminance on the color gradient has a positive correlation with the node performance value or the node performance change value.

[0031] In one embodiment, the step of rendering the graph-formatted topology for the artificial intelligence model may include: a step of rendering the topology so that the first node and the second node are visually integrated by applying common visual attributes to the first node and the second node belonging to corresponding layers within the topology based on layer identification information inherent in the nodes within the topology; or a step of rendering the topology so that the edges of the first node and the second node are visually integrated by applying common visual attributes to the edges connected to each of the first node and the second node belonging to corresponding layers within the topology based on layer identification information inherent in the nodes within the topology.

[0032] In one embodiment, the method may further include: receiving a user input including a request to change the rendered topology; determining the type of the change request based on the user input; and determining whether to interact with the server for re-rendering the topology based on the type of the change request.

[0033] In one embodiment, the step of determining whether to interact with the server may include: if the type of the change request is determined to be a first type in which nodes and edges are not changed, re-rendering the topology in which the positions of the nodes and edges are changed according to the change request without interacting with the server; if the type of the change request is determined to be a second type including the creation of nodes, determining transmission information to be transmitted to the server based on an operator corresponding to the node to be created; if the type of the change request is determined to be a third type including the deletion of nodes, transmitting node identification information corresponding to the node to be deleted to the server; and if the type of the change request is determined to be a fourth type including the creation of edges, transmitting identification information for the start node and end node corresponding to the edge to be created to the server.

[0034] In one embodiment, a server for visualizing an artificial intelligence model is disclosed. The server may include at least one processor. The processor may perform the steps of: receiving an artificial intelligence model from a user terminal; generating an intermediate representation structure by abstracting the artificial intelligence model; converting the intermediate representation structure into structured text data for the visualization of the artificial intelligence model using attribute values ​​included in the intermediate representation structure; and transmitting the structured text data to the user terminal so that the user terminal renders a graph-format topology for the artificial intelligence model.

[0035] In one embodiment, a computer program stored on a computer-readable storage medium is disclosed. When executed by a computing device, the computer program allows the computer device to perform a method for visualizing an artificial intelligence model, the method may include: receiving an artificial intelligence model from a user terminal; generating an intermediate representation structure by abstracting the artificial intelligence model; converting the intermediate representation structure into structured text data for the visualization of the artificial intelligence model using attribute values ​​included in the intermediate representation structure; and transmitting the structured text data to the user terminal so that the user terminal renders a graph-format topology for the artificial intelligence model.

[0036] In one embodiment, a user terminal for visualizing an artificial intelligence model is disclosed. The user terminal includes at least one processor, and the processor may perform the steps of: receiving structured text data corresponding to an artificial intelligence model from a server; extracting nodes and edges from the structured text data; reconstructing the structured text data by changing a first schema of the structured text data into a second schema for graph rendering using the nodes and edges; and rendering a graph-format topology for the artificial intelligence model using the reconstructed structured text data.

[0037] In one embodiment, a method for providing a user interface for visualizing an artificial intelligence model, which is performed by a user terminal, is disclosed. The method may include: receiving an artificial intelligence model to be visualized; rendering a graph-type topology on the user interface that visualizes and represents the abstracted structure of the artificial intelligence model, wherein the graph-type includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes; when a user selection input selecting a region on the topology of the user interface is obtained, displaying a topology change option object having a correlation with the selected region on the user interface, wherein the change option displayed on the topology change option object is determined differently based on the location of the selected region; and re-rendering the topology with the change reflected on the user interface based on a user change input requesting a change on the topology change option object.

[0038] In one embodiment, if the selected area corresponds to a node area occupied by a node on the topology, the topology change option object indicates the movement of a node object associated with the node area, the deletion of the node object, and the creation of an edge object associated with the node object; if the selected area corresponds to an edge area occupied by an edge on the topology, the topology change option object indicates the deletion of an edge object associated with the edge area; and if the selected area corresponds to a remaining area not occupied by both a node and an edge on the topology, the topology change option object may indicate the creation of a new node object.

[0039] In one embodiment, when a user change input for changing a node on the topology change option object is obtained, the target node object to be changed, the target edge object connecting the target node object, and the neighbor node object connected to the target node object through the target edge may be recognized together as targets for topology change.

[0040] In one embodiment, the step of re-rendering the topology reflecting the change on the user interface may include: when a user change input creating a new node on the topology change option object is obtained, displaying a new node object at a location related to the selected area and displaying a new node option object requesting additional input related to the new node object; and in response to receiving additional input requesting the creation of a target edge object related to the new node object on the new node option object, re-rendering the modified topology on the user interface to include target edge objects connecting the neighbor node objects of the new node object and the new node object.

[0041] In one embodiment, the step of re-rendering a modified topology on the user interface to include target edges connecting the neighbor node objects of the new node object and the new node object may include the step of re-rendering a topology on the user interface in which the new node object and the neighbor node objects are highlighted to be distinguished from other node objects on the modified topology.

[0042] In one embodiment, the user selection input includes a first selection input and a second selection input different from the first selection input, and when the user selection input is the first selection input, the topology change option object is displayed on the user interface, and when the user selection input is the second selection input, at least one of the attributes and metadata of a node corresponding to the second selection input may be displayed.

[0043] In one embodiment, the step of re-rendering the topology reflecting the change on the user interface may include: when a user change input to delete a node object on the topology change option object is obtained, deleting the node object associated with the selected area and replacing the first edge objects associated with the deleted node object with a second edge object connecting neighboring node objects associated with the deleted node object; and re-rendering the topology reflecting the deleted node object and the second edge object on the user interface.

[0044] In one embodiment, the step of re-rendering the topology reflecting the change on the user interface may include: when a user change input to delete an edge object on the topology change option object is obtained, deleting the edge object associated with the selected area and moving at least one of the node objects and edge objects on the topology based on the location of the deleted edge object; and re-rendering the topology reflecting the deleted edge object and the moved object on the user interface.

[0045] In one embodiment, the user interface may include a first area for displaying the graph-format topology, a search input object for receiving a search input within the topology, a second area including a display option object for determining the display method of components within the topology and a display option object for determining the exposure or hiding of components within the topology, and a third area for displaying at least one of attributes and metadata embedded in a node within the topology and allowing modification input for at least one of the attributes and metadata.

[0046] In one embodiment, when user input is received on a display method selection activation object for determining a display method of nodes within the topology, the user interface may display a display method selection window for determining at least one of a first node display method that distinguishes and displays nodes according to structural characteristics of nodes within the topology and a second node display method that distinguishes and displays nodes according to performance of nodes within the topology.

[0047] In one embodiment, the method may further include: receiving user input information identifying the hardware on which the artificial intelligence model is to be executed; and, among the node objects included in the topology, displaying unsupported node objects that are not supported by the hardware or inefficient node objects whose performance degrades when executed on the hardware in a way that is distinguishable from other node objects on the topology.

[0048] In one embodiment, the method further comprises: receiving a user input selecting the unsupported node object or the inefficient node object; displaying a list of recommended replacement candidate nodes to replace the unsupported node object or the inefficient node object; and re-rendering a topology in which the unsupported node object or the inefficient node object is changed to a replacement node object in response to receiving a user input determining a replacement node within the list of replacement candidate nodes, and the artificial intelligence model may be converted in such a way that the unsupported node or the inefficient node object is changed to the replacement node as change information including the unsupported node object or the inefficient node object and the replacement node object is transmitted to a server.

[0049] In one embodiment, the list of replaceable candidate nodes may display identification information of the replaceable candidate nodes, similarity between the original node to be replaced and the replaceable candidate nodes, and performance information of the replaceable candidate nodes.

[0050] In one embodiment, the method may further include the step of re-rendering the topology in response to a user input selecting a region of a node within the rendered topology on the user interface, so that the selected node and a plurality of sub-nodes constituting the node are displayed together while maintaining the connection structure between the selected node and neighboring nodes.

[0051] In one embodiment, the step of re-rendering the topology reflecting the change on the user interface may include: determining whether to interact with a server for re-rendering the topology based on a change type determined according to the user change input.

[0052] In one embodiment, a computer program stored on a computer-readable storage medium is disclosed. When executed by a user terminal, the computer program enables the user terminal to perform a method for providing a user interface (UI) for visualizing an artificial intelligence model, the method may include: receiving an artificial intelligence model to be visualized; rendering a graph-type topology on the user interface that visualizes and represents an abstract structure of the artificial intelligence model, wherein the graph-type includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes; when a user selection input is obtained for selecting a region on the topology of the user interface, displaying a topology change option object having a correlation with the selected region on the user interface, wherein the change option displayed on the topology change option object is determined differently based on the location of the selected region; and re-rendering the topology with the change reflected on the user interface based on a user change input requesting a change on the topology change option object.

[0053] In one embodiment, a user terminal is disclosed that provides a user interface (UI) for visualizing an artificial intelligence model. The user terminal includes at least one processor, and the processor may perform the steps of: receiving an artificial intelligence model to be visualized; rendering a graph-type topology that visualizes and represents the abstracted structure of the artificial intelligence model on the user interface, wherein the graph-type includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes; when a user selection input selecting a region on the topology of the user interface is obtained, displaying a topology change option object having a correlation with the selected region on the user interface, wherein the change option displayed on the topology change option object is determined differently based on the location of the selected region; and re-rendering the topology with the change reflected on the user interface based on a user change input requesting a change on the topology change option object.

[0054] A method for providing a user interface (UI) for visualizing an artificial intelligence model, which is performed by a user terminal, is disclosed. The method may include: receiving a first artificial intelligence model and a second artificial intelligence model to be visualized; generating a first topology in a graph format that visualizes and represents the abstracted structure of the first artificial intelligence model and generating a second topology in a graph format that visualizes and represents the abstracted structure of the second artificial intelligence model—the graph format includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes—and rendering the first topology in a first area of ​​the user interface and rendering the second topology in a second area of ​​the user interface, thereby rendering the first topology and the second topology so as to be visually comparable on the user interface.

[0055] In one embodiment, the step of rendering the first topology and the second topology so as to be visually comparable on the user interface may include: identifying a differential node representing a difference between the first artificial intelligence model and the second artificial intelligence model by comparing the first topology of the first artificial intelligence model and the second topology of the second artificial intelligence model; and rendering the first topology and the second topology so as to be visually comparable on the user interface in a manner such that the differential node is visually distinguished from common nodes existing in common within the first topology and the second topology by applying predefined visual attributes to the differential node.

[0056] In one embodiment, the step of rendering the first topology and the second topology so as to be visually comparable on the user interface may include: a step of identifying a first difference node that exists in the first AI model but is deleted in the second AI model and a second difference node that does not exist in the first AI model but is added in the second AI model by comparing the first topology of the first AI model and the second topology of the second AI model; and a step of rendering the first topology and the second topology so as to be visually comparable on the user interface by applying a first visual attribute to the first difference node, applying a second visual attribute to the second difference node, and applying a third visual attribute to a common node that is commonly included in the first AI model and the second AI model— wherein at least one of the first visual attribute, the second visual attribute, and the third visual attribute is different from each other. there is.

[0057] In one embodiment, the step of rendering the first topology and the second topology so as to be visually comparable on the user interface may include: generating a comparison result between identification information of nodes included in the first artificial intelligence model and identification information of nodes included in the second artificial intelligence model by traversing the nodes of the first artificial intelligence model and the nodes of the second artificial intelligence model; identifying a difference node representing a difference between the first artificial intelligence model and the second artificial intelligence model based on the comparison result; and reconstructing at least one of the first topology and the second topology using the difference node.

[0058] In one embodiment, the second artificial intelligence model of the second topology is a derived model generated based on user change input regarding the first topology of the first artificial intelligence model, which is the original model, on the user interface, and the step of rendering the first topology and the second topology so as to be visually comparable on the user interface may include: the step of reconstructing the second topology by adding a first node deleted due to the user change input and a first edge associated with the first node on the second topology, and the step of rendering the reconstructed second topology and the first topology so as to be visually comparable on the user interface by applying a visual attribute to the first node to identify the deletion.

[0059] In one embodiment, the step of rendering the reconstructed second topology and the first topology so as to be visually comparable on the user interface may include: a step of rendering the reconstructed second topology and the first topology so as to be visually comparable on the user interface in such a way that the display position of the first node on the first topology corresponds to the display position of the first node on the reconstructed second topology and the visual attributes of the first node on the first topology and the visual attributes of the first node on the reconstructed second topology are different from each other.

[0060] In one embodiment, the second AI model of the second topology is a derived model generated on the user interface based on user change input regarding the first topology of the original model, the first AI model, and the step of rendering the first topology and the second topology so as to be visually comparable on the user interface comprises: a step of reconstructing the second topology by adding a first node deleted due to the user change input and a first edge associated with the first node on the second topology; a step of applying a first visual attribute for identifying deletion to the first node, a step of applying a second visual attribute for identifying addition to the second node added due to the user change input, and a step of rendering the reconstructed second topology and the first topology so as to be visually comparable on the user interface by applying a third visual attribute to a common node existing in common with the first AI model and the second AI model, and the first visual The attributes, the second visual attribute, and the third visual attribute may differ in at least one of color and shape.

[0061] In one embodiment, the step of rendering the first topology and the second topology so as to be visually comparable on the user interface comprises: a first comparison step of determining whether a corresponding node in the first AI model exists for each of the nodes of the second AI model by traversing the nodes of the second AI model; a step of, based on the result of the first comparison step, adding a common node among the nodes of the second AI model where a corresponding node of the first AI model exists to the virtual model structure, and adding a second difference node among the nodes of the second AI model where a corresponding node of the first AI model does not exist, along with a second visual attribute, to the virtual model structure; a second comparison step of identifying a first difference node among the nodes of the first AI model where a corresponding node of the second AI model does not exist by traversing the nodes of the first AI model; identifying a neighbor common node connected to the output of the identified first difference node in the first AI model, and within the virtual model structure to maintain the connectivity relationship of the first topology The method may include the steps of adding the first differential node with a first visual attribute at a location adjacent to the neighbor common node, and generating a differential edge connecting the first differential node and the neighbor common node in the virtual model structure, and rendering the first topology and the second topology so as to be visually comparable on the user interface based on the completed virtual model structure.

[0062] In one embodiment, the second artificial intelligence model of the second topology is a derived model generated based on user change input regarding the first topology of the first artificial intelligence model, which is the original model, on the user interface, and the step of rendering the first topology and the second topology so as to be visually comparable on the user interface may include: applying a first visual attribute to identify deletion to a first node deleted due to the user change input and displaying the first node to which the first visual attribute is applied on the first topology, and applying a second visual attribute to identify addition to a second node added due to the user change input and displaying the second node to which the second visual attribute is applied on the second topology, thereby rendering the first topology and the second topology so as to be visually comparable on the user interface.

[0063] In one embodiment, the method may further include: when a user selection input is obtained for selecting a region on the first topology rendered on the user interface, displaying a topology change option object correlated with the selected region on the user interface; reconstructing the first topology to reflect the change based on a user change input requesting a change on the topology change option object; determining a node to change visual attributes in the second topology by comparing the reconstructed first topology with the second topology; and re-rendering the reconstructed first topology and the second topology reflecting the change in visual attributes on the user interface.

[0064] In one embodiment, the method further includes the step of transmitting change information related to the user change input to a server, and as the change information related to the user change input is transmitted to the server, a modified first artificial intelligence model corresponding to the re-rendered first topology may be generated.

[0065] In one embodiment, the user interface may output one of a first scroll operation for scrolling the first topology, a second scroll operation for scrolling the second topology, and a third scroll operation for scrolling both the first topology and the second topology, based on the starting position of a user scroll input on the user interface, or output one of a first scroll operation for scrolling the first topology, a second scroll operation for scrolling the second topology, and a third scroll operation for scrolling both the first topology and the second topology, based on a user selection input regarding whether topology binding is performed on the user interface.

[0066] In one embodiment, the user interface may output a first scroll operation for scrolling a first topology rendered on the first area when it detects that a user scroll input has started on the first area of ​​the user interface, output a first scroll operation for scrolling a second topology rendered on the second area when it detects that a user scroll input has started on the second area of ​​the user interface, and output a third scroll operation for scrolling the first topology rendered on the first area and the second topology rendered on the second area together when it detects that a user scroll input has started on the third area between the first area and the second area of ​​the user interface.

[0067] In one embodiment, the user interface may output a fourth scroll operation that automatically scrolls the second topology on the second area in response to a user selection input selecting a third node on the first topology of the user interface, so that a target node corresponding to the third node among the nodes of the second topology is displayed at one location in the second area.

[0068] In one embodiment, the user interface may include a search area comprising a first area displaying the first topology, a second area displaying the second topology, and a search input object receiving a search input within the topologies. The user interface may perform an automatic scrolling operation so that each of the nodes of the first topology and the nodes of the second topology corresponding to the search results according to the search input are highlighted at predefined locations in the first area and the second area.

[0069] In one embodiment, the method may further include the step of determining a rendering method of the first topology and the second topology based on one view mode selected by a user among a plurality of predefined view modes. The step of rendering the first topology and the second topology so as to be visually comparable on the user interface comprises:, when the rendering method is determined to be a first view mode, displaying a first difference node that exists in the first AI model but not in the second AI model as a first visual attribute on the first topology, and displaying a second difference node that does not exist in the first AI model but exists in the second AI model as a second visual attribute on the second topology; when the rendering method is determined to be a second view mode, displaying the first difference node as the first visual attribute and the second difference node as the second visual attribute on the second topology so that the difference nodes are integrated into the second topology and visualized; and when the rendering method is determined to be a third view mode, placing the common nodes of the first topology and the second topology at corresponding positions so that the first topology and the second The method may include the step of matching the visual layout of the common nodes by readjusting the position of at least one node of at least one topology.

[0070] In one embodiment, the second artificial intelligence model is a derivative model in which a part of the first artificial intelligence model is modified, and the method may further include the step of displaying at least one of the similarity between the first artificial intelligence model and the second artificial intelligence model, the number of added nodes, and the number of deleted nodes based on the result of comparing the node identifier and edge identifier of the first artificial intelligence model with the node identifier and edge identifier of the second artificial intelligence model.

[0071] In one embodiment, the difference node may include a node representing a structural difference between the first artificial intelligence model and the second artificial intelligence model, and at least one of nodes having different attributes among the nodes corresponding to each other in the first artificial intelligence model and the second artificial intelligence model.

[0072] In one embodiment, a computer program stored on a computer-readable storage medium is disclosed. When executed by a user terminal, the computer program enables the user terminal to perform a method for providing a user interface (UI) for visualizing an artificial intelligence model, wherein the method may include: receiving a first artificial intelligence model and a second artificial intelligence model to be visualized; generating a first topology in a graph format that visualizes and represents the abstracted structure of the first artificial intelligence model and generating a second topology in a graph format that visualizes and represents the abstracted structure of the second artificial intelligence model—the graph format includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes—; and rendering the first topology in a first area of ​​the user interface and rendering the second topology in a second area of ​​the user interface, thereby rendering the first topology and the second topology so as to be visually comparable on the user interface.

[0073] In one embodiment, a user terminal is disclosed that provides a user interface (UI) for visualizing an artificial intelligence model. The user terminal includes at least one processor, and the processor may perform the steps of: receiving a first artificial intelligence model and a second artificial intelligence model to be visualized; generating a first topology in a graph format that visualizes and represents the abstracted structure of the first artificial intelligence model and generating a second topology in a graph format that visualizes and represents the abstracted structure of the second artificial intelligence model—the graph format includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes—and rendering the first topology in a first area of ​​the user interface and rendering the second topology in a second area of ​​the user interface, thereby rendering the first topology and the second topology so as to be visually comparable on the user interface.

[0074] A technique according to one embodiment of the present disclosure can efficiently provide graph visualization of various artificial intelligence models.

[0075] The present disclosure can enhance the user experience through a user interface (UI) for visualizing artificial intelligence models.

[0076] FIG. 1 schematically illustrates a block diagram of a computing device according to one embodiment of the present disclosure.

[0077] FIG. 2 illustrates an exemplary structure of an artificial intelligence model according to one embodiment of the present disclosure.

[0078] FIG. 3 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0079] FIG. 4 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0080] FIG. 5 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0081] FIG. 6 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0082] FIG. 7 illustrates, in accordance with one embodiment of the present disclosure, the topology re-rendering of a user interface based on user change input.

[0083] FIG. 8 exemplarily illustrates the topology re-rendering of a user interface in response to user change input according to one embodiment of the present disclosure.

[0084] FIG. 9 illustrates, in accordance with one embodiment of the present disclosure, the topology re-rendering of a user interface based on user change input.

[0085] FIG. 10 illustrates an exemplary method for providing a user interface for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0086] FIG. 11 illustrates an exemplary user interface for comparing a plurality of artificial intelligence models according to one embodiment of the present disclosure.

[0087] FIG. 12 illustrates an exemplary user interface that provides comparison results of a plurality of artificial intelligence models according to one embodiment of the present disclosure.

[0088] FIG. 13 illustrates an exemplary user interface that provides comparison results of a plurality of artificial intelligence models according to one embodiment of the present disclosure.

[0089] FIG. 14 illustrates an exemplary user interface in which a sub-node for a node of an artificial intelligence model is displayed according to one embodiment of the present disclosure.

[0090] FIG. 15 illustrates an exemplary user interface for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0091] FIG. 16 illustrates an exemplary user interface for determining a display method related to a node of an artificial intelligence model according to one embodiment of the present disclosure.

[0092] FIG. 17 illustrates an exemplary user interface for determining a display method related to a node of an artificial intelligence model according to one embodiment of the present disclosure.

[0093] FIG. 18 is a drawing for understanding color changes related to nodes of an artificial intelligence model according to one embodiment of the present disclosure.

[0094] FIG. 19 illustrates an exemplary user interface for visualizing an artificial intelligence model using hierarchical identification information for nodes of the artificial intelligence model according to one embodiment of the present disclosure.

[0095] FIG. 20 illustrates an exemplary user interface for converting an operator using platform information on which an artificial intelligence model is to be executed, according to one embodiment of the present disclosure.

[0096] FIG. 21 illustrates an exemplary user interface for converting an operator using platform information on which an artificial intelligence model is to be executed, according to one embodiment of the present disclosure.

[0097] FIG. 22 illustrates an exemplary user interface that renders topologies comparablely according to a view mode according to one embodiment of the present disclosure.

[0098] FIG. 23 presents an exemplary methodology for performing comparative rendering according to one embodiment of the present disclosure.

[0099] FIG. 24 presents an exemplary methodology for performing comparative rendering according to one embodiment of the present disclosure.

[0100] FIG. 25 is a schematic diagram of an exemplary computing environment of a computing device according to one embodiment of the present disclosure.

[0101] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions.

[0102] In this specification, a component refers to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or an execution of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. A component may reside within a processor and / or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from one component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).

[0103] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the terms "or" and "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.

[0104] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”

[0105] And, the term "at least one of A or B" should be interpreted to mean "a case including only A," "a case including only B," or "a combination of A and B."

[0106] Terms expressed as N, such as the first, second, or third in the present disclosure, are used to distinguish at least one entity. For example, the entities expressed as the first and second may be identical or different from each other.

[0107] An artificial intelligence model in this disclosure may refer to a set of programs or functions designed with a mathematical and statistical structure to automatically perform tasks such as prediction, classification, generation, and / or inference on a given input by learning patterns from data. An artificial intelligence model may be constructed by modeling the relationship between input and output in the form of a function. The operators and layers of the artificial intelligence model in this disclosure may be used interchangeably depending on the implementation mode. Specific examples of the artificial intelligence model will be described later in FIG. 2.

[0108] An intermediate representation structure in this disclosure may refer to a standardized representation format used in an intermediate stage to optimize a model's computational graph or structure for a specific execution environment or platform. An intermediate representation structure in this disclosure may refer to a temporary data structure for an artificial intelligence model used to visualize and / or optimize the artificial intelligence model. An intermediate representation structure in this disclosure may refer to an intermediate data representation format generated as a result of abstraction that extracts and processes the core features of the artificial intelligence model. An intermediate representation structure may include attribute values ​​for determining the inputs, outputs, nodes, the relationships between nodes, and metadata of the artificial intelligence model.

[0109] For example, an intermediate representation structure may include multiple nodes and edges connecting them. For example, an intermediate representation structure may consist of a graph corresponding to an artificial intelligence model, a module representing a layer (or operator) defined dependently with respect to a framework, and a node representing the smallest unit of operators constituting the module. In one embodiment, the intermediate representation structure may be configured such that all nodes within the intermediate representation structure (or constituting the artificial intelligence model) are included in one module. In one embodiment, the intermediate representation structure may be configured such that all nodes within the intermediate representation structure are included in one or more modules. In one embodiment, the intermediate representation structure in this disclosure may take the form of a graph. For example, the graph associated with the intermediate representation structure in this disclosure may represent the result of abstracting the artificial intelligence model based on the connectivity of modules, nodes, and tensors. The intermediate representation structure in this disclosure may represent an intermediate or temporary structure for modification for execution on various frameworks and / or various platforms of the artificial intelligence model. The intermediate representation structure in this disclosure may include a graph structure. An intermediate representation structure may be created such that nodes independent of the framework and modules, which are sets of nodes, have a hierarchical structure or dependency relationship. The intermediate representation structure of the present disclosure may be configured to include a node corresponding to a framework-independent atomic operator, a module dependent on the framework, and a tensor that connects the nodes and / or the modules. A tensor is a data structure used to represent the inputs, outputs, weights, intermediate values, and / or quantization values ​​of an artificial intelligence model. For example, a tensor may have a multidimensional array structure. For example, a tensor may contain reference information for the transformation or optimization of an artificial intelligence model.For example, a tensor may be stored at an edge connecting nodes, and / or a tensor may be located at an edge connecting modules. For example, a tensor may be located at an input to a module or node, and / or a tensor may be located at an output from a module or node.

[0110] An intermediate representation structure in the present disclosure may refer to a logical structure for expressing the functions of an artificial intelligence model. An intermediate representation structure in the present disclosure may include a combination of nodes corresponding to a minimum unit of an atomic operator that is universally applicable to a plurality of frameworks. For example, if the frameworks of the artificial intelligence models performing the same operation are different, different modules may be included in the intermediate representation structure, and the same nodes may be included in the intermediate representation structure even if the frameworks of the artificial intelligence models performing the same operation are different.

[0111] Abstraction in the present disclosure may be used to express the internal configuration of an artificial intelligence model as a more simplified intermediate representation structure. For example, abstraction may mean a method for representing an artificial intelligence model from a physical or code-based entity into a concept of a logical architecture. For example, abstraction may include transforming an artificial intelligence model into an intermediate representation structure consisting only of nodes, transforming it into an intermediate representation structure consisting of operators, and / or transforming it into a hierarchical intermediate representation structure of nodes and modules. As a result of abstraction of a received artificial intelligence model, an intermediate representation structure in the present disclosure may be generated.

[0112] In the present disclosure, topology may refer to a data structure that defines the arrangement and connection relationships of nodes constituting the structure of an artificial intelligence model. For example, topology may represent how the nodes and edges of an artificial intelligence model are configured in a graph format.

[0113] The performance of an artificial intelligence model in the present disclosure may include the accuracy, efficiency, resource usage, inference speed, degree of model lightweighting, inference latency, and Signal-to-Noise Ratio (SNR) of the artificial intelligence model. In one embodiment, performance may include the performance of the model itself and / or the performance when the model is executed on a platform (e.g., memory usage, latency, model accuracy, and / or fan usage).

[0114] In one embodiment, a performance change value of an artificial intelligence model may represent the degree of change in performance for an artificial intelligence model or a node within the model as a result of model transformation, model modification, and / or model comparison. SNR may be an example of a performance change value. For example, an SNR related to performance change may be used when comparing the artificial intelligence model before and after modification or optimization.

[0115] A platform in the present disclosure may include hardware on which an artificial intelligence model is to be executed and / or a compiler that links the artificial intelligence model to the hardware. In one embodiment, the platform may include hardware. In one embodiment, the platform may include a compiler. In one embodiment, the platform may represent a combination of hardware and a compiler. In one embodiment, the platform may be referred to as an execution environment, a deployment target, a system configuration, a runtime environment, and / or an inference platform on which an artificial intelligence model can be executed.

[0116] The user interface in the present disclosure may represent a graphical user interface (GUI).

[0117] FIG. 1 schematically illustrates a block diagram of a computing device according to one embodiment of the present disclosure.

[0118] The configuration of the computing device (100) illustrated in FIG. 1 is merely a simplified example. In one embodiment of the present disclosure, the computing device (100) may include other configurations for performing the computing environment of the computing device (100), and only some of the disclosed configurations may constitute the computing device (100).

[0119] A computing device (100) according to some embodiments of the present disclosure may be a device for visualizing an artificial intelligence model. The computing device (100) may be used in a sense that encompasses a server and a user terminal.

[0120] A server according to one embodiment of the present disclosure may convert an artificial intelligence model into an intermediate representation structure by abstracting the artificial intelligence model and / or perform operations for various performance transformations on the intermediate representation structure. A server according to some embodiments of the present disclosure may be a device for converting an artificial intelligence model to optimize the performance of the artificial intelligence model. The server may communicate with a user terminal for visualizing the artificial intelligence model, and, for example, the server may generate structured text data and transmit the structured text data to the user terminal so that the user terminal can render a graph-format topology of the artificial intelligence model. The server may include any type of computer system or computer device, such as, for example, a microprocessor, a mainframe computer, a digital processor, a portable device, a device controller, a web server, an application server, or a database server.

[0121] In one embodiment, the user terminal may correspond to a user device that receives data resources generated from a server, performs additional information processing, and provides a user interface. The user terminal may include a personal computer (PC), notebook, mobile terminal, smartphone, tablet PC, etc., owned by the user.

[0122] The computing device (100) may include a processor (110), memory (130), and a network unit (150).

[0123] The processor (110) may be composed of one or more cores and may include a processor for performing operations related to data processing, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a digital signal processor (DSP), a tensor processing unit (TPU), and a neural processing unit (NPU) of a computing device (100). The processor (110) may read a computer program stored in memory (130) and perform data processing according to embodiments of the present disclosure. That is, a computer program stored in a computer-readable storage medium (e.g., memory (130)) may cause one or more processors (e.g., processor (110)) to perform operations according to embodiments of the present disclosure. Additionally, the processor (110) may implement any component for performing data processing according to embodiments of the present disclosure.

[0124] According to one embodiment of the present disclosure, the processor (110) may perform operations for training a neural network. For example, the processor (110) may perform operations for training a neural network, such as processing input data for training in deep learning (DL), extracting features from input data, calculating errors, and updating the weights of the neural network using backpropagation. At least one of the CPU, GPGPU, DSP, TPU, and NPU of the processor (110) may process the training of a network function. For example, the CPU and GPGPU may together process the training of a network function and data classification using the network function. Additionally, in one embodiment of the present disclosure, processors of a plurality of computing devices may be used together to process the training of a network function and data classification using the network function.

[0125] The processor (110) can typically control the overall operation of the computing device (100). For example, the processor (110) can perform steps included in the flowcharts described below. The processor (110) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through components included in the computing device (100) or by running an application stored in memory (130).

[0126] In one embodiment, the computing device (100) may include a System on Chip (SoC). For example, the computing device (100) may include a single chip in which the processor (110), memory (130), and network unit (150) shown in FIG. 1 are all mounted. In this case, the SoC may further include any modules for implementing the operation of the computing device (100), such as an I / O interface, a Power Management IC (PMIC), and a security module. Meanwhile, it will be obvious to a person skilled in the art that the SoC may be referred to as a processor. Therefore, in this embodiment, operations performed by the processor (110) in the present disclosure should be interpreted as being able to be performed on the SoC.

[0127] In one embodiment of the present disclosure, the memory (130) may store any form of information generated or determined by the processor (110) and / or any form of information received by the network unit (150). In one embodiment, the memory (130) may store information related to the visualization of an artificial intelligence model, information regarding the artificial intelligence model, information regarding the platform, information regarding an intermediate representation structure, and information regarding structured text, etc.

[0128] In one embodiment of the present disclosure, the memory (130) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, a magnetic disk, and / or an optical disk. The computing device (100) may operate in conjunction with web storage that performs the storage function of the memory (130) on the internet. The memory (130) may include an embedded memory mounted inside the processor (110) and / or a memory mounted outside the processor (110) within the computing device (100). The description of the memory described above is merely an example, and the present disclosure is not limited thereto. The memory (130) may be operated by the processor (110).

[0129] A network unit (150) according to one embodiment of the present disclosure may implement any wired or wireless communication network capable of transmitting and receiving any type of data and signals, etc. The technologies described in this specification may be implemented not only by the networks mentioned above but also by other networks.

[0130] In one embodiment, the computing device (100) (e.g., user terminal) may further include an input unit and an output unit. The input unit may receive input from a user that is necessary for outputting information for visualizing an artificial intelligence model. The input unit may include a camera or video input unit for video signal input, a microphone or audio input unit for audio signal input, and a user input unit for receiving information from a user (e.g., a keyboard and touch key, a mechanical key, etc.).

[0131] The term "user input" in the present disclosure may refer to any form of input from a user related to user requests and information provided by the user performed within a user interface (or within a user terminal).

[0132] An output unit according to one embodiment of the present disclosure is intended to generate output related to sight, hearing, or touch, and may include at least one of a display unit, an audio output unit, a haptic module, and an optical output unit. The display unit may implement a touch screen by forming a layered structure with a touch sensor or by being formed integrally. Such a touch screen functions as a user input unit that provides an input interface between a user terminal and a user, and at the same time, may provide an output interface between users. For example, the display unit may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display. The display unit displays (outputs, displays) data processed at the user terminal.

[0133] FIG. 2 illustrates an exemplary structure of an artificial intelligence model according to one embodiment of the present disclosure.

[0134] Throughout this specification, artificial intelligence model, artificial intelligence-based model, computational model, machine learning model, deep learning model, neural network, network function, and neural network may be used interchangeably.

[0135] A neural network can be composed of a set of interconnected computational units, which can generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node. The nodes (or neurons) constituting the neural networks may be interconnected by one or more links. In FIG. 2, the term "node" is used to refer to a neuron within an artificial intelligence model, while the term "node" in the intermediate representation structure is used to represent an atomic operator. Nodes in the intermediate representation structure are used to constitute modules and may have a different meaning from the nodes corresponding to neurons in a neural network. While the set of nodes in FIG. 2 may represent a layer or an operator, the nodes in the intermediate representation structure correspond to an indivisible atomic operator.

[0136] In a neural network, one or more nodes connected via links can form relative input and output node relationships. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As described above, the input node versus output node relationship can be generated based on links. One or more output nodes may be connected to a single input node via links, and vice versa.

[0137] In a relationship between an input node and an output node connected through a single link, the value of the output node's data can be determined based on the data input to the input node. Here, the link interconnecting the input node and the output node may have a weight. The weight can be variable and can be varied by the user or an algorithm to enable the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node's value can be determined based on the values ​​input to the input nodes connected to the output node and the weights set on the links corresponding to each input node.

[0138] As described above, a neural network consists of one or more nodes interconnected through one or more links, forming input-output node relationships within the network. The characteristics of a neural network can be determined by the number of nodes and links within the network, the relationships between the nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values ​​for the links, the two neural networks may be recognized as different from each other.

[0139] A neural network can be composed of a set of one or more nodes. A subset of nodes constituting a neural network can form a layer. Some of the nodes constituting a neural network can form a layer based on their distances from an initial input node. For example, a set of nodes with a distance of n from an initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be traversed to reach that node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within a neural network can be defined in a way different from that described above. For example, a layer of nodes may be defined by its distance from a final output node.

[0140] In one embodiment of the present disclosure, a set of neurons or nodes may be defined by the expression a layer.

[0141] Initial input nodes may refer to one or more nodes within a neural network to which data is directly input without passing through links in their relationships with other nodes. Alternatively, in terms of link-based relationships between nodes within the neural network, they may refer to nodes that do not have other input nodes connected by links. Similarly, final output nodes may refer to one or more nodes within a neural network that do not have output nodes in their relationships with other nodes. Furthermore, hidden nodes may refer to nodes constituting the neural network that are neither initial input nodes nor final output nodes.

[0142] A neural network according to one embodiment of the present disclosure may have the same number of nodes in the input layer as the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. Additionally, a neural network according to another embodiment of the present disclosure may have the same number of nodes in the input layer as the number of nodes in the output layer as the number of nodes increases as it progresses from the input layer to the hidden layer. Furthermore, a neural network according to yet another embodiment of the present disclosure may have the same number of nodes in the input layer as the number of nodes in the output layer as the number of nodes decreases as it progresses from the input layer to the hidden layer. A neural network according to yet another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer is greater than the number of nodes in the output layer.

[0143] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to input and output layers. Using a deep neural network allows for the identification of latent structures in data. Deep neural networks may include convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, Generative Adversarial Networks (GAN), Restricted Boltzmann Machines (RBM), deep belief networks (DBN), Q networks, U networks, Siamese networks, etc. The description of deep neural networks described above is merely illustrative and the present disclosure is not limited thereto.

[0144] The artificial intelligence-based model of the present disclosure can be represented by a network structure of any structure described above, including an input layer, a hidden layer, and an output layer.

[0145] A neural network that can be used in an artificial intelligence-based model of the present disclosure may be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, federated learning for distributed deep learning, or incremental learning. Training of a neural network may be a process of applying knowledge to the neural network to perform a specific operation.

[0146] Neural networks can be trained to minimize the error in their output. The training process involves repeatedly inputting training data into the network, calculating the error between the network's output and the target for the training data, and updating the weights of each node by backpropagating the error from the output layer to the input layer in a direction that reduces the error. In supervised learning, training data is used where the correct answer is labeled for each data point (i.e., labeled training data), whereas in unsupervised learning, the correct answer may not be labeled for each training data point. For instance, in the case of supervised learning regarding data classification, the training data may consist of data where each training data point is labeled with a category. Labeled training data is input into the neural network, and the error can be calculated by comparing the network's output (category) with the labels of the training data.

[0147] As another example, in the case of unsupervised learning for data classification, errors can be calculated by comparing the input training data with the neural network output. The calculated errors are backpropagated within the neural network (i.e., from the output layer to the input layer), and this backpropagation updates the connection weights of each node in each layer of the neural network. The amount of change in the connection weights of each updated node can be determined by the learning rate. The neural network's calculation of the input data and the backpropagation of errors can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations in the neural network's learning cycle. For instance, a high learning rate can be used during the early stages of training to quickly achieve a certain level of performance and increase efficiency, while a low learning rate can be used during the later stages to improve accuracy.

[0148] In the training of neural networks, the training data is generally a subset of the real-world data (i.e., the data intended to be processed by the trained neural network). Consequently, a training cycle may exist where errors decrease on the training data but increase on the real-world data. Overfitting is a phenomenon where the network learns excessively on the training data, leading to increased errors on the real-world data. For example, a neural network trained on yellow cats might fail to recognize cats when seeing anything other than yellow, which can be considered a type of overfitting. Overfitting can act as a cause for increased errors in machine learning algorithms. Various optimization methods can be used to prevent this overfitting. To prevent overfitting, methods such as increasing the training data, regularization, dropout (which disables some nodes in the network during training), and the use of batch normalization layers can be applied.

[0149] According to one embodiment of the present disclosure, a computer-readable medium storing a data structure is disclosed. The data structure described above may be stored in a memory (130) in the present disclosure, executed by a processor (110), and transmitted and received by a network unit (150).

[0150] A data structure can refer to the organization, management, and storage of data that enables efficient access and modification of data. A data structure can refer to the organization of data for solving specific problems (e.g., data retrieval, data storage, data modification in the shortest possible time). A data structure may also be defined by physical or logical relationships between data elements designed to support specific data processing functions. Logical relationships between data elements may include connections between user-defined data elements. Physical relationships between data elements may include actual relationships between data elements physically stored on a computer-readable storage medium (e.g., a permanent storage device). Specifically, a data structure may include sets of data, relationships between data, and functions or instructions applicable to the data. Through an effectively designed data structure, a computing device can perform operations while minimizing the use of its resources. Specifically, through an effectively designed data structure, a computing device can increase the efficiency of operations, reading, insertion, deletion, comparison, exchange, and retrieval.

[0151] Data structures can be classified into linear and non-linear data structures based on their form. A linear data structure is one where only one piece of data is connected to the next. Linear data structures can include lists, stacks, queues, and deques. A list can refer to a set of data that maintains an internal order. Lists can include linked lists. A linked list is a data structure where data is connected in a line, with each piece of data possessing a pointer. In a linked list, the pointer can contain information regarding the connection to the next or previous data. Depending on its form, a linked list can be represented as a singly linked list, a doubly linked list, or a circular linked list. A stack is a data arrangement structure that allows for restricted access to data. A stack can be a linear data structure where data can be processed (e.g., insertion or deletion) only at one end. Data stored in a stack can be a Last-In, First-Out (LIFO) data structure, meaning that the later an item is entered, the sooner it is retrieved. A queue is a data sequence structure that allows for limited access to data; unlike a stack, it can be a FIFO (First in First Out) data structure where data stored later is retrieved later. A deque is a data structure that can process data at both ends.

[0152] Non-linear data structures can be structures where multiple data are connected after a single piece of data. Non-linear data structures may include graph data structures. A graph data structure can be defined by vertices and edges, and an edge may include a line connecting two different vertices. Graph data structures may include tree data structures. A tree data structure may be a data structure where there is only one path connecting two different vertices among the multiple vertices included in the tree. In other words, it may be a data structure that does not form a loop in a graph data structure.

[0153] The data structure may include a neural network. The data structure including the neural network may be stored on a computer-readable medium. The data structure including the neural network may also include data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, loss functions for learning the neural network, etc. The data structure including the neural network may include any of the components disclosed above. That is, the data structure including the neural network may be composed of all or any combination thereof, such as data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for learning the neural network. In addition to the aforementioned components, the data structure including the neural network may include any other information that determines the characteristics of the neural network. Additionally, the data structure may include all forms of data used or generated during the computational process of the neural network, and is not limited to the foregoing. The computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. The neural network may be composed of a set of interconnected computational units that may generally be referred to as nodes. These nodes may also be referred to as neurons. The neural network is composed of at least one node.

[0154] A data structure may include data input to a neural network. A data structure including data input to a neural network may be stored on a computer-readable medium. Data input to a neural network may include training data input during the neural network learning process and / or input data input to a neural network after training is complete. Data input to a neural network may include pre-processed data and / or data subject to pre-processing. Pre-processing may include a data processing process for inputting data into a neural network. Accordingly, a data structure may include data subject to pre-processing and data generated by pre-processing. The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0155] The data structure may include weights of the neural network. (In this specification, weights and parameters may be used interchangeably.) The data structure including the weights of the neural network may be stored on a computer-readable medium. The neural network may include multiple weights. The weights may be variable and may be varied by a user or an algorithm to enable the neural network to perform a desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node may determine the data value output from the output node based on values ​​input to the input nodes connected to the output node and weights set on the links corresponding to each input node. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0156] As an example rather than a limitation, weights may include weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Weights that vary during the neural network learning process may include weights at the start of the learning cycle and / or weights that vary during the learning cycle. Weights for which neural network learning is completed may include weights for which the learning cycle is completed. Accordingly, a data structure containing the weights of a neural network may include a data structure containing weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Therefore, the weights and / or combinations of each weight described above are included in the data structure containing the weights of a neural network. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0157] Data structures containing the weights of a neural network may be stored on a computer-readable storage medium (e.g., memory, hard disk) after undergoing a serialization process. Serialization may be a process of converting a data structure into a form that can be stored on the same or different computing devices and later reconstructed for use. A computing device may serialize the data structure to transmit and receive data over a network. A serialized data structure containing the weights of a neural network may be reconstructed on the same or different computing devices through deserialization. Data structures containing the weights of a neural network are not limited to serialization. Furthermore, data structures containing the weights of a neural network may include data structures designed to increase computational efficiency while minimizing the use of computing device resources (e.g., B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree in non-linear data structures). The foregoing is merely an example and the present disclosure is not limited thereto.

[0158] The data structure may include hyperparameters of the neural network. The data structure including the neural network hyperparameters may be stored on a computer-readable medium. The hyperparameters may be variables that are varied by the user. The hyperparameters may include, for example, a learning rate, a cost function, the number of learning cycle iterations, weight initialization (e.g., setting the range of weight values ​​subject to weight initialization), and the number of hidden units (e.g., the number of hidden layers, the number of nodes in the hidden layers). The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0159] An AI-based model according to one embodiment of the present disclosure may include a Large Language Model (LLM). In the present disclosure, a Large Language Model may refer to an AI-based model trained using a vast amount of training data to perform Natural Language Processing. The Large Language Model may include a Transformer, a Transformer encoder family model, and / or a Transformer decoder family model. A Transformer encoder family model may correspond to an AI model that uses a Transformer encoder structure. A Transformer decoder family model may correspond to an AI model that uses a Transformer decoder structure. For example, the AI-based model of the present disclosure may include a GPT family Large Language Model, a BERT (Bidirectional Encoder Representations from Transformers) family Large Language Model, etc.

[0160] In one embodiment, the transformer may be composed of an encoder that encodes input data and a decoder that decodes the encoded data. The transformer may have a structure that takes a series of input data as input, undergoes encoding and decoding steps, and outputs a series of output data. In one embodiment, the series of input data may be processed into a form that the transformer can compute. The process of processing the series of input data into a form that the transformer can compute may include a tokenizing process and an embedding process. The tokenizing process may refer to the process of dividing the series of input data into tokens of a certain unit. For example, the certain unit may include word units. The embedding process may refer to the process of converting at least one token tokenized from the series of input data into an embedding vector.

[0161] In one embodiment, the transformer can obtain an embedding vector to be input to an encoder by combining a token embedding vector that embeds at least one token corresponding to a series of input data, a segment embedding vector that distinguishes sentences containing the tokens for each token, and a position embedding vector that reflects the position of the token. The encoder family model and the decoder family model of the transformer can also obtain embedding vectors by performing the same method.

[0162] In one embodiment, for the transformer to encode and decode a series of input data, the encoder and decoder within the transformer may utilize an attention algorithm. An attention algorithm may refer to an algorithm that, for a given query, calculates similarity by applying a softmax function to an attention score obtained by matrix multiplying the query with a key, and calculates an attention value for the query by matrix multiplying the calculated similarity with a value.

[0163] In one embodiment, the self-attention algorithm may refer to an attention algorithm that uses a query, a key, and a value generated by multiplying the same embedding vector by a query weight, a key weight, and a value weight, respectively. The cross-attention algorithm may refer to an attention algorithm that uses a query generated by multiplying a first embedding vector by a query weight, and a key and a value generated by multiplying a second embedding vector by a key weight and a value weight, respectively. The query weight, the key weight, and the value weight may be trainable parameters that are updated through the training process of a large-scale language model.

[0164] In one embodiment, the encoder of the transformer may include an embedding layer, a self-attention layer that applies a self-attention algorithm to an embedding vector, a normalization layer, and a feed-forward neural network (FFN). Additionally, the encoder may have a form in which N unit structures including a self-attention layer, a normalization layer, and a feed-forward neural network are connected. The decoder of the transformer may include an embedding layer, a masked self-attention layer, a normalization layer, a cross-attention layer that applies a cross-attention algorithm, and a feed-forward neural network. Additionally, the decoder may have a form in which N unit structures including a masked self-attention layer, a normalization layer, a cross-attention layer, and a feed-forward neural network are connected. The masked self-attention layer may correspond to a layer that calculates an attention value for each of the sequences in which words are sequentially included among a plurality of words included in a series of input data.

[0165] A transformer may include additional components such as a linear layer and a softmax layer, in addition to an encoder and a decoder. The encoder family model and the decoder family model of a transformer may also each include the aforementioned additional components in addition to the encoder and decoder. A method for constructing a transformer using an attention algorithm may include the method disclosed in Vaswani et al., Attention Is All You Need, 2017 NIPS, which is incorporated herein by reference.

[0166] In one embodiment, attention layers such as a self-attention layer, a masked self-attention layer, and a cross-attention layer may correspond to a multi-head attention layer that includes a plurality of attention layers in parallel. The multi-head attention layer may concatenate the attention values ​​output from each of the plurality of attention layers and output an output attention value by matrix multiplying the concatenated matrix by an output weight. The output attention value output from the multi-head attention layer may have the same size as the attention value output from a single attention layer.

[0167] In one embodiment, the transformer may be trained through a Masked Language Model (MLM) process, a Next Sentence Prediction (NSP) process, etc. The MLM process may refer to a training process that predicts a masked word through a series of training data in which some words are masked. The NSP process may refer to a training process that determines whether two sentences are connected sentences in a series of training data containing any two sentences.

[0168] In one embodiment, a large-scale language model can process various data formats, such as natural language text as well as image data, audio data, and video data. The large-scale language model can embed data to convert data of various data formats into a series of computationally operable data. The large-scale language model can process additional data that represents the relative positional or topological relationships between a series of input data. Alternatively, a series of input data may be embedded by additionally reflecting vectors that represent the relative positional or topological relationships between the input data. In one example, the relative positional relationships between a series of input data may include, but are not limited to, word order within a natural language sentence, the relative positional relationships of each segmented image, and the temporal order of segmented audio waveforms. The process of adding information that represents the relative positional or topological relationships between a series of input data may be referred to as positional encoding.

[0169] An example of a large-scale language model for processing image data (Vision Transformer, ViT) is disclosed in Dosovitskiy, et al., AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE, which is incorporated herein by reference.

[0170] An artificial intelligence model according to one embodiment of the present disclosure may include a multimodal large-scale language model. A multimodal large-scale language model may refer to a large-scale language model capable of understanding and processing relationships between different data formats, such as natural language text data, image data, audio data, and video data. A multimodal language model may include a plurality of encoders that encode input data corresponding to each data format. A multimodal language model may be trained to calculate similarity between embedding vectors encoded by encoders of each data format through training data containing data of different data formats, such that similarity between identical pairs is calculated to be higher and similarity between different pairs is calculated to be lower.

[0171] An example of a large-scale multimodal language model that understands and processes the relationship between image data and natural language text data (Contrastive Language-Image Pre-training, CLIP) is disclosed in Alec Radford, et al., *LEARNING TRANSFERABLE VISUAL MODELS FROM NATURAL LANGUAGE SUPERVISION*, which is incorporated herein by reference.

[0172] FIG. 3 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0173] The steps in FIG. 3 are exemplified as being performed by a computing device (100) (e.g., a server). Depending on the implementation mode, some of the entities of FIG. 3 may be operated by interacting with the server from outside the server.

[0174] In one embodiment, a computing device (e.g., a server) (100) can receive an artificial intelligence model from a user terminal (310).

[0175] For example, an artificial intelligence model may refer to an artificial intelligence model that is the subject of visualization. A computing device (100) may receive information identifying the artificial intelligence model or a file of the artificial intelligence model.

[0176] In one embodiment, the computing device (100) can generate an intermediate representation structure corresponding to the artificial intelligence model by abstracting the artificial intelligence model (320).

[0177] An intermediate representation structure can serve as a blueprint for rendering a result to be visualized in a user interface according to one embodiment of the present disclosure. A computing device (100) can load an intermediate representation structure corresponding to an artificial intelligence model through communication with an internal component or an external server. For example, the computing device (100) can call an API (Application Programming Interface) that provides an intermediate representation structure, and an intermediate representation structure corresponding to an artificial intelligence model can be generated by an internal or external component of the computing device (100) in accordance with the call. For example, an intermediate representation structure may refer to a graph structure represented by nodes and edges. For example, an intermediate representation structure may consist of a hierarchical structure of nodes independent of the framework and modules (sets of nodes) dependent on the framework. For example, an intermediate representation structure may include a combination of nodes corresponding to a minimum unit of atomic operator that is universally applicable to a plurality of frameworks.

[0178] The computing device (100) can read a file of an artificial intelligence model stored in various framework formats (e.g., ONNX, PyTorch, or TensorFlow) and load it into memory (130). For example, the computing device (100) can create a graph structure by using a parser suitable for each format to convert information constituting the artificial intelligence model (e.g., operator types, attributes, and connection relationships) into a data form that the computing device (100) can process. For example, the computing device (100) can extract nodes constituting the artificial intelligence model (e.g., operators) and their attributes and connection relationships by traversing the graph structure (e.g., by using graph traversal algorithms such as depth-first search (DFS) and breadth-first search (BFS)).

[0179] For example, the computing device (100) can identify basic units of operation (e.g., atomic operators that cannot be further decomposed) within an artificial intelligence model by traversing a graph structure and extract their attributes and connections. The computing device (100) can generate an intermediate representation structure in the form of a graph using the identified nodes and their connections.

[0180] For example, the computing device (100) can combine identified nodes into modules, which are higher-level concepts of the nodes, using the hierarchical structure of the framework of the artificial intelligence model. The computing device (100) can generate an intermediate representation structure in the form of a graph using the identified nodes and modules and their connection relationships.

[0181] For example, a computing device (100) can generate an intermediate representation structure by abstracting an artificial intelligence model to include a tensor representing a node independent of the framework (e.g., general-purpose) and the output of the node.

[0182] By example, not a limitation, the intermediate representation structure may include parameters of an artificial intelligence model (e.g., weights, biases, and / or kernels, etc.), activations (or activation values), and nodes. By example, not a limitation, the intermediate representation structure may be a graph structure comprising nodes, modules composed of one or more nodes, and tensors representing the inputs of the module, the outputs of the module, weights input to the module, and activations input to the module on said intermediate representation structure. Information contained within such intermediate representation structures may be used when converted into structured text data.

[0183] In one embodiment, the computing device (100) can convert the intermediate representation structure into structured text data for the visualization of the artificial intelligence model using an attribute value included in the intermediate representation structure (330).

[0184] The computing device (100) can extract information stored in an intermediate representation structure and use the extracted information to generate structured text data for visualizing an artificial intelligence model.

[0185] When generating an intermediate representation structure, the computing device (100) may store attributes or attribute values ​​that define structural characteristics of components (e.g., nodes) of an artificial intelligence model in the modules and / or tensors of the intermediate representation structure. For example, attributes may be defined as parameters that define structural characteristics of components of an artificial intelligence model. For example, attributes may include information related to the input, output, and nodes of the artificial intelligence model, and / or information regarding the connection of the nodes.

[0186] A computing device (100) can query attribute values ​​of an intermediate representation structure to extract from the intermediate representation structure inputs of an artificial intelligence model, outputs of an artificial intelligence model, nodes included in the artificial intelligence model, connection information between the nodes, and metadata included in the artificial intelligence model. For example, the computing device (100) can determine general information for generating structured text using attribute values ​​stored in the intermediate representation structure. Metadata can be defined as information expressing additional descriptions, such as quantization information and / or performance, for components (e.g., nodes) of the artificial intelligence model. In an additional example, metadata is stored in an intermediate representation structure (e.g., nodes, modules, and / or tensors), and the computing device (100) can extract metadata from the intermediate representation structure and generate structured text using the metadata.

[0187] The computing device (100) can convert an intermediate representation structure into a structured text data structure. The computing device (100) can generate structured text data by arranging the extracted information according to a predefined text data structure.

[0188] For example, structured text data is a data structure that is easy to handle during the rendering process on a user terminal, and can take the form of a JSON (JavaScript Object Notation) file. For example, a computing device (100) can query attribute values ​​of an intermediate representation structure to extract the input and output of an artificial intelligence model, nodes, the connection relationships between nodes, and metadata (e.g., information on how to quantize the model), and convert the extracted information into a pre-set text template. The computing device (100) can generate a schema for rendering on a user terminal by configuring a pre-defined JSON template (e.g., a template consisting of graph, modules, in_ports, out_ports, nodes, edges, tensors, and / or metadata) from the intermediate representation structure. A computing device (100) can generate information regarding performance related to an artificial intelligence model (e.g., performance by node or performance change value) through execution of an artificial intelligence model (e.g., simulation on an intermediate representation structure or direct execution of an artificial intelligence model) and generate structured text data by using the information regarding performance together with attribute values ​​extracted from an intermediate representation structure.

[0189] For example, structured text data can refer to a data format used by a server to organize information from an artificial intelligence model according to predetermined rules (or structures) and transmit it to a user terminal. Because structured text data is text-based, it can operate independently of various media or platforms, and since it is structured, machines can easily interpret and process it. A representative example of structured text data is a JSON file.

[0190] The computing device (100) can generate edges not included in the intermediate representation structure by changing tensors included in the intermediate representation structure into edges connecting nodes. Structured text data can be generated using the edges. For example, the intermediate representation structure may include nodes independent of the framework, a first tensor representing parameters of an artificial intelligence model, and a second tensor representing activation values ​​of the nodes. The computing device (100) can generate edges connecting nodes using the second tensor and nodes, and then convert the intermediate representation structure into structured text data using the first tensor, nodes, and edges. For example, the tensors may include weights and activations, and the computing device (100) can generate structured text data by changing specific tensors among the tensors of the intermediate representation structure into edges. For example, a computing device (100) can generate structured text data by identifying a tensor representing an activation among tensors of an intermediate representation structure and changing the identified tensor into an edge.

[0191] In one embodiment, the structured text data may include nodes representing basic operational units of an artificial intelligence model, modules representing sets of nodes, edges connecting nodes, tensors representing parameters, and / or metadata representing quantization-related information. For example, the structured text data may have a text structure in which each of the included pieces of information is represented as a key-value pair. The structured text data may have a JSON data structure represented as a key-value pair.

[0192] In one embodiment, when generating structured text data, the computing device (100) can traverse each of the multiple nodes included in the intermediate representation structure to extract information to be included in the structured text data from each node. For example, the computing device (100) can traverse each of the multiple nodes included in the intermediate representation structure in the direction from input to output of an artificial intelligence model to extract node identification information, input tensor identification information, output tensor identification information, and operator type information from each of the multiple nodes. Structured text data can be generated using this extracted information. For example, the computing device (100) can generate structured text data for each of the multiple nodes within the intermediate representation structure by using the extracted information as a value and mapping it to the key of the structured text data.

[0193] A technique according to one embodiment of the present disclosure can convert a specific tensor in an intermediate representation structure into an edge in order to include an edge that does not exist in the intermediate representation structure in the structured text data. Accordingly, when visualizing (e.g., rendering) an artificial intelligence model on a user terminal, a data structure that is easy to process on the user terminal can be generated in advance by the server. Consequently, the amount of unnecessary data communication with the server during the visualization process of the artificial intelligence model on the user terminal can be reduced, and furthermore, the limited computing resources of the user terminal can be utilized efficiently. Accordingly, the efficiency of resource usage on the user terminal during the visualization process of the artificial intelligence model can be increased, and unnecessary data communication between the server and the user can be reduced.

[0194] In one embodiment, a computing device (100) (e.g., a server) executes an artificial intelligence model to obtain performance values ​​(e.g., accuracy and / or latency) and / or performance change values ​​(e.g., SNR) related to the artificial intelligence model, and may include the received performance values ​​and / or performance change values ​​in metadata. Using metadata containing information related to performance and attribute values ​​stored in an intermediate representation structure, the computing device (100) may convert structured text data from an intermediate representation structure. By storing the performance values ​​and / or performance change values ​​of the executed artificial intelligence model in structured text data (e.g., a first schema), the computing device (100) may allow for efficient visualization of performance related to the artificial intelligence model during the rendering process of a user terminal.

[0195] In one embodiment, the computing device (100) may perform changes, modifications, and / or optimizations (e.g., graph optimization, compression, and / or quantization, etc.) on an artificial intelligence model on an intermediate representation structure. Accordingly, a modified intermediate representation structure corresponding to the artificial intelligence model may be generated. For example, the computing device (100) may generate a modified intermediate representation structure by changing the configuration of nodes and / or modules included in the intermediate representation structure using a model optimization tool comprising at least one of a compressor for lightweighting the model, a graph optimizer for increasing the execution efficiency of the model, and a quantizer for reducing at least one of the size and computational amount of the model. By applying the model optimization tool to the intermediate representation structure, changes, transformations, and / or optimization operations on the intermediate representation structure may be performed, and an optimized artificial intelligence model may be generated by performing conversion to fit the framework on the intermediate representation structure to which these operations have been applied.

[0196] In one embodiment, the computing device (100) can generate a node performance change value that quantitatively represents the performance change of modified nodes within the modified intermediate representation structure (e.g., performance change relative to the node before modification) by performing an inference operation using an intermediate representation structure and performing an inference operation using a modified intermediate representation structure. The node performance change value can be generated by comparing the performance of the node before modification with the performance of the node after modification. The computing device (100) can include this node performance change value in the structured text data when converting the structured text data. For example, the computing device (100) can generate an expected inference simulation result for the case where the AI ​​model is executed on the platform, the input data to be used in the simulation process of the AI ​​model, and the intermediate representation structure, without executing the AI ​​model on the platform, but with the input data as input. The simulation using the intermediate representation structure can be performed, for example, based on a preprocessing method that removes modules within the intermediate representation structure and retains the nodes. Simulation using an intermediate representation structure can be performed, for example, based on a preprocessing method in which a single node is included in a single module within the intermediate representation structure.

[0197] In one embodiment, a computing device (100) may receive a request to modify an artificial intelligence model. For example, a user terminal may receive a request to modify (or change) the model on a user interface representing a visualized artificial intelligence model and transmit the received request to a server. The request to modify and the request to change in this disclosure may be used interchangeably. When there is a request to modify an artificial intelligence model, the computing device (100) may use information received from the user terminal to identify the target for model modification, modify the identified target for modification, and regenerate structured text data based on the modified artificial intelligence model. As the regenerated structured text data is transmitted to the user terminal, the request to modify is synchronized between the server and the user terminal, and the result reflecting the request to modify can be output on the user interface.

[0198] In one embodiment, the computing device (100) can respond to receiving a change request for an artificial intelligence model from a user terminal and identify information included in the change request, such as a node identification number, a node name, a node operator type, an edge identification number, and an edge name, from the change request. Using the identified information, the computing device (100) can identify the target of the change request within the artificial intelligence model and change the intermediate representation structure of the artificial intelligence model according to the change request. The computing device (100) can change structured text data based on the changed intermediate representation structure. The changed structured text data is transmitted to the user terminal, and efficient synchronization between the re-rendering result of the graph-format topology by the user terminal and the intermediate representation structure stored on the server can be achieved.

[0199] The computing device (100) can transmit structured text data to the user terminal so that the user terminal renders a graph-format topology for the artificial intelligence model (340).

[0200] In one embodiment, when structured text data is transmitted to a user terminal, the structured text data can be reconstructed so that it is represented as nodes and edges connecting the nodes, as the first schema of the structured text data is changed to a second schema for graph rendering. Based on the reconstructed structured text data, the topology of the graph format for the artificial intelligence model can be rendered.

[0201] In the present disclosure, topology may represent a logical structure expressing the arrangement and connection relationships of nodes (e.g., operators) constituting an artificial intelligence model. Topology may represent the result of expressing which nodes are connected to which other nodes. Topology may represent the visualization result regarding the structure of an artificial intelligence model. Topology may refer to a structural diagram of an artificial intelligence model.

[0202] In the present disclosure, a schema may represent rules that define the format and rules by which data, such as structured text data, is organized. In the above embodiment, the first schema is a schema for data transmission and may define rules or structures for efficiently transmitting data from a server to a user terminal. The second schema is a schema for rendering and may define rules or structures for the user terminal to efficiently visualize (e.g., render) an artificial intelligence model. For example, the first schema may represent a structure in which nodes, edges, metadata, etc., are stored separately in their respective lists, and the second schema may represent a structure in which additional information, such as metadata, is integrated and stored in the related nodes and edges.

[0203] In one embodiment, the computing device (100) can transmit the structured text data to a user terminal so that the user terminal can change the schema of the structured text data generated according to the first schema to the second schema.

[0204] In one embodiment, when a computing device (100) receives an artificial intelligence model having a single input multi-destination structure, it can generate multiple edges by branching each edge corresponding to multiple outputs from a single edge information. Management of these edges can be achieved by displaying an index for each of these multiple edges.

[0205] FIG. 4 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0206] The steps in FIG. 4 are exemplified as being performed by a computing device (100) (e.g., a user terminal). Depending on the implementation mode, some of the entities of FIG. 4 may be operated by interacting with the computing device outside the user terminal.

[0207] In one embodiment, a computing device (e.g., a user terminal) (100) may receive structured text data corresponding to an artificial intelligence model from a server (410). A specific description of the structured text data is replaced by the description above.

[0208] In one embodiment, the computing device (100) can extract nodes and edges from structured text data (420).

[0209] For example, a computing device (100) can extract information regarding nodes and edges from structured text data in which the data are classified according to classification criteria. This information regarding nodes and edges can be utilized in the process of changing the schema of the structured text data. This information regarding nodes and edges can be utilized in the process of reconstructing the structured text data.

[0210] In one embodiment, the computing device (100) can reconstruct the structured text data by changing the first schema of the structured text data to a second schema for graph rendering using nodes and edges (430).

[0211] The computing device (100) may, in the step of reconstructing structured text data, convert the first schema, which includes nodes, edges, tensors, and metadata arranged separately, into the second schema represented by the nodes and edges by integrating tensors and metadata in the structured text data into at least one of nodes and edges. Reconstruction of structured text data may represent an operation of changing the schema of the structured text to a different schema.

[0212] In a JSON file, components of an artificial intelligence model may be listed according to classifications such as sub-graph, modules, in_ports, out_ports, nodes, edges, tensors, metadata, etc. During the process of rendering the graph topology of the artificial intelligence model by a user terminal, the JSON file may have a form in which the connection structure of the operators of the artificial intelligence model cannot be intuitively grasped. Accordingly, the computing device (100) may reorganize the information in the JSON file into modules, nodes, and edges according to a second schema in order to render the graph. In this reorganization process, tensors and metadata included in the JSON file may be modified to be included in modules, nodes, and edges. Accordingly, an operation may be performed to integrate a first schema (e.g., backend schema) divided into several components into a second schema for rendering the graph so that model modification on the user terminal is easy.

[0213] In one embodiment, the computing device (100) can render a graph-format topology for an artificial intelligence model using reconstructed structured text data (440).

[0214] The computing device (100) can generate or render a topology based on a second schema reconstructed with only modules, nodes, and edges, and generate detailed information (e.g., metadata and attributes) corresponding to the topology.

[0215] In the present disclosure, a graph or graph structure may refer to a graph in which an artificial intelligence model is abstracted into a structure of nodes and edges based on connectivity including nodes and tensors. A graph or graph structure may refer to a graph in which an artificial intelligence model is rendered into a structure of nodes and edges based on connectivity including modules, nodes, and tensors.

[0216] Rendering in the present disclosure may refer to the process of converting a topology representing the logical connection structure of an artificial intelligence model into a visual graph image and displaying it on a user interface. Such rendering may refer to a computing process that generates a digital image by assigning specific visual attributes (e.g., location, color, shape, etc.) to abstract data. For example, a computing device (100) may read structured text data received from a server and interpret information such as nodes, edges, and metadata contained therein. Then, the computing device (100) may reconstruct the interpreted data into a format (e.g., a second schema) that can be understood by the visualization library of the computing device (100). For example, a method of integrating separated metadata into a corresponding node object may be used in the reconstruction process. Then, the computing device (100) may calculate where to place each node and edge from the reconstructed structured text data on the screen of the user interface and determine the display method (e.g., color), shape, and size for rendering the nodes and edges. Here, different display methods (e.g., colors) can be determined according to metadata (e.g., SNR) values. Then, the computing device (100) can call a graphics API according to the calculated visual attributes to fill pixels on the screen of the user interface to render a final graph image and output it.

[0217] In one embodiment, the computing device (100) may receive user input including a request to change at least part of the rendered topology (e.g., nodes and edges, etc.). The computing device (100) may re-render the changed topology corresponding to the change request without waiting for a response from a server related to the change request. For example, the computing device (100) may re-render the changed topology corresponding to the change request before transmitting information related to the change request to a server and receiving a response from the server (e.g., modified structured text data).

[0218] The computing device (100) can roll back the changed topology (i.e., the re-rendered topology) to the topology prior to the change (i.e., prior to the re-rendering) in response to receiving a change failure response from the server.

[0219] In response to receiving a change success response from a server, the computing device (100) can integrate metadata and tensors contained in the changed structured text into at least one of the nodes and edges of the re-rendered changed topology using the changed structured text data included in the change success response. That is, the computing device (100) can reconstruct the changed structured text data into a second schema. That is, if the computing device (100) receives a change success response from the server after re-rendering without waiting for a response from the server to a user change request, it can update detailed information (e.g., metadata and tensors, etc.) inherent in the nodes and edges using the information included in the change success response. Additionally, as change information related to the change request is transmitted to the server, a changed artificial intelligence model or a changed intermediate representation structure corresponding to the re-rendered topology can be generated by the server.

[0220] In this way, synchronization can be achieved between the artificial intelligence model and / or intermediate representation structure stored on the server and the topology rendered by the computing device (100). In this way, as the result reflecting the user's change request is output immediately, the speed of interaction with the user at the computing device (100) (i.e., user terminal) can be ensured.

[0221] In one embodiment, the computing device (100) can implement comparative rendering for a plurality of topologies. The computing device (100) can perform comparative rendering by matching the positions of nodes and / or edges belonging to corresponding layers based on the hierarchical structure of artificial intelligence models corresponding to the plurality of topologies. For example, the computing device (100) can receive first structured text data corresponding to a first artificial intelligence model and second structured text data corresponding to a second artificial intelligence model. The computing device (100) can identify differential nodes representing the difference between the first topology of the first artificial intelligence model and the second topology of the second artificial intelligence model by using identification information of nodes included in each of the first structured text data and the second structured text data. The computing device (100) can re-render at least one of the first topology and the second topology using the differential nodes. In the re-rendering process, the computing device (100) can apply predefined visual attributes to the differential nodes so that the differential nodes are re-rendered visually distinguished from the common nodes that exist in common in the first topology and the second topology.

[0222] A difference node may refer to a node that exists in only one of two artificial intelligence models when comparing their structures, indicating a structural difference between the two models. A difference node can be used as a visualization unit to indicate whether operators (e.g., nodes) have been added or deleted between two versions of the models or between the two models. A difference node can be defined as a term encompassing nodes that are not common to all models, identified during the process of comparing the topologies of multiple artificial intelligence models. For example, a first difference node may represent a node that exists in the reference first artificial intelligence model but not in the second artificial intelligence model being compared. For example, a second difference node may represent a node that does not exist in the first artificial intelligence model but exists in the second artificial intelligence model. For example, a difference node may be used to compare structural differences between the original artificial intelligence model and the modified artificial intelligence model when a modified artificial intelligence model exists due to the modification, transformation, and / or optimization of the original artificial intelligence model.

[0223] In one embodiment, the computing device (100) receives user input including a request to change a rendered topology and can determine the type of the change request based on the received user input. Based on the type of the change request, the computing device (100) can determine whether to interact with a server for re-rendering the topology and / or the method of interaction with the server. There may be a correlation between the type of the change request and whether or not there is interaction with the server. There may be a correlation between the type of the change request and the method of interaction with the server.

[0224] More specifically, the computing device (100) may decide to re-render the topology in which the positions of nodes and edges are changed according to the change request without interaction with the server when the type of the change request is determined to be a first type in which nodes and edges are not changed. For example, when a change request is received in which nodes are maintained and there is no change in the connection relationships between nodes, such as a change in the size and relocation of nodes and / or edges, the computing device (100) may perform re-rendering corresponding to the change request independently without interaction with the server. The independent re-rendering of the computing device (100) may mean changing the position state of nodes and / or edges on the user interface without communication with the server. If user input for a refresh is received from the user after such re-rendering is performed, the computing device (100) may output the rendering result prior to the re-rendering on the user interface by rolling back the result of the re-rendering.

[0225] When the type of change request is determined to be a second type including the creation of a node (e.g., creation of a new node), the computing device (100) can determine the transmission information to be sent to the server based on the operator corresponding to the node to be created. Each of the operators may be mapped to what information to be sent to the server. For example, in the case of the Convolution operator, the fact that values ​​related to Padding and Stride must be sent together to the server may be mapped to the Convolution operator. For example, some information may not be sent to the server and may be generated on the server itself.

[0226] When the computing device (100) determines that the type of the change request is a third type including the deletion of a node, it can transmit node identification information (e.g., node ID) corresponding to the node to be deleted to the server. In this case, since only the node ID is transmitted, synchronization between the server and the user terminal can be achieved in a manner that efficiently uses communication resources.

[0227] When the type of change request is determined to be a fourth type including the creation of an edge, the computing device (100) may transmit identification information for a start node and an end node corresponding to the edge to be created to the server. For example, the computing device (100) may transmit information for a start node and an end node connected to an edge to the server so that the server can connect the edge between nodes on an intermediate representation structure.

[0228] When the type of change request is determined to be a fourth type that changes the attribute value and / or metadata of a node, the computing device (100) can transmit the changed value (e.g., node ID and changed padding value) for the padding change to the server. When the type of change request is determined to be a fourth type that changes the attribute value and / or metadata of a node, the computing device (100) may also be implemented to transmit not only the changed value for the padding change but also all information regarding the related node (e.g., Stride value, Kernel value, etc.) to the server.

[0229] In one embodiment, the computing device (100) can determine the visual attributes of the node being rendered based on the performance of the node during the rendering process. For example, the node color may be dynamically determined based on metadata inherent in the node during the rendering process. For example, the node color may be dynamically determined based on the SNR score inherent in the node during the rendering process. The computing device (100) determines a display color corresponding to the node based on the node performance value or node performance change value inherent in the node within the topology, and by rendering the node within the topology with the determined display color, the performance information or performance change information of the nodes can be intuitively verified within the rendering result. For example, the computing device (100) may determine a display color corresponding to the node such that the saturation or luminance on the color gradient has a positive correlation with the node performance value or node performance change value. In this way, the performance of the nodes can be efficiently verified on the rendered topology through the distinction between luminance or saturation, thereby enhancing the user experience.

[0230] In one embodiment, the computing device (100) may assign edges of the same color or assign the same visual attributes to nodes of the same layer during the rendering process. For example, the computing device (100) may render the topology so that the first node and the second node are visually integrated by applying common visual attributes to the first node and the second node belonging to corresponding layers within the topology, based on layer identification information inherent in the nodes within the topology. By rendering nodes belonging to the same layer using layer identification information corresponding to the nodes in the same display method, a technical effect can be achieved in which the internal structure of the artificial intelligence model can be checked more intuitively. As another example, the computing device (100) may render the topology so that the edges of the first node and the second node are visually integrated by applying common visual attributes to the edges connected to each of the first node and the second node belonging to corresponding layers within the topology, based on layer identification information inherent in the nodes within the topology. By using layer identification information corresponding to the nodes, and unifying the display method using layer identification information not only for the nodes but also for the edges associated with the nodes, a technical effect can be achieved in that the internal structure of the artificial intelligence model can be verified more intuitively.

[0231] As illustrated in FIGS. 3 and 4, a technique according to one embodiment of the present disclosure can achieve visualization (e.g., rendering) of the structure of an artificial intelligence model in a manner that efficiently uses computing resources and network resources through interaction between a server and a user terminal, and furthermore, efficient synchronization between data stored on the server (e.g., intermediate representation structure) and data rendered on the user terminal (e.g., reconstructed structured text data) can be implemented during this visualization process.

[0232] FIG. 5 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0233] FIG. 5 illustrates an exemplary interaction between a user terminal (300) and a server (200) for implementing the visualization of an artificial intelligence model. The user terminal (300) and the server (200) may be included within the scope of the computing device (100) described above. The server (200) may receive an artificial intelligence model from the user terminal (300), generate an intermediate representation structure through abstraction, perform a transformation of the structured text data, and transmit the structured text data to the user terminal (300). The user terminal (300) may interact with the server (200), render a graph-format topology based on the received structured text data, and transmit user input to the server (200).

[0234] The user terminal (300) can transmit an artificial intelligence model file for visualization to the server (200) (505). For example, the user terminal (300) can transmit an artificial intelligence model file such as ONNX and Torch to the server. For example, step 505 can correspond to a visualization request step.

[0235] The server (200) can load a model file (510). The server (200) can load the received artificial intelligence model file and generate or obtain an intermediate representation structure corresponding to the loaded model file. The server (200) can prepare the loaded data to an analyzable state by parsing the received artificial intelligence model file and loading it into memory. For example, the server (200) can load an artificial intelligence model file received from a user terminal (300) and identify and / or analyze the loaded artificial intelligence model file. An intermediate representation structure can be generated and / or loaded through operations within the server (200) or through communication with an entity existing outside the server (200). For example, at step 510, the server (200) can call an API to obtain an intermediate representation structure corresponding to the received artificial intelligence model.

[0236] The server (200) can generate or obtain an intermediate representation structure corresponding to the received model file (515). For example, step 515 may be performed by an entity outside or inside the server that generates and manages the intermediate representation structure. The intermediate representation structure may be included in the response from the entity outside or inside the server. For example, the response may have a file format (e.g., a Python object of the intermediate representation structure). For example, the server (200) can convert the artificial intelligence model file received in step 505 into a corresponding intermediate representation structure and return the conversion result (515). For example, the server (200) can abstract the artificial intelligence model to generate an intermediate representation structure represented by a framework-independent node and a framework-dependent module (a set of nodes).

[0237] The server (200) can extract artificial intelligence model information using the return result (520). For example, the server (200) can extract attribute values ​​on an intermediate representation structure corresponding to the artificial intelligence model using the return result. The server (200) can extract information stored in the intermediate representation structure. For example, the server (200) can extract information such as nodes, tensors, and / or metadata that are necessary for visualization (e.g., necessary for reconstructing the intermediate representation structure) using the attribute values ​​included in the intermediate representation structure.

[0238] Specifically, the server (200) can iterate through each component (e.g., node, tensor, module, etc.) stored hierarchically within the intermediate representation structure and extract information necessary for generating structured text data. For example, the server (200) can iterate through a set of nodes included in the intermediate representation structure and, for each node ID, retrieve the corresponding node object, and then extract information necessary for visualization from the attributes of the object and store it in a temporary dictionary. Accordingly, the unique identifier of the node, the name of the node, the identifier of the tensor input to the node (which is later used as an input edge), the identifier of the tensor output from the node (which is later used as an output edge), and the type of operation performed by the node can be extracted. For example, the extracted information can be mapped to the keys of the structured text data. This extraction method can be repeated in a corresponding manner for all other components constituting the intermediate representation structure, such as modules, tensors, and / or metadata, in addition to nodes. Accordingly, a JSON file can be generated as information about the components in the intermediate representation structure is converted and mapped to each key. Thus, data in the form of an intermediate representation structure used within the server (200) can be converted into structured text data that is easy to communicate and visualize.

[0239] The server (200) can generate structured text data using the extracted information (525). For example, the server (200) can perform a file-based conversion (e.g., conversion from a Python object to a JSON format) to visualize an artificial intelligence model. The server (200) can change the information contained in the intermediate representation structure into a form that is easy to handle on the user terminal (300). The server (200) can generate a schema for rendering on the user terminal by querying the attribute values ​​of the Python object (e.g., intermediate representation structure) to extract the inputs and outputs of the artificial intelligence model, nodes, the connection relationships between nodes, and / or metadata (e.g., information on how to quantize the model) and arranging the extracted information according to a pre-set JSON template (e.g., in the format of graph, modules, in_ports, out_ports, nodes, edges, tensors, and metadata). For example, since the intermediate representation structure is a data structure that does not include edges and is represented only by tensors, but the JSON format is a structure that includes edges, the task of converting information stored in tensors within the intermediate representation structure into edges of structured text data can be performed. Structured text data may refer to a result generated by the server (200) arranging extracted model information according to a predefined text data structure (e.g., a first schema) so that the user terminal (300) can render it. Step 525 may correspond to the process of exchanging visualization information between the server (300) and the user terminal (200).

[0240] The user terminal (300) can reconstruct structured text data (530). For example, step 530 may correspond to, for example, the rendering preparation step or schema conversion process of the user terminal (300). The user terminal (300) can generate reconstructed structured text data by changing the schema of the structured text data. The user terminal (300) may represent a process of converting the received structured text data into a format suitable for a rendering library (e.g., a second schema). For example, tensors and metadata that were separated in the structured text data of the first schema may be integrated into nodes and edges in the structured text data of the second schema. For example, since the components of the artificial intelligence model are listed in the JSON file according to a classification system such as sub-graph, modules, in_ports, out_ports, nodes, edges, tensors, and metadata, this structured text data has a form in which the connection structure of the operators of the artificial intelligence model cannot be intuitively grasped. Accordingly, the user terminal (300) can reconstruct the information in the JSON file into only modules, nodes, and edges to efficiently render a graph-formatted topology. In this reconstruction process, tensors and metadata included in the JSON file can be modified to be included in the modules, nodes, and edges. Accordingly, the user terminal (300) can integrate or change the first schema (e.g., backend schema) into a second schema for rendering the graph in order to facilitate rendering of the user terminal (300) and subsequent modification of the artificial intelligence model.

[0241] The user terminal (300) can complete graph visualization into a graph-format topology (535). For example, step 535 may correspond to the output stage of the visualization result. For example, step 535 may correspond to the rendering process of the user terminal. The user terminal (300) can render a graph-format topology for an artificial intelligence model using reconstructed structured text data and present it visually to the user. For example, the user terminal (300) can render a topology (e.g., model structure diagram) based on a schema reconstructed only with modules, nodes, and edges (e.g., a second schema) and generate detailed information (e.g., metadata and attributes) corresponding to the topology. This detailed information may be displayed in the rendering result in relation to the rendered topology according to user input.

[0242] As such, steps 505 to 535 in FIG. 5 exemplarily illustrate features in which visualization (e.g., rendering) of an artificial intelligence model is performed through interaction between a server (200) and a user terminal (300) in response to a model visualization request. Steps 540 to 570 in FIG. 5, to be described later, exemplarily illustrate interaction between a user terminal (300) and a server (200) performed upon receiving user input (e.g., change (modification) input) on the rendered topology. For example, steps 540 to 570 may represent interactive editing steps of the artificial intelligence model.

[0243] A user terminal (300) can receive user input on a rendered topology (540). The user terminal (300) can receive a request from the user to modify the model (e.g., a request to change the topology through visual interaction with the user). Depending on the interaction between the user terminal (300) and the user on the rendered topology, events related to changes in the rendered topology and / or changes in the artificial intelligence model may occur. On the rendered topology, user input related to deleting nodes, adding nodes, adding edges, deleting edges, changing nodes, and changing edges may be received. For example, the user terminal (300) can detect user input selecting a specific area (e.g., a node, an edge, or an empty space) on the rendered topology (e.g., right-clicking the mouse). In response to this, the user terminal (300) can display a topology change option object (e.g., a context menu) that corresponds to the selected area. For example, the location information of the selected area and the information displayed in the topology translation option object (e.g., change options) may be correlated. For example, the information displayed in the topology translation option object (e.g., change options) may be determined differently based on the location information of the selected area. For example, the information displayed in the topology translation option object (e.g., change options) may vary based on the location information of the selected area. User input requesting specific changes, such as node deletion and edge creation, can be additionally obtained through the topology translation option object.

[0244] The user terminal (300) determines the type of change request based on user input and, depending on the determined type, determines whether to interact with the server (200) and / or the information to be transmitted for the interaction. In the case of a simple node location change where the connection structure of the node and edge is not changed, it is determined that interaction with the server (200) is unnecessary, and re-rendering (570) can be performed immediately within the user terminal (300) without a request to the server (200). In the case where a structural change of the artificial intelligence model occurs, such as the creation, deletion, and attribute change of nodes and / or edges, it is determined that interaction with the server (200) is necessary, and the step (545) of transmitting change information to the server (200) for interaction with the server (200) can be performed.

[0245] The user terminal (300) can transmit change information to the server (200) (545). Step 545 may correspond to the communication step of a change request between the user terminal (300) and the server (200). The user terminal (300) can identify and transmit information necessary for changing the intermediate representation structure and generating the changed structured text data to the server (200) for topology changes based on user input. For example, the user terminal (300) can perform re-rendering (570) on the user interface based on user input and transmit change information asynchronously to the server (200). The transmitted information may vary depending on the type of change. For example, the user terminal (300) can transmit the ID of the node to be deleted when deleting a node, and transmit the IDs of the start node and the end node when creating an edge.

[0246] The server (200) can perform backend model modification and data regeneration in accordance with a change request. The server (200) can request modification of the intermediate representation structure based on the change information (550). Such a request (550) can be made through communication between internal entities of the server (200) or through communication between entities outside the server (200) and the server (200). For example, the server (200) can call the API of an entity of the server (200) in response to a received change request and modify the intermediate representation structure using the result of the call.

[0247] The server (200) can obtain a modified intermediate representation structure in accordance with a change request (555). The server (200) can receive change information (550) from the user terminal (300), call an API, and modify the intermediate representation structure (e.g., a Python object). An entity within the server (200) returns the modified intermediate representation structure (555), and the server (200) can generate a new intermediate representation structure (i.e., the modified intermediate representation structure of step 555) that reflects the changes from the modified object. In this way, the server (200) can generate an artificial intelligence model that reflects the changes, so that synchronization between the artificial intelligence model on the server (200) side and the rendered topology on the user terminal (300) side can be implemented.

[0248] The server (200) can extract modified model information from the modified intermediate representation structure (560). Step 560 can be implemented in a manner corresponding to step 520. For example, the server (200) can extract information (e.g., attribute values) from the modified intermediate representation structure (555) which is the return result. The server (200) can use the attribute values ​​included in the intermediate representation structure to extract information such as nodes, tensors, and / or metadata necessary for the reconstruction of the intermediate representation structure.

[0249] The server (200) can regenerate structured text data reflecting change information and transmit the regenerated structured text data to the user terminal (300) (565). Step 565 can be implemented in a manner corresponding to step 525.

[0250] The user terminal (300) can perform re-rendering of a graph-format topology (570). The user terminal (300) can perform immediate re-rendering. For example, immediately after user input (540), the user terminal (300) can perform re-rendering of the graph-format topology (570) by immediately reflecting the requested changes on the user interface without waiting for a response from the server (200) (e.g., the response at step 565). This can provide the user with quick visual feedback. After re-rendering (570), the user terminal (300) can perform change processing in different ways depending on the results included in the response (565) from the server (200). For example, if the response (565) from the server (200) includes a successful change (or a successful response), the user terminal (300) can use the structured text data received from the server (200) to integrate or update detailed information of the already re-rendered topology (e.g., a new node ID generated by the server (200) and / or changed metadata, etc.). For example, if the response (565) from the server (200) includes a failed change (or a failed response), the user terminal (300) can perform a rollback to return the re-rendered topology to its state prior to the modification.

[0251] Accordingly, the technique according to one embodiment of the present disclosure implements an optimistic update method in which the UI immediately reflects changes without waiting for a response from the server (200) when the user directly manipulates the topology on the GUI, thereby achieving the technical effect of providing immediate and seamless visual feedback to the user. This reduces the user's cognitive load and can significantly improve productivity by maximizing the workflow during iterative model modification and experimentation processes.

[0252] In addition, the technique according to one embodiment of the present disclosure implements an asynchronous synchronization mechanism that integrates detailed information upon successful change or rolls back to a previous state upon failed change, based on the final response from the server (200) after immediate re-rendering of the user terminal (300). This provides fast responsiveness to the user and simultaneously ensures that the final model state always matches the valid state verified by the server (200), thereby achieving a technical effect. Accordingly, the effect of ensuring data integrity and the stability of the entire system can be achieved by fundamentally blocking potential errors or abnormal states on the UI from being reflected in the actual model.

[0253] Furthermore, the technique according to one embodiment of the present disclosure can achieve the technical effect of significantly lowering the barrier to specialized knowledge in modeling, as users are allowed to directly select and modify familiar graphic elements, such as nodes and edges, on a visually rendered topology instead of directly modifying complex code. The technique according to one embodiment of the present disclosure can achieve the technical effect of increasing accessibility to AI model development and promoting collaboration by shifting the task of modifying the internal structure of an AI model from a code-centric to a visual-centric approach.

[0254] In addition, the logic for changing the structure of an actual artificial intelligence model in a technique according to one embodiment of the present disclosure can be centrally processed by a server (200). Accordingly, even if a user attempts a logically impossible connection on the UI, the server (200) verifies and processes it, thereby ensuring that an AI model with a valid structure is always generated, thus achieving a technical effect. This can lead to preventing errors that may occur during the complex model modification process and ensuring the quality and reliability of the modified model.

[0255] A user terminal (300) can receive a model download request from a user and transmit the received download request to a server (200) (575). The server (200) can transmit a model file or transmit a link (e.g., a URL) for model download to the user terminal (300). For example, if the user terminal (300) requests a download (575) to save a modified artificial intelligence model completed through visualization and modification as an actual file, the server (200) can respond by transmitting (580) the corresponding model file or download link corresponding to an intermediate representation structure stored in the server (200) to the user terminal.

[0256] FIG. 6 illustrates an exemplary method for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0257] The steps illustrated in FIG. 6 are exemplary steps, and depending on the implementation mode, additional steps may be included, or some of the steps in FIG. 6 may be omitted or replaced.

[0258] For example, the computing device (100) in FIG. 6 may correspond to a user terminal.

[0259] In one embodiment, the computing device (100) can receive an artificial intelligence model that is the subject of visualization (610).

[0260] For example, a computing device (100) may receive structured text data from a server (200) in which an intermediate representation structure is reconstructed into a first schema. For example, the artificial intelligence model received by the computing device (100) (e.g., a user terminal) in FIG. 6 may correspond to the structured text data transmitted from the server (200). In step 610, the computing device (100) may receive information related to the AI ​​model to be analyzed and visualized.

[0261] In one embodiment, the computing device (100) can render a graph-type topology on a user interface that visualizes and represents the abstracted structure of an artificial intelligence model (620).

[0262] For example, the computing device (100) can generate a visual representation corresponding to an artificial intelligence model by abstracting the artificial intelligence model using received structured text data. This may represent a process of converting and displaying the abstracted AI model structure (e.g., intermediate representation structure) into a graphic form that can be perceived by the user. Based on the received model-related data, the computing device (100) can render a graph-type topology composed of nodes (computation units) and edges (data flows) on a user interface. For example, the computing device (100) can visually represent the basic architecture of the AI ​​model, that is, the connection relationships between nodes and edges. The computing device (100) can render unsupported nodes that are not supported on a specific platform (e.g., hardware) or inefficient nodes that cause performance degradation during execution so that they can be visually distinguished from other nodes (e.g., by displaying a warning color or icon). When comparing two models, the computing device (100) can identify a difference node that exists only in one model and render the difference node so that it is clearly distinguished from the common node by applying predefined visual attributes, such as green for the added node and red for the deleted node. In the present disclosure, the term difference node may be used to encompass not only added and deleted nodes but also optimized nodes (e.g., pruning, quantization, operator type change, operator fusion, etc.). In another embodiment, the computing device (100) can render the node by dynamically varying the color of the node based on the node performance value (e.g., SNR score) included in the node's metadata, such that the saturation or brightness of the color gradient is proportional to the magnitude of the performance value.

[0263] In one embodiment, when a user selection input is obtained in which a region on the topology of a user interface is selected, the computing device (100) may display a topology change option object that has a correlation with the selected region on the user interface (630).

[0264] The computing device (100) can interact with the user on the rendered topology to identify the change intent corresponding to the user input and generate a re-rendering result according to the change intent. For example, the computing device (100) can provide context-ware editing tools on the user interface in response to user input.

[0265] For example, there may be a correlation between the location of user input (e.g., a selected area) on the rendered topology and topology change options. Topology change options may be mapped to and managed for each area on the rendered topology. The computing device (100) may display a topology change option object (e.g., a context menu for change) containing actionable change options, depending on the location of the area corresponding to the user's selected input on the rendered topology. The display of such topology change option objects may be performed on the computing device (100) without interaction with the server (200).

[0266] In one embodiment, the rendering result on the user interface may be classified into multiple regions, and a corresponding topology change option object may be assigned to each of the multiple regions. For example, if the user's selection input selects a specific node region within the user interface, the topology change option object may display options such as moving or deleting the node, or creating an edge starting from the node. As another example, if the user's selection input selects a specific edge region within the user interface, the topology change option object may display an option to delete the edge. As yet another example, if the user's selection input selects a remaining region (empty space) within the user interface that has no nodes or edges, the topology change option object may display an option to create a new node. Thus, depending on the location selected by the user on the user interface, different objects containing different content related to topology changes may be output.

[0267] In an additional embodiment, an implementation that displays different topology change option objects depending on the type of user selection input may also be possible. For example, if the user's selection input is a first selection input such as a right-click of the mouse, a topology change option object may be displayed, and if it is a second selection input such as a left-click, a detailed information window displaying the attributes and / or metadata of the corresponding node may be displayed. For example, the computing device (100) may re-render the topology in response to a user input selecting a region of a node within the rendered topology (e.g., an additional information display input), so that the selected node and a plurality of sub-nodes constituting the node are displayed together while maintaining the connection structure between the selected node and neighboring nodes. Specific examples of these features will be described later in FIG. 14.

[0268] Modifications in the present disclosure may be used to encompass not only changes to the structure of the artificial intelligence model but also optimization (e.g., compression and graph optimization, etc.) and quantization of the artificial intelligence model.

[0269] In one embodiment, the computing device (100) can re-render the topology with the changes reflected on the user interface based on a user change input requesting a change on a topology change option object (640).

[0270] In one embodiment, when a user change input requesting a structural change, such as deleting or creating a node, is received on a topology change option object, the computing device (100) may re-render the topology according to the user change input. For example, the computing device (100) may reflect the changes in the user interface without waiting for a response from the server (200) and may vary additional actions based on the response from the server (200) according to the response from the server (200). For example, if a change success response is received from the server (200) after an immediate re-rendering, the computing device (100) may integrate the detailed information (e.g., ID of a newly created node) transmitted from the server (200) into the previously re-rendered topology to determine the final state of the re-rendered topology. For example, if a change failure response is received from the server (200) after an immediate rendering, the computing device (100) may cancel the changes reflected on the user interface and perform a rollback to return to the topology state prior to the change. For example, if the user's change request input is a simple node position movement or a simple edge position movement that does not require interaction with the server (200), the computing device (100) can perform re-rendering only within the user interface of the computing device (100) without a request to the server (200) and terminate the re-rendering process.

[0271] In one embodiment, in response to a user change input, the nodes before and after the node to be changed and the related edges may be considered together during the process in which the computing device (100) performs re-rendering. For example, when a user change input to change a node on a topology change option object is obtained, the target node object to be changed, the target edge object connecting the target node object, and the neighbor node object connected to the target node object through the target edge may be recognized together as targets for topology change. For example, the computing device (100) may transmit information about the recognized objects to the server (200). In another example, the computing device (100) may transmit only information about the target node object to the server (200), and the server (200) may use the transmitted information to additionally recognize the target edge object connecting the target node object and the neighbor node object connected to the target node object through the target edge, thereby changing or regenerating the corresponding intermediate representation structure and / or structured text data.

[0272] In one embodiment, when a user change input to create a new node on a topology change option object is obtained, the computing device (100) may display a new node object at a location associated with a selected area and display a new node option object requesting additional input associated with the new node object. For example, the new node option object may include a first option object for moving the new node object, a second option object for creating an edge associated with the new node object, and / or a third option object for deleting the new node object or deleting another node. In response to receiving additional input requesting the creation of a target edge object associated with the new node object on the new node option object, the computing device (100) may re-render the modified topology on the user interface to include target edge objects connecting the new node object and neighbor node objects of the new node object.

[0273] In one embodiment, when a user change input on a topology change option object includes the creation of a new node, once the target node to be created is determined, a target edge connecting the target node and neighbor nodes can be automatically created by the computing device (100). For example, the computing device (100) can automatically identify neighbor nodes connected to the target node using location information within the topology of the target node to be created, and can automatically create and render a target edge connecting the identified neighbor nodes and the target node.

[0274] In an additional embodiment, the target edge may be automatically generated through a method of automatically recognizing the neighbor node(s) of the new node when the new node is created or determined.

[0275] In one embodiment, when a new node object is rendered, the re-rendering of the topology may be implemented by highlighting the connection results between the new node object and neighbor node objects. For example, the computing device (100) may re-render the topology on the user interface so that the new node object and neighbor node objects are highlighted to distinguish them from other node objects on the changed topology. For example, methods such as distinguishing the colors of the node objects, distinguishing the shapes of the node objects, or adding distinguishing text information to the related node objects may be considered.

[0276] In one embodiment, when a user change input to delete a node object on a topology change option object is obtained, the computing device (100) can delete the node object associated with the selected area and replace the first edge objects associated with the deleted node object with second edge objects that connect neighboring node objects associated with the deleted node object. Accordingly, the edges associated with the deleted node can be configured to be connected to neighboring nodes other than the deleted node. The computing device (100) can implement automatic connection of edges following node deletion by re-rendering the topology reflecting the deleted node object and the second edge object on the user interface.

[0277] In one embodiment, when a user change input to delete an edge object on a topology change option object is obtained, the computing device (100) can delete the edge object associated with the selected area and move at least one of the node objects and edge objects on the topology based on the location of the deleted edge object. Accordingly, the position of the rendered topology can be rearranged according to the deleted edge. The computing device (100) can reconstruct the topology reflecting the edge deletion by re-rendering the topology reflecting the deleted edge object and the moved object on the user interface.

[0278] As described above in the previous drawings, the computing device (100) determines whether to interact with the server for re-rendering the topology based on a change type determined by user change input determined by interaction with the user, and by performing interaction with the server (200) according to this determination, efficient synchronization between backend data stored in the server (200) and frontend data output on the user interface can be implemented.

[0279] A technique according to one embodiment of the present disclosure can implement context-aware interaction in which, when a user selects a specific area on a topology, only modification options related to that selected area are selectively provided on the user interface. For example, the content of the topology change option object displayed may differ when a node is selected compared to when an empty space is selected. This technical feature can achieve the technical effect of reducing the user's cognitive load by not exposing unnecessary or invalid options to the user, and preventing operational errors in a complex GUI environment. Through the technique according to one embodiment of the present disclosure, the user can focus only on tasks that can be performed in the current context, thereby deriving the technical effect of significantly increasing user convenience and task accuracy.

[0280] FIG. 7 illustrates, in accordance with one embodiment of the present disclosure, the topology re-rendering of a user interface based on user change input.

[0281] In the user interface (700) illustrated in FIG. 7, a plurality of objects (705 to 750) are exemplarily represented. The objects described below may correspond to UI objects. The computing device (100) in FIG. 7 may correspond to a user terminal.

[0282] The user interface (700) may include a first area displaying a graph-format topology, a search input object (705) receiving a search input within the topology, a display option object (710) for determining how components within the topology are displayed, and a display option object (715) for determining whether components within the topology are exposed or hidden, and a third area displaying at least one of attributes and metadata embedded in a node within the topology and allowing modification input for at least one of the attributes and metadata. As an example, not a limitation, the second area including the search input object (705), the display option object (710), and the display option object (715) may occupy an area of ​​the upper or upper-left part of the user interface (700), the third area may occupy a right area of ​​the user interface (700), and the first area displaying the topology may generally occupy the remaining areas of the user interface (700).

[0283] The user interface (700) may include a toolbar object (701) that allows searching within a rendered topology, provides viewing options, and provides exposure option filters. For example, the toolbar object (701) may include a search input object (705) that receives a search input to search for whether a specific representation exists within the topology. Depending on the search result found on the search input object (705), screen navigation to a node and / or edge corresponding to the search result within the rendered topology may be performed automatically. The node and / or edge corresponding to the search result may be highlighted in a different form than objects that do not correspond to the search result. For example, the toolbar object (701) may include a display option object (710). The display option object (710) may be used to set information to be displayed and information not to be displayed within the user interface (700). For example, when a display option object (710) is selected, a user selection option object may be output on the user interface (700) that allows a selection as to whether to display edge indices, whether to display tensors or edges, and / or whether to display weights. For example, a toolbar object (701) may include an exposure option object (715). The exposure option object (715) may be used to determine data to filter on the user interface (700). When a user selection input for the exposure option object (715) is received, the exposure option (715) may be output as a user selection option object that allows a selection as to whether to filter specific metadata, for example.

[0284] The toolbar object (701) may further include a model comparison mode activation object (720) for activating the model comparison mode. When a user selection input for the model comparison mode activation object (720) is received, the currently rendered topology is output in the first area of ​​the user interface (700), and an input area for allowing input for a new artificial intelligence model may be displayed on the user interface (700). A specific description of this will be provided later in FIG. 11.

[0285] The toolbar object (701) may further include a model editing mode activation object (725) for activating the model editing mode. When a user selection input for the model editing mode activation object (725) is received, the user interface (700) may allow receiving user input for model change or model modification according to one embodiment of the present disclosure. For example, prior to receiving a user selection input for the model editing mode activation object (725), when a specific area of ​​the rendered topology is selected, information about the selected area (e.g., metadata and attributes, etc.) may be output. When a user selection input for the model editing mode activation object (725) is received, when a specific area or remaining area of ​​the rendered topology is selected, a topology change option object related to that area (e.g., correlated with the selected area) may be output on the user interface (700).

[0286] In one embodiment, the user interface (700) may be configured to enable search expression-based search through a search input object (705). For example, a computing device (100) may be configured to filter internal components of an artificial intelligence model, such as specific operations or nodes, through the user interface (700). The search expression may include a combination of multiple predefined keys and values, such as op_type, kernel_size, and params, and logical operators (e.g., AND, OR). The keys used in the search expression may be defined as identifiers that generalize the unique attributes of each element constituting the artificial intelligence model. These may include structural keys indicating the type of operation or structural characteristics (e.g., op_type, kernel_size), tensor keys indicating the form or type of data input or output to the operation (e.g., input_shape, dtype), and parameter keys related to performance, such as trainable weights or computational amounts (e.g., params, flops). Specifically, the user can accurately identify only the desired nodes by simultaneously specifying the type of operation and the kernel size, such as 'op_type: Conv AND kernel_size: [3,3]', or by setting the number of parameters and the activation function as conditions, such as 'params > 10000 AND activation: ReLU'. The user interface (700) can output a rendered topology. The rendered topology can represent a graph-visualized result of the artificial intelligence model. The rendered topology can be represented by multiple nodes (730, 735, 745, and 750) and edges connecting the nodes, as in the example of FIG. 7. The topology can be composed of multiple nodes representing the operation units of the artificial intelligence model and edges connecting them.

[0287] In one embodiment, the computing device (100) may group nodes so that the nodes are distinguished and displayed according to the computational characteristics or attributes of the nodes. For example, nodes (730 and 745) may represent a Rectified Linear Unit (ReLU) operator. ReLU is an activation function widely used in deep learning models that performs a non-linear transformation to increase the expressiveness of the model by outputting 0 if the input value is less than 0 and outputting the input value as is if it is greater than 0. Nodes (730 and 745) may form a topology on the user interface (700) in a form that displays identification information about the nodes. Nodes (735 and 750) may represent a Convolution operator. The Convolution operation is an operation mainly used in computer vision models that performs the function of extracting a feature map by applying a learnable filter (e.g., a kernel) to input data (e.g., an image). In the user interface (700) of FIG. 7, a node (735) may be provided in a form that displays together attribute information (735a) defining detailed operations such as kernel_shape and strides, and identification information (735b) of the node. The computing device (100) may vary the display information for the node on the user interface (700) according to the operational characteristics of the nodes. The nodes (730 and 745) and nodes (735 and 750) of FIG. 7 are grouped with corresponding nodes, and the user interface (700) may display the nodes in a manner where the groups are distinguished from one another.

[0288] According to one embodiment of the present disclosure, the user interface (700) can edit the structure of the artificial intelligence model based on the interaction between the rendered topology and the user. Editing, changing, and modifying in the present disclosure may be used interchangeably depending on the mode of implementation.

[0289] In one embodiment, when a user selection input selecting an area on the user interface (700) is obtained, the computing device (100) may display a topology change option object having a correlation with said selected area on the user interface (700). The correlation may refer to a logical association for selecting and providing only valid editing options applicable to an element according to the type and / or location of the graphic element selected by the user on the user interface (700). There may be a correlation between the location of the user input on the user interface (700) and the editing options output on the user interface (700). That is, there may be a correlation between the type and / or location of the user input on the user interface (700) and the editing options output on the user interface (700). If the type and / or location of the user input on the user interface (700) is different, editing options of different content may be output. Based on this correlation, the user interface (700) can achieve the technical effect of filtering out unnecessary or invalid options in advance and intelligently presenting only the editing options that best suit the user's intent. In addition, this topology change option object (740) can serve to assist the user in editing complex topology structures more intuitively and efficiently.

[0290] As illustrated in FIG. 7, when a user selection input is received from a user to select a node area occupied by a Conv node (735), the computing device (100) may recognize that the user selection input indicates a context of a node based on the type of UI object associated with the location of the selected area. When a user selection input (e.g., mouse click and touch input) is received from a user to select a node area occupied by a specific node (735) on the topology, the computing device (100) may display a topology change option object (740) correlated with the selected area. Since the selected area is detected as a node area, the topology change option object (740) may include change options such as moving the node object (Move node), deleting the node object (Delete node), and creating an edge object associated with the node object (Create edge). Subsequently, when the computing device (100) receives a user change input from the user selecting a specific option (e.g., Delete node) on the topology change option object (740), the computing device (100) may re-render the topology with the changes reflected, such as deleting the node and reconnecting the associated edges based on the user change input. This process may include, for example, replacing first edge objects associated with the deleted node object with second edge objects connecting neighboring node objects.

[0291] For example, if user input selects a move option for a node object, the node can be moved to a desired location on the topology via drag-and-drop by re-rendering. In this example, re-rendering can be performed by the computing device (100) without communication with a server, and if input regarding a refresh is received from the user, the moved node can be returned to its original location. For example, if user input requests the deletion of a node object, the computing device (100) can perform re-rendering to delete the selected node object and automatically recognize the broken connection point caused by the deletion of the node, and create a new edge that directly connects neighboring nodes. For example, if user input requests the user to select and delete an edge, the edge is deleted and the topology is re-rendered accordingly. For example, re-rendering can be implemented in which a new node object is created through user selection input, such as a right-click of the mouse, in an empty remaining area within the topology graph. The newly created node object can be connected to a new edge through additional input in which the user selects another neighboring node object they wish to connect to, and the topology can be re-rendered to reflect these changes.

[0292] When input is received from a user selecting an edge area occupied by an edge object connecting two nodes (e.g., 730 and 735), the computing device (100) may recognize the context of the user's selection as an edge based on the location of the selected area and / or the type of UI object. In this case, depending on the correlation with the edge, the topology change option object (740) may display an option specialized for the edge, such as the deletion of the edge object, either preferentially or exclusively.

[0293] Additionally, when user input is received from a user selecting a residual area (e.g., the background portion of FIG. 7) that is not occupied by both nodes and edges on the topology, the computing device (100) may perceive the context of the selected area as empty space. Depending on the correlation with the perceived context, the user interface (700) may display an option to create a new node object that adds a new element to the space through a topology change option object (740).

[0294] FIG. 8 illustrates, in accordance with one embodiment of the present disclosure, a topology re-rendering of a user interface (800) based on user change input.

[0295] FIG. 8 exemplarily illustrates, in steps (800a, 800b, 800c), the process of creating a new node object among a topology change method according to one embodiment of the present disclosure. The computing device (100) in FIG. 8 may correspond to a user terminal.

[0296] Referring to reference numeral 800a of FIG. 8, a portion of a topology including a first node (810) and a second node (830) and an edge (820) connecting them is rendered on the user interface (800). A user selection input may be received from the user to select a remaining area between the first node (810) and the second node (830) that is not occupied by other nodes or other edges. In response, the computing device (100) may display a topology change option object (840) on the user interface (800) that includes an option to create a new node object (Create node) based on the correlation with the selected area.

[0297] When a user change input is received from a user selecting the Create node option on the topology change option object (840), the computing device (100) can re-render the topology with the change reflected, as shown in reference number 800b of FIG. 8. Specifically, the re-rendering step can display a new node object (850) (e.g., a node representing a Conv operator) at a location associated with the selected area. Along with the creation of the new node object (850), the computing device (100) can display a new node option object (860) that can request additional changes associated with the new node object (850). The new node option object (860) may include, for example, an option to move the location of the new node object (850) (Move node), an option to delete the object (850) (Delete node), and an option to create an edge object associated with the object (850) (Create edge).

[0298] When the computing device (100) receives additional input from the user requesting the creation of a target edge object, such as "Create edge" on a new node option object (860), the computing device (100) may re-render the modified topology as shown in reference number 800c of FIG. 8. The re-rendering step may change and display the topology to include neighbor node objects of the new node object (850), namely the first node (810) and the second node (830), and target edge objects (870) connecting the new node object (850). Additionally, the re-rendering step may apply a highlight (e.g., a red border) so that the new node object (850) created on the modified topology and the neighbor node objects (810, 830) connected thereto are visually distinguished from other node objects within the topology. This may lead to a technical effect that improves the usability of the user interface (800) by clearly perceiving the scope of the topology change to the user.

[0299] Text displayed around an edge object (870) in the user interface (800) may indicate, for example, the identifier of the data flowing between two nodes, i.e., the tensor. For example, the upper edge object (870) may indicate that it is the first (0-indexed) output of a node (810) named ' / patch_emb_bed.2 / Relu'. Such a tensor identifier can generally be composed of a combination of identification information of the source node generating the data and an index identifying the output of that node. Some operators (e.g., nodes) of an artificial intelligence model may have multiple outputs, and in this situation, the tensor identifier may indicate which output of the corresponding node it is. Accordingly, the path of the data can be clearly identified even within a topology of a complex structure.

[0300] FIG. 9 illustrates, in accordance with one embodiment of the present disclosure, a topology re-rendering of a user interface (900) based on user change input.

[0301] FIG. 9 is an example diagram illustrating, in steps (900a, 900b), the process of deleting edge objects in a re-rendering method according to a topology change in one embodiment of the present disclosure.

[0302] Referring to reference numeral 900a in FIG. 9, a topology (900) is rendered on a user interface (900) and includes a plurality of node objects (e.g., a Conv node (930) and an Add node (940), etc.) and edge objects connecting them. A computing device (100) (e.g., a user terminal) may receive a user selection input from a user to select an edge area occupied by a specific edge object (910) on the topology (900). In the example of FIG. 9, the selected edge object (910) corresponds to an edge directly connecting the first Conv node (930) and the Add node (940). When the user selection input is obtained, the computing device (100) may display a topology change option object (920) on the user interface (900) that includes an option to delete the edge object (910) based on the correlation with the selected edge area. For example, as illustrated in FIG. 9, a topology change option object (920) can be displayed around a selected edge object (910).

[0303] Reference numeral 900b exemplarily illustrates the result of topology re-rendering with edge deletion reflected. When a user change input is received from a user selecting the Delete edge option on the topology change option object (920), the computing device (100) can re-render the topology (900b) with the changes reflected according to the user's change input. In the re-rendered topology (900b), it can be seen that the edge object (910) associated with the selected area has been deleted in accordance with the user's change request. Consequently, the data flow path that directly connected the first Conv node (930) and the Add node (940) is removed, and the connection structure of the topology can be changed accordingly. Thus, the present disclosure can provide a user interface (900) that allows the user to easily edit the topology of an artificial intelligence model by allowing an intuitive interaction in which the user visually selects and deletes edges directly.

[0304] FIG. 10 illustrates an exemplary method for providing a user interface for visualizing an artificial intelligence model according to one embodiment of the present disclosure. FIG. 10 exemplarily shows a flowchart illustrating the overall operation of a method for comparing and visualizing a plurality of artificial intelligence models according to one embodiment of the present disclosure.

[0305] Depending on the implementation mode, some steps of the flowchart in FIG. 10 may be omitted or replaced, and an embodiment in which additional steps are added to the flowchart may also be possible. The computing device (100) in FIG. 10 may correspond to a user terminal.

[0306] In one embodiment, the computing device (100) can receive a first artificial intelligence model and a second artificial intelligence model that are the subject of visualization (1010).

[0307] Step 1010 may correspond to an initial data acquisition step for visually comparing the structural differences between the two models. The first artificial intelligence model and the second artificial intelligence model are models to be compared and may represent two models selected by the user and / or may represent an original model and a derived model to which modifications or optimizations have been applied.

[0308] In one embodiment, depending on various user scenarios, step 1010 may include the following examples. In one example, the computing device (100) may receive an original model as a first artificial intelligence model and receive a model (e.g., a derived model) to which optimization techniques such as quantization or pruning have been applied to the original model as a second artificial intelligence model to compare structural changes before and after optimization. In another example, the computing device (100) may receive two models with different architectures (e.g., receiving ResNet as the first artificial intelligence model and EfficientNet as the second artificial intelligence model) to compare and analyze topological differences between the architectures of the models. In yet another example, the second artificial intelligence model may be a derived model created based on changes made by the user to the first artificial intelligence model that the user had previously visualized and edited.

[0309] In one embodiment, the computing device (100) can generate a first topology in a graph format that visualizes and represents the abstracted structure of a first artificial intelligence model, and generate a second topology in a graph format that visualizes and represents the abstracted structure of a second artificial intelligence model (1020).

[0310] For example, a graph format may refer to a structure that includes nodes representing computational units of an artificial intelligence model and edges connecting the nodes.

[0311] In one embodiment, the computing device (100) can generate a plurality of topologies in a graph format that visualize and represent the abstracted structure of each of the received plurality of artificial intelligence models.

[0312] In one embodiment, the specific method for generating a graph-format topology of the computing device (100) is replaced with the description above.

[0313] In an additional embodiment, the computing device (100) may identify differential nodes representing structural differences between models by traversing the nodes of the models and comparing node identification information (e.g., unique ID of the node) during the process of generating a topology.

[0314] In one embodiment, the computing device (100) can render the first topology and the second topology so as to be visually comparable on the user interface by rendering the first topology in a first area of ​​the user interface and rendering the second topology in a second area of ​​the user interface (1030).

[0315] For example, the first area and the second area are predetermined partitioned spaces on the user interface, where the first area may occupy the left area of ​​the user interface and the second area may occupy the right area of ​​the user interface.

[0316] In the present disclosure, the expression "visually comparable rendering" may mean not only simply displaying two or more model topologies side by side, but also configuring and providing visual information in a manner that assists a user in intuitively and efficiently perceiving similarities and differences between the two models. This may refer to a user interface actively processing and rendering information related to topologies to achieve the specific purpose of comparison. Such comparative rendering may include any GUI-based technical processing that provides visual cues to enable a user to discover structural differences between two models with minimal cognitive effort. This may be implemented through methods such as spatial arrangement, highlighting of differences, and interactive linkage.

[0317] In the process of comparing and rendering topologies, the computing device (100) can identify differential nodes representing structural differences between models by traversing the nodes of the models and comparing node identification information (e.g., unique ID of the node).

[0318] For example, the computing device (100) may traverse the nodes of the second artificial intelligence model to generate a second topology, identify a node that does not exist in the first artificial intelligence model as an added node (e.g., a second differential node), and apply a second visual attribute (e.g., a green highlight) to indicate the addition thereto. The computing device (100) may traverse the nodes of the first artificial intelligence model, identify a node that does not exist in the second artificial intelligence model as a deleted node (e.g., a first differential node), and visually restore and add the node to the second topology with a first visual attribute (e.g., a red highlight). The computing device (100) may traverse both the first and second artificial intelligence models to create and reconfigure new connections so that the edges (e.g., connections) of the added first differential node maintain the connections prior to deletion. The computing device (100) can readjust the node positions of the first topology (or the second topology) to align them with the common node positions of the second topology (or the first topology) in order to facilitate visual comparison between the two topologies.

[0319] For example, the computing device (100) may traverse the first artificial intelligence model and identify nodes not present in the second artificial intelligence model as deleted nodes (e.g., first difference nodes), and display them on the first topology as first visual attributes (e.g., red). In parallel (or serially), the computing device (100) may traverse the second artificial intelligence model and identify nodes not present in the first artificial intelligence model as added nodes (e.g., second difference nodes), and display them on the second topology as second visual attributes (e.g., green).

[0320] In the process of comparing and rendering topologies, the computing device (100) can identify differential nodes by considering not only node identifiers or edge identifiers, but also the history of modifications made in the process of modifying or optimizing the first artificial intelligence model into the second artificial intelligence model. For example, if some nodes are deleted or added during the graph optimization and / or quantization process, a history reflecting which nodes were deleted, added, or changed is stored, and the computing device (100) can identify the deleted, added, or changed nodes by retrieving the history stored during the model comparison process.

[0321] The computing device (100) can identify a node that is common to the models but has changed attributes as a type of difference node. For example, when comparing models before and after quantization, if the same Conv node exists but the data type of the weights has changed, the computing device (100) can identify it as a node with different attributes and include it in the difference nodes.

[0322] In the present disclosure, a difference node may refer to a node where differences exist between artificial intelligence models. For example, a difference node may be used to encompass not only added and deleted nodes but also optimized nodes (e.g., pruning, quantization, operator type change, operator fusion, etc.). For example, a difference node may represent a node indicating structural differences between artificial intelligence models and / or a node with different attributes among corresponding nodes in artificial intelligence models. For example, a difference node may be displayed in a manner that distinguishes it from common nodes in shape or color, etc., during rendering. For example, among the difference nodes, a deleted difference node and an added difference node may be assigned different visual attributes, and each of these difference nodes may be assigned visual attributes different from the common node. Different visual attributes may indicate that at least one of color and appearance (e.g., shape) is different.

[0323] A computing device (100) can implement re-rendering of topologies by identifying common nodes and difference nodes within a topology through comparison between multiple topologies and reconstructing at least one topology based on the identification results. Reconstruction of a topology may refer to a process of creating a new visual structure optimized for comparison purposes by visually adding nodes or edges that do not exist in the original topology of one of the models being compared, or by changing existing connection relationships. That is, reconstruction of a topology may mean redrawing a visual blueprint of the models based on the identified differences so that the user can clearly and intuitively perceive the differences between the two models. For example, reconstructing deleted nodes by visually restoring them, connecting edges according to the connection relationships of deleted nodes, and / or creating a new virtual model structure by synthesizing information from the two models may be included within the scope of reconstruction of a topology.

[0324] The computing device (100) displays the two topologies so that they can be visually compared by rendering the generated first topology in a first area of ​​the user interface and the second topology in a second area, and this comparison rendering may include various rendering methods so that the user can intuitively recognize the difference between the two models.

[0325] In one embodiment, the computing device (100) can perform rendering by automatically identifying structurally different difference nodes by comparing a plurality of topologies and applying predefined visual attributes to the difference nodes. For example, when the computing device (100) compares a modified model (second topology) with an original model (first topology), nodes that exist only in the original model and have been deleted can be highlighted in red, and nodes newly added to the modified model can be highlighted in green. Through this, a technical effect can be achieved in which the user can immediately identify which nodes have been changed by color alone, without having to compare the two models one by one.

[0326] In the present disclosure, the artificial intelligence model can be modified in two main ways. The first method is modification through model editing, and such modification may be used to encompass node deletion, node addition, edge deletion, edge addition, and attribute modification. The second is modification through model optimization, and such modification may be used to encompass graph optimization, quantization, and compression. Accordingly, user modification input may be intended to be user input in a form that encompasses such model editing and model optimization. For example, if the user modification input relates to model editing, nodes (or attribute values) selected or specified by the user may be deleted, added, and / or modified. For example, if the user modification input relates to model optimization, nodes (or attribute values) may be deleted, added, and / or modified by applying a model optimization tool on the server or user terminal to the artificial intelligence model (e.g., intermediate representation structure) in response to a model optimization request from the user.

[0327] In one embodiment, if there exists a derived model in which a specific node has been deleted due to user change input to the original model, the computing device (100) may reconstruct the second topology by adding the node deleted due to user change input and the associated edge on the second topology corresponding to the derived model. By applying a visual attribute for identifying deletion to the deleted node on the reconstructed second topology, a visual comparison rendering between the second topology reflecting the user change input and the first topology corresponding to the original model may be implemented. The computing device (100) may correspond the display position of the first node (deleted node) on the first topology with the display position of the first node (deleted node) on the reconstructed second topology, and display the visual attributes of the first node on the first topology and the visual attributes of the first node on the reconstructed second topology as being different from each other.

[0328] In one embodiment, if a derivative model exists in which a first node is deleted and a second node is added due to user change input to an original model, the computing device (100) can reconstruct the second topology by adding the first node deleted due to user change input and the first edge associated with the first node to the second topology. The computing device (100) can perform comparative rendering of a plurality of models by applying a first visual attribute to the first node to identify deletion during the rendering process, applying a second visual attribute to the second node added due to user change input to identify addition, and applying a third visual attribute to a common node that exists in common in the first artificial intelligence model and the second artificial intelligence model.

[0329] In one embodiment, depending on the implementation mode, an implementation may also be possible in which a deleted node is displayed in the topology of the original model and an added node is displayed in the topology of the derived model (or modified model). For example, a computing device (100) may perform a visually comparable rendering by applying a first visual attribute to identify deletion to a first node deleted due to user change input, displaying the first node with the first visual attribute applied on the first topology, and applying a second visual attribute to identify addition to a second node added due to user change input, displaying the second node with the second visual attribute applied on the second topology.

[0330] In one embodiment, a computing device (100) may compare multiple topologies using a virtual model structure. In the present disclosure, a virtual model structure may refer to an integrated temporary data structure generated during the process of comparing multiple artificial intelligence models. The virtual model structure may serve as a standard for rendering the final comparison visualization result by integrating the components (common nodes, added nodes, and deleted nodes) and modified connection relationships of multiple models into a consistent format. The virtual model structure may represent a data structure that is dynamically generated to include the comparison results of multiple artificial intelligence models. The computing device (100) may set a matching method to determine whether the nodes of multiple models are identical to each other during the comparison process. The matching method may include a method of comparison based on the unique identifier (ID) of each node, and depending on the implementation mode, it may also be possible to perform the comparison based on the name or operator type of the node. The computing device (100) can identify nodes added through traversal of the second artificial intelligence model, identify nodes deleted through traversal of the first artificial intelligence model, reconstruct connection relationships accordingly, and perform layout adjustment and visual comparison rendering based on this reconstruction. For example, to implement comparison rendering for the first artificial intelligence model and the second artificial intelligence model, the computing device (100) can perform a first comparison step of determining whether a corresponding node exists in the first artificial intelligence model for each of the nodes of the second artificial intelligence model by traversing the nodes of the second artificial intelligence model.Then, based on the result of the first comparison step, the computing device (100) may add a common node to the virtual model structure in which a corresponding node of the first artificial intelligence model exists among the nodes of the second artificial intelligence model, and may add a second differential node to the virtual model structure in which a corresponding node of the first artificial intelligence model does not exist among the nodes of the second artificial intelligence model, along with a second visual attribute. Then, the computing device (100) may perform a second comparison step of identifying a first differential node in which a corresponding node of the second artificial intelligence model does not exist among the nodes of the first artificial intelligence model by traversing the nodes of the first artificial intelligence model. Then, the computing device (100) may identify a neighbor common node connected to the output of the first differential node identified in the first artificial intelligence model, add a first differential node with a first visual attribute at a location adjacent to the neighbor common node within the virtual model structure to maintain the connection relationship of the first topology, and generate a differential edge connecting the first differential node and the neighbor common node in the virtual model structure. Then, the computing device (100) can render the first topology and the second topology so as to be visually comparable on the user interface based on the completed virtual model structure. Further examples of the corresponding embodiment will be described later in FIGS. 23 and 24.

[0331] In one embodiment, when a user change input is applied to a specific node of one model while multiple topologies corresponding to multiple models are rendered, the computing device (100) can change the visual attributes of the corresponding node on the topology of another model to which the change input has not been applied, reflecting the difference between the models. For example, when a user change input to delete the first node of the second artificial intelligence model is received while comparison rendering is implemented, the computing device (100) can perform re-rendering to apply visual attributes related to deletion to the first node in the first artificial intelligence model. The computing device (100) transmits change information related to the user change input to a server, and as the change information related to the user change input is transmitted to the server, a modified first artificial intelligence model corresponding to the re-rendered first topology can be generated at the server.

[0332] In one embodiment, the computing device (100) can automatically readjust the layout so that nodes common to both topologies are placed at the same screen location. For example, when the layouts of Model A and Model A' are aligned and rendered, most nodes can be displayed at corresponding or the same location on the user interface. If a node is added to Model A' compared to Model A, the new added node is displayed at that location instead of an empty space, thereby achieving the technical effect that the user can immediately detect structural discrepancies between the models.

[0333] In one embodiment, the computing device (100) can enhance the convenience of comparison by rendering the views of two topologies so that they operate in conjunction with each other. For example, when user input is received from a user to scroll the first topology displayed in the first area, the computing device (100) can allow corresponding parts to be displayed on a single screen by scrolling the second topology in the second area together. As another example, when user selection input is received from a user to click a specific node in the first topology, the computing device (100) can automatically scroll the second topology to position the node on the second topology corresponding to the selected node in the first topology in the central part of the second area and highlight it. Through this, the technical effect of allowing the user to easily track and compare the state of a specific node before and after a change, even within a large model, can be achieved.

[0334] In one embodiment, the computing device (100) may provide a plurality of view modes for comparable rendering. The computing device (100) may determine a comparative rendering method for a plurality of topologies based on a view mode selected by a user among a plurality of predetermined view modes. For example, if the view mode is determined to be a first view mode (e.g., Parallel View), the computing device (100) may display a deleted first difference node in the first topology as a first visual attribute (e.g., red) and an added second difference node in the second topology as a second visual attribute (e.g., green). In this way, the computing device (100) may render two topologies in a parallel structure. For example, if the view mode is determined to be a second view mode (e.g., Derived View), the computing device (100) can render the first topology as is and render both the first and second difference nodes by integrating them into the second topology as their respective visual attributes. For example, if the view mode is determined to be a third view mode (e.g., Aligned View), the computing device (100) can perform rendering by readjusting the node positions within at least one topology so that the common nodes of the two topologies are placed at corresponding positions to each other, thereby aligning the visual layout. For example, in the third view mode, a rendering method can be used in which the difference nodes are placed in the second topology and the common nodes are aligned with each other.

[0335] In one embodiment, the computing device (100) may allow interaction between two rendered topologies. For example, when user input is detected from a user selecting a specific node on the first topology, the computing device (100) may automatically scroll the second topology to display a target node corresponding to the selected node in the central part of the screen occupied by the second topology.

[0336] In one embodiment, the computing device (100) can quantitatively display the difference between two artificial intelligence models through a user interface. For example, the computing device (100) can display the similarity between the first artificial intelligence model and the second artificial intelligence model, the number of added nodes, and / or the number of deleted nodes in text form based on the result of comparing the node identifier and edge identifier of the first artificial intelligence model with the node identifier and edge identifier of the second artificial intelligence model. For example, the computing device (100) can output data in text form indicating whether the architecture between the two models is identical (identical or non-identical), whether the graph structure is identical (identical or non-identical), a value quantitatively representing the similarity of the nodes of the topologies (e.g., 63.4% identical), the number of nodes of the topology(s), the number of added nodes, and / or the number of deleted nodes. For example, such data may be displayed on the user interface in the form of a pop-up window.

[0337] FIG. 11 illustrates an exemplary user interface (1100) for comparing a plurality of artificial intelligence models according to one embodiment of the present disclosure.

[0338] In order to compare a plurality of artificial intelligence models according to one embodiment of the present disclosure, an initial state of a user interface (1100) for receiving a second artificial intelligence model while the first artificial intelligence model is rendered is illustrated as an example. Although two models are exemplified as targets for comparison rendering in this specification, comparison rendering for three or more models may also be possible depending on the implementation mode, provided that it does not deviate from the scope of the present disclosure.

[0339] Referring to FIG. 11, the user interface (1100) may be divided into at least a first area and a second area so that two models can be visually compared. For example, a model comparison mode activation object (1110) for activating a model comparison mode may be displayed on the top of the first area. In response to a user selection input for the model comparison mode activation object (1110), the computing device (100) may divide the user interface (1100) into a first area and a second area. When a user selection input for the model comparison mode activation object (1110) is received, the currently rendered topology is output to the first area of ​​the user interface (1100), and an input area for allowing input for a new artificial intelligence model may be additionally displayed on the user interface (1100).

[0340] In the first area (left side of the drawing), a first topology (1120) in the form of a graph of the previously received first artificial intelligence model is rendered and displayed. The second area (right side of the drawing) may represent a space for receiving and rendering the second artificial intelligence model. In the initial creation state of the space, the second area may display guidance text (1130) instructing to upload the second model to be compared, and a model upload button object (1140) that allows the user to select a model file.

[0341] Accordingly, the following input actions may be received from the user to visualize a second artificial intelligence model to be compared with the first artificial intelligence model (1120). First, user input may be received from the user selecting a model upload button object (1140, Browse model) displayed in the second area. When a model file is selected or input from the user, the computing device (100) (e.g., user terminal) may receive the second artificial intelligence model and generate a second topology representing the abstracted structure of the model. The computing device (100) may render the generated second topology in the second area instead of the existing guide text (1130) and upload button object (1140).

[0342] Through the series of processes described above, the user interface (1100) can switch the two models to a visually comparable state by displaying the first topology of the first area and the second topology of the second area in parallel.

[0343] FIG. 12 illustrates an exemplary user interface (1200) that provides comparison results of a plurality of artificial intelligence models according to one embodiment of the present disclosure.

[0344] FIG. 12 is an exemplary illustration of a user interface (1200) rendered by comparing two topologies according to an embodiment of the present invention. The user interface (1200) in FIG. 12 exemplarily illustrates, for example, a method of visually distinguishing and displaying difference nodes within a topology and a method of controlling scrolling operations between two topologies.

[0345] Referring to FIG. 12, the user interface (1200) may include a first area (1200a) displaying a first topology corresponding to a first artificial intelligence model and a second area (1200b) displaying a second topology corresponding to a second artificial intelligence model. In the example of FIG. 12, a situation is given in which a node (1210a) in the first topology is retained, the remaining nodes (1215a, 1220a, 1225a, and 1230a) are deleted, and new nodes (1235) are added to the second topology.

[0346] A computing device (100) (e.g., a user terminal) can compare a first topology and a second topology to identify difference nodes (1235, 1215a, 1215b, 1220a, 1220b, 1225a, 1225b, 1230a and 1230b) representing structural differences or differences in attributes between the two models, and common nodes (1210a, 1210b) present in both models. Subsequently, the computing device (100) can render the differences so that the user can intuitively perceive them by applying predefined visual attributes according to the type of difference node. Additionally, the computing device (100) can assign distinct visual attributes to the difference nodes and the common nodes.

[0347] FIG. 12 illustrates an example of rendering in which visual attributes are assigned to differential nodes only in the second region (1200b) among the differential nodes and common nodes identified on the first topology and the second topology, and the nodes in the first region (1200a) are rendered while maintaining the visual attributes of the first topology. For example, the computing device (100) may assign different visual attributes to the differential node (1235) added relative to the first topology and the differential node (1215b, 1220b, 1225b, and 1230b) among the differential nodes (1235, 1215b, 1220b, 1225b, and 1230b) of the second region (1200b).

[0348] In another embodiment, a rendering method can also be implemented in which the difference nodes of the first area (1200a) and the second area (1200b) all have visual attributes different from the common nodes. For example, the common nodes (1210a, 1210b) on the first area (1200a) and the second area (1200b) (i.e., nodes that exist in common in the first model and the second model) may be displayed with a visual attribute of, for example, blue, the difference nodes in the first area (1200a) may be displayed with a visual attribute of green or brown depending on the characteristics of the nodes, deleted nodes among the difference nodes in the second area (1200b) may be displayed in red, and added nodes among the difference nodes in the second area (1200b) may be displayed in green.

[0349] One embodiment of the present disclosure may provide various scrolling actions to increase user convenience in navigating two topologies. The user interface (1200) may determine whether to perform scrolling in a first area (1200a), scrolling in a second area (1200b), or performing synchronized scrolling in the first area (1200a) and the second area (1200b) based on the starting position of the scroll input. The user interface (1200) may establish a correlation between the starting position of the scroll input and the scrolling action. If the starting position of the scroll is included within the first area (1200a), the user's scrolling action may affect the topology of the first area (1200a). If the starting position of the scroll is included within the second area (1200b), the user's scrolling action may affect the topology of the second area (1200b). When the start of a scroll input is detected in a third area between the first area (1200a) and the second area (1200b), a third scroll operation may be performed in which the first topology and the second topology are scrolled together according to the scroll input. The third area here may include any area between the first area (1200a) and the second area (1200b). The third area here may include the surrounding area of ​​the topology binding selection object (1200c). This allows for synchronized scrolling of both the first area (1200a) and the second area (1200b) to be implemented through a single scroll operation. This may be useful when comparing and navigating identical parts of two models.

[0350] Additionally, the user interface (1200) may include a topology binding selection object (1200c) that determines whether to synchronize the scrolling actions of two topologies. Based on whether the topology binding selection object (1200c) is enabled, it may be determined whether synchronized scrolling actions are performed or whether individual area scrolling actions are performed. The user can select a scrolling method through the topology binding selection object (1200c). When the topology binding selection object (1200c) is selected, the object may be enabled. When the topology binding selection object (1200c) is selected again while it is enabled, the object may be switched to a disabled state. When the topology binding is disabled, or when a scroll start input is received from the user within the first area (1200a), a first scrolling action in which only the first topology is scrolled may be performed. Likewise, when a scroll input is initiated within the second area (1200b), a second scroll action may be performed in which only the second topology is scrolled. When a selection input for a topology binding selection object (1200c) is received from the user, a binding scroll action may be activated. When the topology binding selection object (1200c) is activated, the user's scroll input may be recognized as a synchronized scroll action for both the first area (1200a) and the second area (1200b), regardless of its starting position.

[0351] In another embodiment, the user interface (1200) can perform a focusing action through automatic scrolling. The user interface (1200) can provide a function to automatically adjust the view of another topology based on a specific node. For example, when a user selection input is received from a user selecting a specific node (e.g., 1210a) on the first topology, the computing device (100) can output a fourth scrolling action to automatically scroll the second topology so that a target node (1210b) corresponding to said node in the second topology is displayed at a specific location in the second area (1200b) (e.g., the central part based on the vertical dimension of the second area screen). In the fourth scrolling action, if there is no common node in the first area and the second area, the fourth scrolling action may not be performed. This can be effective when wanting to quickly compare the before and after of a specific node change in two large models.

[0352] In an additional embodiment, the computing device (100) may also allow a selection input for scrolling to move to the next difference node. For example, the computing device (100) may allow an input for performing a scroll from the previous difference node to the next difference node. The user interface (1200) may provide a user input function that allows for sequential movement between identified difference nodes so as to efficiently and quickly review the differences between two models. For example, the user interface (1200) may include a difference node traversal selection object that performs functions such as moving to the next difference node or moving to the previous difference node. For example, the user interface (1200) may implement movement between the previous difference node and the next difference nodes according to a predefined mouse input or a predefined keyboard shortcut for moving between difference nodes. The computing device (100) may detect a list of pre-identified difference nodes and the location of the node displayed on the current view of the user interface (1200) during the comparison process between topologies. Based on the user's input of movement between differential nodes, the computing device (100) may determine the next or previous differential node closest to the current position as the target node. The computing device (100) may perform an automatic scrolling operation on the topology so that the target node is displayed at a predefined location (e.g., the central part of the screen where the topology is rendered). Additionally, the computing device (100) may enhance user focus on the scrolling results on the user interface (1200) by applying temporary visual attributes (e.g., highlighting) to the moved target node as the scrolling operation is completed. Additionally, if multiple topologies are set to a synchronized scrolling mode, when movement between differential nodes is performed in one topology, another topology may also be scrolled together to a corresponding position.

[0353] FIG. 13 illustrates an exemplary user interface (1300) that provides comparison results of a plurality of artificial intelligence models according to one embodiment of the present disclosure.

[0354] FIG. 13 is an example of a user interface (130) rendered by comparing two topologies according to one embodiment of the present disclosure, and in particular illustrates a rendering method in a Derived View mode, i.e., a second view mode.

[0355] Referring to FIG. 13, the user interface may include a first area (1300a) in which the first topology of the first artificial intelligence model is rendered, and a second area (1300b) in which the second topology of the second artificial intelligence model is rendered. This may be useful for focusing on changes in a derived model (e.g., the second model) in which a part of the original model (e.g., the first model) has been changed, based on the original model.

[0356] Specifically, in the example of FIG. 13, the MatMul node (1320a) and Add node (1330b) of the first topology were replaced with nodes of a complex structure leading to Reshape (1350) - Transpose (1360) - Conv (1370) - Transpose (1380) in the second topology. These new nodes of the replaced structure can be identified as differential nodes (e.g., second differential nodes). Additionally, since the MatMul node (1320a) and Add node (1330b) of the first topology were deleted from the second topology, they can be identified as differential nodes (e.g., first differential nodes).

[0357] In one embodiment, the first topology displayed in the first area (1300a) is rendered as is to have the structure of the original model, so that it can serve as a clear reference point for comparison between models. On the other hand, the second topology displayed in the second area (1300b) can be reconstructed so that the difference nodes identified based on the comparison results between models (e.g., additional nodes and deleted nodes) are integrated and visualized.

[0358] The node (1310a) of the first topology and the node (1310b) of the second topology are common nodes representing the LayerNormalization operator. The node (1340a) of the first topology and the node (1340b) of the second topology are also common nodes representing the Div operator. Since these common nodes exist in both models in an unchanged state when the derived model is created, they can be rendered with visual attributes (e.g., the same color) corresponding to the common nodes.

[0359] In the second topology, all nodes (1320b, 1330b, 1350, 1360, 1370, 1380, and 1390), excluding the common nodes (1310b and 1340b), correspond to difference nodes. Among these, the second difference nodes (added nodes) that do not exist in the first model but exist in the second model may correspond to reference numbers 1350, 1360, 1370, 1380, and 1390. These second difference nodes may be rendered with a second visual attribute (e.g., green) applied to indicate that they have been added. Among the difference nodes, the first difference nodes that exist in the first model but not in the second model may correspond to reference numbers 1320b and 1330b. These first difference nodes can be rendered with a first visual attribute (e.g., red, etc.) applied to indicate that they have been deleted.

[0360] In one embodiment of the present disclosure, the results of changes to first difference nodes (e.g., 1320a and 1330a) that exist only in the first model and are considered to have been deleted may be integrated and displayed on the second topology (1300b). If a specific node of the first model is deleted from the second model, the node may be reconfigured and displayed on the second topology with a first visual attribute (e.g., red) indicating deletion. One embodiment of the present disclosure achieves the technical effect that, by concentrating all change history in the second topology, the user can clearly identify how operators were added, deleted, or replaced relative to the original model.

[0361] Depending on the implementation mode, embodiments that reflect changes in both the first topology and the second topology, and embodiments that reflect changes only in the first topology, may also be included within the scope of the present disclosure.

[0362] FIG. 14 illustrates an exemplary user interface (1400) in which a sub-node for a node of an artificial intelligence model is displayed according to one embodiment of the present disclosure.

[0363] FIG. 14 is an exemplary diagram illustrating, in steps (1400a, 1400b), the process of displaying sub-nodes constituting a selected node in response to a selection input for a specific node on a user interface (1400) according to one embodiment of the present disclosure.

[0364] Referring to reference number 1400a in FIG. 14, a topology representing the abstracted structure of an artificial intelligence model is rendered on the user interface (1400). This topology includes multiple nodes (1410a and 1420a), some of which may include multiple sub-nodes. For example, a node (1420) may be a module type node. In the illustrated example, the resample_p6 (1420a) node corresponds to a module node that is abstracted and displayed at a higher level. In this state, the user can grasp the overall structure of the artificial intelligence model briefly and intuitively.

[0365] Reference numeral 1400b illustrates the extension of a node (1420b) and the subsequent re-rendering. A user selection input may be received from a user to select a region of a specific modular node (e.g., the resample_p6 node) (1420a) within the rendered topology. The user selection input may be of a different type from the user selection input related to editing the node, for example. For example, the user selection input related to editing the node may be exemplified by a right-click of the mouse, and the user selection input related to extending the node may be exemplified by a left-click of the mouse. In response to the user input, the computing device (100) (e.g., a user terminal) may re-render the topology in a manner that outputs the sub-nodes (1430 and 1440), as shown in reference numeral 1400b of FIG. 14. The selected node (1420a) is displayed as an expanded node (1420b), and a plurality of sub-nodes (1430 and 1440) constituting the selected node may be displayed together inside the expanded node (1420b). Inside the expanded node (1420b), a conv_2d sub-node (1430) that performs a two-dimensional convolution operation and a max_pool_2d sub-node (1440) that reduces the size of the feature map may be displayed.

[0366] In one embodiment, as illustrated in reference numeral 1400b, the computing device (100) may perform re-rendering while maintaining the existing connection structure. During re-rendering, the child nodes (1430 and 1440) may be displayed while maintaining the existing connection structure between the selected node (1420a) and the neighbor node (1410a). As an example, but not a limitation, depending on the implementation mode, the edge that previously ran from the efficientnet-lite0 node (1410a) to the resample_p6 node (1420a) may be reconfigured in the re-rendered topology to connect to the first child node (conv_2d, 1430) inside the node (1420b) extended from the neighbor node (1410b).

[0367] Through the user interface (1400) in Fig. 14, the user can first grasp the overall structure of a complex model, and then select a specific module of interest to gradually explore the detailed structure in a drill-down manner, thereby achieving a technical effect that greatly improves the efficiency and convenience of artificial intelligence model analysis.

[0368] FIG. 15 illustrates an exemplary user interface (1500) for visualizing an artificial intelligence model according to one embodiment of the present disclosure.

[0369] FIG. 15 illustrates a layout displayed on a single screen of a user interface (1500) according to one embodiment of the present invention. In one embodiment, the user interface (1500) may be configured so that a search function, a topology visualization function, and a detailed information display or editing function are organically linked.

[0370] Referring to FIG. 15, the user interface (1500) can be divided into at least a first area, a second area, and a third area according to function.

[0371] The first area is a topology visualization area, where the graph-format topology (1520) of the artificial intelligence model is rendered. This topology (1520) may consist of nodes representing the structure of the model and edges connecting them.

[0372] The second area may include a toolbar object (1501) comprising a search input object (1505), a display option object (1510), and an exposure option object (1515). The second area may further include a search result display object (1550) that displays search results when a search input (e.g., Conv) is received from the search input object (1505) in the toolbar object (1501). For example, the search result display object (1550) may have a layered window structure displayed over the first area on the user interface (1500). The search result display object (1550) may include an area for listing search results and display objects for filtering or highlighting specific types of data, such as nodes, edges, and weights within the search results. This second area may provide an interface for finding specific components within the topology. The search input object (1505) may allow the user to input the name or type of the node, edge, or weight they wish to find as text. An example of the user interface (1500) shows an example where the search term "Conv" is entered. A display option object (1510) can be used to determine how components within the topology are displayed. The display option object (1510) allows the user to filter the display targets by limiting them to nodes, edges, or weights. For example, the display option object (1510) can receive input to limit the search results and can limit the search results based on such input. For example, the display option object (1510) can allow the user to limit the search targets to nodes, edges, or specific types thereof.

[0373] The third area may represent a detailed information display and modification area. The third area may display at least one of the attributes and metadata embedded in a node within the topology and allow modification input for at least one of the attributes and metadata. The third area may provide an interface that displays detailed information embedded in a node selected by the user within the topology and allows modification input thereto. For example, the third area may include an attributes panel (1530). The attributes panel (1530) may display attribute information (e.g., name, layer type, and shape) representing the structural characteristics of the selected node. The user may change these attribute values ​​by selecting a modification icon (1560). The third area may include a metadata panel (1540). The metadata panel (1540) may display additional information of the selected node (e.g., performance information, quantization information, etc.). Metadata can display, for example, quantization-related information such as x_scale or x_zero_point. The user can change the metadata value through the edit icon (1570).

[0374] A user interface (1500) according to one embodiment of the present disclosure can provide a more enhanced user experience to the user through integrated operation. Each area of ​​the user interface (1500) can operate in conjunction with each other. For example, a computing device (100) (e.g., a user terminal) can receive a search term input such as "Conv" from the user on a search input object (1505) in a second area. The user interface (1500) can list nodes corresponding to the entered search term in the search result (1550) area, and can display the corresponding nodes (1510 and 1520) on the topology (1520) of the first area so as to be visually distinguished from other nodes by applying a predefined visual attribute such as highlighting. When receiving user selection input from a user, such as selecting a specific item from a list of search results (1550) or directly selecting a node highlighted on the topology (1520), the computing device (100) may display detailed information about the selected node in a third area (1560 and / or 1570). Specifically, when receiving user selection input, such as selecting a specific item from a list of search results (1550) or directly selecting a node highlighted on the topology (1520), the attribute value and metadata value of the corresponding node may be loaded and displayed in the attribute panel (1530) and the metadata panel (1540), respectively.

[0375] In an additional embodiment, the user interface (1500) may produce different outputs depending on the type of user input to the components within the topology (1520) within the first area. For example, if the user selection input is a first selection input such as a mouse right-click, a topology change option object may be displayed through the first area on the user interface (1500), and if the user selection input is a second selection input (e.g., a mouse left-click), at least one of the attributes and metadata of the node corresponding to the second selection input may be displayed through the third area.

[0376] Through an integrated operation method between regions within this user interface (1500), the user can quickly search for a desired node while viewing the overall structure of the model, and immediately check and edit detailed information of the selected node, thereby achieving a technical effect that greatly improves the efficiency of model analysis and modification work.

[0377] FIG. 16 illustrates an exemplary user interface (1600) for determining a display method related to a node of an artificial intelligence model according to one embodiment of the present disclosure.

[0378] Referring to FIG. 16, the user interface (1600) may include a graph visualization area (1610) in which the topology of the artificial intelligence model is rendered, and a toolbar object (1620) containing various control functions. The toolbar object (1620) may correspond, for example, to the toolbar object (701) of FIG. 7 and / or the toolbar object (1501) of FIG. 15. The user interface (1600) may further include a display method selection window (1630) (Node Color Styler) for dynamically changing the display method of nodes within the topology.

[0379] When a user input is received from a user to select a display method selection activation object (1680) on the user interface (1600), the computing device (100) (e.g., user terminal) may generate and output a display method selection window (1630) as illustrated on the right side of the drawing. For example, the size of the graph visualization area (1610) may be reduced, and the display method selection window (1630) may occupy an area of ​​the user interface (1600) that is reduced. For example, an embodiment in which the size of the graph visualization area (1610) is fixed and content is generated on the display method selection window (1630) is also possible. The user may select and apply various methods for determining the display colors of nodes within the topology through the display method selection window (1630). The display method selection window (1630) may include display method selection objects (1640 and 1650) for a user to select a node display method, a category legend (1660) according to the selected method, and a color palette (1670) for specifying a color.

[0380] A display method selection window (1630) according to one embodiment of the present disclosure may guide a user's selection by displaying major categories such as attributes (first node display method) and metadata (second node display method). A display method selection window (1630) according to one embodiment of the present disclosure may include a first node display method that provides distinction based on the structural characteristics of a node. A display method selection object (1640) for selecting the first node display method may be displayed as, for example, an attribute. The first node display method may refer to a method of distinguishing and displaying nodes based on the structural characteristics of a node within a topology. When a user selection input corresponding to an attribute is received on the display method selection window (1630), the user interface (1600) may output a plurality of options related to the attribute in a manner that lists them. On this list, a user input selecting a structural characteristic such as an 'Operator' may be received from the user. For example, a display method selection object (1640) is displayed as an attribute, and in response to a user selection input for the display method selection object (1640) representing the attribute, the display method selection object (1640) may display sub-objects such as operator type, Precision, Dilations, Kernel shape, Pads, and Strides as drop-downs. The example in FIG. 16 illustrates the result where a user selects an 'attribute' from the display method selection object (1640) (i.e., an attribute is selected from the attribute and metadata in the main category), and subsequently selects an 'operator type' in the sub-category related to the attribute. For reference, in the example drawing in FIG. 17, when a display method selection object (1740) corresponding to the 'attribute' is selected and an operator (1640) is selected, a user interface (1600) like FIG. 16 may be output.

[0381] In response to user input leading to attributes and operator types for the display method selection object (1640), the computing device (100) can list all types of operators present in the current topology (e.g., Conv, Mul, Slice, and Batch Normalization) in the category legend (1660) area. The category legend (1660) can display the result of assigning a predefined color to each of the operator types. User color selection input can be received from the user to select each operator type on the category legend (1660) and assign or change the desired color using the color palette (1670). Once the user's settings are complete, the topology of the graph visualization area (1610) can be updated by combining the selection input on the color palette (1670) and the category legend (1660) so that all nodes with the same operator type are rendered in the same color.

[0382] The user interface (1600) of the present disclosure can provide the ability to group nodes within a topology according to structural characteristics, such as operator types, and assign different colors to each through a first node display method. This technical feature can have the effect of enabling a quick grasp of the distribution and patterns of key operators in the topology of a complex and large artificial intelligence model. For example, by setting the Convolution (Conv) layer to purple and the Activation function to yellow, the user can immediately recognize how the Convolution operation and the Activation function are repeated in a specific pattern within the model. This can provide an enhanced user experience for quickly understanding and analyzing the overall architectural design philosophy of the model.

[0383] FIG. 17 illustrates an exemplary user interface (1700) for determining a display method related to a node of an artificial intelligence model according to one embodiment of the present disclosure.

[0384] The user interface (1700) of FIG. 17 exemplarily shows a display method selection window (1730), namely a 'Node Color Styler', which determines the display method of a node based on the presence or absence of metadata values ​​or categories. The display method selection window (1730) displays major categories such as attributes and metadata, allowing the user to make specific selections leading from major categories to medium categories (small categories).

[0385] A display method selection window (1730) according to one embodiment of the present disclosure may include a second node display method that provides distinction based on the performance of the nodes. The second node display method may refer to a method of distinguishing and displaying nodes based on the performance of the nodes within a topology. The second node display method may be determined by selecting a display method selection object (1650) representing metadata in FIG. 16, and when a display method selection object (1650) corresponding to the metadata in FIG. 16 is selected, the computing device (100) in FIG. 17 may display Latency, SNR Score, Target precision, Memory Usage, and Output tensor size in a dropdown form. The user interface (1700) of FIG. 17 exemplarily illustrates a case where the SNR Score (1750) is subsequently selected from a dropdown list after the display method selection object (1650) is selected as metadata. For example, a dropdown method may refer to a graphical user interface (GUI) method in which a hidden list of submenus unfolds in a specific direction (e.g., downward) in response to the interaction of a button click, allowing the user to select desired options. An exemplary UI operation may be performed in such a way that when an attribute object and a metadata object in FIGS. 16 and 17 are selected, their list of submenus is displayed.

[0386] In response to a user selection input from the user selecting 'Metadata' from the display method selection object (1650), the computing device (100) may enable an additional metadata selection object for selecting which performance value to use as a standard. The user interface (1700) shows an example of the case where specific metadata (e.g., SNR score) is selected to be the standard for this node performance value or performance change value.

[0387] Through the first mode selection object (1740 and 1650), a major category of display criteria, such as 'Attributes' or 'Metadata', can be selected. Subsequently, after the selection of the first mode selection object (1740 and 1650) is completed, a specific item for determining the color, such as a node performance value, 'SNR Score', can be selected through the second mode selection object (1750 and 1640) corresponding to the medium or small category. The first mode selection object (1740 and 1650) and the corresponding second mode selection object (1750 and 1640) may represent a hierarchical user input structure used to determine criteria for displaying the topology. When the SNR score is selected, the computing device (100) can determine the display color for the node based on the corresponding node performance value or performance change value inherent in each node. In one embodiment, the process of determining the display color may be performed using a color gradient (1770). That is, the display color may be determined such that the node performance value or performance change value and the saturation or luminance on the color gradient (1770) have a positive correlation. For example, nodes with high SNR scores may be rendered in dark green, and nodes with low SNR scores may be rendered in light green. This allows the user to intuitively understand the performance level of each node in the model solely by the color of the node.

[0388] When user input selecting 'SNR Score' is received from the user, the computing device (100) can analyze all nodes within the topology rendered in the graph visualization area (1710) and automatically classify the state of the value for the item. In the example of FIG. 17, the values ​​were classified into two categories: nodes where the 'SNR Score' value exists ('Not NULL') and nodes where it does not exist ('NULL') (1760). These classified categories can be displayed in the category legend area (1760).

[0389] User input can be obtained from the user by selecting each category (e.g., 'Not NULL') from the category legend (1760) and specifying a desired display color (e.g., blue) using the color palette (1770). Once the color specification for the category is complete, the user can press the 'Done' button to finally apply the changes.

[0390] When applied in the final stage, the topology of the graph visualization area (1710) can be re-rendered according to the determined display method. For example, all nodes with an SNR Score can be displayed in blue, and nodes without such a value can be displayed in red. The intensity of the blue color can be varied according to the quantitative value of the SNR Score inherent in the node (e.g., it can be varied to have a positive correlation).

[0391] The present disclosure may provide a function to distinguish and display nodes based not only on static structural features such as the operator type of a node, but also on the presence or absence of specific metadata values ​​(NULL or Not NULL) or categories of specific attribute values. This may lead to a technical effect that allows the user to visually and quickly verify the integrity and consistency of data within the model. For example, a computing device (100) may immediately identify nodes where the SNR Score is non-existent (NULL) after the quantization process, thereby quickly finding parts where quantization has failed or is missing, which can be utilized as a powerful anomaly detection means. Furthermore, the present disclosure may provide a flexible and multifaceted analysis environment based on user definition. The display method selection window of the present disclosure is not limited to a few predefined criteria, but may dynamically analyze all attributes and metadata of the model to use as criteria for color display. Consequently, the user interface of the present disclosure may have the technical effect of providing a flexible and scalable visualization analysis platform capable of multifaceted data analysis according to the user's needs, going beyond a standardized viewer.

[0392] FIG. 18 is a drawing for understanding color changes related to nodes of an artificial intelligence model according to one embodiment of the present disclosure.

[0393] As an example rather than a limitation, when a region of the Node Color Styler in FIG. 16 and FIG. 17 is selected, the user interface (1800) of FIG. 18 may be displayed. For example, when an SNR score is selected on the Node Color Styler in FIG. 17, the exemplary user interface (1800) of FIG. 18 may be displayed.

[0394] A computing device (100) (e.g., a user terminal) can automatically and dynamically determine the colors mapped to the nodes of the topology based on the color type of metadata selected in FIG. 17 and the performance value (or performance change value) inherent in the node. The higher the SNR score, the darker the color is displayed, thereby allowing the performance of the nodes within the topology to be visually verified.

[0395] A computing device (100) can determine the display color of a node based on a performance value inherent in the node. Referring to FIG. 18, a gradient-based node coloring method (1800) may include a color gradient (1810) and a plurality of node objects (1820, 1830, and 1840) to which the gradient is applied.

[0396] A color gradient (1810) may refer to a visual display object that serves as a reference for mapping numerical node performance values ​​into a continuous color change. For example, a low performance value or a high performance change value may be represented in a light color, and a high color. Node objects (1820, 1830, and 1840) are objects rendered on the topology, and each node object (1820, 1830, and 1840) may include a node performance value (in this example, an SNR score) representing its performance as metadata.

[0397] A rendering step according to one embodiment of the present disclosure can determine a display color corresponding to each node based on performance values ​​inherent in the nodes and automatically and dynamically render the node with the determined color. The display color may be determined such that the node performance value and the saturation or luminance on the color gradient (1810) have a positive correlation. Node (1820) has the lowest SNR score (SNR 12), so it can be rendered as the lightest saturated color corresponding to the beginning of the color gradient (1810). Node (1840) has the highest SNR score (SNR 45), so it can be rendered as the darkest saturated color corresponding to the end of the color gradient (1810). Node (1830) has the median value (SNR 30), so it can be rendered as the color corresponding to the middle of the gradient.

[0398] The computing device (100) can effectively show performance changes resulting from optimization when the user change input is model optimization. In an embodiment where the topologies of multiple models are comparatively rendered, when node coloring according to reference numeral 1800 is applied, a technical effect can be achieved in which the performance of the derived model is intuitively determined on a node-by-node basis based on the color of the node, whether the derived model is superior in performance to the original model.

[0399] This rendering method, exemplified in Fig. 18, can have the effect of converting complex numerical data into intuitive visual information that users can grasp at a glance. Users can immediately recognize which nodes have high or low optimization quality just by looking at the color of each node, allowing for rapid analysis of performance bottlenecks or problems in the artificial intelligence model.

[0400] FIG. 19 illustrates an exemplary user interface (1900) for visualizing an artificial intelligence model using hierarchical identification information for nodes of the artificial intelligence model according to one embodiment of the present disclosure.

[0401] The user interface (1900) represents a rendering method that clearly displays the hierarchical relationships of nodes by integrating the visual attributes of nodes belonging to the same layer within the rendered topology.

[0402] Referring to FIG. 19, the rendered topology (1900) may include multiple nodes and edges. In one embodiment, four slice node groups (1930) branched from an 'images' node are shown. For example, a slice operator may perform the function of extracting a specific part of an input tensor. Each slice node of the slice node group (1930) may be rendered to include a mark indicating the function or identification information of the node (e.g., Slice) and a mark indicating the detailed configuration of the node (data, starts, ends, axes, and steps). These slice nodes may be connected to the images node via an edge (1910) and to the Concat node via an edge (1920).

[0403] A computing device (100) (e.g., a user terminal) can determine a rendering method based on layer identification information inherent in the nodes within the topology. The layer identification information may include the name of the module to which the node belongs, the depth level on the graph, or user-defined group information. The layer identification information may be information inherent in or mapped to the node. The computing device (100) can analyze the layer identification information of each node and determine that all four slice nodes belong to corresponding layers. Based on this, the computing device (100) can render the topology using the layer identification information to visually and hierarchically integrate the nodes or edges.

[0404] As illustrated in FIG. 19, the computing device (100) can render by applying a common visual attribute (e.g., the same red color) to the input edges (1910) connected to each slice node belonging to the same layer. The computing device (100) can also apply another common visual attribute (e.g., the same gray and thickness) to the output edges (1920) connected to the output of each slice node. Accordingly, the input edges (1910) and output edges (1920) can be rendered so that they are visually clearly distinguishable within a group of nodes (1930) belonging to the same layer. According to this UI method, the layer information of the nodes is reflected so that the edges associated with the nodes can be visually integrated. For example, the computing device (100) can display not only the color of the edges but also the arrow position and shape of the edges so that they correspond to each other by reflecting the layer information.

[0405] Additionally, the computing device (100) can implement visual integration of nodes during the topology rendering process. For example, the computing device (100) can render four slice nodes belonging to the same layer by applying a common visual attribute (e.g., the same green color) to each of them so as to be visually integrated. For example, the computing device (100) can unify the rendering positions of each of the four slice nodes belonging to the same layer. For example, the computing device (100) can unify the rendering positions of the slice nodes belonging to the same layer on the user interface (1900) so that they have corresponding coordinate values ​​on the Y-axis. The layout of the nodes of the same layer can be adjusted so that they are placed on the same vertical or horizontal line on the screen to aid visual perception. In an additional embodiment, the computing device (100) may also implement topology rendering by assigning visual attributes corresponding to the slice nodes to nodes (e.g., Concat nodes) that have a similar layer relationship or a relationship requiring inclusion into the same layer. In this way, topology rendering can be implemented so that groups of nodes belonging to the same layer have visual attributes distinct from nodes belonging to other layers and are arranged at corresponding positions.

[0406] A technique according to one embodiment of the present disclosure can visually enhance the readability and comprehensibility of the hierarchical structure of an artificial intelligence model. In complex artificial intelligence models, particularly those containing many parallel structures, the technical effect of enabling users to grasp the model's overall architecture and hierarchical structure at a glance can be achieved by visually grouping and displaying nodes and edges belonging to the same layer. This can assist in intuitively understanding the big picture of the model's design patterns and data flow without having to analyze the functions of individual nodes separately.

[0407] Additionally, a user interface (1900) according to one embodiment of the present disclosure may allow for intuitive tracking of data flow. A computing device (100) can clearly visually distinguish which data flows into which layer group by applying a common color or style to edges entering or leaving a specific layer. This allows the user to easily track and analyze the path of data passing through a specific layer among complexly intertwined edges, which can lead to a technical effect that significantly improves the efficiency of model debugging and analysis.

[0408] FIG. 20 illustrates an exemplary user interface (2000) for converting an operator using platform information on which an artificial intelligence model is to be executed, according to one embodiment of the present disclosure.

[0409] The user interface (2000) may provide an operator converter function that optimizes an artificial intelligence model for a specific hardware platform. Referring to FIG. 20, the user interface (2000) may consist of a platform input area (2010), an operator list area (2020), a topology visualization area (2040), and a node detail area (2050). Operators and nodes may be used interchangeably.

[0410] The computing device (100) can receive platform information from the user through the platform input area (2010) on which the artificial intelligence model will be executed. In the example in FIG. 20, 'RZ / V2M' was selected as the Device and 'DRP-AI v0.1' as the Runtime.

[0411] In one embodiment, platform information may include information for identifying hardware, information for identifying a runtime, and / or information for identifying a compiler. Platform information may define an environment in which an artificial intelligence model and / or an optimized (modified) artificial intelligence model will be executed. Platform information may include at least one of hardware in which the artificial intelligence model will be executed, a compiler for connecting the artificial intelligence model to the hardware, and a runtime representing the environment of the hardware in which the artificial intelligence model will be executed. For example, information regarding hardware included in platform information (305) may include information regarding, for instance, a CPU, a GPU, an NPU, and an FPGA. Information regarding hardware may refer to any form of information that identifies the hardware in which the artificial intelligence model can be executed. For example, information regarding hardware may include information related to the physical components of a system and their characteristics, structure, performance, compatibility, etc. For example, information regarding hardware may include at least one of hardware identification information, supported framework information, supported data type information, supported operator information, or supported granularity information. As another example, information regarding hardware may include hardware constraints. Hardware constraints may include at least one of support for each of the operator's data types, support for hardware operation acceleration of the operator, or speed improvement for each of the operator's data types. For example, a data type may be a type of value and its representation method that can be used when storing and processing data in hardware or a programming language. A data type may be a type of data distinguished by the method of data storage and operation. For example, a data type may include integer types, floating-point types, character types, etc.The hardware supports operations for specific data types, and each data type may have different memory sizes and representation ranges. For example, information regarding a compiler included in the platform information may refer to any form of information that identifies an entity connecting the artificial intelligence model and the hardware. For example, the platform may be identified and / or managed as a combination of hardware and a compiler. For example, the platform may be identified and / or managed as a combination of hardware and a compiler within the computing device (100), and information regarding the related platform may be extracted or generated by the computing device (100) upon receiving the platform information.

[0412] When the computing device (100) receives platform information, it may generate a list of nodes (e.g., operators) that require optimization or conversion on the rendered topology and display this through the operator list area (2020). The computing device (100) may perform re-rendering of the topology by outputting the operator list and assigning distinguishable visual attributes to the nodes that require optimization or conversion on the rendered topology.

[0413] When the computing device (100) receives platform information, it may generate a list of nodes (e.g., operators) that require optimization or conversion during the topology rendering process and display this through the operator list area (2020). Distinguishable visual attributes may be assigned to the nodes that require optimization or conversion during the topology rendering process. Nodes within the topology may be displayed with visual attributes (e.g., colors) that are distinguished according to the analysis results.

[0414] In one embodiment, the operator list area (2020) may display a node selected by the user on the topology visualization area (2040). For example, when the user selects a specific node on the topology, the selected node may be added to the operator list area (2020), and the added node may also be displayed on the operator list area (20200) in a form that inherits the visual attributes of the node on the topology.

[0415] In an additional embodiment, the operator list area (2020) may be displayed to automatically include unsupported nodes (2080) and inefficient nodes (2090) displayed on the topology visualization area (2040).

[0416] The computing device (100) can implement visual linkage between the topology visualization area (2040) and the operator list area (2020) by unifying the visual attributes of unsupported nodes (2080) among the nodes in the topology visualization area (2040) and the visual attributes of unsupported operators (Conv1, conv56, etc.) in the operator list area (2020) (e.g., red highlighting). The computing device (100) can implement visual linkage between the topology visualization area (2040) and the operator list area (2020) by unifying the visual attributes of inefficient nodes (2090) among the nodes in the topology visualization area (2040) and the visual attributes of inefficient operators (Conv2, conv100, etc.) in the operator list area (2020) (e.g., yellow).

[0417] An optimization application button (2030) may be displayed on the operator list area (2020) of the user interface (2000). For example, when the optimization application button (2030) is selected, the computing device (100) may output a list of candidate nodes representing alternative nodes (operators) for the selected node (operator). The list of candidate nodes may include alternative candidate nodes recommended to replace the unsupported node object or the inefficient node object. A detailed description of the list of candidate nodes will be provided later in FIG. 21. For reference, the term "node" in this specification may be used as a term to encompass a unit of operation, an operator, or a layer of the model.

[0418] As another example, the optimization application button (2030) may refer to a button for finally applying a model conversion based on user interaction on the user interface (2000). When the optimization application button (2030) is selected, the computing device (100) (e.g., user terminal) transmits information for operator conversion to the server, and the conversion of the artificial intelligence model can be performed by the server. The converted artificial intelligence model is received from the server to the computing device (100), and the computing device (100) can re-render the converted artificial intelligence model on the user interface (2000).

[0419] As such, the user interface (2000) according to one embodiment of the present disclosure can operate in conjunction with optimization tools for an artificial intelligence model in the backend (e.g., converter, compressor, quantizer and / or graph optimizer), and accordingly, the technical effect of more easily achieving optimization, compatibility, and efficiency of the artificial intelligence model through visual interaction on the user interface (2000) can be secured.

[0420] The topology visualization area (2040) is an area where the graph-format topology of the artificial intelligence model is rendered. Each node within the topology visualization area (2040) can be distinguished and displayed with different visual attributes depending on compatibility and efficiency on the target platform (e.g., the platform input in 2010). Unsupported node objects (2080) that are not supported on the target hardware can be rendered with a first visual attribute (e.g., red), and inefficient node objects (2090) that are supported on the hardware but experience performance degradation during execution can be rendered with a second visual attribute (e.g., yellow). Normal nodes without issues can be rendered with a third visual attribute (e.g., blue).

[0421] The node detailed information area (2050) may refer to an area that displays detailed information of a node when a user selects a specific node in the topology. This area may provide information by dividing it into node Properties and Attributes. Additionally, the node detailed information area (2050) may also include information regarding the input and output of the node.

[0422] Node Properties refer to high-level information indicating a node's identity and membership. For example, this includes the node's operator type (type: Conv), module (module: ai.onnx.v11), and distinguished name (name: Conv_0). Node Properties can be represented as fixed-form information related to the node's attributes. Attributes refer to parameters that define the specific behavior of node operations. For example, this includes the kernel size (kernel_shape), stride (strides), and padding (pads) values ​​of the Conv operator. Attributes can be represented as variable-form information related to the node's attributes.

[0423] In one embodiment, the computing device (100) may implement a linkage between the node detail information area (2050) and the topology visualization area (2040). Through this linkage, the topology visualization area (2040), the node detail information area (2050), and the operator list area (2020) may be visually linked and unified. For example, when the computing device (100) receives user input from a user selecting an unsupported node (2080) on the topology visualization area (2040), it may visually display the cause of the determination of the unsupported node (2080). For example, the computing device (100) determines the first cause element (2060) (e.g., Type (conv)) and the second cause element (2070) (e.g., Dilation (1, 1)) on the node detail information area (2050) as the cause of the determination of this unsupported node, and these elements (2060 and Visual attributes (e.g., red highlighting) corresponding to the visual attributes of unsupported nodes can be assigned to the user interface (2070). The cause of inefficient nodes can also be visualized in the same way. Through this, the user interface (2000) can intuitively present which nodes are not supported by the hardware in the artificial intelligence model, which nodes cause inefficiency in the hardware, and which information among the details of the nodes causes such unsupport and / or inefficiency.

[0424] In an additional embodiment, the computing device (100) may display the unsupported cause factor for the unsupported node (2080) and the inefficient cause factor for the inefficient node (2090) as distinct visual attributes on the node detail information area (2050).

[0425] In one embodiment, when a user provides user input information identifying the platform on which the artificial intelligence model will be executed in the platform input area (2010), the computing device (100) automatically analyzes the node objects included in the topology (2040). This analysis may be implemented through communication with a server or may be performed through an algorithm within the computing device (100). Based on the analysis results, the computing device (100) may classify each node as an unsupported node object (2080), an inefficient node object (2090), or a general node. The computing device (100) may re-render the topology by assigning different visual attributes to each of the classified nodes, and may also re-render the operators in the operator list area (2020) using visual attributes linked to the re-rendering method of the topology. Additionally, when receiving user input to select an unsupported node (2080) or an inefficient node (2090) through the user interface (2000), the computing device (100) may display the problem regarding the selected node through the node detail information area (2050).

[0426] A user interface (2000) according to one embodiment of the present disclosure can provide the technical effect of being able to intuitively identify which operator causes problems on specific hardware by clearly distinguishing and displaying unsupported / inefficient nodes on the model topology by color. This can provide the effect of quickly finding points requiring optimization without a complex profiling process for the artificial intelligence model.

[0427] FIG. 21 illustrates an exemplary user interface (2100) for converting an operator using platform information on which an artificial intelligence model is to be executed, according to one embodiment of the present disclosure.

[0428] In one embodiment, the user interface (2100) may be a user interface that is output in response to a user input requesting conversion on the user interface (2000) of FIG. 20. For example, the user interface (2100) may be output according to a selection input for nodes that want to be converted on the user interface (2000) and a selection input for an optimization application button (2030). The user interface (2100) may be used to replace unsupported or inefficient nodes in an efficient manner to implement the Operator Converter function.

[0429] Referring to FIG. 21, the user interface (2100) includes a platform input area (2010), a list of operators to be optimized (2020), and an optimization application button (2030). In one embodiment, when a user selects nodes to be transformed on the topology and selects the optimization application button (2030), the computing device (100) may automatically output a list of recommended candidate nodes (2140) (Replaceable Operator Map) to replace the selected nodes. In this specification, the list of candidate nodes and the replaceable operator map may be used interchangeably. The list of replaceable candidate nodes (2140) may display identification information for each candidate node, similarity to the original node, and / or performance information of the candidate node (e.g., latency) (2150). Information (2150) within the list of candidate nodes (2140) may be displayed in a table format. For example, score represents a recommendation score calculated by combining speed, accuracy, and similarity, Name represents the operator name of the candidate node, latency represents expected performance information (e.g., latency) when the candidate node is applied, and similarity represents a quantitative value indicating functional similarity with the original node.

[0430] In one embodiment, performance information of a candidate node may be displayed with different visual attributes depending on the comparison result between the performance of the node prior to replacement and the performance of the candidate node. For example, if the performance of the candidate node is lower than the performance of the node prior to replacement or if the increase in performance is less than a first threshold, the performance of the candidate node may be displayed in red. For example, if the performance of the candidate node is higher than the performance of the node prior to replacement but the increase in performance is less than a second threshold, the performance of the candidate node may be displayed in yellow. For example, if the performance of the candidate node is higher than the performance of the node prior to replacement by more than a second threshold, the performance of the candidate node may be displayed in green.

[0431] An embodiment may also be implemented in which, upon receiving user input selecting an unsupported node (2080) or an inefficient node (2090) through a user interface (2000), the computing device (100) displays a list of recommended alternative candidate nodes to replace the node.

[0432] Information regarding the list of candidate nodes (2140) can be obtained from the computing device (100) through communication between the computing device (100) (e.g., user terminal) and the server.

[0433] When the user receives input from the user to determine a specific replacement node within a list of candidate nodes (2140) (e.g., input selecting an Apply object (2160)), the user interface (2100) can re-render the topology in which unsupported nodes and / or inefficient nodes have been changed to selected replacement nodes. Along with this, the computing device (100) transmits the relevant change information to a server so that the process of converting the changed artificial intelligence model can be performed in the background. For example, the computing device (100) can receive the conversion result from the server after re-rendering the changed topology. If the conversion result is a conversion failure, the computing device (100) can roll back the re-rendered result. If the conversion result is a conversion success, the computing device (100) can use the re-rendered result as is or add node details (attributes, metadata, etc.) related to the conversion to the re-rendered result on the user interface (2100).

[0434] The computing device (100) can display the topology before conversion (2040) and the topology after conversion (2110) for comparison as a result of re-rendering. The computing device (100) can display the rendered topology (2040) before conversion and the re-rendered topology (2110) resulting from (or expected from) the conversion execution on a single screen. That is, the original topology (2040) before conversion and the re-rendered topology (2110) after conversion, in which nodes are replaced according to the user's selection, can be displayed. In the process of comparable rendering, the computing device (100) can implement an efficient comparison of the two rendering results by unifying the arrangement positions of the nodes (e.g., by unifying them based on the Y-axis).

[0435] The converted node (2120) is a node in the post-conversion topology (2110) in which the existing unsupported node (2080) has been converted into an efficient node that is supported on the target platform (2010), and can be rendered to have visual attributes that distinguish it from the pre-conversion node (2080) and the unconverted node (2130). The unconverted node (2130) represents another inefficient node to which the conversion has not yet been applied, and can be rendered to have the same visual attributes as the node (2090) of the pre-conversion topology (2040).

[0436] The user interface (2100) may include a plurality of control buttons (2160, 2170, 2180, and 2190). The control button (2160) is an object for receiving input to request conversion for a selected candidate node. The control button (1270) is an object for canceling a previous conversion. The control button (2180) is an object for receiving input to automatically convert the candidate node with the highest score. And the control button (1290) can perform the function of finally saving the model with the changes applied.

[0437] As described above, when the computing device (100) provides user input in which the user selects a specific unsupported node object (2080) or inefficient node object (2090) from the topology (2040) or operator list (2020) (or subsequently selects the Apply Optimization button (2030)), the computing device (100) can analyze hardware-supported operators that can replace the selected node and display a list of replaceable candidate nodes (2140). For example, information (2150) in the list can be sorted in descending order based on the score value. The user interface (2100) can provide the user with a manual selection mode and an automatic selection mode. Manual selection can be performed by directly selecting a desired candidate node from the list of candidate nodes (2140) and then selecting the control button (2160). Automatic selection can be achieved by selecting the control button (2180), for example, the top candidate node with the highest score can be automatically selected and conversion to that candidate node can be performed.

[0438] The user interface (2100) re-renders the topology (2110) in which the unsupported node (2080) has been changed to a selected alternative node object (2120). At the same time, the relevant change information is transmitted to the server, and the process of converting the changed artificial intelligence model at the backend is performed in parallel. In this example, the conversion of the unsupported node (2080) is used as an example, but it will be obvious to those skilled in the art that the inefficient node (2090) can also be converted and re-rendered in the same way.

[0439] A user interface (2100) according to one embodiment of the present disclosure can go beyond simply notifying of problems and provide specific solutions (a list of replaceable candidate nodes) including quantitative data such as score, latency, and similarity. This can lead to a technical effect that enables developers to select the most efficient optimization method based on data without relying on guesswork or iterative experiments.

[0440] A user interface (2100) according to one embodiment of the present disclosure can achieve optimization workflow automation and time reduction. By providing a function to automatically apply the optimal alternative through a control button (2180), the complex operator conversion process can be simplified and automated. The technique of the present disclosure can achieve a technical effect of significantly reducing the time and effort required to port a model to specific hardware, as the entire process from analysis of the operators of an artificial intelligence model to recommendation, application, and final model creation is integrated within a single user interface (2100).

[0441] In addition, a user interface (2100) according to one embodiment of the present disclosure can ensure stability for an artificial intelligence model that is generated and transformed while maintaining functional accuracy. The computing device (100) can assist a developer in selecting an optimal alternative by providing a similarity with the original node along with a list of candidate nodes, thereby considering a balance between performance (e.g., latency) and functional accuracy (e.g., similarity). This can lead to technical effects such as minimizing the risk of accuracy degradation that may occur after the transformation of the artificial intelligence model and ensuring the stability of the optimized model.

[0442] Additionally, the user interface (2100) can efficiently identify intuitive optimization bottlenecks. By clearly distinguishing unsupported nodes and inefficient nodes by color on the model topology, the user interface (2100) allows developers to intuitively identify which operators are causing problems on specific hardware. This provides the technical effect of quickly finding points requiring optimization without a complex profiling process.

[0443] Additionally, the user interface (2100) can improve inefficient workflows that overuse computing resources during the modeling process by providing a workflow that integrates analysis, modification, and verification. The user interface (2100) can enable the user to perform the entire optimization process, from hardware constraint analysis, problem node identification, alternative selection, and final model conversion, in an integrated manner within a single UI. This can lead to a technical effect that innovatively improves existing inefficient workflows that required switching between fragmented tools.

[0444] FIG. 22 illustrates an exemplary user interface (2200) that renders topologies comparablely according to a view mode according to one embodiment of the present disclosure.

[0445] A user terminal according to one embodiment of the present disclosure may provide a plurality of visualization options (e.g., a plurality of view modes) so that the user can more clearly and efficiently perceive structural differences between two artificial intelligence models. The user may select an optimized view mode based on the similarity or degree of change between the two models, and accordingly, the first topology and the second topology may be compared and rendered in different ways. In the comparison rendering process, contents that overlap with the descriptions above (e.g., features regarding the visualization of difference nodes, etc.) are replaced by the descriptions above.

[0446] The computing device (100) can provide multiple view modes in the process of performing comparison rendering for multiple models, induce the user to select a desired comparison rendering method, and vary the comparison rendering method according to the selected result.

[0447] In one embodiment, as illustrated in reference numeral 2210, a computing device (100) (e.g., a user terminal) can implement comparative rendering of multiple artificial intelligence models using a Parallel View mode (2210). The first view mode (2210) may be particularly useful when there is a large structural difference between two models. In the first view mode (2210), the first topology (2240) and the second topology (2250) can be rendered side by side in independent first and second regions of the user interface (2200), respectively. At this time, in the first topology (2240) (e.g., the original model), only the difference nodes deleted from the second model (2250) (e.g., the derived model) can be displayed as distinguishable visual attributes (e.g., red highlights). In addition, in the second topology (2250) (e.g., a derived model), only the difference nodes added from the first model (2240) can be displayed with different visual attributes (e.g., a green highlight). This allows the user to clearly compare and analyze in parallel which elements have been removed and added from the perspective of each model. The first view mode (2210) can clearly separate and show which nodes have been changed from the perspective of each model when there is a large structural difference between the two models.

[0448] In one embodiment, as illustrated in reference numeral 2220, a computing device (100) (e.g., a user terminal) can implement comparative rendering of multiple artificial intelligence models using a Derived View mode (2220). The second view mode (2220) may be useful when the difference between two models is relatively small, such as when comparing a derived model that has been partially modified from an original model. In the second view mode (2220), the first topology (2260) can be rendered as the original structure without any indication of change. On the other hand, the second topology (2270) can be rendered so that both deleted difference nodes and added difference nodes are integrated and distinguished from common nodes. That is, since all changes between the two models are concentrated and expressed on the second topology (2270), the user can track and understand at a glance what changes have occurred in the modified model relative to the original model.

[0449] In one embodiment, as illustrated in reference numeral 2230, a computing device (100) (e.g., a user terminal) can implement comparative rendering of a plurality of artificial intelligence models using an Aligned View mode (2230). The third view mode (2230) may be most effective when two models are nearly similar but have subtle structural differences. When the third view mode (2230) is selected, the computing device (100) (e.g., a user terminal) performs a layout alignment algorithm that readjusts the on-screen coordinates of nodes included in at least one topology so that common nodes existing in both the first topology (2280) and the second topology (2290) are placed at the same or corresponding visual positions (e.g., Y-axis positions and / or X-axis positions). Additionally, in the third view mode (2230), difference nodes are displayed in the second topology (2290), and the original model can be rendered as is on the first topology (2280). When a user selects the third view mode (2230) and the rendering method is determined to be the third view mode (2230), the computing device (100) may perform a step of unifying the layout so that common nodes of the two topologies (2080 and 2090) are placed in corresponding positions. That is, the computing device (100) may align the visual layout of the two topologies (2080 and 2090) by readjusting the node position and / or edge position of at least one of the first topology (2280) and the second topology (2290). By matching the layouts of the two topologies (2280 and 2290) in this way, the user can minimize eye movement when comparing the two models side by side and immediately discover nodes that have been slightly added or deleted between common structures.

[0450] Additionally, the computing device (100) can automatically analyze the topologies of the artificial intelligence models being compared and automatically establish relationships between the artificial intelligence models. For example, the relationship between the models can be determined based on structural similarity between two models, the number of nodes, the locations of similar nodes, and / or the locations of similar edges. A mapping relationship between the relationship between the models and the corresponding view mode can be established. Accordingly, based on the determined relationship, the computing device (100) can automatically perform comparison rendering according to the view mode optimized (mapped) to the determined relationship among a plurality of view modes. Additionally, based on the determined relationship, the computing device (100) may create and display a UI object that recommends the view mode mapped to the determined relationship among a plurality of view modes.

[0451] The present disclosure provides various view modes, thereby offering a visual analysis environment optimized according to the user's analysis objectives and the degree of difference between models, going beyond simple parallel comparison. Since the user can select an appropriate view mode depending on whether the difference between two models is large, small, or subtle, technical effects can be achieved that reduce the user's cognitive burden during the comparative analysis process and maximize the efficiency of the comparative analysis.

[0452] FIGS. 23 and 24 present an exemplary methodology for performing comparative rendering according to one embodiment of the present disclosure.

[0453] FIGS. 23 and 24 illustrate a specific methodology for identifying difference nodes and reconstructing a topology by comparing two artificial intelligence models according to an embodiment of the present invention. The methodology may include a process of systematically analyzing the structural differences between the two models to generate a virtual model structure optimized for comparison purposes, and a process of comparing and rendering the final topology based thereon.

[0454] In the present disclosure, the virtual model structure (2330) may refer to an integrated temporary data structure generated during the process of comparing the first topology (2310) and the second topology (2320). The virtual model structure (2330) may not refer to a copy of the first or second topology itself, but rather to a result of integrating all components identified through each step of the comparison algorithm into a consistent structure that reflects the comparison results. The virtual model structure (2330) may include a common node existing in both topologies, a second differential node identified as added because it exists only in the second topology, a first differential node identified as deleted because it exists only in the first topology, edges connecting the nodes, and / or differential edges reconstructed to represent the connection relationship of the first differential node prior to deletion. The completed virtual model structure holds information regarding the state of each node (e.g., common, added, deleted) and can serve as a direct blueprint for rendering the final comparison visualization result to the user based on this information. Each node within the virtual model structure may include visual attributes assigned according to the node's state.

[0455] FIG. 23 illustrates a node traversal and matching process (2300) for comparison rendering of a computing device (100). For example, the matching process (2300) of FIG. 23 may include a first comparison step and a topology reconstruction therefrom. As illustrated in FIG. 23, the computing device (100) (e.g., a user terminal) may receive a first artificial intelligence model (2310, Model A) and a second artificial intelligence model (2320, Model B) as inputs and generate a result (2330, RESULT C) by comparing them. The computing device (100) may generate a virtual model structure (2330) containing the result of comparing a first topology (2310) generated from the first artificial intelligence model and a second topology (2320) generated from the second artificial intelligence model. Before starting the comparison, the computing device (100) may determine a matching method (e.g., based on ID, name, or operator type) to determine whether the nodes of the two models are identical based on user input or automatically configured information. Depending on the determined matching method, the following comparison steps may be performed. Depending on the determined matching method among a matching method based on the node identifier, a matching method based on the node name, and a matching method based on the node operator type, a matching process between the first topology (2310) and the second topology (2320) may be performed.

[0456] The computing device (100) can perform a first comparison step. Through the first comparison step, a node added on the second topology (2320) can be identified. The computing device (100) can identify a node that exists only in the second artificial intelligence model (2320) by performing the first comparison step. Referring to FIG. 23, in the first comparison step, the computing device (100) can traverse the nodes included in the second topology (2320) of the second artificial intelligence model. The computing device (100) can traverse the nodes in order from the input to the output of the second topology (2320). The computing device (100) can perform a matching process to determine whether a node corresponding to the first topology (2310) exists based on the nodes included in the second topology (2320).

[0457] In step 2340, the computing device (100) may perform a matching process to identify a node A, which is the starting node of the second topology (2320), and to determine whether the identified node A exists in the first topology (2310). Since the node A exists in the first topology (2310), it is determined to be matched, and the computing device (100) may include the node A in the virtual model structure (2330). According to this matching process, the node A may be identified as a common node.

[0458] In step 2350, since node C of the second topology (2320) exists in the first topology (2310), node C can be identified as a common node and added to the virtual model structure (2330).

[0459] In step 2360, since node D connecte...

Claims

1. A method for providing a user interface (UI) for visualizing an artificial intelligence model performed by a user terminal, wherein A step of receiving an artificial intelligence model to be the subject of visualization; A step of rendering a graph-type topology on the user interface that visualizes and represents the abstracted structure of the artificial intelligence model - the graph-type includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes -; When a user selection input is obtained for selecting a region on the topology of the user interface, a step of displaying a topology change option object correlated with the selected region on the user interface - the change option displayed in the topology change option object is determined differently based on the location of the selected region -; and A step of re-rendering the topology reflecting the change on the user interface based on a user change input requesting a change on the topology change option object; including, method.

2. In Paragraph 1, If the selected area corresponds to a node area occupied by a node on the topology, the topology change option object indicates the movement of a node object associated with the node area, the deletion of the node object, and the creation of an edge object associated with the node object. If the selected area corresponds to an edge area occupied by an edge on the topology, the topology change option object indicates the deletion of an edge object associated with the edge area, and If the selected area corresponds to a remaining area not occupied by both nodes and edges on the topology, the topology change option object indicates the creation of a new node object. method.

3. In Paragraph 1, When a user change input for changing a node on the topology change option object is obtained, the target node object to be changed, the target edge object connecting the target node object, and the neighbor node object connected to the target node object through the target edge are recognized together as targets for the topology change. method.

4. In Paragraph 1, The step of re-rendering the topology reflecting the above changes on the user interface is: When a user change input creating a new node on the topology change option object is obtained, a step of displaying a new node object at a location related to the selected area and displaying a new node option object requesting additional input related to the new node object; and In response to receiving additional input requesting the creation of target edge objects associated with the new node object on the new node option object, a step of re-rendering on the user interface a modified topology that includes target edge objects connecting the neighbor node objects of the new node object and the new node object; Includes, and In the above modified topology, the new node object and the neighbor node objects are highlighted to distinguish them from other node objects, method.

5. In Paragraph 1, The above user selection input includes a first selection input and a second selection input different from the first selection input, and If the above user selection input is the first selection input, the above topology change option object is displayed on the user interface, and If the above user selection input is a second selection input, at least one of the attributes and metadata of the node corresponding to the second selection input is displayed, method.

6. In Paragraph 1, The step of re-rendering the topology reflecting the above changes on the user interface is: When a user change input to delete a node object on the topology change option object is obtained, the step of deleting the node object associated with the selected area and replacing the first edge objects associated with the deleted node object with second edge objects connecting neighboring node objects associated with the deleted node object; and A step of re-rendering the topology reflecting the deleted node object and the second edge object on the user interface: including, method.

7. In Paragraph 1, The step of re-rendering the topology reflecting the above changes on the user interface is: When a user change input to delete an edge object on the topology change option object is obtained, the method comprises the steps of deleting the edge object associated with the selected area and moving at least one of the node objects and edge objects on the topology based on the location of the deleted edge object; and A step of re-rendering the topology reflecting the deleted edge object and the moved object on the user interface; including, method.

8. In Paragraph 1, The above user interface is, A first area displaying the topology in the above graph format; A second region comprising a search input object for receiving a search input within the topology, a display option object for determining the display method of components within the topology, and a display option object for determining the exposure or hiding of components within the topology; and A third area that displays at least one of the attributes and metadata embedded in a node within the above topology and allows modification input for at least one of the attributes and metadata; including, method.

9. In Paragraph 1, The above user interface is, When user input is received on a display method selection activation object for determining a display method of nodes within the topology, a display method selection window for determining at least one of a first node display method that distinguishes and displays nodes according to the structural characteristics of the nodes within the topology and a second node display method that distinguishes and displays nodes according to the performance of the nodes within the topology is displayed. method.

10. In Paragraph 1, The above rendering step is, Among the node objects included in the above topology, a step of indicating unsupported node objects that are not supported by the hardware or inefficient node objects whose performance degrades when executed on the hardware in a way that is distinguishable from other node objects on the topology; including, method.

11. In Paragraph 10, A step of receiving user input selecting the above unsupported node object or the above inefficient node object; A step of displaying a list of recommended alternative candidate nodes to replace the above-mentioned unsupported node object or the above-mentioned inefficient node object; and In response to receiving user input determining a replacement node within a list of the above-mentioned replaceable candidate nodes, a step of re-rendering a topology in which the above-mentioned unsupported node object or the above-mentioned inefficient node object is changed to a replacement node object; Includes more, As change information including the above-mentioned unsupported node object or the above-mentioned inefficient node object and the above-mentioned replacement node object is transmitted to the server, the artificial intelligence model is converted in such a way that the unsupported node or the inefficient node is changed to the above-mentioned replacement node, and, The above list of replaceable candidate nodes is, Identification information of replaceable candidate nodes; Similarity between the original node to be replaced and the replaceable candidate node; and Performance information of replaceable candidate nodes; Displaying, method.

12. In Paragraph 1, A step of re-rendering the topology in response to a user input selecting a region of a node within the rendered topology on the user interface, such that the selected node and a plurality of child nodes constituting the node are displayed together while maintaining the connection structure between the selected node and neighboring nodes; including, method.

13. In Paragraph 1, The step of re-rendering the topology reflecting the above changes on the user interface is: A step of determining whether to interact with a server for re-rendering the topology based on a change type determined according to the above user change input; including, method.

14. A computer program stored on a computer-readable storage medium, wherein the computer program, when executed by a user terminal, enables the user terminal to perform a method for providing a user interface (UI) for visualizing an artificial intelligence model, and the method comprises: A step of receiving an artificial intelligence model to be the subject of visualization; A step of rendering a graph-type topology on the user interface that visualizes and represents the abstracted structure of the artificial intelligence model - the graph-type includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes -; When a user selection input is obtained for selecting a region on the topology of the user interface, a step of displaying a topology change option object correlated with the selected region on the user interface - the change option displayed in the topology change option object is determined differently based on the location of the selected region -; and A step of re-rendering the topology reflecting the change on the user interface based on a user change input requesting a change on the topology change option object; including, A computer program stored on a computer-readable storage medium.

15. A user terminal that provides a user interface (UI) for visualizing an artificial intelligence model, It includes at least one processor, and The above processor is: A step of receiving an artificial intelligence model to be the subject of visualization; A step of rendering a graph-type topology on the user interface that visualizes and represents the abstracted structure of the artificial intelligence model - the graph-type includes nodes representing computational units of the artificial intelligence model and edges connecting the nodes -; When a user selection input is obtained for selecting a region on the topology of the user interface, a step of displaying a topology change option object correlated with the selected region on the user interface - the change option displayed in the topology change option object is determined differently based on the location of the selected region -; and A step of re-rendering the topology reflecting the change on the user interface based on a user change input requesting a change on the topology change option object; performing, User terminal.

Images