Method for acquiring data image reflecting intrinsic characteristics of data set, and computing device and system in which method is implemented

Abstract

According to one embodiment of the present disclosure, the provided computing device comprises: a memory including at least one machine learning module; and at least one processor electronically connected to the memory, wherein the at least one processor is configured to: acquire a first data set; identify a first embedding vector by mapping, using a first encoding module, the first data set to a first embedding space of a first dimension on the basis of the first data set; input the first embedding vector into a first adapter module and / or a second adapter module; and acquire, on the basis of the first embedding vector, a first data image corresponding to the first data set using the first encoding module and the first adapter module and / or the second adapter module.

Description

A method for obtaining a data image reflecting the inherent characteristics of a data set, and a computing device and system implementing such a method.

[0001] The present disclosure relates to a computing device for processing data using an artificial intelligence model. More specifically, the present disclosure relates to a computing device that provides data clinic solutions, such as training data, to enhance the capabilities of an artificial intelligence model.

[0002] Recently, deep learning-based artificial intelligence algorithms are being utilized across most technological fields. In particular, unstructured data without regularities are beginning to be utilized in deep learning, raising concerns about the quantitative nature of the data used for learning.

[0003] The industry is proposing various solutions to address the quantitative challenges of data. In particular, as synthetic data generation technology advances, synthetic data is being utilized to train deep learning models in various technical fields.

[0004] However, as virtual data is generated indiscriminately and artificial neural network-based deep learning models have become more advanced, the need to improve the quality of data rather than strengthening the quality of learning models is growing stronger.

[0005] For this reason, accurately assessing the quality of data used to train deep learning models is crucial. However, existing data quality assessment methods have clear limitations, such as limiting themselves to verifying the integrity of structured data. Therefore, a data solution that can be applied universally across data used in various technical fields is needed.

[0006] One task of the present disclosure is to provide a data processing model that maintains the inherent characteristics of a data set.

[0007] Another object of the present disclosure is to provide a learning data processing model for improving the learning efficiency of a machine learning model.

[0008] Another task of the present disclosure is to provide a data clinic solution using a language-based model.

[0009] Meanwhile, the problems to be solved in the present disclosure are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the invention included in the present disclosure belongs from this specification and the attached drawings.

[0010] According to one embodiment of the present disclosure, a computing device may be provided, comprising: a memory including at least one machine learning module; and at least one processor electronically connected to the memory; wherein the at least one processor is configured to obtain a first data set, identify a first embedding vector by mapping the first data set to a first embedding space of a first dimension using a first encoding module, and input the first embedding vector to at least one of a first adapter module or a second adapter module, wherein the first adapter module is configured to map data to a second embedding space of a second dimension, and the second adapter module is configured to map data to a third embedding space of a third dimension; and obtain a first data image corresponding to the first data set based on the first embedding vector using the first encoding module and at least one of the first adapter module or the second adapter module.

[0011] In addition, according to one embodiment of the present disclosure, a data processing method performed by at least one processor included in a computing device may be provided, the method including: obtaining a first data set by the at least one processor; identifying a first data point set by mapping the first data set to a first embedding space; obtaining a first characteristic of the first data set based on at least one distance value between a plurality of data points included in the first data point set; identifying at least one condensation space on the first data point set based on the first characteristic; obtaining a second data point set by removing at least some of the plurality of data points included in the at least one condensation space; and obtaining a processed first data set based on the second data point set.

[0012] Also, according to one embodiment of the present disclosure, a computing device may be provided, comprising: a memory; and at least one processor electronically connected to the memory; wherein the at least one processor is configured to generate a first virtual data set based on a latent code using a pre-learned generative model, identify a first manifold corresponding to the target data set by mapping the target data set to a first embedding space, identify a second manifold corresponding to the first virtual data set by mapping the first virtual data set to the first embedding space, and adjust a parameter of the generative model based on a determination that a similarity between the first manifold and the second manifold does not satisfy a predetermined condition.

[0013] In addition, according to one embodiment of the present disclosure, a data processing method performed by at least one processor may be provided, the method including: obtaining a first data set by the at least one processor; identifying a first data point set by mapping the first data set to a first embedding space using a pre-learned first model; obtaining first diagnostic data associated with at least one characteristic of the first data set based on the first data point set; obtaining prompt data based on the first diagnostic data; and generating a first virtual data set based on the prompt data using a pre-learned second model.

[0014] The solutions to the problems of the present invention are not limited to the solutions described above, and solutions not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0015] According to the present disclosure, a data image reflecting the inherent characteristics of a data set can be provided.

[0016] Additionally, according to the present disclosure, the learning efficiency of a machine learning model can be improved.

[0017] Additionally, according to the present disclosure, a data clinic solution linked to a language-based model can be provided.

[0018] The effects of the present invention are not limited to the effects described above, and effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0019] FIG. 1 is a drawing for explaining a device and system for performing a data clinic method according to various embodiments of the present disclosure.

[0020] FIG. 2 is a block diagram of a computing device that performs a data clinic method and a model learning method for a data clinic according to various embodiments of the present disclosure.

[0021] FIG. 3 is a diagram illustrating various operational methods performed by a computing device performing a data clinic method according to various embodiments of the present disclosure.

[0022] FIG. 4 is a diagram illustrating various systems for providing data clinic services according to various embodiments, and artificial intelligence models and algorithms for constructing the systems.

[0023] FIG. 5 is a diagram illustrating a data lens processing system and a data imaging system according to various embodiments.

[0024] FIG. 6 is a flowchart illustrating a method by which a computing device acquires a lens system based on a data set, according to various embodiments.

[0025] FIG. 7 is a flowchart illustrating an example of a lens processing algorithm performed by a computing device according to various embodiments.

[0026] FIG. 8 is a diagram illustrating an example of a lens processing model constructed for a computing device to perform a lens processing algorithm according to various embodiments.

[0027] FIG. 9 is a diagram illustrating an example of a neural network structure of a lens processing system included in a computing device according to various embodiments.

[0028] FIG. 10 is a diagram illustrating a method for a computing device to enhance a lens processing model by utilizing an auxiliary network according to various embodiments.

[0029] FIG. 11 is a diagram illustrating a method for a computing device to determine a characteristic associated with a dimension for optimizing parameters, according to various embodiments.

[0030] FIG. 12 is a flowchart illustrating a method for a computing device to obtain a data image reflecting inherent characteristics of a data set, according to various embodiments.

[0031] FIG. 13 is a diagram illustrating a method for a computing device to image a data set and determine a task performance capability, according to various embodiments.

[0032] Figure 14 is a diagram illustrating experimental data on the correlation between the amount of learning data and the learning efficiency of an artificial intelligence model.

[0033] FIG. 15 is a diagram illustrating a method for a computing device to remove at least a portion of a data set using a pre-trained artificial intelligence model, according to various embodiments.

[0034] FIG. 16 is a flowchart illustrating an example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0035] FIG. 17 is a diagram illustrating an example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0036] FIG. 18 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0037] FIG. 19 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0038] FIG. 20 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0039] FIG. 21 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0040] FIG. 22 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0041] FIG. 23 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0042] FIG. 24 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0043] FIG. 25 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0044] FIG. 26 is a diagram illustrating an example of a computing device using at least one data processing model to improve the quality of a data set, according to various embodiments.

[0045] FIG. 27 is a diagram illustrating an example of a computing device using at least one data processing model to improve the quality of a data set, according to various embodiments.

[0046] FIG. 28 is a diagram illustrating a pipeline for a computing device to input at least one data processing model based on characteristics of a data set, according to various embodiments.

[0047] FIG. 29 is a diagram illustrating a method for a computing device to generate virtual data based on data diagnostic data, according to various embodiments.

[0048] FIG. 30 is a flowchart illustrating a method for a computing device to generate virtual data using a pre-trained machine learning model, according to various embodiments.

[0049] FIG. 31 is a flowchart illustrating an example of a method for a computing device to verify the suitability of a virtual data set, according to various embodiments.

[0050] FIG. 32 is a flowchart illustrating another example of a method for a computing device to verify the suitability of a virtual data set, according to various embodiments.

[0051] FIG. 33 is a diagram illustrating a method for a computing device to generate virtual data by modifying a data image based on language input, according to various embodiments.

[0052] FIG. 34 is a flowchart illustrating a method for a computing device to generate virtual data by modifying a data image based on language input, according to various embodiments.

[0053] FIG. 35 is a diagram illustrating a method for a computing device to generate virtual data based on speech data, according to various embodiments.

[0054] FIG. 36 is a flowchart illustrating a method for a computing device to generate virtual data based on speech data and provide comparison information according to various embodiments.

[0055] FIG. 37 is a diagram illustrating a language-based generative model and a clinic model included in a computing device according to various embodiments.

[0056] FIG. 38 is a diagram illustrating an example of a computing device using a data clinic model and a language-based model according to various embodiments.

[0057]

[0058] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In describing the embodiments, descriptions of technical details that are well-known in the technical field to which the present disclosure pertains and are not directly related to the present disclosure will be omitted. This is to avoid obscuring the gist of the present disclosure by omitting unnecessary explanations and to convey the gist more clearly.

[0059] Since the embodiments described in this specification are intended to clearly explain the idea of ​​the present invention to a person having ordinary skill in the art to which the present invention pertains, the present invention is not limited to the embodiments described in this specification, and the scope of the present invention should be interpreted to include modified or altered examples that do not depart from the idea of ​​the present invention.

[0060] The terms used in this specification have been selected from widely used terms, taking into account the functions of the present invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, or the emergence of new technologies. However, if a specific term is defined and used with an arbitrary meaning, the meaning of that term will be described separately. Therefore, the terms used in this specification should be interpreted based on the actual meaning of the term and the overall content of this specification, rather than simply the name of the term.

[0061] The drawings attached to this specification are intended to facilitate explanation of the present invention, and the shapes depicted in the drawings may be exaggerated as necessary to help understanding of the present invention, and therefore the present invention is not limited by the drawings.

[0062] If a detailed description of the composition or function of a known disclosure related to the present invention in this specification is deemed to obscure the gist of the present invention, a detailed description thereof will be omitted as necessary. Furthermore, the numbers (e.g., "first," "second," etc.) used throughout the description of this specification are merely identifiers used to distinguish one component from another.

[0063] In addition, the suffixes "part" and "part" for components used in the following description are given or used interchangeably only for the convenience of writing the specification, and do not have distinct meanings or roles in themselves.

[0064] That is, the embodiments of the present disclosure are provided to make the present disclosure complete and to inform those skilled in the art of the scope of the present disclosure, and the invention of the present disclosure is defined solely by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

[0065] Terms such as “first” and / or “second” may be used to describe various components, but the components should not be limited by the terms. The terms are only for the purpose of distinguishing one component from another, for example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component, without departing from the scope of the present disclosure.

[0066] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. Conversely, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

[0067] Each block of the flowchart drawings and combinations of flowchart drawings in the drawings can be performed by computer program instructions. These computer program instructions can be installed in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create a means for performing the functions described in the flowchart block(s). These computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes an instruction means for performing the functions described in the flowchart block(s). Since the computer program instructions may be installed on a computer or other programmable data processing device, a series of operational steps may be performed on the computer or other programmable data processing device to create a computer-executable process, so that the instructions that cause the computer or other programmable data processing device to perform the steps for performing the functions described in the flowchart block(s) may also be able to provide steps for performing the functions described in the flowchart block(s).

[0068] Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative implementation examples, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on their respective functions.

[0069] The term 'unit' as used in this disclosure means a software or hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC). The 'unit' performs specific roles, but is not limited to software or hardware. The 'unit' may be configured to reside on an addressable storage medium and may be configured to play one or more processors. Accordingly, according to some embodiments, the 'unit' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'units' may be combined into a smaller number of components and 'units' or further separated into additional components and 'units'. Additionally, the components and '~parts' may be implemented to activate one or more CPUs within a device or secure multimedia card. Furthermore, according to various embodiments of the present disclosure, the '~part' may include one or more processors.

[0070] The operating principles of the present disclosure are described in detail below with reference to the attached drawings. In the following description of the present disclosure, detailed descriptions of related known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the present disclosure. Furthermore, the terms described below are defined based on the functions of the present disclosure and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0071] FIG. 1 is a drawing for explaining a device and system for performing a data clinic method according to various embodiments of the present disclosure.

[0072] The data clinic method of the present disclosure can be implemented on a platform system (100) based on a communication network. Specifically, a server device that collects and processes data, a learning device that trains learning models for various purposes, and multiple client devices can be connected to each other on a communication network to transmit and receive data.

[0073] For example, a server device may receive data from at least one of a plurality of client devices and transmit the received data to a learning device to train a specific learning model. Furthermore, the server device may process the received data to generate processed data and transmit the processed data to the plurality of client devices.

[0074] Additionally, for example, multiple client devices can access a server implemented by the server device via a communication network, and exchange data with other client devices or use functions implemented by the server via the server.

[0075] Additionally, the server device, multiple client devices, and the learning device may be implemented as a single computing device. Specifically, depending on the embodiment, a computing device may be provided that performs operations such as training a deep learning model, collating and processing data, or transmitting and receiving data.

[0076] Additionally, the server device, the learning device, and the plurality of client devices may include at least one computing device, which may include at least one processor (or controller).

[0077] Below, a computing device for providing a data clinic method is described in more detail.

[0078]

[0079] FIG. 2 is a block diagram of a computing device that performs a data clinic method and a model learning method for a data clinic according to various embodiments of the present disclosure.

[0080] Referring to FIG. 2, the computing device (1000) may include various components for providing a data clinic method. Specifically, the computing device (1000) may include a memory (1010) that stores data and various instructions to be transmitted to the processor, a processor (1020) that performs operations based on instructions transmitted from the memory (1010), and a communication unit (1030) that allows data to be communicated internally within the computing device (1000) or enables communication between the computing device (1000) and an external device.

[0081] Additionally, optionally or alternatively, the computing device (1000) may further include an input device (not shown). In this case, the input device is a device through which user input from an external source is initially received. For example, the computing device (1000) may further include at least one input device such as a keyboard or mouse.

[0082] In addition, optionally or alternatively, the computing device (1000) may further include an output device (not shown). At this time, the output device is a device for displaying specific information externally from the processor (1020). For example, the computing device (1000) may further include at least one output device such as a display, a VR device, AR glasses, an AR projector, or a printing device.

[0083] FIG. 3 is a diagram illustrating various operational methods performed by a computing device performing a data clinic method according to various embodiments of the present disclosure.

[0084] Referring to FIG. 3, at least one processor (1020) of a computing device for a data clinic may perform various operation methods to perform a data clinic method. At this time, the various operation methods may be coded and stored in the memory of the computing device. Specifically, at least one processor may process an input data set received based on the various operation methods and output an output data set. At this time, details regarding the input data set and data included in the output data set are described below (description of FIGS. 4 to 33).

[0085] For example, a computing device according to various embodiments of the present disclosure may perform, but is not limited to, an operating method for data imaging, an operating method for data enhancement, an operating method for data generation, an operating method for data feature extraction, or an operating method for data evaluation.

[0086] Additionally, each of the above-described operating methods can be performed based on operating algorithms of at least one processor included in the computing device.

[0087] For example, a computing device according to various embodiments of the present disclosure may perform, but is not limited to, a data imaging algorithm, a data enhancement algorithm, a data generation algorithm, a data feature extraction algorithm, or a data evaluation algorithm.

[0088] At this time, the names of each operation method and algorithm are arbitrarily named according to the output results for the convenience of explanation, so each operation method or algorithm is only defined based on the operations performed by the processor, and the name of the operation method or algorithm itself does not limit the invention.

[0089] More specifically, a computing device according to various embodiments of the present disclosure can process an input data set according to a data imaging algorithm to generate an image for the input data set.

[0090] Additionally, a computing device according to various embodiments of the present disclosure can improve data by processing an input data set according to a data improvement algorithm, and can generate a result of the improvement.

[0091] Additionally, a computing device according to various embodiments of the present disclosure can generate synthetic data by processing an input data set according to a data generation algorithm.

[0092] Additionally, a computing device according to various embodiments of the present disclosure can process an input data set according to a data feature extraction algorithm to extract a property of the input data set.

[0093] Additionally, a computing device according to various embodiments of the present disclosure can process an input data set according to a data evaluation algorithm to evaluate the quality of the input data set.

[0094] Details of each of the above algorithms are explained below.

[0095] Furthermore, the computing device according to various embodiments of the present disclosure can perform the various operation methods or algorithms described above in parallel, sequentially, or selectively. Specifically, the computing device can use the same input data as input values ​​for different algorithms in parallel, can use the result values ​​output by a specific algorithm sequentially as input values ​​for another algorithm, or can selectively perform some of the algorithms among a plurality of algorithms according to a predetermined method.

[0096] In addition, the various operation methods or algorithms for the data clinic described above can be performed in a deep learning model included in a computing device according to various embodiments of the present disclosure. Specifically, the computing device according to various embodiments of the present disclosure may include one deep learning model for performing the various operation methods or algorithms described above, but is not limited thereto, and may include multiple deep learning models for performing each of the operation methods or algorithms described above, or may include one or more deep learning models for performing at least some of the various operation methods or algorithms described above.

[0097] Figure 4 is a diagram illustrating various systems for providing data clinic services according to various embodiments, as well as artificial intelligence models and algorithms for constructing such systems. Here, a system may refer to a system that includes at least one software or hardware configuration to perform a specific function.

[0098] A computing device according to the present disclosure can provide a data clinic service based on various machine learning frameworks performed by at least one processor and a memory electronically connected to at least one processor.

[0099] In this regard, there are various types of machine learning (AI) frameworks that can be trained to perform a given task. Support vector machines, decision trees, and neural networks are just a few examples of machine learning frameworks used in various applications, such as image processing and natural language processing. Some machine learning frameworks, such as neural networks, utilize layers of nodes that perform specific operations.

[0100] In a neural network, nodes are connected to each other through one or more edges. A neural network may include an input layer, an output layer, and one or more intermediate layers. Each node may process its inputs according to a predefined function and provide output to subsequent layers, or in some cases, previous layers. The input to a particular node may be multiplied by a weight value corresponding to the edge between the input and the node. Additionally, each node may have a separate bias value used to generate the output. Various learning procedures can be applied to learn the edge weights and / or bias values ​​(parameters).

[0101] A neural network architecture may have multiple layers that perform different specific functions. For example, one or more node layers may collectively perform specific operations, such as pooling, encoding, or convolution operations. As used herein, the term "layer" may refer to a group of nodes that share inputs and outputs, such as communicating with external sources or other layers of the network. The term "calculation" may refer to a function that can be performed by one or more node layers. The term "model structure" may refer to the overall architecture of a layered model, including the number of layers, the connectivity of the layers, and the types of operations performed by individual layers. The term "neural network structure" may refer to the model structure of a neural network. The terms "trained model" and / or "tuned model" may refer to the model structure along with the parameters for the trained or tuned model structure. For example, two trained models may have different values ​​for parameters even though they share the same model structure, such as when they are trained on different training data or when the training process has an underlying probabilistic process.

[0102] "Transfer learning" is a broad approach for training models with limited task-specific training data for a specific task. In transfer learning, a model is first pretrained on another task for which valuable training data is available, and then the model is adapted for the specific task using task-specific training data.

[0103] The term "pre-training," as used herein, refers to training a model on a pre-training dataset to adjust model parameters in a manner that allows subsequent adjustments of those model parameters to tailor the model to one or more specific tasks. In some cases, pre-training may involve a self-supervised learning process on unlabeled training data, where the "self-supervised" learning process involves learning from the structure of pre-training examples in the absence of explicit (e.g., manually provided) labels. Subsequent modification of the model parameters obtained through pre-training is referred to herein as "tuning." Tuning may be performed for one or more tasks using supervised learning on explicitly labeled training data, and in some cases, a task different from pre-training may be used for tuning.

[0104] Referring to FIG. 4, a computing device according to the present disclosure may include a data clinic system composed of various machine learning models to provide clinic services.

[0105] For example, the computing device may include, but is not limited to, a data imaging system, a data diagnostic system, and a data treatment system.

[0106] Here, the data imaging system may include, but is not limited to, a lens processing model for determining an optimal dimension for representing the characteristics of the data, an imaging model for obtaining a data image reflecting the inherent characteristics of the data, or a visualization model for visually representing the data.

[0107] Additionally, the data diagnosis system may include, but is not limited to, a diagnosis model for diagnosing at least one characteristic of data or a quality assessment model for evaluating the quality of data.

[0108] Additionally, the data treatment system may include, but is not limited to, a synthetic model (or generation model) for generating targeted virtual data as needed, a data diet model for removing at least a portion of the data, or a data correction model for adjusting the characteristics of at least a portion of the data.

[0109] Various machine learning models included in a computing device may be composed of multiple modules stored in memory. In the present disclosure, the term "module" may be used to refer to the functional units that constitute a machine learning model. For example, a module may include, but is not limited to, an encoder, a decoder, a generator, a discriminator, an adapter, a natural language processing (NLP) module, or a large language model (LLM).

[0110] The computing device can store the plurality of modules described above, and can construct a machine learning framework based on at least some of the modules to obtain an artificial intelligence model for a data clinic. For example, a data lens included in a data imaging system can be implemented as a machine learning model including at least one encoder or at least one adapter, but is not limited thereto.

[0111] Additionally, the computing device can build a database for storing various data acquired based on the data clinic system.

[0112]

[0113] [Improving quality of Imaging - Data Lens Processing]

[0114] To obtain a high-quality image of data, a data lens (or imaging device, imaging manifold generation model, encoder, etc.) designed to preserve the inherent characteristics of the data is required. For example, a computing device can determine the optimal dimensionality of the embedding space to preserve the distribution of the input data set, and then obtain a data image by representing the data set based on that dimensionality (e.g., projection, mapping, dimensionality reduction, etc.).

[0115] Therefore, below, a data lens processing (or design) method is proposed to improve the quality of data imaging.

[0116]

[0117] FIG. 5 is a diagram illustrating a data lens processing system and a data imaging system according to various embodiments.

[0118] Referring to (a) of FIG. 5, a computing device may acquire a data lens system based on a data set. At this time, the data lens system may be a term meaning at least one configuration for mapping the data set to a specific embedding space. For example, the data lens system may include at least one encoder for mapping the data set to an embedding space (or latent space) of a specific dimension and / or at least one adapter for adjusting parameters. In addition, for example, the data lens system may include a neural network layer composed of at least one node for identifying a latent variable (or latent feature vector) corresponding to the data set.

[0119] The computing device (3500) can determine a data lens system that processes the data set to preserve inherent characteristics of the data set based on the data set.

[0120] For example, a computing device can acquire a lens system corresponding to a data set based on a database. Specifically, the computing device can search the database for a lens system corresponding to the input data set based on the characteristics of the input data set.

[0121] As another example, a computing device can obtain a lens system corresponding to a data set based on a lens processing algorithm. Specifically, the computing device can calculate the optimal dimensionality that preserves the inherent characteristics of the input data set.

[0122] Details of the lens processing algorithm are described in the description for Fig. 37.

[0123] Referring to (b) of FIG. 5, a computing device may process a data set based on a data imaging system (3510) including a determined data lens system to obtain a data image. At this time, the imaging system (3510) may include a lens system composed of at least one module (e.g., an encoder, an adapter, etc.).

[0124] In this case, the computing device can obtain a data image representing the intrinsic characteristics of the data set using the imaging system (3510).

[0125]

[0126] FIG. 6 is a flowchart illustrating a method by which a computing device acquires a lens system based on a data set, according to various embodiments.

[0127] Referring to FIG. 6, a computing device can acquire a first data set (S3601). At this time, the first data set may include a learning data set for training an artificial intelligence model, but is not limited thereto.

[0128] Additionally, the computing device can search for a lens system corresponding to the first data set based on the database (S3603).

[0129] When a lens system corresponding to the first data set is searched, the computing device can obtain the first imaging model by loading the first lens system from the database (S3605).

[0130] If a lens system corresponding to the first data set is not found, the computing device can obtain a second imaging model by generating a second lens system according to a pre-stored algorithm (S3607).

[0131] Additionally, in this case, the computing device can store information associated with the first data set and the second lens system in a database (S3609).

[0132] Through this, when a second data set similar to the first data set is input, the computing device can build a second imaging model for obtaining a data image reflecting the intrinsic characteristics of the second data set based on the second lens system pre-stored in the database.

[0133]

[0134] A computing device according to one embodiment of the present disclosure may include a lens processing algorithm for determining lens properties most suitable for imaging a data set and a lens processing model in which the algorithm is implemented.

[0135] For example, a computing device can design a data lens system by selecting at least one of a plurality of encoders, each of which is configured based on different dimensionality, to determine the optimal dimensionality for representing a data set.

[0136]

[0137] FIG. 7 is a flowchart illustrating an example of a lens processing algorithm performed by a computing device according to various embodiments.

[0138] FIG. 8 is a diagram illustrating an example of a lens processing model constructed for a computing device to perform a lens processing algorithm according to various embodiments.

[0139] Referring to FIG. 7, a computing device can obtain a data set (S3701).

[0140] Additionally, the computing device may obtain a first embedding vector based on a data set using a first module (S3703). At this time, the first module may be an encoder (or multiple encoders). Additionally, the first embedding vector may include a set of data points appearing in an embedding space of a specific dimension. Alternatively, the first embedding vector may include a set of instances of a specific dimension. For example, the first embedding vector may be a 512-dimensional feature vector, but is not limited thereto.

[0141] For example, referring to FIG. 8, a computing device may input an acquired data set into a first encoder (E0). In addition, the first encoder (E0) may be implemented to output a first embedding vector (3801) based on the input data set. At this time, the first embedding vector (3801) may have N dimensions (e.g., 512). The first encoder (E0) may include a plurality of neural network layers implemented to dimensionally reduce the data set into an N-dimensional embedding.

[0142] Referring again to FIG. 7, the computing device can input the first embedding vector into the first adapter module and the second adapter module (S3703). At this time, the adapter module may be configured to adjust the parameters of the encoder. Specifically, the computing device can adjust the parameters of the artificial intelligence model by implementing the adapter module to be connected to the encoder.

[0143] Additionally, in this case, the first adapter module and the second adapter module may be connected in parallel. Specifically, the computing device (or the lens processing model included in the computing device) may be implemented such that data output from the first module is input to the first adapter module and the second adapter module, respectively.

[0144] Additionally, the computing device (or lens processing model) may be implemented to include three or more adapter modules. Specifically, the computing device (or lens processing model) may be implemented such that data output from the first module is input to each of the three or more adapter modules.

[0145] For example, referring to FIG. 8, the computing device may input a first embedding vector (3801) into a set of adapter modules, each of which includes a first adapter module (3810) and a second adapter module (3820). In this case, the adapter modules may be implemented to include at least one encoding layer and at least one decoding layer. Specifically, the first adapter module (3810) may include, but is not limited to, a first encoding layer (E1) and a first decoding layer (D1) corresponding to the first encoding layer.

[0146] Referring again to FIG. 7, the computing device can obtain a second embedding vector based on the first embedding vector using the first adapter module, and can obtain a third embedding vector based on the first embedding vector using the second adapter module (S3707). In this case, the computing device can be implemented to perform operations by the first adapter module and operations by the second adapter module simultaneously. The computing device can be implemented to perform parallel operations based on the same input inputted to a plurality of adapter modules.

[0147] For example, referring to FIG. 8, the first adapter module (3810) can identify the second embedding vector (3802) by processing the first embedding vector (3801) using the first encoding layer (E1). In this case, the second embedding vector (3802) can be implemented to be output to the hidden layer of the first adapter module (3810). In addition, the second adapter module (3820) can identify the third embedding vector (3803) by processing the first embedding vector (3801) using the second encoding layer (E2). In this case, the third embedding vector (3803) can be implemented to be output to the hidden layer of the second adapter module (3810).

[0148] Additionally, the second embedding vector (3802) and the third embedding vector (3803) may have different dimensions. For example, the second embedding vector (3802) may have a dimension a1, and the third embedding vector (3803) may have a dimension a2, in which case a1 may be smaller than a2. Additionally, the dimension (N) of the first embedding vector (3801) may be larger than the dimension (a1) of the second embedding vector (3802) and the dimension (a2) of the third embedding vector (3803).

[0149] The computing device can configure multiple adapter modules so that the dimensions of the output embedding vectors increase. The computing device can process a data lens for imaging a data set by selecting at least one of the multiple adapter modules that output embedding vectors of different dimensions.

[0150] Referring again to FIG. 7, the computing device can output a fourth embedding vector based on the second embedding vector using the first adapter module, and can output a fifth embedding vector based on the third embedding vector using the second adapter module (S3709). Specifically, the computing device can obtain the fourth embedding vector by restoring the second embedding vector using the first adapter module, and can obtain the fifth embedding vector by restoring the third embedding vector using the second adapter module.

[0151] For example, referring to FIG. 8, the first adapter module (3810) can identify the fourth embedding vector (3804) based on the second embedding vector (3802) using the first decoding layer (D1). In this case, the fourth embedding vector (3804) can be implemented to be output to the output layer of the first adapter module (3810). In addition, the second adapter module (3820) can identify the fifth embedding vector (3805) based on the third embedding vector (3803) using the second decoding layer (D2). In this case, the fifth embedding vector (3805) can be implemented to be output to the output layer of the second adapter module (3820).

[0152] Additionally, in this case, the fourth embedding vector (3804) and the fifth embedding vector may have the same dimension. Furthermore, the fourth embedding vector, the fifth embedding vector, and the first embedding vector may have the same dimension (N). This is because the adapter module is configured to include an encoding layer and a decoding layer paired with the encoding layer.

[0153] Referring again to FIG. 7, the computing device can use the second module corresponding to the first module to obtain a first restored data set based on the fourth embedding vector and a second restored data set based on the fifth embedding vector (S3711). At this time, the second module may be a decoder (or multiple decoders). Alternatively, the second module may be a generator. Also, the restored data set may be generated to have similar characteristics to the input data set.

[0154] For example, referring to FIG. 8, the computing device may input a fourth embedding vector (3804) and a fifth embedding vector (3805) to a first decoder (D0) corresponding to a first encoder (E0). In this case, the first decoder (D0) may be implemented to output a first restored data set based on the fourth embedding vector (3804) and a second restored data set based on the fifth embedding vector (3805).

[0155] Referring again to FIG. 7, the computing device may select at least one adapter module based on at least one of a first parameter defined based on the data set, the first restored data set, and the second restored data set, and a second parameter defined based on the second embedding vector and the third embedding vector (S3713).

[0156] Specifically, the computing device can select at least one adapter module based on parameters defined based on at least one of a distribution of embedding vectors and a similarity between data, and can build an imaging model based on the selected adapter module.

[0157] For example, the computing device can determine a characteristic associated with at least one dimensionality that optimizes the at least one parameter, and select an adapter module corresponding to the determined characteristic.

[0158] A specific method for optimizing the dimension of a data image by a computing device is described in FIG. 41.

[0159]

[0160] FIG. 9 is a diagram illustrating an example of a neural network structure of a lens processing system included in a computing device according to various embodiments.

[0161] Referring to FIG. 9, the lens processing model included in the computing device may include an artificial neural network structured into multiple layers.

[0162] The computing device may include at least one neural network for acquiring a data image. Specifically, the computing device may include a sub-imaging neural network (3910) for outputting a sub-data image based on input data, and at least one main imaging neural network (3920) for outputting a main data image based on the sub-data image.

[0163] For example, the computing device can obtain a first data point set defined in an N-dimensional embedding space based on an input data set using a sub-imaging neural network (3910). In addition, for example, the computing device can obtain a main data image (or a second data point set) defined in an M-dimensional embedding space (wherein, M>N) based on an N-dimensional sub-data image (or a first data point set) using a main imaging neural network (3920). At this time, the main imaging neural network (3920) can include an encoding neural network, a latent space, and a decoding neural network. The main imaging neural network (3920) can output a restored data image having similar characteristics to the sub-data image through a decoding neural network.

[0164] At this time, the computing device can construct a lens system based on at least one of the plurality of main imaging neural networks. For example, the computing device can be implemented to acquire a lens system including a main imaging neural network based on the algorithm described in FIG. 37.

[0165] Additionally, the computing device may include a restoration neural network (3930) paired with a sub-imaging neural network (3910). The restoration neural network (3930) may output a restoration data set based on the restoration data image.

[0166] Additionally, the lens processing model may include multiple layers.

[0167] Specifically, the lens processing model may be configured as a neural network model including multiple layers composed of multiple nodes. For example, the lens processing model may include an input layer (3901) into which a data set is input, a first latent layer (3902) for identifying a sub-data image, a second latent layer (3903) for identifying a main data image, a third latent layer (3904) for identifying data restored in N dimensions from the main data image, and an output layer (3905) for outputting restored data.

[0168] At this time, the first latent layer (3903) may be implemented as an input layer of the main imaging neural network (3920), but is not limited thereto. The second latent layer (3903) may be implemented as a hidden layer of the main imaging neural network (3920), but is not limited thereto. In addition, the third latent layer (3903) may be implemented as an output layer of the main imaging neural network (3920), but is not limited thereto.

[0169] To determine the main imaging neural network candidate among multiple main imaging neural network candidates, the dimension of the embedding space that best reflects the inherent characteristics of the input data set must be determined. This is described in detail in the description of Figure 41.

[0170]

[0171] FIG. 10 is a diagram illustrating a method for a computing device to enhance a lens processing model by utilizing an auxiliary network according to various embodiments.

[0172] Referring to FIG. 10, after step S3709 of FIG. 7, the computing device may obtain at least one output value associated with task execution using at least one auxiliary network connected to the first module or a plurality of adapter modules (S4001). At this time, the auxiliary network may be an artificial intelligence model for performing the task using an input data set. For example, the auxiliary network may be a classification model composed of a Fully Connected Layer (FC) and a Softmax layer, but is not limited thereto.

[0173] Additionally, the computing device can output task parameters defined based on at least one output value (S4003). Additionally, the computing device can select at least one adapter module based on at least one of the first parameter, the second parameter, or the third parameter (S4005).

[0174] For example, referring to FIG. 8, the computing device may include a first auxiliary network (Aux1) connected to a latent layer of a first module (E0) in which a first embedding vector (3801) appears.

[0175] In this case, the computing device can obtain an output value indicating the result of task execution by inputting the first embedding vector (3801) into the first auxiliary network (Aux1) and performing a task (e.g., classification).

[0176] Additionally, in this case, the computing device can derive task parameters for optimizing the output value. For example, the computing device can optimize the output value by minimizing (or maximizing) a task parameter representing the task performance capability of the first auxiliary network (Aux1).

[0177] Additionally, in this case, the computing device can select at least one adapter module based on at least one of the first parameter defined based on the data set, the first restored data set, and the second restored data set, the second parameter defined based on the second embedding vector and the third embedding vector, or the task parameter.

[0178] As another example, referring to FIG. 8, the computing device may include a plurality of auxiliary networks connected to each of the plurality of adapter modules. Specifically, the computing device may include, but is not limited to, a second auxiliary network (Aux2) connected to a latent layer of the first adapter module (E1) in which a second embedding vector (3802) appears, and a third auxiliary network (Aux3) connected to a latent layer of the second adapter module (E2) in which a third embedding vector (3803) appears.

[0179] In this case, the computing device can obtain a first output value indicating the task performance result by inputting the second embedding vector (3802) into the second auxiliary network (Aux2) and performing the task. In addition, the computing device can obtain a second output value indicating the task performance result by inputting the third embedding vector (3803) into the third auxiliary network (Aux3) and performing the task.

[0180] Additionally, in this case, the computing device can calculate task parameters for optimizing the plurality of output values ​​(the first output value and the second output value). Specifically, the computing device can define the task parameters based on the sum of the plurality of output values.

[0181] Additionally, in this case, the computing device can select at least one adapter module based on at least one of the first parameter defined based on the data set, the first restored data set, and the second restored data set, the second parameter defined based on the second embedding vector and the third embedding vector, or the task parameter.

[0182] According to one embodiment of the present disclosure, a computing device can build an imaging model to jointly learn parameters for a task performance capability of an input learning data set, and can acquire a task-based data image through the imaging model built in this way.

[0183]

[0184] A computing device according to one embodiment of the present disclosure can calculate an optimal dimension reflecting an inherent characteristic of a data set based on a correlation between at least one parameter described above and a dimension by which a data image is defined.

[0185] FIG. 11 is a diagram illustrating a method for a computing device to determine a characteristic associated with a dimension for optimizing parameters, according to various embodiments.

[0186] Referring to FIG. 11, a computing device may use a plurality of modules that output embedding vectors of different dimensions to calculate at least one correlation between the dimensions of the output embedding vectors and at least one parameter (S4101). For example, the computing device may calculate at least one of a first correlation of a similarity parameter (e.g., Mean Squared Error or reconstruction error, etc.) according to a dimension, a second correlation of a distribution parameter (e.g., KL Divergence, etc.) according to a dimension, or a third correlation of a task parameter (e.g., Binary cross-entropy or Categorical cross-entropy, etc.) according to a dimension.

[0187] Additionally, the computing device can produce a dimensionality range for optimizing at least one parameter based on at least one correlation (S4103).

[0188] Additionally, the computing device can determine optimal dimensionality to reflect the inherent characteristics of the data set based on the dimensional range (S4105).

[0189] Additionally, the computing device can build an imaging model including at least one module for mapping the data set onto an embedding space of optimal dimension (S4107).

[0190] A computing device according to one embodiment of the present disclosure can significantly reduce the time required to determine the optimal dimension for representing a data set by implementing the parallel computing algorithm described above, and increase the learning efficiency of an artificial intelligence model comprised of multiple modules. Consequently, the computational cost of at least one processor included in the computing device can be reduced.

[0191]

[0192] FIG. 12 is a flowchart illustrating a method for a computing device to obtain a data image reflecting inherent characteristics of a data set, according to various embodiments.

[0193] Referring to Figure 12, the computing device can build a lens system by determining the optimal dimension of the embedding space that reflects the inherent characteristics of the data set (S4201). The detailed algorithm by which the computing device builds the lens system has been previously described, so it will be omitted.

[0194] Additionally, the computing device can input a data set by loading an imaging model including the constructed lens system (S4203).

[0195] Additionally, the computing device can acquire a data point set by mapping the data set to an optimal dimension embedding space using an imaging model (S4205). At this time, the data point set can form a manifold representing the data set and include multiple data points (or instances) corresponding to each piece of data included in the data set.

[0196] In addition, the computing device can obtain a data image by representing a set of data points in an imaging space (S4207). At this time, the data image may be identical to the set of data points, but is not limited thereto. For example, the computing device can obtain the data image by visualizing the set of data points in an imaging space using a visualization model. As a specific example, the computing device can obtain the data image by visualizing an N-dimensional set of data points in a three-dimensional space, but is not limited thereto.

[0197]

[0198] FIG. 13 is a diagram illustrating a method for a computing device to image a data set and determine a task performance capability, according to various embodiments.

[0199] Referring to FIG. 13, a computing device can obtain a data set (S4301).

[0200] Additionally, the computing device can identify a first set of data points by mapping the data set to a first embedding space using at least one pre-trained model (S4303). At this time, the first embedding space may be a feature map with a predetermined N dimension (e.g., 512).

[0201] Additionally, the computing device can identify a second set of data points by mapping the first set of data points to a second embedding space using at least one pre-trained model (S4305). At this time, the computing device can determine an optimal dimension corresponding to the input data set, and identify the second set of data points by mapping the first set of data points to the second embedding space defined by the optimal dimension.

[0202] Additionally, the computing device can perform a task based on the first data point set using at least one auxiliary network to obtain an output value (S4307).

[0203] In this case, the computing device can evaluate the task performance capability of the data set based on the first data point set. Specifically, the computing device can load an auxiliary network that reflects the task of the artificial intelligence model to be learned by the data set, and can evaluate the task performance capability by inputting the first data point set or the data set into the loaded auxiliary network.

[0204] Additionally, the computing device can acquire a data image based on a second data point set using at least one pre-learned model (S4309).

[0205]

[0206] [Improving a quality of dataset - Data Diet]

[0207] To optimize the framework of an artificial intelligence model and produce highly accurate results, the quality of training data is very important.

[0208] As mentioned above, the quality of training data encompasses both quantitative and qualitative aspects. Specifically, for the successful training of an AI model, it is necessary to (i) secure a sufficient amount of training data to train the AI ​​model, (ii) secure training data with high-quality inherent characteristics (e.g., unbiased distribution), and (iii) secure training data with characteristics appropriate to the learning objective (e.g., task-dependent properties).

[0209] A computing device according to one embodiment of the present disclosure can synthesize or modify (or adjust) data in a direction that improves the inherent characteristics and task-dependent characteristics of the data set to obtain high-quality learning data.

[0210] In addition, the computing device according to the present disclosure can improve the overall quality of a data set by removing at least some data from the data set.

[0211] Typically, data downsampling or undersampling techniques are used to address data imbalance. However, existing undersampling methods have the disadvantage of negatively impacting the learning performance of machine learning models, as they remove data without considering the characteristics of the machine learning dataset.

[0212] A computing device according to the present disclosure can improve the learning efficiency of an artificial intelligence model that is learned by appropriately removing at least some data from a data set.

[0213] Figure 14 is a diagram illustrating experimental data on the correlation between the amount of learning data and the learning efficiency of an artificial intelligence model.

[0214] Referring to Figures 14 (a) and (b), it can be seen that even when the size of the training data set is reduced to a certain level, the accuracy according to training is maintained at a high level, and the time required for training is dramatically reduced. Specifically, even when the amount of data is reduced from 100% to 10%, the classification accuracy (the upper graph of the two graphs shown in Figures 44 (a) and (b)) decreases gradually by about 1%, from 0.9925 to 0.9800, while the training time decreases by more than 500%, from 40 seconds to 5 seconds.

[0215] In this disclosure, we propose a data diet method that improves the quality of a data set by dramatically reducing the time and cost required for learning while maintaining learning efficiency, in addition to solving the problem of imbalance in learning data.

[0216]

[0217] FIG. 15 is a diagram illustrating a method for a computing device to remove at least a portion of a data set using a pre-trained artificial intelligence model, according to various embodiments.

[0218] Referring to FIG. 15, the computing device can sample and remove at least some of the data included in the data set in a predetermined manner using a pre-learned artificial intelligence model (4500).

[0219] For example, referring to (a) of FIG. 15, a computing device can input first input data (4510) into an artificial intelligence model (4500), and the artificial intelligence model (4500) can obtain first output data (4520) by removing at least some of the data included in the first input data (4510).

[0220] At this time, the first input data (4510) may include a first data set (4511) and / or a first data point set (4512) (or a first data image) corresponding to the first data set (4511). Here, the first data point set (4512) may be data obtained by mapping the first data set (4511) to a specific embedding space.

[0221] Additionally, the first output data (4520) may include a first processed data set (4521) and / or a first processed data point set (4522) (or a first processed data image).

[0222] Specifically, the computing device can obtain the first processed data set (4521) by removing at least some of the data included in the first data set (4511). Alternatively, the computing device can obtain the first processed data point set (4522) by removing at least some of the data points included in the first data point set (4512), and obtain the first processed data set (4521) based on the first processed data point set (4522).

[0223] Similarly, referring to (b) of FIG. 15, the computing device can obtain a second processed data set (4541) by removing at least some of the data included in the second data set (4531). Alternatively, the computing device can obtain a second processed data point set (4542) by removing at least some of the data points included in the second data point set (4532), and can obtain the second processed data set (4541) based on the second processed data point set (4542).

[0224] Referring to FIG. 15, the input data sets (4511, 4531) may be data including annotation data or labeling, or may be data not including annotation data or labeling. For example, the first data set (4511) may be data including label data, and the second data set (4531) may be data not including label data.

[0225] In this case, the computing device can perform a data diet process by reflecting label-based annotation information (e.g., class, etc.). This is described in detail below.

[0226]

[0227] FIG. 16 is a flowchart illustrating an example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0228] FIG. 17 is a diagram illustrating an example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0229] Referring to FIG. 16, a computing device can acquire a data set (S4601). Additionally, the computing device can identify a first data point set by mapping the data set to a first embedding space (S4603).

[0230] For example, referring to FIG. 17, the computing device may obtain a first data point set (4720) by mapping a first data set (4710) to a first embedding space using a first pre-trained model (4700). At this time, the first embedding space may be an N-dimensional (N>1) latent space identified from an output layer or a hidden layer of the first pre-trained model (4700). Although the first embedding space is expressed as a two-dimensional space in the drawing, this is for the convenience of explanation, and in reality, it may be a high-dimensional space of four or more dimensions. In addition, the first pre-trained model (4700) may include at least one module (e.g., an encoder) for performing data imaging and representing it in the first embedding space. In this case, the first data point set (4720) may be a data image corresponding to the data set, but is not limited thereto.

[0231] Referring again to FIG. 16, the computing device may acquire a first characteristic of the first data set based on at least one distance value between a plurality of data points included in the first data point set (S4605). At this time, the first characteristic may include an intrinsic characteristic of the data set. For example, the first characteristic may include, but is not limited to, density, distribution, bias, similarity, or uniformity of the data set. In addition, the at least one distance value between the data points may include, but is not limited to, a Euclidean distance between the data points on the first embedding space.

[0232] In addition, the computing device can identify at least one condensed space on the first data point set based on the first characteristic (S4607). In the present disclosure, the condensed space is a concept arbitrarily defined for convenience of explanation, and the term "space" is not intended to limit the invention. Specifically, the condensed space may refer to a specific space in the embedding space where data points are densely packed. Alternatively, the condensed space may refer to at least one data point whose density satisfies a predetermined condition in the embedding space. Alternatively, the condensed space may refer to at least one data point in the embedding space where at least one feature value corresponds to a predetermined range, or a latent space corresponding to at least one data point.

[0233] At this time, the computing device can identify the condensation space based on predetermined conditions.

[0234] Specifically, the computing device can identify a condensation space based on at least one data point whose data density exceeds a threshold value. In this case, the computing device can identify the condensation space based on the magnitude of the absolute density of the data. Alternatively, the computing device can identify the condensation space based on at least one data point whose data density deviation exceeds a threshold value. In this case, the computing device can identify the condensation space by comparing the relative densities of the data.

[0235] Additionally, and not limited thereto, the computing device can identify the condensed space by identifying at least some spaces in the first embedding space where data is biased.

[0236] Additionally, and not limited thereto, the computing device can identify the condensed space by identifying at least some spaces that cause data imbalance in the first embedding space.

[0237] For example, referring to FIG. 17, the computing device may identify at least one condensation space (4725) based on the first data point set (4720). At this time, the at least one condensation space (4725) may include a plurality of data points that are close in distance. That is, the at least one condensation space (4725) may include a plurality of data points corresponding to a plurality of data that are similar to each other.

[0238] Referring again to FIG. 16, the computing device may obtain a second set of data points by removing at least some of the plurality of data points included in at least one condensed space (S4609). For example, the computing device may randomly sample at least some of the plurality of data points included in at least one condensed space and obtain the second set of data points by removing the sampled data points, but is not limited thereto.

[0239] Additionally, the computing device can obtain a processed data set based on the second data point set (S4611). Specifically, the computing device can obtain the processed data set by restoring the second data point set.

[0240] For example, referring to FIG. 17, the computing device can obtain a second data point set (4730) by removing at least one data point included in a condensation space (4725) of the first data point set (4720). In this case, the computing device can remove the at least one data point so that a geometric structure (e.g., a manifold) formed by the first data point set (4720) is maintained. Accordingly, the geometric shape of the second data point set (4730) can correspond to the geometric shape of the first data point set (4720).

[0241] Additionally, the computing device may obtain a first processed data set (4715) based on a second data point set (4730) using a second pre-trained model (4705). At this time, the second pre-trained model (4705) is a model paired with the first pre-trained model (4700) and may include at least one module (e.g., a decoder, etc.) for restoring data defined in the first embedding space into an output domain.

[0242]

[0243] FIG. 18 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0244] FIG. 19 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0245] Referring to FIG. 18, a computing device can acquire a data set (S4801). Furthermore, the computing device can identify a first data point set by mapping the data set to a first embedding space (S4803). The computing device can acquire a first characteristic of the first data set based on at least one distance value between a plurality of data points included in the first data point set (S4805). Furthermore, the computing device can identify at least one condensed space on the first data point set based on the first characteristic (S4807). Since a detailed description of the operations of steps S4801 to S4807 has been described in the description of FIG. 46, a detailed description thereof will be omitted.

[0246] For example, referring to FIG. 19, a computing device can obtain a first data point set (4920) by mapping a first data set (4910) to a first embedding space using a first pre-trained model (4900). The computing device can identify at least one condensed space (4925) based on the first data point set (4920).

[0247] Referring again to FIG. 18, the computing device may identify a first sub-set of data points excluding data points associated with the boundary of at least one condensation space (S4809). At this time, the data points associated with the boundary of the condensation space may include at least one data point defining the condensation space. Alternatively, the data points associated with the boundary of the condensation space may include at least one data point located within a predetermined distance from the boundary of the condensation space. In this case, the computing device may set a range of at least one feature (or embedding vector) defining the boundary of the condensation space.

[0248] For example, referring to FIG. 19, the computing device can identify a first sub-data point set (4921) excluding data points associated with the boundary of a condensed space (4925) of the first data point set.

[0249] Referring again to FIG. 18, the computing device may obtain a second data point set by removing at least some of the data points included in the first sub-data set (S4811). Furthermore, the computing device may obtain a processed data set based on the second data point set (S4813).

[0250] For example, referring to FIG. 19, the computing device can obtain a second data point set (4930) by removing at least some of the data points included in the first sub-data point set (4921). In addition, the computing device can obtain a first processed data set (4915) based on the second data point set (4930) using the second pre-learned model (4905).

[0251]

[0252] FIG. 20 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0253] FIG. 21 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0254] Referring to FIG. 20, a computing device can acquire a data set (S5001). Furthermore, the computing device can identify a first data point set by mapping the data set to a first embedding space (S5003). The computing device can acquire a first characteristic of the first data set based on at least one distance value between a plurality of data points included in the first data point set (S5005). Furthermore, the computing device can identify at least one condensed space on the first data point set based on the first characteristic (S5007). Since a detailed description of the operations of steps S4801 to S4807 has been described in the description of FIG. 46, a detailed description thereof will be omitted.

[0255] For example, referring to FIG. 21, a computing device can obtain a first data point set (5120) by mapping a first data set (5110) to a first embedding space using a first pre-trained model (5100). The computing device can identify at least one condensed space (5125) based on the first data point set (5120).

[0256] Referring again to FIG. 20, the computing device may determine a first sub-set of data points associated with at least one condensed space, but not associated with a boundary of a manifold (S5009). In this case, the manifold may refer to a geometric shape (or boundary of a shape) formed by the first set of data points on the first embedding space.

[0257] Specifically, the computing device can determine a first sub-set of data points that is contained in at least one condensed space, but does not include at least one data point defining a shape of the first set of data points.

[0258] For example, referring to FIG. 21, the computing device can identify a manifold (5127) of a first set of data points (5120). Here, the computing device can identify the manifold (5127) based on at least one geometric characteristic connecting a plurality of data points that determine the shape of the first set of data points (5120).

[0259] At this time, the computing device can determine a first sub-data point set (5121) based on at least one data point associated with at least one condensation space (5125) but not associated with a boundary of a manifold (5127). At this time, the first sub-data point set (5121) can be included in at least one condensation space (5125).

[0260] Referring again to FIG. 20, the computing device may obtain a second data point set by removing at least some of the data points included in the first sub-data point set (S5011). Furthermore, the computing device may obtain a processed data set based on the second data point set (S5013).

[0261] For example, referring to FIG. 21, the computing device can obtain a second data point set (5130) by removing at least some of the data points included in the first sub-data point set (5121). In addition, the computing device can obtain a first processed data set (5115) based on the second data point set (5130) using a second pre-learned model (5105).

[0262]

[0263] FIG. 22 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0264] FIG. 23 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0265] Referring to FIG. 22, a computing device can acquire a data set (S5201). Furthermore, the computing device can identify a first data point set by mapping the data set to a first embedding space (S5203). For example, referring to FIG. 23, the computing device can acquire a first data point set (5320) by mapping the first data set (5310) to the first embedding space using a first pre-trained model (5300).

[0266] Referring again to FIG. 22, the computing device can cluster the first set of data points (S5205). Specifically, the computing device can cluster the first set of data points based on a pre-stored clustering algorithm (e.g., unsupervised learning-based clustering). Through this, the computing device can identify and divide the first set of data points into multiple clusters.

[0267] For example, referring to FIG. 23, a computing device can cluster a first data point set (5320) to identify a first cluster (5321), a second cluster (5322), and a third cluster (5323). At this time, a plurality of data points included in each cluster may exhibit similar characteristics. More specifically, the computing device can cluster a plurality of data points included in the first data point set (5320) based on the similarity of characteristics and divide them into a plurality of clusters.

[0268] Referring again to FIG. 22, the computing device can acquire a plurality of characteristics corresponding to the plurality of clusters based on at least one distance value between data points included in the plurality of clusters (S5207). Specifically, the computing device can acquire characteristics corresponding to the clusters based on inherent characteristics of the data points associated with each cluster. For example, the computing device can acquire, but is not limited to, the number of data points included in the cluster, the average distance between the data points included in the cluster, the radius of the cluster, or the density of the cluster.

[0269] Additionally, the computing device may assign a level to each of the plurality of clusters based on at least one of the plurality of characteristics (S5209). Here, the level is a concept for providing differentiation in the data processing process and may correspond to a weight in a data processing algorithm. Specifically, the computing device may set the level based on at least one characteristic of the plurality of clusters. For example, the computing device may be configured to assign a high level to a cluster with a high density (e.g., the number of data points relative to the radius of the cluster).

[0270] For example, referring to FIG. 23, the computing device can identify at least one characteristic of a plurality of clusters (5321, 5322, 5323). Furthermore, the computing device can assign a first level to a first cluster (5321), a second level to a second cluster (5322), and a third level to a third cluster (5323) based on the characteristics corresponding to the plurality of clusters. For example, when the density of the first cluster (5321) is higher than the density of the third cluster (5323) and the density of the third cluster (5323) is higher than the density of the second cluster (5322), the first level may be greater than the third level, and the third level may be greater than the second level.

[0271] Referring again to Figure 22, the computing device can obtain a second set of data points by removing data based on the assigned level (S5211). Specifically, the computing device can be configured to remove a large number of data points for clusters with high assigned levels.

[0272] For example, referring to FIG. 23, the computing device can obtain a second data point set (5330) by removing data points according to the level assigned to each of the plurality of clusters (5321, 5322, 5323).

[0273] Additionally, the computing device may perform clustering by acquiring a first characteristic (e.g., density) based on the distance between data points based on the first data point set (5320) and identifying at least two condensation spaces based on the first characteristic. For example, the computing device may identify a first cluster (5321), a second cluster (5322), and a third cluster (5323) by identifying a first condensation space, a second condensation space, and a third condensation space where the first characteristic is greater than or equal to a threshold value.

[0274] Referring again to FIG. 22, the computing device can obtain a processed data set based on the second data point set (S5213). For example, referring to FIG. 53, the computing device can obtain a first processed data set (5315) based on the second data point set (5330) using the second pre-trained model (5305).

[0275]

[0276] FIG. 24 is a flowchart illustrating another embodiment in which a computing device obtains a processed data set based on a data set, according to various embodiments.

[0277] FIG. 25 is a diagram illustrating another example of a computing device obtaining a processed data set based on a data set according to various embodiments.

[0278] Referring to FIG. 24, a computing device can acquire a data set (S5401). Furthermore, the computing device can identify a first data point set by mapping the data set to a first embedding space (S5403). For example, referring to FIG. 25, the computing device can acquire a first data point set (5520) by mapping the first data set (5510) to the first embedding space using a first pre-trained model (5500).

[0279] At this time, the data set (5510) may include labeled data. Specifically, the data set (5510) input to the first pre-learning model (5500) may include learning target data and correct answer data (e.g., annotation information) corresponding to the learning target data.

[0280] Referring again to FIG. 24, the computing device can identify a plurality of sub-data point sets based on the first data point set (S5405). Specifically, the computing device can divide the first data point set into a plurality of sub-data point sets based on labeling data included in the data set. For example, referring to FIG. 25, the computing device can identify a first sub-data point set (55210), a second sub-data point set (5522), and a third sub-data point set (5523) based on the first data point set (5520). At this time, the plurality of sub-data point sets can represent different classes.

[0281] Referring again to FIG. 24, the computing device may set at least one boundary area between the plurality of sub-data point sets (S5407). More specifically, the computing device may identify at least one boundary that distinguishes the plurality of sub-data sets on the first embedding space. For example, referring to FIG. 25, the computing device may set at least one boundary area by identifying at least one data point included in the boundary between the plurality of sub-data sets. As a specific example, the computing device may identify, but is not limited to, a first boundary area (5524) between a first sub-data point set (5521) and a third sub-data point set (5523), a second boundary area (5525) between the first sub-data point set (5521) and a second sub-data point set (5522), and a third boundary area (5526) between the second sub-data point set (5522) and the third sub-data point set (5523).

[0282] Referring back to FIG. 24, the computing device may obtain a second data point set by removing at least some data points excluding data associated with at least one boundary region based on the first data point set (S5409). Furthermore, the computing device may obtain a processed data set based on the second data point set (S5411). For example, referring to FIG. 25, the computing device may obtain a second data point set (5530) by removing at least one data point excluding data points included in at least one boundary region (5524, 5525, 5526) among the plurality of data points included in the first data point set (5520). Furthermore, the computing device may obtain a first processed data set (515) based on the second data point set (5530) by using the second pre-trained model (5505).

[0283] A computing device according to various embodiments of the present disclosure can improve the learning efficiency (e.g., learning time, learning cost, etc.) of an artificial intelligence model by selectively removing unnecessary data from a data set according to the data diet algorithm described above.

[0284]

[0285] FIG. 27 is a diagram illustrating an example of a computing device using at least one data processing model to improve the quality of a data set, according to various embodiments.

[0286] Referring to FIG. 27, the computing device can identify a first data point set (6220, or first manifold) based on a data set (6210) using a first model (6200).

[0287] At this time, the computing device can determine at least one characteristic of data included in the data set (6210) based on the first data point set (6220), and can determine a data processing algorithm based on the determined at least one characteristic. Specifically, the computing device can generate data, remove data, or correct data to improve the quality of the data set by improving the characteristics of the data set.

[0288] For example, the computing device may input a first sub-data point set (6230) included in a first sub-space (6221) of an embedding space in which a first data point set (6220) appears into a first model (6201, e.g., a data diet model). In this case, the first sub-space (6221) may be a space in which at least one data point with a relatively high first characteristic (e.g., density) appears. That is, the computing device may identify the first sub-space (6221) by extracting at least one sub-space with a relatively high density from the first data point set (6220).

[0289] The computing device can obtain a corrected first sub-data point set (6235) based on the first sub-data point set (6230) using the first model (6201). Specifically, the computing device can obtain the corrected first sub-data point set (6235) by removing at least some of the plurality of data points included in the first sub-data set (6230).

[0290] Additionally, for example, the computing device may input a second sub-data point set (6240) included in a second sub-space (6222) of the embedding space in which the first data point set (6220) appears into the second model (6202, e.g., data generation model). In this case, the second sub-space (6222) may be a space in which at least one data point with a relatively low first characteristic (e.g., density) appears. That is, the computing device may identify the second sub-space (6222) by extracting at least one sub-space with a relatively low density from the first data point set (6220).

[0291] The computing device can obtain a corrected second sub-data point set (6245) based on the second sub-data point set (6240) using the second model (6202). Specifically, the computing device can obtain the corrected second sub-data point set (6245) by generating data based on a latent code corresponding to an arbitrary area on the second sub-space (6222).

[0292] Additionally, for example, the computing device may input a third sub-data point set (6250) included in a third sub-space (6223) of the embedding space in which the first data point set (6220) appears into a third model (6203, e.g., data adjustment model). In this case, the third sub-space (6223) may be a space in which data points located at the boundary between data groups (e.g., labeling data, clusters, etc.) with different second characteristics (e.g., classes) appear. That is, the computing device may identify the third sub-space (6223) by extracting at least one sub-space in which data points with different classes are located on the first data point set (6220).

[0293] The computing device can obtain a corrected third sub-data point set (6255) based on the third sub-data point set (6250) using the third model (6203). Specifically, the computing device can obtain the corrected third sub-data point set (6255) by adjusting the feature value of at least one data point among a plurality of data points included in the third sub-space (6223).

[0294]

[0295] FIG. 28 is a diagram illustrating a pipeline for a computing device to input at least one data processing model based on characteristics of a data set, according to various embodiments.

[0296] Referring to FIG. 28, a computing device can acquire a data set (S6301). Additionally, the computing device can identify a first data point set by mapping the data set to a first embedding space (S6303).

[0297] Additionally, the computing device can screen the first data point set using a screening module configured with at least one diagnostic metric (S6305). Specifically, the computing device can apply the screening module configured with at least one diagnostic metric to the first data point set to calculate at least one characteristic value of the data. Through this, the computing device can determine at least one characteristic of the data included in the first data point set.

[0298] The computing device can input the at least one sub-data point set into at least one of a first model associated with data generation, a second model associated with data removal, or a third model associated with data correction, based on the diagnostic results for the at least one sub-data point set (S6307).

[0299]

[0300] [Integration Data Clinic Model and Large Language Model]

[0301] Machine learning models for natural language processing include natural language understanding models, which aim to infer information from natural language, and natural language generation models, which aim to generate natural language based on some input. Training examples for a natural language understanding model can be task-oriented. For example, to train a natural language understanding model to understand user utterances requesting travel to multiple destinations, a task-specific corpus of labeled training examples can be used. Such a corpus can contain a variety of human-labeled examples of user utterances, and the labels can include intent labels (e.g., booking a flight, finding public transportation, etc.) and slot labels (e.g., origin and destination). For the purposes of this disclosure, the terms "utterance" or "natural language input" include not only words spoken by a user or machine, but also words conveyed using text, sign language, etc.

[0302] In many cases, insufficient human-labeled training examples are readily available for training task-adaptive language understanding models. In other words, models trained using only available examples are likely to perform poorly on a given task. The disclosed implementation provides an approach that uses a generative model to generate synthetic, task-specific training examples that can be used in place of, or in addition to, human-generated training examples. The term "synthetic" in this disclosure means at least partially machine-generated. As described herein, using a generative model to generate training data for a natural language understanding model can provide a large volume of appropriate training data at a relatively low cost, as the synthetic training examples do not require human labeling.

[0303] Existing techniques for training generative models do not necessarily yield generative models that are particularly useful for generating task-specific training examples. For example, one approach to unsupervised training of generative models is to train them to predict the next word in a sequence given the previous words they have already seen. However, if the training data used for these generative models is a general-purpose corpus (e.g., Wikipedia articles, books, web articles, etc.), the trained generative model will learn to generate text similar to the text in the general-purpose corpus. While this approach can yield generative models that generate reasonable utterances, such models may lack utility in certain natural language scenarios.

[0304] For example, "conversational behavior" has significant utility in user-facing applications such as conversational bots and digital assistants. These automated applications can use natural language understanding models to interpret received user utterances, for example, inferring intent and slot values ​​from the words spoken or typed by the user. Furthermore, these automated applications can use generative models to generate response utterances for the user.

[0305] However, generative models trained on general corpora (e.g., Wikipedia articles) may not be particularly adept at generating synthetic utterances appropriate for conversational behavior in user-facing scenarios. Furthermore, the synthetic data (e.g., synthetic utterances) generated by such models may not closely resemble user requests for conversation-based systems, and thus may not be particularly useful as synthetic training data for natural language understanding models intended to understand user conversations.

[0306]

[0307] A computing device according to one embodiment of the present disclosure can provide synthetic data to improve the quality of learning data by using the above-described natural language processing model (e.g., a natural language understanding model or a generative model).

[0308]

[0309] FIG. 29 is a diagram illustrating a method for a computing device to generate virtual data based on data diagnostic data, according to various embodiments.

[0310] Referring to FIG. 29, a computing device can identify a first data point set (6405) based on a data set (6401) using a first model (6410). Here, the first model (6410) may be a data imaging model for representing the distribution of data included in the first data set (6401).

[0311] In addition, the computing device may obtain first diagnostic data based on the first data point set (6405) using a calculator (6420) in which a diagnostic metric is set. At this time, the first diagnostic data may include utterance data indicating the characteristics of the first data set (6401). Alternatively, the first diagnostic data may include utterance data indicating the quality of the first data set (6401). Alternatively, the first diagnostic data may include utterance data indicating matters requiring treatment (e.g., generation, removal, correction, etc.) of the first data set (6401). For example, the first diagnostic data may obtain first diagnostic data including utterance data indicating a diagnostic result on the quality of the data set, such as "insufficient data having class A," "need to generate class A data," "excessive amount of class B data," "need to remove class B data," "the boundary between class C data and class D data is ambiguous," or "need data to distinguish class C data from class D."

[0312] Additionally, the computing device can obtain prompt data based on the first diagnostic data using a natural language processing model (6430). Here, the prompt data may refer to input data configured to obtain a result using a machine learning model. For example, the prompt data may refer to a set of commands input to generate a response from a data generation model based on a large language model. As a specific example, in the natural language processing model, prompt data is a general term for sequence templates, labels, and words.

[0313] A computing device may be configured to generate prompts appropriate for the type of data to be generated (e.g., domain, modality, class, etc.) using a large language model (6440). The computing device may determine the type of data to be generated based on first diagnostic data and generate prompt data based on the determined data type. Furthermore, the computing device may input the generated prompt data into the large language model (6440). Accordingly, the computing device may generate a virtual data set (6445) based on the prompt data using the large language model (6440).

[0314]

[0315] FIG. 30 is a flowchart illustrating a method for a computing device to generate virtual data using a pre-trained machine learning model, according to various embodiments.

[0316] Referring to FIG. 30, a computing device may acquire a first data set (S6501). Furthermore, the computing device may identify a first data point set by mapping the data set to a first embedding space using a pre-trained first model (S6503). Furthermore, the computing device may acquire first diagnostic data associated with at least one characteristic of the first data set based on the first data point set (S6505). Furthermore, the computing device may generate prompt data based on the first diagnostic data (S6507). Furthermore, the computing device may generate a first virtual data set based on the prompt data using a pre-trained second model (S6509).

[0317]

[0318] Referring again to FIG. 29, the computing device can verify whether suitable data has been generated based on the diagnostic results of the data set (6410). Specifically, the computing device can verify whether the generated virtual data set (6445) satisfies predetermined criteria. For example, the computing device can perform the verification by identifying whether the generated virtual data set (6445) corresponds to the results according to the first diagnostic data.

[0319] For example, a computing device can verify whether data requested for generation has been generated by imaging a virtual data set (6445) using a data imaging model (6410).

[0320]

[0321] FIG. 31 is a flowchart illustrating an example of a method for a computing device to verify the suitability of a virtual data set, according to various embodiments.

[0322] Referring to FIG. 31, the computing device can generate a virtual data set according to operation S6509 of FIG. 30.

[0323] Additionally, the computing device can identify at least one targeting region in the first embedding space (S6601). Specifically, the targeting region can be identified by identifying at least one region requiring data generation in the first embedding space based on the first diagnostic data.

[0324] For example, referring to FIG. 29, the computing device can identify a first region (6407a) in the embedding space as a target region based on first diagnostic data corresponding to a first data point set (6405). Specifically, the computing device can determine characteristics of data to be generated based on the first diagnostic data, and identify the target region by specifying at least one region corresponding to the determined characteristics.

[0325] Referring again to FIG. 31, a first sub-data set can be identified by mapping a first virtual data set to a first embedding space using a pre-trained first model (S6603). Furthermore, the computing device can verify the correlation between the first sub-data set and at least one targeting region (S6605).

[0326] In this case, the computing device can determine whether the correlation between the first sub-data set and at least one targeting region satisfies a predetermined criterion (S6607). For example, the computing device can set the predetermined criterion based on the degree to which the first sub-data set corresponds to the targeting region. Alternatively, the computing device can set the predetermined criterion based on the degree to which the area occupied by the first sub-data set in the first embedding space overlaps the targeting region.

[0327] If the predetermined criteria are met, the computing device can determine that suitable virtual data has been generated and terminate the verification algorithm.

[0328] If the predetermined criteria are not met, the computing device can adjust the parameters of the pre-trained second model to obtain a tuned second model (S6609). Alternatively, the computing device can determine that the prompt data contains an error, in which case the parameters of the natural language processing model can be adjusted. Furthermore, the computing device can use the adjusted second model to generate a second virtual data set based on the first diagnostic data (S6611).

[0329] For example, referring to FIG. 29, the computing device can identify that a sub-data point set corresponding to a virtual data set (6445) appears in a second region (6407b) on the first embedding space. The computing device can determine that the second region (6407b) occupied by the sub-data point set is not associated with the first region (6407a), which is the target region. In this case, the computing device can optimize the model by adjusting at least one parameter of the natural language processing model (6430) or the large language model (6440), judging that a predetermined criterion is not met.

[0330] Alternatively, the computing device may use the calculator (6450) to derive an indicator (e.g., a matching score, etc.) regarding the correlation between the generated virtual data set (6445) and the first data set (6401). The computing device may optimize the model by adjusting at least one parameter of the natural language processing model (6430) or the large language model (6440) based on the indicator regarding the correlation.

[0331]

[0332] As another example, the computing device can determine the characteristics of a data set including the generated virtual data set (6445) to verify whether the quality has improved.

[0333]

[0334] FIG. 32 is a flowchart illustrating another example of a method for a computing device to verify the suitability of a virtual data set, according to various embodiments.

[0335] Referring to FIG. 32, a computing device may input a second data set including a first data set and a first virtual data set into a pre-trained first model (S6701). Furthermore, the computing device may identify a second data point set by mapping the second data set to a first embedding space using the pre-trained first model (S6703). Furthermore, the computing device may obtain second diagnostic data associated with at least one characteristic of the second data set based on the second data point set (S6705). Furthermore, the computing device may determine whether a predetermined criterion is satisfied based on the second diagnostic data (S6707).

[0336] If the predetermined criteria are met, the computing device can terminate the verification algorithm, determining that suitable virtual data has been generated.

[0337] If the predetermined criteria are not met, the computing device can adjust the parameters of the pre-trained second model to obtain a tuned second model (S6709). Alternatively, the computing device can determine that there is an error in the prompt data, in which case the parameters of the natural language processing model can be adjusted. Furthermore, the computing device can use the adjusted second model to generate a second virtual data set based on the first diagnostic data (S6711).

[0338] For example, referring to FIG. 29, the computing device can obtain second diagnostic data based on a second data set including a generated virtual data set (6445) and a first data set (6401).

[0339] The computing device can analyze the second diagnostic data to determine whether the generated virtual data set (6445) satisfies a predetermined criterion (e.g., a generation constraint condition). Specifically, the computing device can compare the first diagnostic data and the second diagnostic data to determine whether the quality of the data has improved. For example, the computing device can compare information related to the quality of data included in the first diagnostic data with information related to the quality of data included in the second diagnostic data. In this case, the computing device can determine that the data satisfies the predetermined criterion if the quality of the data determined based on the second diagnostic data is higher than the quality of the data determined based on the first diagnostic data.

[0340] If the computing device determines that the virtual data set (6445) does not satisfy a predetermined criterion, the model can be optimized by adjusting at least one parameter of the natural language processing model (6430) or the large language model (6440) based on this.

[0341]

[0342] FIG. 33 is a diagram illustrating a method for a computing device to generate virtual data by modifying a data image based on language input, according to various embodiments.

[0343] Referring to FIG. 33, a computing device can acquire a first data image (6805) by representing a first data set (6801) in an imaging space using a data imaging model (6810). At this time, the first data image (6805) can be acquired to reflect the inherent characteristics of the first data set (6801).

[0344] The computing device can receive a speech input (e.g., text input) from a user device and a user input for a first data image (6805). In this case, the user input for the first data image (6805) may be a user input related to an area on the first data image (6805) where data needs to be generated. For example, the computing device can obtain a data image in which at least a portion (6807) of the first data image (6805) is masked.

[0345] In this case, the computing device may generate a prompt based on the input speech data using a natural language processing model (6820). At this time, the prompt may be associated with the generation of data corresponding to at least a portion (6807) of the first data image. Specifically, the computing device may generate prompt data associated with the characteristics (or type) of data to be generated in at least a portion (6807) of the first data image (6805) based on the input speech data. For example, the computing device may generate a prompt from a user input to “[generate] [X] [food image data] representing [ramen],” but is not limited thereto.

[0346] Additionally, the computing device can input a first data image with a prompt and at least some areas (6807) masked into the large language model (6830).

[0347] The computing device can use the large language model (6830) to generate a second data image (6809) based on the first data image in which the prompt and at least a portion (6807) are masked. At this time, the second data image (6809) can include a specific portion (6811) corresponding to the virtual instances. Specifically, the computing device can obtain the second data image (6809) including the specific portion (6811) by generating at least one virtual instance (or data point, feature vector, etc.) in at least a portion (6807) of the first data image. For example, the computing device can generate the virtual instances based on feature values ​​of data corresponding to at least a portion (6807) of the first data image (6805), and can identify the specific portion (6811) by indicating the location of the virtual instances in the embedding space.

[0348] In this case, the computing device can identify latent codes corresponding to virtual instances corresponding to a specific region (6811) of the second data image (6809). Specifically, the computing device can identify the latent code by generating at least one feature value corresponding to a specific region (6811) in a specific embedding space, but is not limited thereto.

[0349] Additionally, the computing device can input the identified latent code into a pre-trained generative model (6840). Using the generative model (6840), the computing device can obtain a virtual data set (6845) based on the latent code.

[0350] Through this, the computing device can accurately identify the user's intent based on input not only in natural language input but also in data images where the distribution of data is visually confirmed, and can generate virtual data corresponding to the intent.

[0351]

[0352] FIG. 34 is a flowchart illustrating a method for a computing device to generate virtual data by modifying a data image based on language input, according to various embodiments.

[0353] Referring to FIG. 34, a computing device can acquire a data set (S6901). In addition, the computing device can acquire a first data image corresponding to the data set using a pre-learned first model (S6903).

[0354] Additionally, the computing device may input prompt data and a first data image obtained from user input into a pre-trained second model (S6905). In this case, at least a portion of the first data image may be masked. Specifically, the computing device may input the first data image, with at least a portion of the masked area, from the user input, together with the prompt data, into the second model.

[0355] Additionally, the computing device can obtain a first modified data image in which at least a portion of the first data image is modified using a pre-learned second model (S6907).

[0356] Additionally, the computing device can acquire a latent code based on the first modified data image and input the latent code into a pre-trained third model (S6909). Furthermore, the computing device can use the third model to acquire a first virtual data set based on the latent code (S6911).

[0357]

[0358] A computing device according to one embodiment of the present disclosure may include at least one machine learning model for generating data based on a user's natural language input. In this case, the computing device may be utilized as an auxiliary network to evaluate and improve the quality of virtual data generated based on language.

[0359] FIG. 35 is a diagram illustrating a method for a computing device to generate virtual data based on speech data, according to various embodiments.

[0360] Referring to FIG. 35, a computing device can generate prompt data based on utterance data using a natural language processing model (7010). At this time, the computing device can determine the characteristics (e.g., modality, domain, etc.) of virtual data to be generated by identifying the intent based on the utterance data. Specifically, the computing device can identify the utterance intent based on at least one entity included in the utterance data, and generate prompt data for generating virtual data based on the intent.

[0361] Additionally, the computing device can generate a virtual data set (7005) based on the prompt data using a large language model (7020).

[0362] In this case, the computing device can evaluate the quality of the virtual data set (7005) using at least one auxiliary network (7030, 7040).

[0363] For example, the computing device can evaluate the quality of the virtual data set (7005) using the first auxiliary network (7030, Aux(t)) for performing a task. At this time, the first auxiliary network (7030) can be implemented to correspond to a machine learning model to be trained using the virtual data set (7005). For example, the computing device can store the first auxiliary network (7030) by loading (or implementing using a method such as transfer learning) a machine learning model designed to perform a specific task. Alternatively, the computing device can receive a pre-trained machine learning model together with the speech data, and obtain the first auxiliary network (7030) based on the received model.

[0364] The first auxiliary network (7030) may be a pre-trained machine learning model using a pre-training data set (7001).

[0365] The computing device can input the generated virtual data set (7005) into the first auxiliary network (7030). The computing device can perform a task based on the virtual data set (7005) using the first auxiliary network (7030) and output a result value based on the task.

[0366] In this case, the computing device can evaluate the quality of the virtual data set (7005) based on the above result value. For example, if the task execution result does not meet the preset condition, the computing device can determine that the quality of the virtual data set (7005) is low and adjust the parameters of at least one language model (7010, 7020).

[0367] Alternatively, the computing device may use the virtual data set (7005) to evaluate the pre-trained first auxiliary network (7030). In this case, the computing device may determine whether to perform additional training on the pre-trained first auxiliary network (7030) based on the evaluation result. For example, if the task execution result does not meet a preset condition, the computing device may determine that the training of the first auxiliary network (7030) is insufficient and perform additional training on the first auxiliary network (7030) using the virtual data set (7005).

[0368]

[0369] As another example, the computing device may use a second auxiliary network (7040, Aux(I)) for data imaging to evaluate the quality of a virtual data set (7005). In this case, the second auxiliary network (7040) may be implemented to correspond to the data imaging model of the present disclosure.

[0370] The computing device can use the second auxiliary network (7040) to provide comparison information indicating a correlation between the virtual data set (7005) and the data to be generated (in accordance with the intent).

[0371] Specifically, the computing device can input the generated virtual data set (7005) and the comparison target data set (7002) into the second auxiliary network (7040). At this time, the comparison target data set (7002) may include data received together with speech data from the user device or pre-training data (e.g., 7001) of a machine learning model to be trained using the virtual data set.

[0372] The computing device may be configured to identify a relationship between the comparison target data set (7002) and the virtual data set (7005) by representing the comparison target data set (7002) and the virtual data set (7005) in a common space.

[0373] The computing device can use the second auxiliary network (7040) to map the comparison target data set (7002) and the virtual data set (7005) to a specific embedding space.

[0374] Specifically, the computing device can identify a first data point set (7031) that reflects the characteristics of the comparison target data set (7002) by mapping the comparison target data set (7002) to a specific embedding space.

[0375] Additionally, the computing device can identify a second data point set (7035) that reflects the characteristics of the virtual data set (7005) by mapping the virtual data set (7005) to the specific embedding space.

[0376] The computing device can provide comparison information between the comparison target data set (7002) and the virtual data set (7005) by representing the first data point set (7031) and the second data point set (7035) in a common space.

[0377] Additionally, the computing device can provide characteristic comparison information between the comparison target data set (7002) and the virtual data set (7005) by acquiring at least one characteristic of the comparison target data set (7002) based on the first data point set (7031) and by acquiring at least one characteristic of the virtual data set (7005) based on the second data point set (7035).

[0378]

[0379] FIG. 36 is a flowchart illustrating a method for a computing device to generate virtual data based on speech data and provide comparison information according to various embodiments.

[0380] Referring to FIG. 36, the computing device can input prompt data obtained from user input into a pre-trained first model (S7101). Furthermore, the computing device can use the first model to generate a virtual data set based on the prompt data (S7103).

[0381] Additionally, the computing device can input the virtual data set, the target data set, into an auxiliary network electronically connected to the first model (S7105). Furthermore, the computing device can map the target data set and the virtual data set into a specific embedding space using the auxiliary network (S7107).

[0382] Additionally, the computing device can identify a first data point set corresponding to the comparison target data set and a second data point set corresponding to the virtual data set (S7109). Additionally, the computing device can provide data comparison information including the first data point set and the second data point set (S7111).

[0383] A computing device according to the present disclosure can improve the learning efficiency of a machine learning model and develop the task performance capability of a machine learning model by accurately providing information on the quality of the generated data to a user who wants to generate learning data based on a large language model.

[0384]

[0385] FIG. 37 is a diagram illustrating a language-based generative model and a clinic model included in a computing device according to various embodiments.

[0386] In order for the computing device according to the present disclosure to generate high-quality virtual data, it is important to accurately identify the intent of the input speech data and generate an appropriate prompt.

[0387] To automatically perform prompt engineering, which constructs appropriate prompts to obtain high-level results from machine learning models, it is necessary to determine the required data characteristics based on the input utterance and generate virtual data that reflects those characteristics.

[0388] Referring to (a) of FIG. 37, the computing device may include a prompt engineering model for configuring an appropriate prompt based on utterance data and a generative model for generating virtual data based on the prompt.

[0389] A computing device can train a prompt engineering model based on utterance data and target data for generation. Specifically, the computing device can be implemented to train a prompt engineering model using data to be generated based on the utterance data, thereby generating prompts corresponding to the characteristics of data associated with the utterance data.

[0390] In order for a computing device to construct a prompt to generate virtual data corresponding to the intent of an utterance, it is necessary to derive minimal information related to the data to be generated based on the utterance data. For example, a generative model can determine at least one characteristic of the data based on the input prompt to fill in slots corresponding to the characteristics required for generating the virtual data. The number of slots required for generating the virtual data may be predetermined, but is not limited thereto.

[0391]

[0392] Referring to (b) of FIG. 37, the computing device according to the present disclosure can be advanced by training language-based models (e.g., a large language model, a prompt engineering model) as an auxiliary network based on input / output data of a clinic model.

[0393] Specifically, to link the data clinic model with pre-trained language-based models, the computing device can further train (e.g., fine tune) the language-based models based on data acquired during the data clinic process.

[0394] For example, as in the embodiment illustrated in FIG. 26, in order to generate virtual data based on diagnostic data, the computing device may additionally train language-based models using at least one corpus included in the diagnostic data and data corresponding thereto.

[0395] Additionally, for example, as in the embodiment illustrated in FIG. 33, the computing device may further train language-based models using the data image and at least one corpus associated with the data image to generate modified data images and virtual data based on the data image.

[0396]

[0397] FIG. 38 is a diagram illustrating an example of a computing device using a data clinic model and a language-based model according to various embodiments.

[0398] Referring to FIG. 38, the computing device may include a clinic model (7300). The clinic model (7300) may include at least one imaging model (L1, L2) and at least one generation model (Decoder or Generator).

[0399] Additionally, the computing device may include at least one auxiliary network (Aux(t), Aux(p)) that communicates with the clinic model (7300).

[0400] The computing device can use at least one auxiliary network to perform a task, generate a prompt to generate virtual data, or generate virtual data based on the prompt.

[0401] The computing device can acquire a first data image (IOD1) based on input data (I) using at least one data imaging model. Furthermore, the computing device can perform a task based on the first data image using a first auxiliary network (Aux(t)) for performing the task. Alternatively, the computing device can further acquire a second data image (IOD2) using at least one data imaging model.

[0402] Additionally, the computing device can generate virtual data (I`) based on input data using at least one generative model. Additionally, the computing device can generate synthetic prompts (P`) based on the virtual data using a second auxiliary network (Aux(p)) for generating prompts.

[0403] At this time, the computing device can train the second auxiliary network based on the generated synthetic prompt (P`) and correct prompt (P). Specifically, the computing device can optimize the parameters of the second auxiliary network so that the synthetic prompt becomes similar to the correct prompt.

[0404] Additionally, the computing device can generate virtual data based on the prompt by utilizing a third auxiliary network (Aux(s)) for generating data based on the prompt. Specifically, the computing device can input a synthetic prompt (P`) into the third auxiliary network and generate first synthetic data (D`). Additionally, the computing device can input a correct prompt (P) into the third auxiliary network and generate second synthetic data (D).

[0405] The computing device may learn at least one of a third auxiliary network or a second auxiliary network based on the first synthetic data (D`) and the second synthetic data (D). Specifically, the computing device may learn the second auxiliary network so that the first synthetic data (D`) generates prompts similar to the second synthetic data (D). Alternatively, the computing device may learn the third auxiliary network so that the first synthetic data (D`) generates prompts similar to the second synthetic data (D).

[0406] The computing device according to the present disclosure provides a data clinic model that accurately diagnoses the characteristics of artificial intelligence learning data and improves the quality of the learning data. Furthermore, a language-based model linked to the data clinic is used to accurately understand natural language input and provide data clinic services based on this understanding.

[0407] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0408] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0409] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In computing devices, A memory comprising at least one artificial intelligence model; and At least one processor electronically connected to said memory; At least one processor, Obtain the first data set, Using the first encoding module, a first embedding vector is identified by mapping the first data set to a first-dimensional first embedding space, Inputting the first embedding vector to at least one of the first adapter module and the second adapter module, wherein the first adapter module is implemented to map data to a second embedding space of a second dimension, and the second adapter module is implemented to map data to a third embedding space of a third dimension, and A computing device configured to obtain a first data image corresponding to the first data set based on the first embedding vector using at least one of the first encoding module and the first adapter module or the second adapter module.

2. In paragraph 1, A computing device wherein said at least one processor is further configured to build a data imaging model based on said first encoding module and at least one of said first adapter module or said second adapter module.

3. In paragraph 1, At least one processor, Using the first adapter module, the second embedding vector of the second dimension is identified based on the first embedding vector, Using the second adapter module, a third embedding vector of the third dimension is identified based on the first embedding vector, A computing device further configured to select at least one of the first adapter module or the second adapter module based on a first parameter defined based on the second embedding vector and the third embedding vector.

4. In paragraph 3, At least one processor, Using the first adapter module, a fourth embedding vector is output based on the second embedding vector, and using the second adapter module, a fifth embedding vector is output based on the third embedding vector, and A computing device further configured to obtain a first restored data set based on the fourth embedding vector and to obtain a second restored data set based on the fifth embedding vector using a first decoding module corresponding to the first encoding module.

5. In paragraph 4, A computing device wherein said at least one processor is further configured to select at least one of said first adapter module or said second adapter module based on a second parameter defined based on said first restoration data set and said second restoration data set.

6. In paragraph 1, A computing device, characterized in that the first adapter module and the second adapter module are connected in parallel with respect to the first encoding model.

7. In paragraph 1, A computing device wherein said at least one processor is configured to perform operations by said first adapter module in parallel with operations by said second adapter module by simultaneously inputting said first embedding vector to said first adapter module and said second adapter module.

8. In paragraph 1, A computing device further comprising an auxiliary network electronically connected to at least one of the first encoding module or the plurality of adapter modules.

9. In paragraph 8, At least one processor, Obtaining at least one output value associated with task execution using the above auxiliary network, A computing device further configured to select at least one of the first adapter module or the second adapter module based on a third parameter defined based on at least one output value.

10. A data processing method performed by at least one processor included in a computing device, wherein, by the at least one processor, Step of obtaining the first data set; A step of identifying a first embedding vector by mapping the first data set to a first embedding space of a first dimension using a first encoding module; A step of inputting the first embedding vector to at least one of a first adapter module or a second adapter module, wherein the first adapter module is implemented to map data to a second embedding space of a second dimension, and the second adapter module is implemented to map data to a third embedding space of a third dimension; and A method comprising: obtaining a first data image corresponding to the first data set based on the first embedding vector using at least one of the first encoding module and the first adapter module or the second adapter module.

11. In paragraph 10, A method further comprising: a step of constructing a data imaging model based on the first encoding module and at least one of the first adapter module and the second adapter module.

12. In paragraph 10, A method further comprising: a step of simultaneously inputting the first embedding vector to the first adapter module and the second adapter module, thereby performing an operation by the first adapter module in parallel with an operation by the second adapter module.

13. In paragraph 10, A method further comprising the step of electronically connecting an auxiliary network to at least one of the first encoding module or a plurality of adapter modules.

14. In paragraph 13, A step of obtaining at least one output value associated with performing a task using the auxiliary network; and A method further comprising: selecting at least one of the first adapter module or the second adapter module based on a third parameter defined based on at least one output value; 15. A server including a computing device according to paragraph 1; and A system comprising a user device that is connected to the server and transmits a first data set to the server.

Images