Electronic device and control method thereof

Abstract

An electronic device is disclosed. The electronic device includes a memory for storing at least one instruction, and a processor connected to the memory to control the electronic device. By executing the at least one instruction, the processor: obtains learning data including a plurality of pieces of learning data; based on the learning data, a plurality of embedding vectors which aim at the multiple pieces of learning data and can be mapped to an embedding space are obtained; based on the plurality of embedded vectors, training an artificial intelligence model used for classifying categories of the plurality of pieces of learning data; identifying the embedded vectors misclassified by the artificial intelligence model in the plurality of embedded vectors; identifying an embedded vector closest to the misclassified embedded vector in the embedded space; obtaining a combined embedding vector corresponding to a path connected with the nearest embedding vector and the misclassified embedding vector in the embedding space; and adding the combined embedded vector to learning data to retrain the artificial intelligence model.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a control method thereof. More specifically, the present disclosure relates to an electronic device and a control method thereof that can train an artificial intelligence model by acquiring new training data based on existing training data. Background Art

[0002] An artificial intelligence (AI) system is one in which a machine itself derives a desired result or performs a desired operation by performing training and making determinations.

[0003] AI technology includes machine learning such as deep learning and basic element technology using machine learning. AI technology is used in a wide range of technical fields such as language understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0004] For example, AI technology can be used in the technical fields of visual understanding and reasoning / prediction. Specifically, AI technology can be used to implement technology for analyzing and classifying input data. In other words, methods and devices that can obtain expected results by analyzing and / or classifying input data can be implemented.

[0005] Here, when the AI ​​model generates output data corresponding to the input data, the accuracy of the output data may vary depending on the training data.

[0006] At this point, the problem is that it takes a lot of time and resources to secure a large amount of training data and improve the performance of the AI ​​model.

[0007] The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above may be applicable as prior art with respect to the present disclosure. Summary of the invention

[0008] Technical issues

[0009] Various aspects of the present disclosure at least solve the above-mentioned problems and / or disadvantages, and at least provide the following advantages. Therefore, one aspect of the present disclosure is to provide an electronic device and a control method thereof that can improve the performance of an artificial intelligence model for classifying input data by synthesizing training data based on an embedding space.

[0010] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments.

[0011] Technical Solutions

[0012] According to one aspect of the present disclosure, an electronic device is provided. The electronic device includes a memory storing at least one instruction, and a processor connected to the memory to control the electronic device by executing at least one instruction, the processor being configured to obtain training data including multiple training data, respectively obtain multiple embedding vectors for the multiple training data that can be mapped to an embedding space, train an artificial intelligence model that classifies the multiple training data based on the multiple embedding vectors, identify misclassified embedding vectors misclassified by the artificial intelligence model among the multiple embedding vectors, identify the embedding vector closest to the misclassified embedding vector in the embedding space, obtain a synthetic embedding vector in the embedding space corresponding to a path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector, and retrain the artificial intelligence model by adding the synthetic embedding vector to the training data.

[0013] The processor may also be configured to obtain a synthesized embedding vector at a point on the path in the embedding space by synthesizing the misclassified embedding vector and the embedding vector closest to the misclassified embedding vector.

[0014] An incorrectly classified embedding vector may be an embedding vector whose labeled category is different from the category predicted by the artificial intelligence model after the embedding vector is input into the artificial intelligence model.

[0015] The embedding vector that is closest to the misclassified embedding vector may be the embedding vector that was successfully classified by the artificial intelligence model.

[0016] The processor may also be configured to label the category of the synthesized embedding vector the same as the labeled category of the misclassified embedding vector.

[0017] The processor may also be configured to obtain multiple embedding vectors by respectively extracting features from multiple pieces of training data.

[0018] The processor can also be configured to re-identify embedding vectors misclassified by the artificial intelligence model when the performance of the retrained artificial intelligence model is lower than or equal to a predetermined standard, and update the artificial intelligence model by obtaining a synthetic embedding vector corresponding to a path in the embedding space connecting the re-identified misclassified embedding vector to the embedding vector closest to the re-identified misclassified embedding vector.

[0019] According to another aspect of the present disclosure, a control method of an electronic device is provided. The control method includes obtaining training data including a plurality of training data, respectively obtaining a plurality of embedding vectors for the plurality of training data that can be mapped to an embedding space, training an artificial intelligence model for classifying the plurality of training data based on the plurality of embedding vectors, identifying an incorrectly classified embedding vector that is incorrectly classified by the artificial intelligence model among the plurality of embedding vectors, identifying an embedding vector that is closest to the incorrectly classified embedding vector in the embedding space, obtaining a synthetic embedding vector in the embedding space corresponding to a path connecting the incorrectly classified embedding vector to the embedding vector that is closest to the incorrectly classified embedding vector, and retraining the artificial intelligence model by adding the synthetic embedding vector to the training data.

[0020] When obtaining the synthesized embedding vector, the synthesized embedding vector at the point located on the path in the embedding space may be obtained by synthesizing the misclassified embedding vector and the embedding vector closest to the misclassified embedding vector.

[0021] An incorrectly classified embedding vector may be an embedding vector whose labeled category is different from the category predicted by the artificial intelligence model after the embedding vector is input into the artificial intelligence model.

[0022] The embedding vector that is closest to the misclassified embedding vector may be the embedding vector that was successfully classified by the artificial intelligence model.

[0023] The control method may further include labeling the category of the synthesized embedding vector the same as the labeled category of the misclassified embedding vector.

[0024] When obtaining multiple embedding vectors, the multiple embedding vectors may be obtained by respectively extracting features from multiple pieces of training data.

[0025] The control method may also include: when the performance of the retrained artificial intelligence model is lower than or equal to a predetermined standard, re-identifying the embedding vector misclassified by the artificial intelligence model, and updating the artificial intelligence model by obtaining a synthetic embedding vector corresponding to a path in the embedding space connecting the re-identified misclassified embedding vector to the embedding vector closest to the re-identified misclassified embedding vector.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium includes a program for executing a control method of an electronic device, the control method including obtaining training data including a plurality of training data, respectively obtaining a plurality of embedding vectors for the plurality of training data that can be mapped to an embedding space, training an artificial intelligence model for classifying the plurality of training data based on the plurality of embedding vectors, identifying an incorrectly classified embedding vector among the plurality of embedding vectors that is incorrectly classified by the artificial intelligence model, identifying an embedding vector in the embedding space that is closest to the incorrectly classified embedding vector, obtaining a synthetic embedding vector in the embedding space corresponding to a path connecting the incorrectly classified embedding vector to the embedding vector closest to the incorrectly classified embedding vector, and retraining the artificial intelligence model by adding the synthetic embedding vector to the training data.

[0027] Other aspects, advantages, and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description which discloses various embodiments of the present disclosure in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0029] Figure 1 is a block diagram for explaining a configuration of an electronic device according to an embodiment of the present disclosure;

[0030] Figure 2 is a diagram for explaining an artificial intelligence model according to an embodiment of the present disclosure;

[0031] Figure 3 is a flowchart for explaining a method for obtaining an embedding vector according to an embodiment of the present disclosure;

[0032] Figure 4 is a diagram for explaining an embedding space according to an embodiment of the present disclosure;

[0033] Figure 5 is a flowchart for explaining a method for obtaining a synthetic embedding vector according to an embodiment of the present disclosure;

[0034] Figure 6a , Figure 6b , Figure 6c , Figure 6d and Figure 6e is a diagram for explaining a method of acquiring a synthetic embedding vector according to various embodiments of the present disclosure;

[0035] Figure 7is a flowchart for explaining a method for retraining an artificial intelligence model according to an embodiment of the present disclosure;

[0036] Figure 8 is a block diagram for explaining the configuration of an electronic apparatus and an external device according to an embodiment of the present disclosure;

[0037] Fig. 9 is a timing diagram for explaining operations of an electronic apparatus, an external device, and a server according to an embodiment of the present disclosure;

[0038] Fig.10a and Fig.10b is a diagram for explaining the performance of an artificial intelligence model according to various embodiments of the present disclosure;

[0039] Fig.11 is a flowchart for explaining a control method of an electronic device according to an embodiment of the present disclosure;

[0040] Fig.12 is a flowchart for explaining a method for obtaining a synthesized sample using a generative adversarial network (GAN) according to an embodiment of the present disclosure; and

[0041] Fig.13 It is a flowchart for explaining a method of obtaining synthesized samples using a variational autoencoder (VAE) and a variational auto decoder (VAD) according to an embodiment of the present disclosure.

[0042] The same reference numerals are used throughout the drawings to denote the same elements. DETAILED DESCRIPTION

[0043] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. The following description includes various specific details to assist in understanding, but these details should be considered as merely exemplary. Therefore, it will be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures may be omitted for clarity and conciseness.

[0044] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it is clear to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.

[0045] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.

[0046] The terminology used herein is for describing particular embodiments only and is not intended to limit the scope of the present disclosure.Singular forms include plural forms unless the context clearly indicates otherwise.

[0047] The expressions “having”, “may have”, “include”, “may include”, etc. used herein indicate the existence of stated features (e.g., components such as values, functions, operations, or parts), and do not exclude the existence of additional features.

[0048] The expressions "A or B", "at least one of A and / or B", "one or more of A and / or B", etc. used herein may include all possible combinations of the items listed therewith. For example, "A or B", "at least one of A and B", or "at least one of A or B" may mean (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0049] The expressions “first,” “second,” etc. used herein may modify various components regardless of order and / or importance and may be used to distinguish one component from another and do not limit the components.

[0050] It should also be understood that when a component (e.g., a first component) is referred to as being "(operably or communicatively) coupled to / (operably or communicatively) coupled to" or "connected to" another component (e.g., the second component), this means that one component is coupled to / is coupled to or connected to another component directly or via an intermediate component (e.g., a third component).

[0051] On the other hand, it should be understood that when a component (e.g., a first component) is referred to as being "directly coupled to / directly coupled to another component (e.g., the second component)" or "directly connected to another component (e.g., the second component)", this means that there is no intermediate component (e.g., a third component) between the one component and the other component.

[0052] As used herein, the expression "configured to (or set to)" may be used interchangeably with the expressions "suitable for," "capable of," "designed to," "adapted to," "manufactured to," or "capable of," depending on the context. The term "configured to (set to)" does not necessarily mean "specifically designed to" in hardware.

[0053] In contrast, the expression “a device configured to…” may mean that the device, together with other devices or components, is “capable of…” in a specific context. For example, the phrase “a processor configured (set to) perform A, B, and C” may represent a dedicated processor (e.g., an embedded processor) for performing the corresponding operations, or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor (AP)) capable of performing the corresponding operations by executing one or more software programs stored in a memory device.

[0054] In an embodiment, a "module" or "unit" performs at least one function or operation and may be implemented as hardware, software, or a combination thereof. In addition, multiple "modules" or multiple "units" may be integrated into at least one module and may be implemented as at least one processor in addition to the "modules" or "units" that need to be implemented in specific hardware.

[0055] Meanwhile, various elements and regions in the drawings are schematically shown. Therefore, the technical spirit of the present disclosure is not limited by the relative sizes or distances shown in the drawings.

[0056] Hereinafter, embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings so that those having ordinary skill in the art to which the present disclosure pertains can easily perform the embodiments.

[0057] Figure 1 is a block diagram for explaining a configuration of an electronic device according to an embodiment of the present disclosure.

[0058] refer to Figure 1 , the electronic device 100 may include a memory 110, a communication interface 120, a user interface 130, a speaker 140, a microphone 150, and a processor 160. Some of the above components may be omitted from the electronic device 100, and other components may also be included in the electronic device 100.

[0059] In addition, the electronic device 100 can be implemented as an audio device such as headphones or a head-mounted device, but this is merely an example, and the electronic device 100 can be implemented in various forms (such as smartphones, tablet personal computers (PCs), PCs, servers, smart TVs, mobile phones, personal digital assistants (PDAs), laptop computers, media players, e-book readers, digital broadcast terminals, navigation devices, self-service kiosks, MP3 players, digital cameras, wearable devices, home appliances, and other mobile or non-mobile computing devices).

[0060] The memory 110 may store at least one instruction related to the electronic device 100. The memory 110 may store an operating system (O / S) for driving the electronic device 100. In addition, the memory 110 may store various software programs or applications for the electronic device 100 to operate according to various embodiments of the present disclosure. In addition, the memory 110 may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0061] Specifically, the memory 110 may store various software modules for the electronic device 100 to operate according to various embodiments of the present disclosure, and the processor 160 may execute the various software modules stored in the memory 110 to control the operation of the electronic device 100. That is, the memory 110 may be accessed by the processor 160, and data may be read / written / modified / deleted / updated by the processor 160.

[0062] Meanwhile, the term "memory 110" herein may be used as meaning including the memory 110, a read-only memory (ROM) (not shown), a random access memory (RAM) (not shown) in the processor 160, or a memory card (not shown) installed in the electronic device 100 (e.g., a micro secure digital (SD) card or a memory stick).

[0063] In addition, the memory 110 may store at least one artificial intelligence model 111. In this case, the artificial intelligence model 111 may be a trained model that classifies input data when the data is input.

[0064] Figure 2 is a diagram for explaining an artificial intelligence model according to an embodiment of the present disclosure.

[0065] refer to Figure 2 , when audio data 2100 is input, the artificial intelligence model 111 may output one of the multiple categories 2200. At this time, for example, the output category may include at least one of human speech in category 1, musical sound in category 2, or noise in category 3. Alternatively, the multiple categories may include the speech of a specific person.

[0066] Meanwhile, the artificial intelligence model 111 may include at least one artificial neural network, and the artificial neural network may include a plurality of layers. Each of the plurality of neural network layers has a plurality of weight values, and the neural network operation is performed using the operation result of the previous layer and the operation between the plurality of weight values. The plurality of weight values ​​of the plurality of neural network layers may be optimized by the learning result of the artificial intelligence model. For example, the plurality of weight values ​​may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. Here, the weight value of each layer may be referred to as a parameter of each layer.

[0067] Here, the artificial neural network may include at least one of various types of neural network models, such as convolution neural network (CNN), one-dimensional convolution neural network (1DCNN), region with convolution neural network (R-CNN), region proposal network (RPN), recurrent neural network (RNN), stacking-based deep neural network (S-DNN), state-space dynamic neural network (S-SDNN), deconvolution network, deep belief network (DBN), restricted Boltzmann machine (RBM), full convolution network, long short-term memory (LSTM) network, bidirectional long short-term memory (Bi-LSTM) network, classification network, ordinary residual network, dense network, hierarchical pyramid network, full convolution network, squeeze and excitation network (squeeze and excitation network). network, SENet), transformer network, encoder, decoder, autoencoder or a combination thereof, and the artificial neural network in the present disclosure is not limited to the above examples.

[0068] The communication interface 120 includes a circuit and is a component capable of communicating with an external device and a server. The communication interface 120 may communicate with the external device or server in a wired or wireless communication manner. In this case, the communication interface 120 may include a Bluetooth TM Module (not shown), Wi-Fi module (not shown), infrared (IR) module, local area network (LAN) module, Ethernet module, etc. Here, each communication module can be implemented in the form of at least one hardware chip. In addition to the above-mentioned communication method, the wireless communication module can also include at least one communication chip that performs communication according to various wireless communication standards (such as zigbee, universal serial bus (USB), mobile industry processor interface camera serial interface (mobile industry processor interface cameraserial interface, MIPI CSI), third generation (3G), third generation partnership project (3GPP), long term evolution (LTE), advanced LTE (LTE-A), fourth generation (4G) and fifth generation (5G)). However, this is only an example, and the communication interface 120 can use at least one communication module in various communication modules.

[0069] The user interface 130 is a component for receiving user instructions for controlling the electronic device 100. The user interface 130 may be implemented as a device such as a button, a touch pad, a mouse, or a keyboard, or may also be implemented as a touch screen capable of performing both a display function and a manipulation input function. Here, the button may be any type of button, such as a mechanical button, a touch pad, or a wheel formed in a specific area (such as a front side, a side edge, or a rear side) outside the body of the electronic device 100. The electronic device 100 may obtain various user inputs through the user interface 130.

[0070] The speaker 140 may not only output various types of audio data processed by the input / output interface but also output various notification sounds or voice messages.

[0071] The microphone 150 may acquire voice data such as user voice. For example, the microphone 150 may be integrally formed with the electronic device 100 in an upward, forward, or lateral direction. The microphone 150 may include various components such as a microphone that collects the user voice in an analog form, an amplifier circuit that amplifies the collected user voice, an analog-to-digital (A / D) conversion circuit that samples the amplified user voice and converts the sampled user voice into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0072] The processor 160 may control the overall operation and functions of the electronic device 100. Specifically, the processor 160 is connected to components of the electronic device 100 including the memory 110, and may control the overall operation of the electronic device 100 by executing at least one instruction stored in the memory 110 as described above.

[0073] The processor 160 may be implemented in various ways. For example, the processor 160 may be implemented as at least one of an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, a hardware control logic, a hardware finite state machine (FSM), and a digital signal processor (DSP). Meanwhile, the term "processor 160" herein may be used to include a central processing unit (CPU), a graphics processing unit (GPU), a main processing unit (MPU), and the like.

[0074] The operation of the processor 160 for implementing various embodiments of the present disclosure may be implemented through the artificial intelligence model 111 and a plurality of modules.

[0075] Specifically, data related to the artificial intelligence model 111 and multiple modules according to the present disclosure may be stored in the memory 110. After accessing the memory 110 and loading the data related to the artificial intelligence model 111 and multiple modules into the memory or buffer in the processor 160, the processor 160 may use the artificial intelligence model 111 and multiple modules to implement various embodiments according to the present disclosure. At this time, the multiple modules may include a training data acquisition module 161, an embedding module 162, a training module 163, a synthesis module 164, and an update module 165.

[0076] However, the artificial intelligence model 111 and at least one of the plurality of modules according to the present disclosure may be implemented as hardware and included in the processor 160 in the form of a system on chip.

[0077] The training data acquisition module 161 can acquire multiple training data. In this case, the training data can be training data for training the artificial intelligence model 111.

[0078] For example, the training data acquisition module 161 may acquire audio data for training the artificial intelligence model 111 through the microphone 150. Meanwhile, the training data may be implemented as various types of data such as images and videos as well as audio data.

[0079] In addition, the training data acquisition module 161 can acquire a plurality of training data whose categories are marked. For example, when the plurality of training data are audio data, each of the plurality of training data can be marked as a human voice category, a music sound category, or an applause sound category.

[0080] Alternatively, the training data acquisition module 161 may acquire training data whose categories are not labeled. At this time, the training data acquisition module 161 may label one of the multiple categories according to the similarity between the features acquired from the training data.

[0081] The embedding module 162 may respectively obtain multiple embedding vectors for multiple training data that can be mapped to the embedding space. In this case, the multiple embedding vectors may respectively correspond to multiple training data.

[0082] Specifically, the embedding module 162 may obtain multiple embedding vectors by respectively extracting features from multiple training data.

[0083] Figure 3 is a flowchart for explaining a method of acquiring an embedding vector according to an embodiment of the present disclosure.

[0084] refer to Figure 3 When a plurality of training data are acquired in operation S310, in operation S320, the embedding module 162 may extract features from each of the plurality of training data. By analyzing the training data frame by frame in the time domain and the frequency domain, the embedding module 162 may extract features such as energy, mel frequency cepstral coefficient (MFCC), centroid, volume, power, subband energy, low short-time energy ratio, zero crossing rate, frequency centroid, frequency bandwidth, spectral flux, cepstrum change flux, or loudness.

[0085] Alternatively, the embedding module 162 may use a principal component analysis (PCA) or an independent component analysis (ICA) method to extract features of the plurality of training data.

[0086] Then, in operation S330, the embedding module 162 may use at least one of the extracted features to obtain a plurality of embedding vectors that may be mapped to the embedding space. At this time, the plurality of embedding vectors may be mapped to the embedding space, such as Figure 3 In this case, training data in the same category or similar training data may be located at a short distance, and training data in different categories or different training data may be located at a long distance. Here, each of the embedding vectors may correspond to each feature point shown in the embedding space.

[0087] Figure 4is a diagram for explaining an embedding space according to an embodiment of the present disclosure.

[0088] At the same time, reference Figure 4 , the multiple embedding vectors can be embedding vectors whose categories are labeled. For example, Figure 3 As shown, category 1, category 2, or category 3 may be labeled to each of the plurality of embedding vectors. In this case, category 1 may correspond to human speech, category 2 may correspond to musical sound, and category 3 may correspond to noise. Alternatively, category 1 may correspond to A's speech, category 2 may correspond to B's speech, and category 3 may correspond to C's speech.

[0089] In this case, the categories of the plurality of embedding vectors may correspond to the labeled categories of the plurality of training data. Meanwhile, the plurality of training data may be data whose categories are labeled, but this is only an example, and the training data acquisition module 161 may acquire a plurality of training data whose categories are not labeled. In this case, the embedding module 162 may label the categories of the plurality of embedding vectors (or the plurality of training data) based on the clusters formed by the embedding vectors in the embedding space.

[0090] Then, based on the multiple embedding vectors, the training module 163 can train the artificial intelligence model 111 to classify the multiple training data. In this case, the data input to the artificial intelligence model 111 can be the training data or the embedding vector obtained from the training data. In addition, the data output by the artificial intelligence model 111 can be the predicted category of the input data.

[0091] In this case, the training module 163 can use at least some of the training data (or embedding vectors) as criteria for determination to train the artificial intelligence model 111 through supervised learning. For example, the training module 163 can train the artificial intelligence model 111 in a supervised learning manner by using the embedding vector as an independent variable and using the class labeled to the embedding vector as a dependent variable.

[0092] Alternatively, the training module 163 can train the artificial intelligence model 111 through unsupervised learning to find the criteria for determining the category by autonomously learning using the training data (or embedding vector) without any specific guidance. In addition, the training module 163 can train the artificial intelligence model 111 through reinforcement learning, for example, using feedback on whether the result is correct based on the learning situation.

[0093] In addition, the training module 163 can train the artificial intelligence model 111 using, for example, a learning algorithm including error back propagation or gradient descent.

[0094] Then, when training data or embedding vectors are input, the trained artificial intelligence model 111 can classify the input data based on its position in the embedding space.

[0095] Meanwhile, the training module 163 may train the artificial intelligence model 111 , but this is merely an example, and the artificial intelligence model 111 may be a model trained by a separate external device or a separate external server and stored in the memory 110 .

[0096] In addition, the synthesis module 164 may identify misclassified data in the results output by the artificial intelligence model 111. In this case, the misclassified data may be data whose labeled category is different from the category predicted by the artificial intelligence model 111 after the data is input to the artificial intelligence model 111. For example, when the artificial intelligence model 111 receives an embedding vector to which "music sound" is labeled as input and outputs "human speech" as a classification result, the embedding vector of the "music sound" may be a misclassified embedding vector.

[0097] Furthermore, based on the identified misclassified data, the synthesis module 164 may retrain the artificial intelligence model 111 by additionally synthesizing training data.

[0098] Figure 5 Detailed description is a flowchart for explaining a method for obtaining a synthetic embedding vector according to an embodiment of the present disclosure. Figure 6a , Figure 6b , Figure 6c , Figure 6d and Figure 6e is a diagram for explaining a method of obtaining a synthetic embedding vector according to various embodiments of the present disclosure.

[0099] refer to Figure 5 , in operation S510 , the synthesis module 164 may identify an embedding vector misclassified by the artificial intelligence model 111 among the plurality of embedding vectors.

[0100] refer to Figure 6a , multiple embedding vectors obtained from the training data can be mapped to the embedding space. In this case, the label category of each of the multiple embedding vectors can be as follows Figure 6a The illustrated category is category 1, category 2, or category 3. Here, the labeled category of the first embedding vector 610 among the plurality of embedding vectors may be category 2.

[0101] Furthermore, the category predicted by the artificial intelligence model 111 trained based on multiple embedding vectors can be as follows Figure 6b Category 1, Category 2, or Category 3 as indicated.

[0102] In this case, the first embedding vector 610 of the plurality of embedding vectors may be input to the artificial intelligence model 111, and the category predicted by the artificial intelligence model 111 may be category 1. That is, the labeled category of the first embedding vector 610 may be category 2, and the predicted category of the first embedding vector 610 may be category 1. In this case, the synthesis module 164 may identify the first embedding vector 610 as an incorrectly classified embedding vector. At the same time, when there are multiple incorrectly classified embedding vectors, the synthesis module 164 may identify multiple incorrectly classified embedding vectors.

[0103] Then, in operation S520 , the synthesis module 164 may identify an embedding vector that is closest to the misclassified embedding vector in the embedding space.

[0104] In this case, the closest embedding vector may be an embedding vector whose category is predicted to be the same as the labeled category of the misclassified embedding vector. That is, the synthesis module 164 may identify an embedding vector that is close to the misclassified embedding vector among the embedding vectors whose labeled category (or predicted category) is the same as the labeled category of the misclassified embedding vector.

[0105] refer to Figure 6b , when the first embedding vector 610 is identified as an incorrectly classified embedding vector, the synthesis module 164 may identify the second embedding vector 620 located at the closest distance to the first embedding vector 610 in the embedding space as the embedding vector closest to the incorrectly classified embedding vector among the embedding vectors whose categories are predicted to be the same as the labeled category (i.e., category 1) of the first embedding vector 610.

[0106] In addition, the closest embedding vector may be an embedding vector successfully classified by the artificial intelligence model 111. That is, the labeled category of the closest embedding vector may be the same as the predicted category of the closest embedding vector. In other words, the synthesis module 164 may identify an embedding vector that is closest to the misclassified embedding vector among the multiple embedding vectors successfully classified by the artificial intelligence model 111.

[0107] Then, in operation S530, the synthesis module 164 may obtain a synthesized embedding vector corresponding to a path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector in the embedding space. Meanwhile, the path between embedding vectors herein may refer to the shortest path connecting the embedding vectors to each other in the embedding space. That is, the path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector may refer to the shortest path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector in the embedding space.

[0108] Specifically, refer to Figure 6b, the synthesis module 164 may identify a path 630 connecting the misclassified embedding vector 610 and the second embedding vector 620 that is closest to the misclassified embedding vector 610 .

[0109] At this time, the synthesis module 164 can obtain a synthesized embedding vector located at a point in the embedding space that is located on a path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector by synthesizing the misclassified embedding vector and the embedding vector closest to the misclassified embedding vector.

[0110] refer to Figure 6c , synthesis module 164 may identify a path 630 connecting first embedding vector 610 and second embedding vector 620 to each other, and obtain a synthesized embedding vector 640 at a point located on path 630. At this time, synthesis module 164 may obtain synthesized embedding vector 640 by synthesizing first embedding vector 610 and second embedding vector 620, but this is only an example, and synthesis module 164 may obtain an embedding vector at a point located on path 630 based on information related to path 630.

[0111] Furthermore, the synthesized embedding vector 640 may be located at a center point between the first embedding vector 610 and the second embedding vector 620 in the embedding space, but this is merely an example and the synthesized embedding vector 640 may be located at any point on the path.

[0112] Meanwhile, the synthesis module 164 may acquire one synthesized embedding vector located at a point of the path, but this is merely an example, and the synthesis module 164 may acquire a plurality of synthesized embedding vectors located at a plurality of points of the path.

[0113] refer to Figure 6d , when the path 630 is identified, the synthesis module 164 can obtain multiple synthesized embedding vectors 641, 642 and 643 at multiple points located on the path 630.

[0114] Meanwhile, when an incorrectly classified embedding vector is identified, the synthesis module 164 may identify one embedding vector closest to the incorrectly classified embedding vector, but this is only an example, and the synthesis module 164 may identify multiple embedding vectors close to the incorrectly classified embedding vector in order of short distance. At this time, the synthesis module 164 may obtain one or more synthesized embedding vectors, each of which is located at a point on a path connecting the incorrectly classified embedding vector to each of the multiple identified embedding vectors.

[0115] refer to Figure 6e, the synthesis module 164 may identify a plurality of embedding vectors 620, 621, and 622 in the order in which the plurality of embedding vectors are close to the first embedding vector 610. Then, the synthesis module 164 may identify a plurality of paths 630, 631, and 632 between the first embedding vector and the plurality of embedding vectors 620, 621, and 622, respectively.

[0116] Then, still refer to Figure 6e , the synthesis module 164 can respectively obtain multiple synthetic embedding vectors 640, 641 and 642 located on multiple paths 630, 631 and 632.

[0117] Alternatively, synthesis module 164 may identify at least one embedding vector within a predetermined distance from the misclassified embedding vector in the embedding space. At this time, synthesis module 164 may obtain one or more synthesized embedding vectors located at points on a path connecting each of the at least one embedding vector and the misclassified embedding vector to each other.

[0118] At the same time, the synthesis module 164 can identify an incorrectly classified embedding vector and obtain a synthesized embedding vector based on the identified incorrectly classified embedding vector, but this is only an example, and the synthesis module 164 can identify multiple incorrectly classified embedding vectors and obtain multiple synthesized embedding vectors based on the multiple incorrectly classified embedding vectors.

[0119] At this time, similar to the above method, the synthesis module 164 can identify an embedding vector close to each of the multiple misclassified embedding vectors, and obtain multiple synthesized embedding vectors located on multiple paths, each of the multiple paths connecting each of the multiple misclassified embedding vectors to each of the embedding vectors close to the misclassified embedding vector.

[0120] At the same time, when the synthesized embedding vector is obtained, the synthesis module 164 can mark the category of the obtained synthesized embedding vector. At this time, the synthesis module 164 can mark the category of the synthesized embedding vector as the same as the category of the closest embedding vector. Alternatively, the synthesis module 164 can mark the category of the synthesized embedding vector as the same as the labeled category of the misclassified embedding vector.

[0121] Then, the update module 165 can update the artificial intelligence model 111 by adding the synthetic embedding vector to the training data and retraining the artificial intelligence model 111. At this time, the synthetic embedding vector added to the training data can be an embedding vector whose category is labeled. In addition, the training module 163 can update the artificial intelligence model 111 using methods such as supervised learning, unsupervised learning, error back propagation, or gradient descent as described above.

[0122] At the same time, when the performance of the trained (or retrained) artificial intelligence model is lower than or equal to a predetermined standard, the update module 165 retrains and updates the artificial intelligence model 111 by re-identifying the embedding vectors misclassified by the artificial intelligence model and additionally obtaining synthetic embedding vectors.

[0123] Figure 7 Detailed description is a flowchart for explaining a method for retraining an artificial intelligence model according to an embodiment of the present disclosure.

[0124] Specifically, refer to Figure 7 , when the artificial intelligence model 111 is trained (or retrained) in operation S710, in operation S720, the update module 165 may identify whether the performance of the trained artificial intelligence model 111 is lower than or equal to a predetermined standard. Using some of the training data as verification data, the update module 165 may identify the performance of the artificial intelligence model 111 based on whether the labeled category included in the verification data is similar to the predicted category, and identify whether the identified performance is higher than or equal to a reference value.

[0125] For example, among all the data input to the artificial intelligence model 111, 80% of the data may be data successfully classified by the artificial intelligence model 111, and 20% of the data may be misclassified data. At this time, the update module 165 can identify that the performance of the artificial intelligence model 111 is lower than or equal to the predetermined standard by comparing the classification success rate of the artificial intelligence model 111 (i.e., 80%) with the predetermined standard (i.e., 90%).

[0126] When it is identified that the performance of the artificial intelligence model 111 is lower than or equal to the predetermined standard (yes (Y) at operation 720 ), in operation S730 , the updating module 165 may re-identify the misclassified embedding vector.

[0127] Then, similar to the above method, in operation S740, the update module 165 can obtain a synthetic embedding vector located at a point on the path connecting the re-identified misclassified embedding vector to the embedding vector close to the re-identified misclassified embedding vector. Then, in operation S750, the update module 165 can update the artificial intelligence model 111 by adding the synthetic embedding vector corresponding to the path connecting the re-identified misclassified embedding vector to the embedding vector closest to the misclassified embedding vector to the training data.

[0128] Meanwhile, the artificial intelligence model 111 may be stored in the memory 110 of the electronic device 100, but this is only an example, and the artificial intelligence model may be stored in an external device. Then, the electronic device 100 may obtain an embedding vector for training or updating the artificial intelligence model, and send the embedding vector to the external device to update the artificial intelligence model stored in the external device.

[0129] Figure 8 is a block diagram for explaining the configuration of an electronic apparatus and external devices according to an embodiment of the present disclosure.

[0130] Specifically, refer to Figure 8 , the electronic device 100 can communicate with the external device 200 to send / receive data. At this time, the electronic device 100 can communicate directly with the external device 200, but this is only an example, and the electronic device 100 can communicate with the external device 200 via a separate external device. For example, in the case where the electronic device 100 is a headset, the electronic device 100 can communicate with the external device 200 as a server through a smartphone. Alternatively, in the case where the electronic device 100 is a headset and the external device is a smartphone, the electronic device 100 can communicate through Bluetooth. TM The module communicates with an external device 200 which is a smartphone.

[0131] The external device 200 may include a memory 210, a communication interface 220, and a processor 230. In this case, the memory 210 may store an artificial intelligence model 211 that classifies input data when the data is input.

[0132] Fig. 9 is a timing diagram for explaining operations of an electronic apparatus, an external device, and a server according to an embodiment of the present disclosure.

[0133] refer to Fig. 9 In operation S910, the electronic device 100 may acquire training data. For example, the electronic device 100 may acquire voice data as training data through the microphone 150.

[0134] Alternatively, the electronic device 100 may acquire training data from a separate external sensor. For example, in the case where the electronic device 100 is a smartphone, the electronic device 100 may receive recorded voice data from an earphone including a microphone.

[0135] In operation S920, based on the acquired training data, the electronic device 100 may acquire an embedding vector. The electronic device 100 may acquire an embedding vector including information on features of the speech data by extracting features from the training data.

[0136] Therefore, in operation S930, the electronic device 100 may transmit the acquired embedding vector to the external device 200. That is, the electronic device 100 may transmit the embedding vector to the external device 200 instead of transmitting the original training data to the external device 200, and thus, the electronic device 100 does not need to transmit the training data including the user's personal information (e.g., audio data in which voice is recorded) to the external device 200.

[0137] When receiving the embedding vector, in operation S940, the external device 200 may input the received embedding vector to the artificial intelligence model 211 to identify an incorrectly classified embedding vector.

[0138] In operation S950, similar to the above method, based on the misclassified embedding vector, the external device 200 may acquire a synthetic embedding vector located on a path connecting the misclassified embedding vector to an embedding vector close to the misclassified embedding vector.

[0139] In operation S960, the external device 200 may retrain the artificial intelligence model 211 using the synthesized embedding vector.

[0140] When the artificial intelligence model is retrained, the external device 200 may transmit the retrained artificial intelligence model 211 to the electronic apparatus 100 in operation S970 .

[0141] Fig.10a and Fig.10b is a diagram for explaining the performance of an artificial intelligence model according to various embodiments of the present disclosure.

[0142] refer to Fig.10a As the training of an AI model that classifies input data progresses, the validation loss for a particular class may increase, leading to overfitting. At this point, adding training data to address the overfitting problem requires a lot of time and resources.

[0143] On the contrary, reference Fig.10b , if the artificial intelligence model is trained or updated by additionally synthesizing training data according to the above method, the overfitting problem occurring in the artificial intelligence model can be solved.

[0144] Fig.11 is a flowchart for explaining a control method of the electronic device 100 according to an embodiment of the present disclosure.

[0145] In operation S1110 , the electronic device 100 may acquire a plurality of training data.

[0146] Then, in operation S1120, the electronic device 100 may respectively obtain a plurality of embedding vectors that can be mapped to the embedding space for the plurality of training data. In this case, the electronic device 100 may respectively obtain a plurality of embedding vectors by extracting features from the plurality of training data.

[0147] Then, in operation S1130, the electronic device 100 may train the artificial intelligence model 111 that classifies the plurality of training data based on the plurality of embedding vectors.

[0148] Then, in operation S1140, the electronic device 100 may identify an embedding vector misclassified by the artificial intelligence model among the plurality of embedding vectors.

[0149] Specifically, the electronic device 100 may specify a first embedding vector among the plurality of embedding vectors. Then, the electronic device 100 may identify whether the labeled category of the first embedding vector is different from the category predicted by the artificial intelligence model 111.

[0150] At this time, when the labeled category of the first embedding vector is the same as the predicted category of the first embedding vector, the electronic device 100 can identify the first embedding vector as a successfully classified embedding vector. When the first embedding vector is identified as a successfully classified embedding vector, the electronic device 100 can specify a second embedding vector from among the plurality of embedding vectors. Then, the electronic device 100 can identify whether the labeled category of the second embedding vector is different from the category predicted by the artificial intelligence model 111.

[0151] When the labeled category of the second embedding vector is different from the predicted category of the second embedding vector, the electronic device 100 may identify the second embedding vector as an incorrectly classified embedding vector. For example, the labeled category of the second embedding vector may be "music sound", and the category predicted by inputting the second embedding vector into the artificial intelligence model 111 may be "human speech". At this time, the electronic device 100 may identify the second embedding vector as an incorrectly classified embedding vector. Then, when the incorrectly classified embedding vector is identified, in operation S1150, the electronic device 100 may identify the embedding vector in the embedding space that is closest to the incorrectly classified embedding vector. At this time, the closest embedding vector may be the embedding vector successfully classified by the artificial intelligence model 111.

[0152] Then, in operation S1160, the electronic device 100 may obtain a synthetic embedding vector corresponding to a path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector in the embedding space. In this case, the synthetic embedding vector may be located at a point in the embedding space at the path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector.

[0153] Specifically, the electronic device 100 can use any of a variety of data synthesis methods to obtain a synthetic embedding vector. In this case, the data synthesis method can be a method using a model capable of generating a spectrogram or an original waveform. In this case, the model for data synthesis can be stored in the memory 110.

[0154] According to an embodiment, the electronic device 100 may generate data using a generative adversarial network (GAN).

[0155] Fig.12 Detailed description is a flowchart for explaining a method for obtaining a synthesized sample using a GAN according to an embodiment of the present disclosure.

[0156] refer to Fig.12 In operation S1210, the electronic device 100 may acquire a Gaussian noise vector. At this time, the Gaussian noise vector may be a vector including values ​​randomly acquired from a Gaussian probability distribution.

[0157] Then, in operation S1220, the electronic device 100 may obtain a synthesized sample by inputting the obtained Gaussian noise vector and the embedding vector into the GAN. That is, when the Gaussian noise vector and the embedding vector are input into the GAN, the GAN may output a synthesized sample.

[0158] At this time, the embedding vector input to the GAN may be at least one of the misclassified embedding vector and a vector closest to the misclassified embedding vector.

[0159] At the same time, the Gaussian noise vector includes values ​​randomly obtained from a Gaussian probability distribution, and at this time, the randomly obtained values ​​can provide data variability within a specific category. This randomness makes it possible to synthesize data more efficiently.

[0160] In addition, the synthesized sample may be a synthesized embedding vector, but this is only an example, and the electronic device 100 may obtain the synthesized embedding vector by extracting features from the synthesized sample. At this time, the synthesized sample may correspond to a point on the shortest path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector.

[0161] According to another embodiment, the electronic device 100 may synthesize data using a variational autoencoder (VAE) and a variational autodecoder (VAD).

[0162] Fig.13 is a flowchart for explaining a method of acquiring synthesized samples using VAE and VAD according to an embodiment of the present disclosure.

[0163] refer to Fig.13In operation S1310, the electronic device 100 may input sample data and an embedding vector into the VAE. Here, the sample data may be training data corresponding to the embedding vector. That is, the embedding vector input into the VAE may be an embedding vector obtained by extracting features from the sample data.

[0164] In addition, in operation S1320, the electronic device 100 may use the data output from the VAE to obtain sampled data. At this time, the sampled data may include values ​​randomly extracted from the data output from the VAE. Alternatively, the sampled data may be data obtained using values ​​randomly extracted from a Gaussian distribution or the like. The random value may provide data variability within a specific category. This randomness makes it possible to synthesize data more efficiently.

[0165] Thereafter, in operation S1330, the electronic device 100 may obtain a synthesized sample by inputting the acquired sampling data into the VAD. The synthesized sample may be a synthesized embedding vector, but this is only an example, and the electronic device 100 may obtain the synthesized embedding vector by extracting features from the synthesized sample. In addition, the synthesized sample may correspond to a point on the shortest path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector.

[0166] Meanwhile, the electronic device 100 may acquire synthesized data in various ways other than the method using GAN or VAE.

[0167] Then, in operation S1170, the electronic device 100 may retrain the artificial intelligence model 111 by adding the synthesized data to the training data.

[0168] Here, the synthesized data can be a synthesized embedding vector or a synthesized sample.

[0169] Meanwhile, when a plurality of synthesized data are acquired, the electronic device 100 may add data corresponding to points on the shortest path connecting the misclassified embedding vector to the embedding vector closest to the misclassified embedding vector in the plurality of synthesized data to the training data.

[0170] At the same time, the electronic device 100 may verify the synthesized data and add the synthesized data to the training data based on the verification result. Specifically, the electronic device 100 may compare the synthesized data with the training data pre-stored in the memory 110, and determine whether to add the synthesized data to the training data based on the comparison result. At this time, the electronic device 100 may obtain a value indicating the similarity between the pre-stored training data and the synthesized data, and when the obtained value indicating the similarity is greater than or equal to a predetermined value, the synthesized data is added to the training data.

[0171] Alternatively, the electronic device 100 may identify whether to add the synthesized data to the training data using a result value obtained by inputting the synthesized data into the artificial intelligence model 111. That is, the electronic device 100 may verify the artificial intelligence model 111 using the synthesized data and retrain the artificial intelligence model 111 based on the verification result.

[0172] Specifically, the electronic device 100 may identify whether the labeled category of the synthesized data is different from the category predicted by the artificial intelligence model 111. When the labeled category of the synthesized data is different from the category predicted by the artificial intelligence model 111, the electronic device 100 may retrain the artificial intelligence model 111 using the synthesized data.

[0173] Alternatively, the electronic device 100 may verify the artificial intelligence model 111 using multiple pieces of synthesized data, and retrain the artificial intelligence model 111 based on the verification result. Specifically, the electronic device 100 may identify the accuracy of the artificial intelligence model 111 based on the result value output by inputting the multiple pieces of synthesized data into the artificial intelligence model 111. At this time, when the accuracy is lower than or equal to a predetermined value, the electronic device 100 may retrain the artificial intelligence model 111 by adding the multiple pieces of synthesized data to the training data. Alternatively, the electronic device 100 may compare the identified accuracy with the accuracy of a second artificial intelligence model stored in the memory 110 or an external server, and retrain the artificial intelligence model 111 when the identified accuracy is lower than or equal to the accuracy of the second artificial intelligence model.

[0174] In addition, when the performance of the retrained artificial intelligence model 111 is lower than or equal to a predetermined standard, the electronic device 100 can update the artificial intelligence model 111 by re-identifying the embedding vector misclassified by the artificial intelligence model and obtaining a synthetic embedding vector corresponding to a path in the embedding space that connects the re-identified misclassified embedding vector to the embedding vector closest to the re-identified misclassified embedding vector. At the same time, the functions related to artificial intelligence according to the present disclosure can be operated by the processor 160 and the memory 110 of the electronic device 100.

[0175] The processor 160 may include one or more processors. In this case, the one or more processors may include a general-purpose processor such as a central processing unit (CPU) or an application processor (AP), a graphics-specific processor such as a graphics processing unit (GPU) or a visual processing unit (VPU), or an artificial intelligence-specific processor such as a neural processing unit (NPU) or a tensor processing unit (TPU).

[0176] As an embodiment of the present disclosure, in the case where the system on chip (SoC) included in the electronic device 100 includes a plurality of processors, the electronic device 100 may use a graphics-dedicated processor or an artificial intelligence-dedicated processor among the plurality of processors to perform operations related to artificial intelligence (e.g., operations related to learning or reasoning of an artificial intelligence model), and may use a general-purpose processor among the plurality of processors to perform general operations of the electronic device. For example, the electronic device 100 may use at least one of a GPU, a VPU, an NPU, and a TPU dedicated to convolution operations among the plurality of processors to perform operations related to artificial intelligence, and may use at least one of a CPU and an AP among the plurality of processors to perform general operations of the electronic device 100.

[0177] In addition, the electronic device 100 can use multiple cores (e.g., dual cores, quad cores, etc.) included in one processor to perform operations related to artificial intelligence functions. Specifically, the electronic device can use multiple cores included in the processor to perform convolution operations in parallel. One or more processors control the input data to be processed according to predefined operating rules or artificial intelligence models stored in the memory. The predefined operating rules or artificial intelligence models are created by learning.

[0178] Here, creation by learning means creating a predefined operating rule or artificial intelligence model with desired characteristics by applying a learning algorithm to a plurality of training data. Such learning can be performed in the device itself that performs artificial intelligence according to the present disclosure, or can be performed by a separate server / system.

[0179] The artificial intelligence model may include multiple neural network layers. Each layer has multiple weight values, and the layer operation is performed using the operation result of the previous layer and the operation between the multiple weight values. Examples of neural networks include convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), and deep Q networks, and unless otherwise specified herein, neural networks are not limited to the above examples.

[0180] A learning algorithm is a method of training a predetermined target device (e.g., a robot) using multiple training data to allow the predetermined target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning, and unless otherwise specified herein, learning algorithms are not limited to the above examples.

[0181] Meanwhile, the term "unit" or "module" used herein refers to a unit configured in hardware, software, or firmware, and can be used interchangeably with the terms "logic", "logic block", "component", "circuit", etc., for example. A "unit" or "module" can be an integrated component, or a minimum unit for performing one or more functions or a part thereof. For example, a module can be configured as an application specific integrated circuit (ASIC).

[0182] Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium (e.g., a computer-readable storage medium). A machine is a device that calls stored instructions from a storage medium and can operate according to the called instructions, and may include an electronic device 100 according to an embodiment disclosed herein. If the instruction is executed by a processor, the function corresponding to the instruction may be executed directly by the processor or using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" simply means that the storage medium is a tangible device that does not include a signal, regardless of whether the data is stored in the storage medium semi-permanently or temporarily.

[0183] According to an embodiment, the method according to various embodiments disclosed herein may be included in a computer program product for supply. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be distributed via an application store (e.g., PlayStore). TM If the computer program product is distributed online, at least a part of the computer program product may be temporarily generated or at least temporarily stored in a storage medium (such as a memory of a manufacturer's server, an application store's server, or a relay server).

[0184] Each of the components (e.g., modules or programs) according to various embodiments may include a single entity or multiple entities, and some of the above-mentioned subcomponents may be omitted, or other subcomponents may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, and the integrated entity may perform the same or similar functions performed by the corresponding components before integration. According to various embodiments, the operations performed by modules, programs or other components may be performed sequentially, in parallel, repeatedly or heuristically, or at least some of the operations may be performed or omitted in a different order, or other operations may be added.

[0185] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. An electronic device, comprising: A memory storing at least one instruction; and a processor connected to the memory to control the electronic device, Wherein, by executing the at least one instruction, the processor is configured to: Acquire training data including multiple pieces of training data; Based on the training data, respectively obtain a plurality of embedding vectors for the plurality of training data that can be mapped to the embedding space; Based on the multiple embedding vectors, training an artificial intelligence model for classifying the multiple training data; identifying misclassified embedding vectors among the plurality of embedding vectors that are misclassified by the artificial intelligence model; identifying an embedding vector in the embedding space that is closest to the misclassified embedding vector; obtaining a synthetic embedding vector in the embedding space corresponding to a path connecting the misclassified embedding vector to the embedding vector that is closest to the misclassified embedding vector; and The artificial intelligence model is retrained by adding the synthetic embedding vector to the training data.

2. The electronic device according to claim 1, wherein: By executing the at least one instruction, the processor is further configured to: The synthesized embedding vector at a point on the path in the embedding space is obtained by synthesizing the misclassified embedding vector and the embedding vector that is closest to the misclassified embedding vector.

3. The electronic device according to claim 1, wherein: The misclassified embedding vector includes an embedding vector whose labeled category is different from the category predicted by the artificial intelligence model after the embedding vector is input into the artificial intelligence model.

4. The electronic device according to claim 1, wherein: The embedding vectors that are closest to the misclassified embedding vectors include embedding vectors successfully classified by the artificial intelligence model.

5. The electronic device according to claim 1, wherein: By executing the at least one instruction, the processor is further configured to: The category of the synthesized embedding vector is labeled with the same category as the labeled category of the misclassified embedding vector.

6. The electronic device according to claim 1, wherein: By executing the at least one instruction, the processor is further configured to: The multiple embedding vectors are obtained by respectively extracting features from the multiple pieces of training data.

7. The electronic device according to claim 1, wherein: By executing the at least one instruction, the processor is further configured to: re-identifying embedding vectors misclassified by the artificial intelligence model based on the performance of the retrained artificial intelligence model being less than or equal to a predetermined criterion; and The artificial intelligence model is updated by obtaining a synthetic embedding vector in the embedding space corresponding to a path connecting the re-identified misclassified embedding vector to the embedding vector that is closest to the re-identified misclassified embedding vector.

8. A control method for an electronic device, the control method comprising: Acquire training data including multiple pieces of training data; Based on the training data, respectively obtain a plurality of embedding vectors for the plurality of training data that can be mapped to the embedding space; Based on the multiple embedding vectors, training an artificial intelligence model for classifying the multiple training data; identifying misclassified embedding vectors among the plurality of embedding vectors that are misclassified by the artificial intelligence model; identifying an embedding vector in the embedding space that is closest to the misclassified embedding vector; obtaining a synthetic embedding vector in the embedding space corresponding to a path connecting the misclassified embedding vector to the embedding vector that is closest to the misclassified embedding vector; and The artificial intelligence model is retrained by adding the synthetic embedding vector to the training data.

9. The control method according to claim 1, wherein: When obtaining the synthesized embedding vector, the synthesized embedding vector at the point of the path in the embedding space is obtained by synthesizing the misclassified embedding vector and the embedding vector closest to the misclassified embedding vector.

10. The control method according to claim 1, wherein: The misclassified embedding vector includes an embedding vector whose labeled category is different from the category predicted by the artificial intelligence model after the embedding vector is input into the artificial intelligence model.

11. The control method according to claim 1, wherein: The embedding vectors that are closest to the misclassified embedding vectors include embedding vectors successfully classified by the artificial intelligence model.

12. The control method according to claim 1, further comprising: The category of the synthesized embedding vector is labeled with the same category as the labeled category of the misclassified embedding vector.

13. The control method according to claim 1, wherein: When obtaining the multiple embedding vectors, the multiple embedding vectors are obtained by respectively extracting features from the multiple training data.

14. The control method according to claim 1, further comprising: re-identifying embedding vectors misclassified by the artificial intelligence model based on the performance of the retrained artificial intelligence model being less than or equal to a predetermined criterion; and The artificial intelligence model is updated by obtaining a synthetic embedding vector in the embedding space corresponding to a path connecting the re-identified misclassified embedding vector to the embedding vector that is closest to the re-identified misclassified embedding vector.

15. A non-transitory computer-readable recording medium comprising a program for executing a control method of an electronic device, the control method comprising: Acquire training data including multiple pieces of training data; Based on the training data, respectively obtain a plurality of embedding vectors for the plurality of training data that can be mapped to the embedding space; Based on the multiple embedding vectors, training an artificial intelligence model for classifying the multiple training data; identifying misclassified embedding vectors among the plurality of embedding vectors that are misclassified by the artificial intelligence model; identifying an embedding vector in the embedding space that is closest to the misclassified embedding vector; obtaining a synthetic embedding vector in the embedding space corresponding to a path connecting the misclassified embedding vector to the embedding vector that is closest to the misclassified embedding vector; and The artificial intelligence model is retrained by adding the synthetic embedding vector to the training data.

Images