Apparatus and method for processing natural language using boosting keyword sets and suppressing keyword sets

Abstract

To provide an apparatus and method of processing a natural language using a boosting keyword set and suppressing keyword set.SOLUTION: A natural language processing method performed in an electronic device including at least one processor and at least one memory storing commands to be executed by the at least one processor includes the steps of: acquiring a boosting keyword set including at least one boosting keyword that is an object of generation boost when generating a sentence using an artificial neural network model; acquiring a suppressing keyword set including at least one suppressing keyword that is an object of generation suppression when generating a sentence using the artificial neural network model; and generating sentences through the artificial neural network model based on the boosting keyword set and the suppressing keyword set.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a technique for processing natural language using a keyword set. [Background Art]

[0002] Recently, in the field of natural language processing technology, there is a trend of using an LLM (Large Language Model) as a base model, and then performing post-processing techniques such as fine-tuning or few-shot learning in accordance with a user's purpose.

[0003] However, according to the prior art, when the base LLM model is a model trained with a foreign language, there have been problems such as foreign language appearing in fields that require Korean generation, or unnecessary keywords or keywords undesired by the user being generated because the user cannot modify the training data set of the base LLM model.

[0004] Accordingly, while conventional LLM models are used as base models, demand is increasing for a technique capable of controlling sentences generated in an inference process according to the types of keywords desired by a user.

[0005] Korean Registered Patent No. 10-2668859 discloses a "natural language processing-based control system, an operation method thereof, and a communication method thereof". [Prior Art Documents] [Patent Documents]

[0006] [Patent Document 1] Korean Patent Laid-Open Publication No. 10-2023-0112514 [Brief Summary of the Invention] [Problem to be Solved by the Invention]

[0007] This disclosure provides a technique for processing natural language using boosting keyword sets and suppressing keyword sets. [Means for solving the problem]

[0008] As one aspect of this disclosure, a natural language processing method may be proposed. The method is a natural language processing method performed in an electronic device including one or more processors and one or more memories storing instructions to be executed by the one or more processors, and may include the steps of: acquiring a boosting keyword set, wherein the boosting keyword set includes at least one boosting keyword that is subject to generation promotion when generating text using an artificial neural network model; acquiring an inhibitory keyword set, wherein the inhibitory keyword set includes at least one inhibitory keyword that is subject to generation suppression when generating text using an artificial neural network model; and generating text through an artificial neural network model based on the boosting keyword set and the inhibitory keyword set.

[0009] Another aspect of this disclosure may be proposed: an electronic device for natural language processing. The electronic device includes one or more processors and one or more memories storing instructions to be executed by the one or more processors, the one or more processors acquiring a boosting keyword set, the boosting keyword set including at least one boosting keyword to be promoted when generating text using an artificial neural network model, acquiring a suppression keyword set, the suppression keyword set including at least one suppression keyword to be suppressed when generating text using an artificial neural network model, and generating text through an artificial neural network model based on the boosting keyword set and the suppression keyword set.

[0010] Another aspect of this disclosure may be proposed of a non-temporary computer-readable recording medium on which instructions for natural language processing are recorded. In a non-temporary computer-readable recording medium on which instructions for one or more processors to perform an operation are recorded, the instructions may cause the one or more processors to perform the steps of: acquiring a boosting keyword set, wherein the boosting keyword set includes at least one boosting keyword that is to be promoted when generating text using an artificial neural network model; acquiring an inhibitory keyword set, wherein the inhibitory keyword set includes at least one inhibitory keyword that is to be suppressed when generating text using an artificial neural network model; and generating text through an artificial neural network model based on the boosting keyword set and the inhibitory keyword set. [Effects of the Invention]

[0011] The natural language processing method disclosed herein can improve natural language processing speed.

[0012] The natural language processing method disclosed herein can generate text by adjusting the generation probability according to the type of keyword.

[0013] The natural language processing method disclosed herein can increase the probability that the generated text contains boosting keywords.

[0014] The natural language processing method disclosed herein can reduce the probability that the generated text contains suppression keywords. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows a system including a server (100), a user terminal (200), and a communication network (300) according to one embodiment of the present disclosure. [Figure 2] This is a block diagram of a server (100) according to one embodiment of the contents disclosed herein. [Figure 3] This is a block diagram of a user terminal (200) according to one embodiment of the contents disclosed herein. [Figure 4] This figure illustrates a flowchart of a natural language processing method according to one embodiment of the present disclosure. [Figure 5] This is a conceptual diagram illustrating a keyword tryout according to one embodiment of the present disclosure. [Figure 6] This diagram illustrates a flowchart of a first embodiment that generates text using keyword tryouts. [Figure 7] This diagram illustrates a flowchart of a second embodiment that generates text using keyword tryouts. [Figure 8] This diagram illustrates a flowchart of a third embodiment that generates text using keyword tryouts. [Modes for carrying out the invention]

[0016] <Summary of the Invention>

[0017] In one embodiment, the boosting keyword set is a keyword set relating to a first language, and the suppression keyword set may include a keyword set relating to a second language.

[0018] In one embodiment, the boosting keyword set or the suppression keyword set may be generated based on the word distribution in publicly available data relating to the target language and the word distribution in user-inputted unique data.

[0019] In one embodiment, the boosting keyword set may include words that appear in the public data at a frequency lower than a first threshold frequency, but that appear in the proprietary data at a frequency higher than a second threshold frequency.

[0020] In one embodiment, the suppression keyword set may include words that appear in the public data at a frequency higher than the third threshold frequency, and that appear in the proprietary data at a frequency lower than the fourth threshold frequency.

[0021] In one embodiment, the step of generating text through the artificial neural network model may include the step of calculating the generation probability for each of a plurality of tokens based on the output of the artificial neural network model for an input token sequence, and the step of determining the subsequent token.

[0022] In one embodiment, the generation probability for each of the plurality of tokens may be calculated to be different from one another according to the classification of each token.

[0023] In one embodiment, the generation probability for each of the plurality of tokens may be calculated using a first probability distribution adjustment parameter (factor) to increase the generation probability when the token is included in the token set of the boosting keyword set, and a second probability distribution adjustment parameter to decrease the generation probability when the token is included in the token set of the suppression keyword set.

[0024] In one embodiment, the step of generating text through the artificial neural network model may be performed using a keyword trie comprising one or more nodes, each of which may include a keyword state value relating to a token and a token sequence that sequentially includes tokens from each node included in the path from the root node to the current node.

[0025] In one embodiment, the keyword try may be generated based on the boosting keyword set or the suppression keyword set.

[0026] In one embodiment, the step of generating text through the artificial neural network model includes the steps of generating a first token sequence using a first probability distribution adjustment parameter, generating a second token sequence using the first and second probability distribution adjustment parameters, and replacing one of the first and second token sequences with the other token sequence if a predetermined condition is met, wherein the predetermined condition may be a condition whose validity is determined based on the keyword try.

[0027] In one embodiment, the step of replacing one of the first token sequence and the second token sequence with the other token sequence when the predetermined conditions are met may include the step of replacing the first token sequence with the second token sequence if it is determined that the first token sequence contains a suppression keyword, and the step of replacing the second token sequence with the first token sequence if it is determined that the first token sequence contains a boosting keyword or does not contain a suppression keyword.

[0028] In one embodiment, the step of generating text through the artificial neural network model may include the step of generating a first token sequence using a first probability distribution adjustment parameter, and, if it is determined that the first token sequence contains a suppression keyword, the step of generating a second token sequence using the first probability distribution adjustment parameter and the second probability distribution adjustment parameter.

[0029] In one embodiment, the step of generating text through the artificial neural network model may include the steps of generating multiple candidate token sequences by determining multiple subsequent tokens for each of N (where N is a natural number of 2 or more) token sequences using the artificial neural network model; calculating the cumulative probability for each of the multiple candidate token sequences according to a predetermined calculation method; and determining N token sequences from the multiple candidate token sequences based on the cumulative probability.

[0030] In one embodiment, the step of calculating the cumulative probability for each of a plurality of candidate token sequences according to the predetermined calculation method may include the step of increasing the cumulative probability if it is determined that the candidate token sequence contains a boosting keyword, and the step of decreasing the cumulative probability if it is determined that the candidate token sequence contains a suppression keyword.

[0031] <Detailed description of the invention>

[0032] The various embodiments described herein are illustrative for the purpose of clearly illustrating the technical concept of this disclosure and are not intended to limit it to any particular embodiment. The technical concept of this disclosure includes various modifications, equivalents, alternatives, and embodiments selectively combined from all or part of the embodiments described herein. Furthermore, the scope of rights relating to the technical concept of this disclosure is not limited to the various embodiments shown below or their specific descriptions.

[0033] Unless otherwise defined, terms used herein, including technical or scientific terms, may have meanings that are generally understood by a person with ordinary skill in the art to which this disclosure pertains.

[0034] The expressions "includes," "may include," "possess," "may possess," "have," and "may have" used in this book mean that the feature in question (e.g., function, operation, or component) exists, but do not exclude the existence of other additional features. In other words, these expressions should be understood as open-ended terms that inherently include the possibility of including other embodiments.

[0035] The singular expressions used in this book may imply plural meanings unless the context indicates otherwise, and this also applies to singular expressions used in claims.

[0036] The terms "first," "second," or "first," "second," etc., used in this document, unless otherwise indicated by context, are used to distinguish one object from another when referring to multiple similar objects, and do not limit the order or importance of those objects. For example, the multiple keywords in this disclosure can be distinguished from each other by being expressed as "first keyword," "second keyword," etc. Similarly, terms used in this disclosure, such as "threshold frequency" and "probability distribution adjustment parameter," can be distinguished from each other by expressions such as "first," "second," etc.

[0037] Expressions used in this book such as "A, B, and C," "A, B, or C," "at least one of A, B, and C," or "at least one of A, B, or C" can mean the listed items or all possible combinations of the listed items. For example, "at least one of A or B" could include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0038] As used in this book, the term "part" may refer to software or hardware components such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). However, "part" is not limited to hardware and software. A "part" may be configured to be stored in an addressable storage medium, or to run one or more processors. In one embodiment, a "part" may include components such as software components, object-oriented software components, class components, task components, as well as processors, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0039] As used in this book, the expression "based on" is used to describe one or more factors that influence the act or action of a decision, judgment, or action described in the phrase or sentence containing the expression, and the expression does not exclude additional factors that influence the act or action of that decision, judgment, or action.

[0040] As used in this book, the expression that one component (e.g., component 1) is "linked" or "connected" to another component (e.g., component 2) may mean not only that component is directly linked or connected to the other component, but also that it is linked or connected through a new other component (e.g., component 3).

[0041] The expression "configured to" as used in this book can mean, depending on the context, "set to do," "capable of doing," "modified to do," "made to do," or "capable of doing." This expression is not limited to the meaning of "specifically designed in terms of hardware." For example, a processor configured to perform a specific operation could mean a generic-purpose processor capable of performing that specific operation by running software, or a special-purpose computer structured through programming to perform that specific operation.

[0042] In this disclosure, artificial intelligence (AI) refers to technologies that mimic human learning, reasoning, and perceptual abilities and realize them using computers, and may include the concepts of machine learning and symbolic logic. Machine learning (ML) can be an algorithmic technology that classifies or learns the features of input data on its own. Artificial intelligence technologies can analyze input data as machine learning algorithms, learn the results of that analysis, and make judgments or predictions based on those learned results. Furthermore, technologies that utilize machine learning algorithms to mimic the cognitive and judgment functions of the human brain can also be understood as falling under the category of artificial intelligence. For example, this may include technologies such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0043] In this disclosure, machine learning may mean the process of training a neural network model using experience in processing data. Through machine learning, computer software can improve its own data processing capabilities. A neural network model is constructed by modeling the correlations between data, and these correlations can be represented by multiple parameters. Machine learning can be defined as the process by which an artificial neural network model extracts and analyzes features from given data to derive correlations between the data, and then optimizes the parameters of the neural network model by repeating this process. For example, an artificial neural network model can learn the mapping (correlation) between inputs and outputs for data given in input-output pairs. Alternatively, even if only input data is given, an artificial neural network model can derive regularities between the given data and learn their relationships.

[0044] In this disclosure, an artificial neural network, an artificial intelligence learning model, a machine learning model, or an artificial neural network model may be designed to implement the structure of the human brain on a computer and may include multiple network nodes that mimic human neurons and have weights. The multiple network nodes may be interconnected, mimicking the synaptic activity of neurons where neurons exchange signals via synapses. In an artificial neural network, the multiple network nodes may be located in layers of different depths and exchange data according to convolutional connectivity. An artificial neural network may be, for example, an artificial neural network model, a convolutional neural network model, etc.

[0045] Various embodiments of this disclosure will be described below with reference to the attached drawings. In the attached drawings and the description thereof, identical or substantially equivalent components may be denoted by the same reference numerals. In the following description of various embodiments, redundant descriptions of identical or corresponding components may be omitted, but this does not mean that the component is not included in that embodiment.

[0046] Figure 1 shows a system including a server (100), a user terminal (200), and a communication network (300) according to one embodiment of the present disclosure. The server (100) and the user terminal (200) can exchange information with each other via the communication network (300).

[0047] The server (100) may be an electronic device that performs the natural language processing operations related to this disclosure. The server (100) is an electronic device that transfers information to a wired or wirelessly connected user terminal (200) or transfers the results of natural language processing, and may be, for example, an application server, a proxy server, a cloud server, etc.

[0048] The user terminal (200) may be the terminal of a user attempting to receive the results of natural language processing. The user terminal (200) may be at least one of the following: a smartphone, a tablet computer, a personal computer (PC), a mobile phone, a personal digital assistant (PDA), an audio player, or a wearable device. The communication network (300) may include all wired and wireless communication networks.

[0049] The communication network (300) may be configured to exchange data between a server (100) and a user terminal (200). The wired communication network may include, for example, a communication network using USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface) (registered trademark), RS-232 (Recommended Standard-232), or POTS (Plain Old Telephone Service). Wireless communication networks may include, for example, communication networks relating to technologies such as eMBB (enhanced Mobile Broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (Long-Term Evolution), LTE-A (LTE Advance), NR (New Radio), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (Code Division Multiple Access), WCDMA® (Wideband CDMA), WiBro (Wireless Broadband), WiFi (Wireless Fidelity), Bluetooth® (Registered Trademark), NFC (Near Field Communication), GPS (Global Positioning System), or GNSS (Global Navigation Satellite System). The communication network (300) in this specification may include, without limitation, a variety of communication networks that enable data exchange between multiple entities or devices, and is not limited to the examples given above.

[0050] In the disclosures of this specification, when describing the configuration or operation of a device, the term "device" may be used to refer to the device being described, and the term "external device" may be used to refer to a device that exists externally from the perspective of the device being described. For example, when a server (100) is described as a "device," a user terminal (200) may be called an "external device" from the perspective of the server (100). Also, for example, when a user terminal (200) is described as a "device," a server (100) may be called an "external device" from the perspective of the user terminal (200). In other words, the server (100) and the user terminal (200) may be called a "device" and an "external device," respectively, or they may be called an "external device" and a "device," respectively, depending on the perspective of their respective operating entities.

[0051] Figure 2 is a block diagram of a server (100) according to one embodiment of the contents disclosed herein. The server (100) may include one or more processors (110), communication interfaces (120), or memory (130) as components. In some embodiments, at least one of these components of the server (100) may be omitted, or other components may be added to the server (100). In some embodiments, some components may be implemented as an integrated entity or as one or more entities, either additionally or alternatively. At least some of the internal or external components of the server (100) may be interconnected via buses, GPIO (General Purpose Input / Output), SPI (Serial Peripheral Interface), MIPI (Mobile Industry Processor Interface), etc., to exchange data or signals.

[0052] One or more processors (110) may be referred to as processor (110). The term processor (110) can mean a collection of one or more processors unless the context clearly indicates otherwise. A processor (110) can drive software (e.g., instructions, programs, etc.) to control at least one component of a server (100) connected to the processor (110). A processor (110) can also perform various operations such as calculations, processing, data generation, or manipulation. A processor (110) can also load data, etc., from memory (130) or store data, etc., in memory (130).

[0053] The communication interface (120) can perform wireless or wired communication between the server (100) and other devices (e.g., a user terminal (200) or another server). For example, the communication interface (120) can perform wireless communication in accordance with methods such as eMBB, URLLC, MMTC, LTE, LTE-A, NR, UMTS, GSM, CDMA, WCDMA, WiBro, WiFi, Bluetooth, NFC, GPS, or GNSS. In addition, for example, the communication interface (120) can perform wired communication in accordance with methods such as USB (Universal Serial Bus) (registered trademark), HDMI (registered trademark) (High Definition Multimedia Interface), RS-232 (Recommended Standard-232), or POTS (Plain Old Telephone Service).

[0054] Memory (130) can store a variety of data. The data stored in memory (130) is data acquired, processed, or used by at least one component of the server (100), and may include software (e.g., instructions, programs, etc.). Memory (130) may include volatile or non-volatile memory. The term memory (130) may mean one or more memory collections unless the context clearly indicates otherwise. The expressions “a collection of instructions stored in memory (130)” or “a program stored in memory (130)” as used herein may be used to refer to an operating system, application for controlling the resources of the server (100), or middleware that provides a variety of functions to an application so that the application can utilize the resources of the server (100). In one embodiment, when the processor (110) performs a particular operation, memory (130) may store instructions executed by the processor (110) that correspond to the particular operation.

[0055] In one embodiment, the server (100) can transfer data corresponding to the calculation results of the processor (110), data received by the communication interface (120), or data stored in memory (130) to an external device. The external device may be a device for displaying, showing, or outputting the received data.

[0056] In one embodiment, the server (100) may further include an input unit (140). The input unit (140) may be a component that transmits data received from an external source to at least one component included in the server (100). For example, the input unit (140) may include at least one of a mouse, a keyboard, or a touchpad.

[0057] In one embodiment, the server (100) may further include an output unit (150). The output unit (150) may display (output) or transfer (send) information processed by the server (100) to an external source. For example, the output unit (150) may visually display the information processed by the server (100). The output unit (150) may display UI (User Interface) information or GUI (Graphic User Interface) information, etc. In this case, the output unit (150) may include at least one of the following: liquid crystal display (LCD), thin film transistor-liquid crystal display (TFT-LCD), organic light-emitting diode (OLED), flexible display, 3D display, or e-ink display. Also, for example, the output unit (150) may audibly display the information processed by the server (100). The output unit (150) can display audio data conforming to any audio file format (e.g., MP3, FLAC, WAV, etc.) through an audio device. In this case, the output unit (150) may include at least one of a speaker, a headset, or headphones. Alternatively, for example, the output unit (150) can transfer information processed by the server (100) to an external output device. The output unit (150) can transfer or send information processed by the server (100) to an external output device using a communication interface (120). The output unit (150) can also transfer or send information processed by the server (100) to an external output device using another output communication interface.

[0058] Figure 3 is a block diagram of a user terminal (200) according to one embodiment of the contents disclosed herein. The user terminal (200) may include one or more processors (210), communication interfaces (220), or memory (230) as components. The user terminal (200) may also further include at least one of an input unit (240) or an output unit (250).

[0059] The processor (210) can drive software (e.g., instructions, programs, etc.) to control at least one component of a user terminal (200) connected to the processor (110). The processor (210) can also perform various operations such as calculations, processing, data generation, or manipulation. Furthermore, the processor (210) can load data, etc., from memory (230) or store data, etc., in memory (230).

[0060] The communication interface (220) can perform wireless or wired communication between a user terminal (200) and other devices (e.g., a server (100) or other user terminals). For example, the communication interface (220) can perform wireless communication according to methods such as eMBB, URLLC, MMTC, LTE, LTE-A, NR, UMTS, GSM, CDMA, WCDMA, WiBro, WiFi, Bluetooth, NFC, GPS, or GNSS. Alternatively, for example, the communication interface (220) can perform wired communication according to methods such as USB, HDMI, RS-232, or POTS.

[0061] Memory (230) can store a variety of data. The data stored in memory (230) is data acquired, processed, or used by at least one component of the user terminal (200), and may include software (e.g., instructions, programs, etc.). Memory (230) may include volatile or non-volatile memory. The term memory (230) may mean one or more memory collections unless the context clearly indicates otherwise. The expressions “a collection of instructions stored in memory (230)” or “a program stored in memory (230)” as used herein may be used to refer to an operating system, application for controlling the resources of the user terminal (200), or middleware, etc., that provides a variety of functions to an application so that the application can utilize the resources of the user terminal (200). In one embodiment, when the processor (210) performs a specific operation, memory (230) may store instructions executed by the processor (210) that correspond to the specific operation.

[0062] In one embodiment, the user terminal (200) may further include an input unit (240). The input unit (240) may be a component that transmits data received from an external source to at least one component included in the user terminal (200). For example, the input unit (240) may include at least one of a mouse, a keyboard, or a touchpad.

[0063] In one embodiment, the user terminal (200) may further include an output unit (250). The output unit (250) may display (output) or transfer (send) information processed by the user terminal (200) to an external source. For example, the output unit (250) may visually display the information processed by the user terminal (200). The output unit (250) may display UI (User Interface) information or GUI (Graphic User Interface) information, etc. In this case, the output unit (250) may include at least one of the following: liquid crystal display (LCD), thin-film transistor liquid crystal display (TFT-LCD), organic light-emitting diode (OLED), flexible display, three-dimensional display (3D display), or electronic ink display (E-ink display). Also, for example, the output unit (250) may audibly display the information processed by the user terminal (200). The output unit (250) can display audio data conforming to any audio file format (e.g., MP3, FLAC, WAV, etc.) through an audio device. In this case, the output unit (250) may include at least one of a speaker, headset, or headphones. Alternatively, the output unit (250) can also transfer information processed by the user terminal (200) to an external output device. The output unit (250) can transfer or send information processed by the user terminal (200) to an external output device using a communication interface (220). The output unit (250) can also transfer or send information processed by the user terminal (200) to an external output device using another output communication interface.

[0064] In the following explanation, for the sake of clarity, the operating entity may be omitted, but in this case, each operation can be understood to be performed by the server (100). However, the method according to this disclosure may also be performed by a user terminal (200), and some of the operations included in the method may be performed by the user terminal (200) and the rest by the server (100).

[0065] Figure 4 is a diagram illustrating an illustrative flowchart of a natural language processing method according to one embodiment of the present disclosure.

[0066] The server (100) may acquire a boosting keyword set (S410). The boosting keyword set may include at least one boosting keyword that is targeted for generation enhancement when generating text using an artificial neural network model.

[0067] In natural language processing, text data, which is the data to be processed, is converted into a form that a computer can recognize and process. In this conversion process, tokenization, which divides the text data into certain units, and embedding, which converts individual tokens into vector values ​​that a computer can recognize and process, may be performed. Byte pair encoding (BPE) techniques may be used for tokenization. Generally, BPE is a technique that generates tokens by breaking down words into character or Unicode units to create a vocabulary, and then merging characters or Unicode that appear consecutively in the vocabulary according to their frequency of appearance. In one embodiment, the tokenization process may be byte-level BPE. Assuming that individual characters contained in natural language data are encoded in UTF-8, byte-level BPE means a technique that breaks down all 1 to N (where N is an integer greater than or equal to 1) bytes that make up each character to create an initial vocabulary on a byte-by-byte basis, and then sequentially performs merge steps to generate a vocabulary with a predetermined number of elements. On the other hand, embedding is the process of converting each token generated by tokenization into an embedding vector, which can be generated using various techniques such as Glove, FastText, and Word2Vec.

[0068] JPEG0007917210000001.jpg58150

[0069] In this disclosure, “keyword” is used as a term to refer to a sequence of tokens having a predetermined length, and is a concept that includes boosting keywords and suppressing keywords. A “boosting keyword” may be used as a term to refer to a specific sequence of tokens whose generation is facilitated so that it is included in a sentence when the server (100) generates a sentence using an artificial neural network model. A “suppressing keyword” may be used as a term to refer to a specific sequence of tokens whose generation is suppressed so that it is not included in a sentence when the server (100) generates a sentence using an artificial neural network model.

[0070] Next, the server (100) may acquire a set of suppression keywords (S420). The set of suppression keywords may include at least one suppression keyword that is subject to generation suppression when text is generated by the artificial neural network model.

[0071] In one embodiment of this disclosure, the boosting keyword set may be a keyword set relating to a first language (e.g., Korean), and the suppression keyword set may be a keyword set relating to a second language (e.g., English, Chinese, or Japanese). Specifically, the boosting keyword set may be configured to include token sequences that satisfy specific occurrence conditions within the overall token sequence set after performing tokenization on publicly available text data relating to the first language. Similarly, the suppression keyword set may be configured to include token sequences that satisfy specific occurrence conditions within the overall token sequence set after performing tokenization on publicly available text data relating to the second language. In this case, the specific occurrence conditions may be, for example, conditions that are satisfied if they occur a predetermined number of times (n times) or more, or conditions that are satisfied if they occur with a predetermined probability (n%) or more. When generating text using an artificial neural network model with a boosting keyword set for a specific language (first language) and a suppression keyword set for another language (a second language other than the first language), the generation of keywords relating to the first language is promoted while the generation of keywords relating to the second language is suppressed, thus enabling the generation of text that is more suitable for usage examples of the first language. The specific text generation method is described in detail below.

[0072] In one embodiment of the present disclosure, a boosting keyword set or a suppressing keyword set may be generated by comparing the word distribution in publicly available data relating to the target language with the word distribution in user-inputted proprietary data. In this case, publicly available data means publicly known data that is available to anyone via the internet, and proprietary data means data entered by the user.

[0073] In one embodiment, the boosting keyword set may include words that appear a predetermined number of times or more in the unique data. In another embodiment, the boosting keyword set may include words that appear a predetermined frequency or more in the unique data. In yet another embodiment, the boosting keyword set may include words that appear at a frequency lower than the first threshold frequency in the public data but at a frequency higher than the second threshold frequency in the unique data. Specifically, the boosting keyword set may include words (i.e., token sequences) that appear at a low frequency (e.g., 0.01% or less) in the public data but at a high frequency (e.g., 0.05% or more) in the unique data entered by the user. The first threshold frequency or the second threshold frequency can be appropriately set based on statistics of the word distribution relating to the public data or the unique data, respectively. For example, the first threshold frequency may be set as the first quartile of the probability of each word appearing in the public data, and the second threshold frequency may be set as the third quartile of the probability of each word appearing in the unique data, etc. Additionally, the boosting keyword set can include words that appear at a frequency lower than the first threshold frequency in the public data, but that appear a certain number of times or more in the unique data.

[0074] In one embodiment, the suppression keyword set may include words that appear in the public data at a frequency higher than the third threshold frequency but appear in the unique data at a frequency lower than the fourth threshold frequency. Specifically, the suppression keyword set may include words that appear at a high frequency (e.g., 0.05% or less) in the public data but appear at a low frequency (e.g., 0.01% or less) in the unique data entered by the user. The third threshold frequency or the fourth threshold frequency can be appropriately set based on statistical data of the word distribution for the public data or unique data, respectively, similar to the first threshold frequency or the second threshold frequency described above. Furthermore, the suppression keyword set may also include words that appear in the public data at a frequency higher than the third threshold frequency but appear in the unique data below a certain number of times.

[0075] As described above, this disclosure can generate a boosting keyword set or a suppressing keyword set by comparing the word distribution in publicly available data for the target language with the word distribution in user-inputted unique data. This has the effect of selecting words that are used more frequently or less frequently in the unique data based on the basic word distribution of the publicly available data for the target language, and efficiently reflecting this in text generation using an artificial neural network model.

[0076] Next, the server (100) may generate a text through an artificial neural network model based on a boosting keyword set and an inhibitory keyword set (S430). In this disclosure, a text may consist of one or more consecutive tokens and may include keywords. In this disclosure, a text may be a sequence of tokens in which subsequent tokens, which are repeatedly determined at each step by the artificial neural network model, are arranged in a sequence.

[0077] Specifically, the server (100) can calculate the generation probability for each of several tokens based on the output of the artificial neural network model for the input token sequence. The generation probability can be expressed, for example, as shown in Equation 1 below.

[0078]

number

[0079] Equation 1 represents the probability of generating the t-th token based on the input token sequence. In this case, x t x1 represents the t-th token, t-1 This is from the 1st token to the t-1th token (where t is 1 or greater, x1 0 This indicates the input token sequence at step t, which includes the token array up to φ). α This represents the embedding vector for a specific token (α) that is an element included in the entire token set (vocabulary, V), and h trepresents the output value of the last layer of the model (e.g., the logit function). The T on the right-hand side may be called temperature, as a probability distribution adjustment parameter for adjusting the probability distribution. The probability distribution adjustment parameter (T) in this disclosure is a parameter that is applied comprehensively to the entire token set (V), and as its value becomes infinitely large, the effect appears that the overall probability values ​​converge to similar values, regardless of the model output for individual tokens. Therefore, compared to not using the probability distribution adjustment parameter, fewer tokens with high generation probabilities and more tokens with low generation probabilities are selected relative to the conventional probability distribution. On the other hand, as the value of the probability distribution adjustment parameter approaches 0, the effect appears that the model output for individual tokens is amplified. Therefore, compared to not using the probability distribution adjustment parameter, more tokens with high generation probabilities and fewer tokens with low generation probabilities are selected relative to the conventional probability distribution. As illustrated in Equation 1 above, the server (100) in this disclosure can calculate the generation probability for each of the multiple tokens that are elements of the entire token set (V). The server (100) may determine the token with the highest calculated generation probability among the tokens to be the successor token.

[0080] According to one embodiment of the present disclosure, the server (100) may calculate generation probabilities that differ from each other depending on the classification of each token. A method for calculating generation probabilities that differ from each other depending on the classification of tokens can be expressed, for example, as shown in the following equations 2-3.

[0081]

number

[0082]

number

[0083] Equation 2 is obtained by changing T in Equation 1 to T·T x This is a replacement for T xrepresents the token-level temperature (i.e., probability distribution adjustment parameter) for the x-th token. Formula 3 is an equation representing the token-level temperature, where T B denotes the boosting probability distribution parameter used when the token (x) is included in the token set (V B ) of the boosting keyword set, and T S denotes the suppression probability distribution parameter used when the token (x) is included in the token set (V S ) of the suppression keyword set. In this case, the token set of the boosting keyword set may be a set composed of tokens included in the token sequences constituting each boosting keyword. Also, the token set of the suppression keyword set may be a set composed of tokens included in the token sequences constituting each suppression keyword. That is, when calculating the generation probability for each of a plurality of tokens, if a token is included in the token set of the boosting keyword set, the server (100) calculates the generation probability using the boosting probability distribution adjustment parameter (T B ) for increasing the generation probability; and if a token is included in the token set of the suppression keyword set, the server (100) can calculate the generation probability using the suppression probability distribution adjustment parameter (T S ) for decreasing the generation probability. Specifically, the value of the boosting probability distribution parameter (T B ) may be a real number smaller than 1 for increasing the selection probability of tokens included in the boosting keyword, and the value of the suppression probability distribution parameter (T S ) may be a real number larger than 1 for decreasing the selection probability of tokens included in the suppression keyword. As described above, by using mutually different probability distribution adjustment parameters according to the classification of tokens, the server (100) calculates mutually different generation probabilities for each token, and can determine subsequent tokens following the input sequence accordingly.

[0084] The server (100) of this disclosure may generate text through an artificial neural network model using a keyword try (trie) that includes one or more nodes.

[0085] Figure 5 is a conceptual diagram illustrating a keyword try according to one embodiment of the present disclosure. Each node in the keyword try may contain a token and a keyword state value. The keyword state value of a node is set to the topmost node, the root node (N root This can be a value that indicates the keyword state of a token sequence formed when the tokens of each node included in the path from the root node (N) are arranged in sequence. root ) from t node (N t If the sequence of consecutive tokens arranged by traversing the tri structure up to ) corresponds to a boosting keyword, then the t node (N) included in the keyword try is t The keyword state value of ) can be 1. Also, for example, the root node (N root ) from t node (N t If the token sequence up to ) matches the suppression keyword, then the t node (N t The keyword state value of ) can be -1. Also, for example, the root node (N root ) from t node (N t If the token sequence up to ) does not correspond to either a boosting keyword or a suppression keyword, then t node (N t The keyword state value of ) may be 0. Hereinafter in this disclosure, each node in a keyword try may be represented as (v,s), where v is the token the node possesses and s is the keyword state value of the node. Each node in a keyword try may have different state values ​​depending on whether the token sequence of nodes placed in the path from the root node to that node corresponds to a boosting keyword, a suppression keyword, or neither a boosting keyword nor a suppression keyword.

[0086] In this disclosure, keyword tries may be generated based on a boosting keyword set or a suppressing keyword set. For illustrative purposes, refer to the keyword tries in Figure 5, where the boosting keyword set is {[v2,v 22 ',[v2,v 21 ,v 212 It is a set of token sequences like ]}, and the suppression keyword set is {[v2,v 21 ,v 211 ',[v2,v 22 ,v 221 Assume that the set consists of token sequences like ]}. In this case, the first token of each token sequence included in the overall token sequence set is common to v2, so the common token, token v2, can be located at the first parent node. Also, the root node (N root The token sequence ([v2]) from ) to token v2 is not included in the boosting keyword set or the suppression keyword set, so the keyword state value of the node with token v2 can be 0. Therefore, the parent node of the entire set of token sequences that has token v2 can be represented as (v2,0). Next, the token that follows token v2 in the entire set of token sequences is token v 21 or token v 22 Therefore, as child nodes of the parent node (i.e., (v2,0)), each token v 21 or token v 22 A node containing the token v can be placed. In this case, token v can be placed from the root node. 21 Token sequence based on the path to ([v2,v 21 ]) is not included in the boosting keyword set or the suppressing keyword set, therefore token v 21 The keyword state value of a node with this can be 0. Therefore, token v 21 The node that has (v 21 It can be represented as ,0). On the other hand, from the root node, token v 22 Token sequence based on the path to ([v2,v 22]) is a token sequence included in the boosting keyword set, therefore token v 22 The keyword state value of a node with this can be 1. Therefore, token v 22 The node that has (v 22 It can be expressed as shown in 1). The above is merely an example of how keyword tries can be generated based on a boosting keyword set or a suppressing keyword set, and does not limit the scope of this disclosure.

[0087] Figure 6 is an illustrative flowchart of the first embodiment of generating text using keyword tryouts.

[0088] The server (100) can generate a first token sequence using a first probability distribution adjustment parameter (i.e., a boosting probability distribution adjustment parameter) (S610). Specifically, in step t (where t is a natural number greater than or equal to 1), the server (100) inputs a first token sequence of length t-1, including the first token to the (t-1)th token, into an artificial neural network model, calculates the generation probability for each of the multiple tokens using the first probability distribution adjustment parameter on the output of the artificial neural network model, determines the subsequent token according to the calculated generation probabilities, and concatenates the determined subsequent token by following it to the (t-1)th token, thereby generating a first token sequence of length t. The generation probabilities that the server (100) calculates for each of the multiple tokens using the boosting probability distribution parameter can be calculated, for example, according to equation 2 above and equation 4 below.

[0089]

number

[0090] Equation 4 is obtained by removing the suppression probability distribution parameter from Equation 3, and the boosting probability distribution parameter (T B ) using token level T x The formula for calculating this is shown below. Equation 4, token level T xWhen applying this to equation 2, the server (100) can calculate the generation probability using the boosting probability distribution parameter.

[0091] The server (100) can generate a second token sequence using a first probability distribution adjustment parameter (i.e., a boosting probability distribution parameter) and a second probability distribution adjustment parameter (i.e., a suppression probability distribution parameter) (S620). That is, in step t (where t is a natural number greater than or equal to 1), the server (100) inputs a token sequence of length t-1, including the first token to the (t-1)th token, to an artificial neural network model, calculates the generation probability for each of the multiple tokens using the first and second probability distribution adjustment parameters on the output of the artificial neural network model, determines the subsequent token according to the calculated generation probabilities, and generates a second token sequence of length t by concatenating the subsequent token after the (t-1)th token. The generation probabilities that the server (100) calculates for each of the multiple tokens using the boosting probability distribution parameter and the suppression probability distribution parameter can be calculated, for example, by equations 2 and 3 described above.

[0092] The server (100) may replace one of the first and second token sequences with the other token sequence if a predetermined condition is met (S630). In this case, the predetermined condition may be a condition whose validity is determined based on keyword tryouts.

[0093] If the server (100) determines through keyword tryouts that the first token sequence contains a suppression keyword, it may replace the first token sequence with the second token sequence.

[0094] In one embodiment, the server (100) may determine in step t that the first token sequence of length t contains a suppression keyword if the node corresponding to the newly added token in the first token sequence of length t-1 is a leaf node in the keyword try and the keyword state value of that node indicates a suppression keyword (e.g., -1).

[0095] In one embodiment, if, in step t, the server (100) determines that the first token sequence of length t may contain a suppression keyword, and sets a variable indicating that the first token sequence of length t may contain a suppression keyword to True, then executes the next step. That is, in such a case, in step t, the server (100) reserves judgment on whether the first token sequence contains a suppression keyword, but only preserves the possibility that it contains a suppression keyword, and proceeds to the subsequent steps (i.e., from step t+1 onwards). At this time, if, in the subsequent steps, the variable indicating that the first token sequence may contain a suppression keyword has a True value, and the node corresponding to the newly added token in the first token sequence does not exist in the keyword try, or is a node that is a leaf node in the keyword try and has a state value indicating a suppression keyword, then the server (100) may determine that the first token sequence contains a suppression keyword.

[0096] The server (100) may replace the second token sequence with the first token sequence if keyword try determines that the first token sequence contains a boosting keyword or does not contain a suppressing keyword.

[0097] In one embodiment, the server (100) may determine in step t that the first token sequence of length t contains a boosting keyword if the node corresponding to the newly added token in the first token sequence of length t-1 is a leaf node in the keyword try, and the keyword state value of that node indicates a boosting keyword (for example, 1).

[0098] In one embodiment, the server (100) can determine that the first token sequence contains a boosting keyword if, in step t, there is no node in the keyword try corresponding to a token newly added to the first token sequence of length t-1, and a variable indicating that the first token sequence may contain a suppression keyword has a value of False.

[0099] As described above, the server (100) can generate a token sequence that contains more boosting keywords and fewer suppressing keywords by replacing one of the first token sequence and the second token sequence with the other token sequence based on predetermined conditions determined on keyword tryouts. That is, the server (100) of this disclosure calculates all the first token sequences generated using the boosting probability distribution parameters and the second token sequences generated using the boosting probability distribution parameters and the suppressing probability distribution parameters in each step. If a suppressing keyword is detected in the first token sequence, the server can replace the first token sequence with a second token sequence generated by further suppressing the suppressing keyword, thereby removing the suppressing keyword. Also, if a boosting keyword is detected in the first token sequence, the server can replace the second token sequence up to that point with the first token sequence to synchronize subsequent calculations. Furthermore, the server (100) that uses keyword tryouts to determine whether a keyword is a suppressing keyword or a boosting keyword has the advantage of faster computation speed. Instead of iterating through the entire token sequence or from the root to the current node each time it determines whether a token sequence contains a boosting keyword or a suppressing keyword, it can immediately determine whether a token sequence contains a boosting keyword or a suppressing keyword using only the node position and keyword state value of the last token, according to the data structure of the keyword tryouts described above.

[0100] Figure 7 is an illustrative flowchart of a second embodiment that generates text using keyword tryouts.

[0101] The server (100) may generate a first token sequence using a first probability distribution adjustment parameter (i.e., a boosting probability distribution adjustment parameter) (S710). Step S710 in Figure 7 can be performed identically or similarly to step S610 in Figure 6, so a redundant explanation is omitted.

[0102] If the server (100) determines that the first token sequence contains a suppression keyword, it may generate a second token sequence using the first probability distribution adjustment parameter and the second probability distribution adjustment parameter (S720).

[0103] If the first token sequence is determined to contain a suppression keyword, the input token sequence for generating the second token sequence may be determined based on the node path of the keyword try at the time the first token sequence is determined to contain a suppression keyword.

[0104] For example, the token sequence corresponding to the suppression keyword is "[v s1 ,v s2 ,v s3 ] is represented as , and the node path corresponding to such a token sequence is represented by keyword try as "[Root,(v s1 ,0),(v s2 ,0),(v s3 Assume that it can be represented as ",-1)". Also, in one step, server (100) has a first token sequence of length 5 "[v1,v2,v3,v s1 ,v s2 ]" followed by a subsequent token (v s3 ) adds a first token sequence of length 6 "[v1,v2,v3,v s1 ,v s2 ,v s3 Assume that the token sequence "[v s1 ,v s2 ,v s3 Remove "[v]" from the first token sequence and add that token sequence to the input token sequence "[v s1 ,v s2 ,v s3A second token sequence can be generated as ]. That is, the input token sequence that is input to the artificial neural network model to generate the second token sequence may be a token sequence obtained by removing the inhibitory keyword from the first token sequence.

[0105] Unlike the first embodiment described above, the server (100) in the second embodiment can generate a second token sequence to replace the first token sequence when it is determined that the first token sequence contains a suppression keyword. This allows the present disclosure to generate text that contains many boosting keywords and few suppression keywords while reducing the amount of memory used. Furthermore, in the case of the second embodiment, the second token sequence is generated by deleting the suppression keyword contained in the first token sequence and then generating the token again, which has the effect of removing suppression keywords that have been detected more than a predetermined number of times.

[0106] Figure 8 is an illustrative flowchart of a third embodiment that generates text using keyword tryouts.

[0107] The server (100) can generate multiple candidate token sequences by determining multiple subsequent tokens for each of N (where N is a natural number greater than or equal to 2) token sequences of length t (where t is a natural number greater than or equal to 1) using an artificial neural network model (S810). That is, in each step, the server (100) performs operations through the artificial neural network model for each of the N token sequences and generates multiple (e.g., M) subsequent tokens, resulting in a total of N × M candidate token sequences being generated.

[0108] The server (100) can calculate the cumulative probability for each of several candidate token sequences according to a predetermined calculation method (S820). The cumulative probability for a token sequence may be the cumulative product of the generation probabilities of subsequent tokens newly added at each step. For example, if the generation probabilities of each token in the token sequence "[v1,v2,v3]" are 0.9, 0.8, and 0.1 respectively, the cumulative probability of the token sequence may be 0.072 (=0.9 × 0.8 × 0.1).

[0109] The server (100) may increase the cumulative probability if it determines that a candidate token sequence contains a boosting keyword. For example, if the sub-token sequence "[v2,v3]" in the candidate token sequence "[v1,v2,v3]" is a boosting keyword, the server may increase the cumulative probability of that candidate token sequence. The increase in cumulative probability can be achieved by adding a predetermined positive weight to the cumulative probability or by multiplying the cumulative probability by a real number greater than 1, etc.

[0110] The server (100) may reduce the cumulative probability if it determines that a candidate token sequence contains a suppression keyword. For example, if the sub-token sequence "[v2,v3]" in the candidate token sequence "[v1,v2,v3]" is a suppression keyword, the server may reduce the cumulative probability of that candidate token sequence. The reduction of the cumulative probability can be performed by adding a predetermined negative weight to the cumulative probability or by multiplying the cumulative probability by a real number less than 1, etc.

[0111] The server (100) can determine N token sequences of length t+1 from among multiple candidate token sequences based on the cumulative probability for each of the multiple candidate token sequences (S830). That is, if the server (100) generates M subsequent tokens for each token sequence in descending order of generation probability in each step, the server (100) can determine N token sequences in descending order of cumulative probability of a total of N × M candidate token sequences in each step.

[0112] The server (100) according to the third embodiment generates text by considering N token sequences in each step, thus not only solving the problem of greedy algorithms that generate text with a single output, but also more efficiently generating text that contains more boosting keywords and fewer suppressing keywords by adjusting the cumulative probability using keyword tries generated based on a boosting keyword set or a suppressing keyword set.

[0113] In the flowcharts or diagrams relating to the disclosures herein, each step of the method or algorithm is described in a sequential order; however, each step may be performed in any order that can be combined as needed, in addition to being performed sequentially. The descriptions of the flowcharts or diagrams herein do not preclude changes or modifications to the method or algorithm, nor do they imply that any particular step is essential or desirable. In one embodiment, at least some steps may be performed in parallel, iteratively, or heuristically. In one embodiment, at least some steps may be omitted, or other steps may be added.

[0114] Various embodiments relating to the disclosure herein may be implemented as software on a machine-readable storage medium. The software may be software for implementing the various embodiments relating herein. The software may be inferred from the various embodiments relating herein by a programmer in the art to which the disclosure relating herein pertains. For example, the software may be a program containing machine-readable instructions (e.g., code or code segment). A machine is a device capable of operating according to instructions called from a storage medium, and may be, for example, a computer. In one embodiment, the machine may be a computing device relating to the various embodiments relating herein. In one embodiment, the processor of the machine may execute a called instruction, causing the components of the machine to perform the function corresponding to the instruction. In one embodiment, the processor may be a processor (110, 210) relating to the embodiments relating herein. A storage medium may mean any type of recording medium that is machine-readable and on which data is recorded. A storage medium may include, for example, ROM, RAM, CD-ROM, magnetic tape, flexible disk, optical data storage device, etc. In one embodiment, the storage medium may be memory (130, 230). In one embodiment, the storage medium may be implemented in a distributed form across a networked computer system, etc. Software may be stored and executed in a distributed manner across the computer system, etc. The storage medium may be a non-transitory storage medium. A non-transitory storage medium means a tangible medium that exists regardless of whether data is stored semi-permanently or temporarily, and does not include transiently propagated signals.

[0115] Although the technical concept relating to the disclosures herein has been explained through various embodiments, the technical concept relating to the disclosures herein includes a variety of substitutions, modifications, and alterations that can be understood by a person with ordinary skill in the art to which the disclosures herein pertain. Furthermore, such substitutions, modifications, and alterations should be understood to be included in the attached claims.

Claims

1. A natural language processing method performed on an electronic device including one or more processors and one or more memories storing instructions to be executed by the one or more processors, A step of acquiring a boosting keyword set, wherein the boosting keyword set includes at least one boosting keyword that is targeted for generation enhancement when generating text using an artificial neural network model. A step of acquiring a set of suppression keywords, wherein the set of suppression keywords includes at least one suppression keyword that is subject to suppression during text generation using an artificial neural network model. The steps include generating text through an artificial neural network model based on the boosting keyword set and the suppression keyword set, Includes, The boosting keyword set or the suppression keyword set is It is generated based on the word distribution in publicly available data for the target language and the word distribution in user-generated unique data. The aforementioned boosting keyword set is A method comprising words that appear at a frequency lower than a first threshold frequency in the publicly available data, and that appear at a frequency higher than a second threshold frequency in the unique data.

2. The method according to claim 1, wherein the boosting keyword set is a keyword set relating to a first language, and the suppression keyword set is a keyword set relating to a second language.

3. A natural language processing method performed on an electronic device including one or more processors and one or more memories storing instructions to be executed by the one or more processors, A step of acquiring a boosting keyword set, wherein the boosting keyword set includes at least one boosting keyword that is targeted for generation enhancement when generating text using an artificial neural network model. A step of acquiring a set of suppression keywords, wherein the set of suppression keywords includes at least one suppression keyword that is subject to suppression during text generation using an artificial neural network model. The steps include generating text through an artificial neural network model based on the boosting keyword set and the suppression keyword set, Includes, The boosting keyword set or the suppression keyword set is It is generated based on the word distribution in publicly available data for the target language and the word distribution in user-generated unique data. The aforementioned set of suppression keywords is: A method comprising words that appear at a frequency higher than the third threshold frequency in the publicly available data, and which appear at a frequency lower than the fourth threshold frequency in the proprietary data.

4. The step of generating text through the aforementioned artificial neural network model is: The steps include: calculating the generation probability for each of a plurality of tokens based on the output of the artificial neural network model for the input token sequence; The step of determining the successor token, The method according to claim 1, including the method described in claim 1.

5. The generation probability for each of the aforementioned multiple tokens is: The method according to claim 4, wherein each token is calculated differently from the others according to its classification.

6. The generation probability for each of the aforementioned multiple tokens is: The method according to claim 4, wherein if the token is a token included in the token set of the boosting keyword set, it is calculated using a first probability distribution adjustment parameter to increase the generation probability, and if the token is a token included in the token set of the suppression keyword set, it is calculated using a second probability distribution adjustment parameter to decrease the generation probability.

7. A natural language processing method performed on an electronic device including one or more processors and one or more memories storing instructions to be executed by the one or more processors, A step of acquiring a boosting keyword set, wherein the boosting keyword set includes at least one boosting keyword that is targeted for generation enhancement when generating text using an artificial neural network model. A step of acquiring a set of suppression keywords, wherein the set of suppression keywords includes at least one suppression keyword that is subject to suppression during text generation using an artificial neural network model. The steps include generating text through an artificial neural network model based on the boosting keyword set and the suppression keyword set, Includes, The step of generating text through the aforementioned artificial neural network model is: It is executed using keyword tries that include one or more nodes. A method comprising one or more nodes, each including a keyword state value relating to a token sequence, each including a token and tokens of each node included in the path from the root node to the current node.

8. The aforementioned keyword tryout is The method according to claim 7, which is generated based on the boosting keyword set or the suppression keyword set.

9. The step of generating text through the aforementioned artificial neural network model is: A step of generating a first token sequence using a first probability distribution adjustment parameter, A step of generating a second token sequence using the first probability distribution adjustment parameter and the second probability distribution adjustment parameter, If predetermined conditions are met, the step of replacing one of the first token sequence and the second token sequence with the other token sequence, The method according to claim 7, wherein the predetermined conditions are conditions whose validity is determined based on the keyword try.

10. If the aforementioned predetermined conditions are met, the step of replacing one of the first token sequence and the second token sequence with the other token sequence is: If it is determined that the first token sequence contains a suppression keyword, the first token sequence is replaced with the second token sequence. If it is determined that the first token sequence contains a boosting keyword or does not contain a suppression keyword, the second token sequence is replaced with the first token sequence. The method according to claim 9, including the method described in claim 9.

11. The step of generating text through the aforementioned artificial neural network model is: A step of generating a first token sequence using a first probability distribution adjustment parameter, If it is determined that the first token sequence contains a suppression keyword, the steps include generating a second token sequence using the first probability distribution adjustment parameter and the second probability distribution adjustment parameter, The method according to claim 7, including the method described in claim 7.

12. The step of generating text through the aforementioned artificial neural network model is: The steps include generating multiple candidate token sequences by determining multiple subsequent tokens for each of N (where N is a natural number greater than or equal to 2) token sequences using the artificial neural network model, The steps include: calculating the cumulative probability for each of the plurality of candidate token sequences according to a predetermined calculation method; The steps include determining N token sequences from the plurality of candidate token sequences based on the cumulative probability, The method according to claim 7, including the method described in claim 7.

13. The step of calculating the cumulative probability for each of the plurality of candidate token sequences according to the predetermined calculation method is: If it is determined that the candidate token sequence contains a boosting keyword, the step of increasing the cumulative probability, If it is determined that the candidate token sequence contains a suppression keyword, the step of reducing the cumulative probability, The method according to claim 12, including the method described in claim 12.

14. In electronic devices, One or more processors, Includes one or more memories storing instructions to be executed by the one or more processors, The one or more processors described above are: A boosting keyword set is acquired, and the boosting keyword set includes at least one boosting keyword that is targeted for generation enhancement when generating text using an artificial neural network model. A set of suppression keywords is acquired, and this set of suppression keywords includes at least one suppression keyword that is subject to generation suppression when generating text using an artificial neural network model. Based on the aforementioned boosting keyword set and the aforementioned suppression keyword set, text is generated through an artificial neural network model. The boosting keyword set or the suppression keyword set is It is generated based on the word distribution in publicly available data for the target language and the word distribution in user-generated unique data. The aforementioned boosting keyword set is An electronic device that includes words that appear at a frequency lower than the first threshold frequency in the publicly available data, and that appear at a frequency higher than the second threshold frequency in the proprietary data.

15. In a non-temporary computer-readable recording medium on which instructions are recorded to cause one or more processors to perform an operation when executed by one or more processors, The instruction is given to one or more processors, A step of acquiring a boosting keyword set, wherein the boosting keyword set includes at least one boosting keyword that is targeted for generation enhancement when generating text using an artificial neural network model. A step of acquiring a set of suppression keywords, wherein the set of suppression keywords includes at least one suppression keyword that is subject to suppression during text generation using an artificial neural network model. The steps include generating text through an artificial neural network model based on the boosting keyword set and the suppression keyword set, Make it run, The boosting keyword set or the suppression keyword set is It is generated based on the word distribution in publicly available data for the target language and the word distribution in user-generated unique data. The aforementioned boosting keyword set is A computer-readable recording medium that includes words that appear at a frequency lower than a first threshold frequency in the publicly available data, and that appear at a frequency higher than a second threshold frequency in the proprietary data.

Images