Information processing device, information processing method and information processing program
The information processing device addresses the challenge of unknown vocabulary in AI models by replacing such terms with alternative vocabulary during additional learning, enhancing the AI model's ability to process and learn new terms effectively.
Patent Information
- Application Number
- JP2023183788
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-10-26
- Publication Date
- 2025-05-13
- Estimated Expiration
- 2043-10-26
AI Technical Summary
Existing AI models struggle to handle unknown vocabulary during simple additional learning, leading to inappropriate output results as unknown vocabulary is processed as unknown.
An information processing device with first and second processing units that replace unknown vocabulary in AI model input sentences with alternative vocabulary, allowing the AI model to learn and process these terms effectively during additional learning.
Enables AI models to handle unknown vocabulary even with simple additional learning, improving output results by effectively learning and processing new terms.
Smart Images
Figure 2025073215000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]
[0002] In recent years, the use of natural language AI (Artificial Intelligence) models has been expanding, and there is a demand for technology that enables the AI models to handle unknown vocabulary that they have not learned. Patent Document 1 discloses a method for detecting unknown words by comparing text before and after conversion. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2022-042033 A Summary of the Invention [Problem to be solved by the invention]
[0004] One possible technique for this is to pre-train unknown vocabulary before using the AI model. However, pre-training requires adjusting many parameters in the entire AI model, including the intermediate layer and output layer, which requires a large amount of memory and computation time.
[0005] To solve this problem, a method called fine-tuning, which is additional learning, can be considered. Additional learning involves additional learning of the output results from the output layer of the AI model, so it requires less memory and calculation time than pre-learning.
[0006] However, with additional learning, vocabulary that the AI model has not learned is not newly learned and is treated as unknown vocabulary, which may result in inappropriate output results.
[0007] The present disclosure has been made in consideration of the above, and aims to provide a technology that enables an AI model to handle unknown vocabulary even through simple additional learning. [Means for solving the problem]
[0008] An information processing device of one embodiment of the present disclosure includes a first processing unit that replaces unknown vocabulary included in a first sentence that an AI model has not learned with another vocabulary, and has the replaced replacement sentence additionally learned by the AI model; and a second processing unit that replaces the unknown vocabulary included in a second sentence to the AI model with the another vocabulary, inputs the replaced replacement sentence to the AI model, and returns the another vocabulary included in a response sentence from the AI model to the unknown vocabulary.
[0009] An information processing method of one embodiment of the present disclosure is an information processing method performed by an information processing device, which replaces unknown vocabulary that has not been learned by an AI model and is included in a first sentence with another vocabulary, additionally learns the replaced replacement sentence to the AI model, replaces the unknown vocabulary included in a second sentence to the AI model with the another vocabulary, inputs the replaced replacement sentence to the AI model, and returns the other vocabulary included in a response sentence from the AI model to the unknown vocabulary.
[0010] An information processing program according to an embodiment of the present disclosure causes a computer to function as the information processing device. Effect of the Invention
[0011] The present disclosure provides a technology that enables an AI model to handle unknown vocabulary even with simple additional learning. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a diagram illustrating a functional block configuration of an information processing device. [Diagram 2] FIG. 2 is a diagram showing a processing flow of the information processing device in the learning stage of the AI model. [Diagram 3]FIG. 3 is a diagram illustrating an example of the correspondence table. [Figure 4] FIG. 4 is a diagram showing a processing flow of an information processing device at the stage of using an AI model. [Diagram 5] FIG. 5 is a diagram illustrating an example of the correspondence table. [Figure 6] FIG. 6 is a diagram showing an example of the configuration and processing of the conventional technology and this embodiment. [Figure 7] FIG. 7 is a diagram illustrating a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.
[0014] [Functions of information processing device] FIG. 1 is a diagram showing a functional block configuration of an information processing device 1 according to the present embodiment.
[0015] The information processing device 1 includes a first processing unit 11 that functions during the learning stage of the AI model, a second processing unit 12 that functions during the use stage of the AI model, and a memory unit 13 that stores various information handled by the information processing device 1.
[0016] The first processing unit 11 has a function of additionally learning a sentence (first sentence) A for additional learning in an AI model, and determining unknown vocabulary that has not been learned by the AI model based on the results of the additional learning.
[0017] The first processing unit 11 also has a function of replacing unknown vocabulary contained in the additional learning sentence A with another vocabulary, and additionally learning the replaced replacement sentence A' in the AI model. For example, the first processing unit 11 replaces technical terms that are unknown vocabulary with words (e.g., "#" + word) that are not used in the usage environment (AI model environment), and additionally learning the replaced replacement sentence in the AI model.
[0018] The first processing unit 11 also has a function of generating a correspondence table in which unknown vocabulary and other vocabulary are associated with each other, and storing the generated correspondence table in the storage unit 13.
[0019] Before inputting the processing request sentence (second sentence) Bin to the AI model, the second processing unit 12 performs the same vocabulary replacement as the vocabulary replacement performed by the first processing unit 11 in the learning stage.
[0020] That is, the second processing unit 12 has a function of replacing unknown vocabulary contained in the processing request text Bin with other vocabulary by using a correspondence table stored in the storage unit 13, and inputting the replaced replaced text Bin' to the AI model. For example, the second processing unit 12 replaces technical terms that are unknown vocabulary with words that are not used in the usage environment.
[0021] The second processing unit 12 also has a function of receiving a response sentence Bout from the AI model in response to the replacement sentence Bin', replacing another vocabulary contained in the received response sentence Bout with unknown vocabulary, and outputting the replaced replacement sentence Bout'. For example, the second processing unit 12 returns a word not used in the usage environment (the replaced vocabulary) to the original technical term (the vocabulary before replacement) and outputs it.
[0022] The first processing unit 11 and the second processing unit 12 may be integrated into one processing unit.
[0023] [Operation of information processing device] First, the operation of the information processing device 1 in the learning stage of the AI model will be described.
[0024] FIG. 2 is a diagram showing a processing flow of the information processing device 1 in the learning stage of the AI model.
[0025] Step S101; User A prepares sentence A for additional learning and inputs it to the information processing device 1. The first processing unit 11 of the information processing device 1 acquires the sentence A. Sentence A is, for example, "Install an insulating module as a lightning protection measure for the CSM."
[0026] Step S102; Next, the first processing unit 11 inputs the above-mentioned sentence A to the AI model, and additionally learns the sentence A in the AI model. As a result, the AI model learns the sentence A's "Install an insulating module as a lightning protection measure for the CSM."
[0027] Step S103; Next, user A inputs a processing request sentence related to sentence A to the information processing device 1. The first processing unit 11 of the information processing device 1 acquires the sentence and inputs it to the AI model (the AI model that has learned sentence A). The processing request sentence is, for example, "Please tell me about lightning damage countermeasures for CSM."
[0028] Step S104; When an AI model is input with a sentence such as, "Please tell me about lightning damage prevention measures for CSM," the AI model, which had correctly learned the original sentence A, will output a response sentence (the same as the additionally learned sentence A) that says, "Install an insulation module to protect CSM from lightning damage."
[0029] On the other hand, if the AI model does not correctly learn the vocabulary "CSM", the AI model will output a response sentence related to "lightning damage countermeasures", but the "CSM" that has not been correctly learned will output an unknown vocabulary " <unk>" and replace it with " <unk>The response text is "Install an insulation module as a measure against lightning damage."
[0030] Therefore, the first processing unit 11 adds the following to the response sentence from the AI model: <unk>Determine whether " is included. <unk>If " is included, proceed to step S105. <unk>If " is not included, proceed to step S106.
[0031] Step S105; The response text from the AI model was " <unk>If " is included, the first processing unit 11 extracts " <unk>vocabulary X corresponding to the position of "," is obtained, and a replacement sentence A' of sentence A is generated by replacing the vocabulary X "CSM" with another vocabulary Y such as "#1#2#3."
[0032] After that, the process returns to step S102, and the first processing unit 11 inputs "Install an insulating module as a lightning damage countermeasure for #1#2#3" from the replacement text A' into the AI model, causing the AI model to perform additional learning (additional learning again regarding text A) (second time step S102). Then, a processing request text such as "Please tell me about lightning damage countermeasures for #1#2#3" from the replacement text A' is input into the AI model (the AI model that has already learned the replacement text A') (second time step S103). After that, the response text from the AI model is updated to " <unk>It is determined again whether or not " is included (second step S104).
[0033] If the AI model correctly learns the replacement sentence A' in step S102 for the second time, the AI model will say, <unk>" is output as a response sentence such as "Install an insulation module as a lightning protection measure for #1, #2, and #3."
[0034] Step S106; After step S105, the response text from the AI model is " <unk>If " is not included, the first processing unit 11 identifies the vocabulary X of "CSM" acquired from the document A in step S105 as an unknown vocabulary.
[0035] Step S107; After step S106, the first processing unit 11 generates a correspondence table in which vocabulary X (unknown vocabulary) is associated with vocabulary Y (another vocabulary), and stores the correspondence table in the storage unit 13. An example of the correspondence table is shown in FIG.
[0036] Step S108; After step S107, the first processing unit 11 judges whether or not there is a next sentence. If there is a next sentence, the first processing unit 11 repeats steps S101 to S107 until there is no next sentence, and updates the correspondence table. If there is no next sentence, the first processing unit 11 ends the additional learning.
[0037] Next, the operation of the information processing device 1 at the stage of using the AI model will be described.
[0038] FIG. 4 is a diagram showing a processing flow of the information processing device 1 at the stage of using the AI model.
[0039] Step S201; User B inputs a processing request text Bin from an AI prompt or the like into the information processing device 1. For example, text Bin is "What are some examples of CSM lightning damage countermeasures?" Note that user B may be the same as user A described in steps S101 and S103, but is generally a third party who uses the AI model.
[0040] Step S202; Next, the second processing unit 12 of the information processing device 1 searches for vocabulary X from the text Bin using the correspondence table stored in the storage unit 13, and replaces "CSM" in the searched vocabulary X with "#1#2#3" in the vocabulary Y. In other words, since "CSM" contained in the text Bin is an unknown vocabulary to the AI model, the unknown vocabulary is replaced with another corresponding vocabulary from the correspondence table.
[0041] Step S203; Next, the second processing unit 12 inputs the replacement sentence Bin' of the sentence Bin "What are examples of lightning damage countermeasures for #1#2#3?", in which "CSM" has been replaced with "#1#2#3", to the AI model. In the replacement sentence Bin', "CSM", which is an unknown vocabulary to the AI model, has been replaced with "#1#2#3", and since the AI model correctly learned "#1#2#3" in the second step S102, the AI model can properly process the replacement sentence Bin'. After that, the AI model outputs a response sentence Bout such as "Install an insulation module as a lightning damage countermeasure for #1#2#3."
[0042] Step S204; Next, the second processing unit 12 refers to the correspondence table and restores "#1#2#3" of the vocabulary Y contained in the response sentence Bout from the AI model to "CSM" of the original vocabulary X.
[0043] Step S205; Finally, the second processing unit 12 outputs a replacement sentence Bout', such as "Install an insulating module as a lightning protection measure for the CSM," in which "#1#2#3" is changed back to the original "CSM," on the screen of user B's user terminal as a response result to sentence Bin.
[0044] [Variation 1] If the unknown vocabulary of the AI model can be grasped in advance, a correspondence table that associates the unknown vocabulary with other vocabulary may be generated in advance. In this case, steps S102 to S104 for identifying the unknown vocabulary with respect to the text A can be omitted.
[0045] In other words, the first processing unit 11 does not need to have a function in advance of additionally learning the additional learning sentence A in the AI model and determining unknown vocabulary based on the additional learning result. The first processing unit 11 may have a function of replacing unknown vocabulary contained in sentence A that has not been learned by the AI model with another vocabulary and additionally learning the replaced replaced sentence A' in the AI model.
[0046] In this case, the first processing unit 11 acquires the sentence A (step S101), skips steps S102 to S104, generates a replaced sentence A' by replacing the vocabulary X contained in the sentence A with the vocabulary Y (step S105), and inputs the replaced sentence A' to the AI model for additional learning (step S102). After that, the first processing unit 11 skips steps S106 to S107 and proceeds to step S108.
[0047] [Variation 2] The method of replacing vocabulary X with vocabulary Y is arbitrary. An example is shown in Figure 5.
[0048] If vocabulary X is an alphabet, then the alphabet replaced with kana (``CSM'' → ``CSM'') is the vocabulary Y. Each alphabet can be replaced with ````#'' + kana'' to make ``#C#S#M'', or the alphabet can be reversed and replaced with kana at the end to make ``CSMMSEASI''.
[0049] If vocabulary X is a kanji with old characters (e.g., "Saito"), the new characters of the kanji (e.g., "Saito") may be used as vocabulary Y. In addition, words that are not used in the usage environment (e.g., "sakemasu") or words that are not normally used (e.g., "ujiesuwara") may be used as vocabulary Y. A combination of multiple different vocabulary Ys may also be used as vocabulary Y.
[0050] If "CSM" is replaced with, for example, "Ujiesuwara" in step S105, the response sentence from the AI model in the second step S104 will again be " <unk>In this case, in step S105, a vocabulary Y other than "Ujiesuwara" is selected.
[0051] In this regard, in order to reduce the number of times going through step S105, it is preferable that the vocabulary Y is a vocabulary that the AI model has already learned from the beginning. However, since it is usually difficult to grasp the vocabulary that the AI model has already learned, it is preferable to add " <unk>Repeatedly select different vocabulary Y until no more words Y contain the term "Y".
[0052] In addition, it was stated that "it is preferable that vocabulary Y is a vocabulary that the AI model has already learned." However, if it corresponds to a vocabulary that the AI model normally handles, the vocabulary itself may be processed by the AI model, and vocabulary Y may not be identified from the response sentence of the AI model. Therefore, vocabulary Y is, <unk>Vocabulary Y shown in Figure 5 is preferred, such as words that are unlikely to be replaced by " and are not commonly used in the AI model environment.
[0053] [Comparison between the conventional technology and the present embodiment] A comparison will be made between the conventional technology without the information processing device 1 and this embodiment with the information processing device 1.
[0054] FIG. 6(a) is a diagram showing a configuration example and a processing example of a conventional technique in which an information processing device 1 is not provided.
[0055] In general, an AI device 2 that handles natural language includes a tokenizer 21 that performs natural language processing such as word decomposition on the sentence to be processed, and an AI model 22 that processes data after the natural language processing and responds to requests.
[0056] When the tokenizer 21 receives the sentence "What is an example of a CSM lightning damage countermeasure?" from the user terminal 3, the tokenizer 21 converts "CSM" into a numeric value because "CSM" is an unassigned vocabulary (unlearned vocabulary) and cannot be converted into a numeric value. <unk>" and the assigned vocabulary (learned vocabulary) is converted to the corresponding numerical value and input into the AI model 22.
[0057] AI model 22 processes the request, but <unk>" is output as is, and the rest of the response is output as a numerical value. <unk>", etc., may cause the tokenizer 21 to output a response that is unrelated to the request. <unk>" is output as is to the screen of the user terminal 3, and the numerical values are replaced with the corresponding characters and output.
[0058] As a result, in response to the question, "What are some examples of CSM lightning protection measures?" <unk>The response "Fever..." is displayed on the screen of the user terminal 3. <unk>" is unknown to the user, and "I have a fever..." is a response that is not relevant to the question.
[0059] On the other hand, in this embodiment, before inputting a sentence to the tokenizer 21, unassigned vocabulary contained in the sentence is expressed by another vocabulary, and the sentence using the another vocabulary is additionally learned by the AI model. This makes it possible for the AI model to respond appropriately even if technical terms, etc. contained in the sentence are vocabulary unassigned by the AI model.
[0060] Specifically, as shown in FIG. 6(b), the information processing device 1 is interposed between the user terminal 3 and the tokenizer 21.
[0061] When the information processing device 1 receives the sentence "What are some examples of CSM's lightning damage prevention measures?" from the user terminal 3, it replaces "CSM", which is an unassigned vocabulary for the AI model 22, with "CSM", which is an assigned vocabulary, and sends it to the tokenizer 21.
[0062] The tokenizer 21 and the AI model 22 perform existing operations.
[0063] When the tokenizer 21 receives the question “What are some examples of CSM’s lightning damage countermeasures?” from the information processing device 1, since there is no unassigned vocabulary, it converts all the vocabulary into corresponding numerical values and inputs them to the AI model 22.
[0064] AI model 22 is the input " <unk>" is not included, and since the AI model 22 has previously learned a sentence in which "CSM" is replaced with "シーエスエム", it is able to process the input. The AI model 22 outputs the processed numerical value to the tokenizer 21 after processing the input.
[0065] Thereafter, the tokenizer 21 replaces the numerical values contained in the output from the AI model 22 with the corresponding characters and transmits the replaced response sentence, such as "Install an insulation module from CSM as a lightning damage countermeasure." to the information processing device 1 that originated the processing request.
[0066] The information processing device 1 restores “CSM” contained in the response sentence from the tokenizer 21 to the original “CSM”, and outputs “Install an insulation module as a lightning protection measure for CSM” to the screen of the user terminal 3.
[0067] As a result, in response to the question "What is an example of a lightning protection measure for CSM?", for example, a response such as "Install an insulation module as a lightning protection measure for CSM" is displayed on the screen of the user terminal 3. Seeing this, the user can correctly understand the answer from the AI model 22 to his / her question.
[0068] [Effects of the embodiment] According to this embodiment, the information processing device 1 is equipped with a first processing unit 11 that replaces unknown vocabulary that has not been learned by the AI model, which is contained in a sentence A for additional learning, with another vocabulary and has the AI model additionally learn the replaced replacement sentence A', and a second processing unit 12 that replaces unknown vocabulary contained in a sentence Bin for a processing request to the AI model with another vocabulary, inputs the replaced replacement sentence Bin' to the AI model, and returns the other vocabulary contained in the response sentence Bout from the AI model to unknown vocabulary.Therefore, it becomes possible for the AI model to process unlearned vocabulary through simple additional learning, making it easy to apply the AI model to various usage environments.
[0069] Furthermore, according to this embodiment, the first processing unit 11 additionally learns the additional learning sentence A in the AI model, and determines unknown vocabulary based on the results of the additional learning. This makes it possible to apply the present invention to cases where the AI model does not know unknown vocabulary that has not been learned, making it easier to apply the present invention to a wider variety of AI model usage environments.
[0070] For example, the information processing device 1 according to the present embodiment is applicable to Generative Pre-trained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT).
[0071] [others] The present disclosure is not limited to the above-described embodiment, and various modifications are possible within the scope of the present disclosure.
[0072] The information processing device 1 of the present embodiment described above can be realized, for example, by using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in Fig. 7. The memory 902 and the storage 903 are storage devices. In the computer system, the CPU 901 executes a predetermined program loaded on the memory 902, thereby realizing each function of the information processing device 1.
[0073] The information processing device 1 may be implemented by one computer. The information processing device 1 may be implemented by multiple computers. The information processing device 1 may be a virtual machine implemented in a computer. The program for the information processing device 1 may be stored in a computer-readable recording medium such as an HDD, SSD, USB memory, CD, or DVD. The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the information processing device 1 may also be distributed via a communication network. [Explanation of symbols]
[0074] 1. Information processing device 11 First processing section 12 Second processing section 13 Storage section 2 AI device 21 Tokenizer 22 AI models 3. User terminal 901 CPU 902 Memory 903 Storage 904 Communication equipment 905 Input Device 906 Output Device< / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk> < / unk>
Claims
1. A first processing unit that replaces unknown vocabulary that is included in a first sentence and that has not been learned by an AI model with another vocabulary, and additionally learns the replaced sentence in the AI model; A second processing unit that replaces the unknown vocabulary included in a second sentence to the AI model with the different vocabulary, inputs the replaced replacement sentence to the AI model, and returns the different vocabulary included in a response sentence from the AI model to the unknown vocabulary; An information processing device comprising:
2. The first processing unit includes: The information processing device according to claim 1 , further comprising: additionally learning the first sentence in the AI model; and determining the unknown vocabulary based on a result of the additional learning.
3. the first processing unit generates a correspondence table in which the unknown vocabulary and the other vocabulary are associated with each other; The information processing apparatus according to claim 1 , wherein the second processing unit replaces the unknown vocabulary included in the second sentence with the different vocabulary by using the correspondence table.
4. An information processing method performed by an information processing device, Replacing unknown vocabulary that is included in the first sentence and that has not been learned by the AI model with another vocabulary, and additionally learning the replaced sentence in the AI model; replacing the unknown vocabulary included in a second sentence to the AI model with the different vocabulary, inputting the replaced replacement sentence to the AI model, and restoring the different vocabulary included in a response sentence from the AI model to the unknown vocabulary; Information processing methods.
5. An information processing program that causes a computer to function as the information processing device according to any one of claims 1 to 3.
Citation Information
Patent Citations
Unknown word detection method and unknown word detection device
JP2022042033A