Training dataset generation system, training dataset generation method, and program

Abstract

To provide a learning dataset generation system, a learning dataset generation method and a program which can efficiently and accurately generate a learning dataset used for learning of a determination model.SOLUTION: A learning dataset generation system 1 comprises: an estimation harmful text acquisition unit 1200 which inputs a harmful text generation prompt GP1 to a language model LM, and acquires an estimation harmful text ET1 outputted from the language model LM; a harmful verification unit 1202 which inputs a harmful verification prompt VP1 to the language model LM, and acquires a harmful verification result VR1 outputted from the language model LM; and an addition determination unit 124 which determines whether or not to add an estimation harmful text ET1 to a learning dataset, on the basis of the harmful verification result VR1, and adds the estimation harmful text ET1 to the learning dataset as a harmful text, when determining to add the estimation harmful text ET1 to the learning dataset.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present invention relates to a training dataset generation system, a training dataset generation method, and a program. [Background technology]

[0002] With the spread of SNS (Social Network Services), in recent years, development has been progressing on machine learning models (hereinafter referred to as "determination models") that accept text to be judged as input and determine whether the text to be judged contains harmful content.

[0003] Furthermore, various techniques are being studied for generating a learning dataset that is used for training the above-mentioned determination model and that includes harmful text that indicates harmful content (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3309-3326, Dublin, Ireland. Association for Computational Linguistics. Summary of the Invention [Problem to be solved by the invention]

[0005] There is a need for a technology that can generate training datasets more efficiently and accurately than the conventional technology. The generation of training datasets by conventional technology involves manual work. Therefore, there is room for improvement in terms of efficiency and accuracy.

[0006] The present invention has been made in consideration of the above-mentioned problems, and one of its objectives is to provide a training dataset generation system, a training dataset generation method, and a program that can efficiently and accurately generate training datasets used for training a determination model. [Means for solving the problem]

[0007] A training dataset generation system according to one embodiment of the present invention is a training dataset generation system that receives a text to be judged as an input, is used to train a judgment model that judges whether the text to be judged indicates harmful content, and generates a training dataset including harmful text that indicates harmful content. The training dataset generation system includes: a text generation means including at least one language model; an estimated harmful text acquisition means that inputs a harmful text generation prompt to the text generation means to generate text that indicates harmful content and acquires estimated harmful text output from the text generation means; a harmful verification means that inputs a harmful verification prompt to the text generation means to verify whether the estimated harmful text indicates harmful content and acquires a harmful verification result output from the text generation means; and an addition determination means that determines whether or not to add the estimated harmful text to the training dataset based on the harmful verification result, and adds the estimated harmful text to the training dataset as the harmful text if it is determined that the estimated harmful text should be added to the training dataset. [Effects of the Invention]

[0008] According to the present invention, a learning data set used for training a determination model can be generated efficiently and accurately. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a hardware configuration of a training dataset generation system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a flow diagram illustrating an example of a harmful text generation process. [Figure 3] FIG. 10 is a diagram illustrating a process for acquiring estimated harmful text. [Figure 4] FIG. 10 is a diagram illustrating a verification process for estimated harmful text. [Figure 5] FIG. 10 is a flow diagram illustrating an example of a harmless text generation process. [Figure 6] FIG. 10 is a diagram illustrating a process for acquiring estimated harmless text. [Figure 7] FIG. 10 is a diagram illustrating a verification process for presumed harmless text. [Figure 8] FIG. 10 is a flow diagram illustrating an example of a non-explicit harmless text generation process. [Figure 9] FIG. 10 is a diagram illustrating a process for acquiring estimated non-explicit harmless text. [Figure 10] FIG. 10 is a diagram illustrating a verification process for estimated non-explicit harmless text. [Figure 11] FIG. 10 is a flow diagram illustrating an example of a non-explicit harmful text generation process. [Figure 12] FIG. 10 is a diagram illustrating a process for acquiring estimated non-explicit harmful text. [Figure 13] FIG. 10 is a diagram illustrating a process of estimating a text type. [Figure 14] FIG. 10 is a diagram illustrating a text type estimation result. [Figure 15] FIG. 1 is a functional block diagram showing an example of functions realized by a training dataset generation system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [1. Hardware configuration of the training dataset generation system] FIG. 1 is a diagram showing an example of the hardware configuration of a training dataset generation system 1 according to an embodiment of the present invention. The training dataset generation system 1 according to this embodiment is a computer such as a personal computer. As shown in FIG. 1, the training dataset generation system 1 includes a text generation server 10 (text generation means) and a user terminal 12. The text generation server 10 and the user terminal 12 are connected to each other via a network N such as the Internet or a LAN.

[0012] In this embodiment, the training dataset generation system 1 is configured by a text generation server 10 and a user terminal 12, which are different computers, but the training dataset generation system 1 may be configured by a single computer. Also, the text generation server 10 and the user terminal 12 may each be configured by multiple computers. When the text generation server 10 or the user terminal 12 is configured by multiple computers, it may be implemented by a cloud system.

[0013] The text generation server 10 is a server computer including a control unit 10a, a storage unit 10b, and a communication unit 10c. For example, the storage unit 10b is a program-controlled device such as a CPU that operates according to a program stored in the storage unit 10b. The storage unit 10b is, for example, a storage element such as a ROM or RAM, or a hard disk drive. The storage unit 10b stores programs and the like executed by the control unit 10a. The communication unit 10c is a communication interface such as a network board or a wireless LAN module.

[0014] The user terminal 12 is a personal computer including a control unit 12a, a storage unit 12b, a communication unit 12c, an operation unit 12d, and a display unit 12e. The physical configurations of the control unit 12a, the storage unit 12b, and the communication unit 12c may be similar to the physical configurations of the control unit 10a, the storage unit 10b, and the communication unit 10c, respectively. The operation unit 12d is an input device such as a keyboard, a mouse, or a touch panel. The display unit 12e is a display such as a liquid crystal display or an organic EL display.

[0015] [2. Processing performed by the training dataset generation system] The following describes the processing executed by the training dataset generation system 1 according to this embodiment with reference to FIGS.

[0016] The training dataset generated by the training dataset generation system 1 according to this embodiment (hereinafter, sometimes simply referred to as the "training dataset") is a dataset that includes one or more of four types of text data: harmful text that explicitly indicates harmful content, implicit harmful text that implicitly indicates harmful content, harmless text that explicitly indicates harmless content, and implicit harmless text that implicitly indicates harmless content. Therefore, the following description will be divided into a harmful text generation process that generates harmful text, a harmless text generation process that generates harmless text, a harmless text generation process that generates implicit harmless text, and a harmless text generation process that generates implicit harmful text.

[0017] [2-1. Harmful text generation process] Fig. 2 is a flow diagram showing an example of harmful text generation processing, Fig. 3 is a diagram explaining processing for acquiring estimated harmful text ET1, and Fig. 4 is a diagram explaining processing for verifying estimated harmful text ET1.

[0018] (1) Obtaining suspected harmful text First, as shown in FIG. 3, the learning dataset generation system 1 inputs a harmful text generation prompt GP1 into the language model LM, and obtains estimated harmful text ET1 output from the language model LM.

[0019] 2, the user terminal 12 transmits a harmful text generation prompt GP1 to the text generation server 10 (S200). The text generation server 10 receives the harmful text generation prompt GP1 and inputs the harmful text generation prompt GP1 into the language model LM to obtain estimated harmful text ET1 output from the language model LM (S202). The text generation server 10 then transmits the estimated harmful text ET1 output from the language model LM to the user terminal 12 (S204).

[0020] The language model LM is a machine learning model that performs natural language processing and is generated by learning using a large amount of text data. The language model LM is a machine learning model that is trained to receive a prompt as input and output a response sentence according to the content indicated by the prompt. The prompt is text data that includes, for example, a directive that instructs the output content of the language model LM, a conditional statement that indicates a condition for restricting the output content of the language model LM, etc.

[0021] Note that some language models are equipped with a mechanism (moderation mechanism) that limits the generation of harmful text, but in this embodiment, a language model LM that does not have such a mechanism is used. Specifically, in this embodiment, a case where the language model LM is Mistral-7B (Mistral AI) is exemplified. However, the language model LM is not limited to this example, and may be a so-called large-scale language model having more parameters than Mistral-7B. Alternatively, the language model LM may be a language model having fewer parameters than Mistral-7B. Furthermore, in this embodiment, a case where the language model LM is in a so-called chatbot format is exemplified, but the language model LM does not have to be in a chatbot format.

[0022] The harmful text generation prompt GP1 is a prompt that instructs the user to generate text that indicates harmful content. Specifically, the harmful text generation prompt GP1 includes a harmful content specifying phrase GP1a that specifies the harmful content indicated by the text to be generated, and a directive phrase GP1b that instructs the user to generate text that indicates the harmful content specified by the harmful content specifying phrase GP1a.

[0023] Harmful content may be, for example, hate speech, harassment, self-harming content, obscene content, violent content, or cruel content. That is, the harmful text generation prompt GP1 may be, for example, a prompt instructing the user to generate text containing hate speech, harassment, self-harming content, obscene content, violent content, or cruel content. In this embodiment, the harmful content is hate speech. Hate speech refers to statements or actions that attack, threaten, insult, or discriminate against individuals or groups belonging to a specific community based on race, gender, ethnicity, religion, country of origin, sexual orientation, disability, caste, economic or social origin, etc. Figure 3 shows the harmful text generation prompt GP1 of this embodiment, but the text is not limited to this example.

[0024] More specifically, the harmful text generation prompt GP1 of this embodiment is a prompt that instructs the user to generate text that explicitly indicates harmful content. The harmful text generation prompt GP1 includes explicit phrases GP1c for making the content indicated by the text explicit. Examples of explicit phrases GP1c include phrases such as "explicitly," "clearly," "obviously," "unequivocally," "unequivocally," and "directly." Note that the harmful text generation prompt GP1 does not have to include explicit phrases GP1c. In other words, the harmful text generation prompt GP1 may simply be a prompt that instructs the user to generate text that indicates harmful content (without specifying whether it is explicit or not).

[0025] (2) Verification of suspected harmful text Next, as shown in FIG. 4, the learning dataset generation system 1 inputs the harmful verification prompt VP1 into the language model LM, and obtains the harmful verification result VR1 output from the language model LM.

[0026] 2, the user terminal 12 transmits a harm verification prompt VP1 to the text generation server 10 (S206). The text generation server 10, having received the harm verification prompt VP1, inputs the harm verification prompt VP1 into the language model LM and obtains a harm verification result VR1 output from the language model LM (S208). The text generation server 10 then transmits the harm verification result VR1 output from the language model LM to the user terminal 12 (S210).

[0027] The harm verification prompt VP1 is a prompt that instructs the user to verify whether the presumed harmful text ET1 indicates harmful content. Specifically, the harm verification prompt VP1 is a prompt that instructs the user to verify whether the presumed harmful text ET1 explicitly indicates harmful content. More specifically, the harm verification prompt VP1 includes a harmful content specifying phrase VP1a for specifying harmful content, and a directive phrase VP1b for instructing the user to verify whether the presumed harmful text ET1 explicitly indicates harmful content specified by the harmful content specifying phrase VP1a. Note that if the harmful text generation prompt GP1 does not include a directive phrase GP1c, the harm verification prompt VP1 may simply be a prompt that verifies whether the presumed harmful text ET1 indicates harmful content (without specifying whether it is explicit or not).

[0028] It is preferable that the harm verification prompt VP1 be a question in the form of a closed question. This increases the likelihood that the harm verification result VR1 output from the language model LM will be text containing either affirmative phrases (e.g., Yes) or negative phrases (e.g., No) (or phrases equivalent thereto), as shown in Fig. 4, which facilitates the processing of step S212, which will be described later. It is also preferable that the harm verification prompt VP1 further includes phrases for instructing the respondent to answer with either affirmative phrases or negative phrases.

[0029] Furthermore, in this embodiment, session information between the user terminal 12 and the text generation server 10 is stored in a storage unit of the text generation server 10 on which the language model LM operates. In S22, it is not necessary to input the estimated harmful text ET1 to be verified again into the language model LM. However, in S22, the estimated harmful text ET1 may be input again into the language model LM. This also applies to the harmless text generation process and the implicit harmless text generation process described below.

[0030] (3) Addition of judgment to the training dataset Finally, the training dataset generation system 1 determines whether to add the estimated harmful text ET1 to the training dataset based on the harmful verification result VR1, and if it determines to add the estimated harmful text ET1 to the training dataset, it adds the estimated harmful text ET1 to the training dataset as harmful text.

[0031] Specifically, as shown in FIG. 2, the user terminal 12 refers to the harm verification result VR1 and determines whether the estimated harmful text ET1 explicitly indicates harmful content (S212). If the user terminal 12 determines that the estimated harmful text ET1 explicitly indicates harmful content (S212; Y), it stores the estimated harmful text ET1 in the storage unit 12b as harmful text (S214) and terminates this processing. If the user terminal 12 determines that the estimated harmful text ET1 does not explicitly indicate harmful content (S212; N), it terminates this processing. In S212, if the user terminal 12 determines that the estimated harmful text ET1 does not explicitly indicate harmful content, it may delete the estimated harmful text ET1.

[0032] Specifically, in S212, the user terminal 12 performs syntactic analysis on the harmful verification result VR1 and determines whether the harmful verification result VR1 includes positive phrases or negative phrases. If the user terminal 12 determines in S212 that the harmful verification result VR1 includes positive phrases, it determines that the estimated harmful text ET1 explicitly indicates harmful content. If the user terminal 12 determines in S212 that the harmful verification result VR1 includes negative phrases, it determines that the estimated harmful text ET1 does not explicitly indicate harmful content.

[0033] According to the harmful text generation process according to the present embodiment described above, harmful text can be generated without human intervention, and therefore a learning dataset can be obtained efficiently and accurately.

[0034] [2-2. Harmless text generation process] Fig. 5 is a flow diagram showing an example of harmless text generation processing, Fig. 6 is a diagram explaining processing for obtaining estimated harmless text ET2, and Fig. 7 is a diagram explaining verification processing for estimated harmless text ET2.

[0035] (1) Obtaining presumed harmless text First, as shown in FIG. 6, the learning dataset generation system 1 inputs a harmless text generation prompt GP2 into the language model LM, and obtains estimated harmless text ET2 output from the language model LM.

[0036] 5, the user terminal 12 transmits a harmless text generation prompt GP2 to the text generation server 10 (S500). The text generation server 10 receives the harmless text generation prompt GP2 and inputs the harmless text generation prompt GP2 into the language model LM to obtain estimated harmless text ET2 output from the language model LM (S502). The text generation server 10 then transmits the estimated harmless text ET2 output from the language model LM to the user terminal 12 (S504).

[0037] The harmless text generation prompt GP2 is a prompt that instructs the user to generate text that indicates harmless content. Specifically, the harmless text generation prompt GP2 includes a harmless content specifying phrase GP2a for specifying the harmless content indicated by the text to be generated, and a directive phrase GP2b for instructing the user to generate text that indicates the harmless content specified by the harmless content specifying phrase GP2a. The harmless content specifying phrase GP2a is, for example, a phrase such as "harmless," "safe," "non-dangerous," or "wholesome."

[0038] More specifically, the harmless text generation prompt GP2 in this embodiment is a prompt that instructs the user to generate text that explicitly indicates harmless content. In this embodiment, in which the harmful content is hate speech, text that explicitly indicates harmless content refers, for example, to text that does not mention a specific community. Text that mentions a specific community is likely to be erroneously determined by the determination model to indicate harmful content, even if it does not contain any offensive, threatening, insulting, or discriminatory remarks or behavior. In this way, text that mentions a specific community can be said to be text that implicitly indicates harmless content. Based on this, in this embodiment, the harmless text generation prompt GP2 includes a non-mention phrase GP2c that instructs the user not to mention the specific community.

[0039] (2) Verification of Presumed Harmless Text Next, as shown in FIG. 7, the learning dataset generation system 1 inputs the harmless verification prompt VP2 to the language model LM, and obtains the harmless verification result VR2 output from the language model LM.

[0040] 5, the user terminal 12 transmits a harmless verification prompt VP2 to the text generation server 10 (S506). The text generation server 10, having received the harmless verification prompt VP2, inputs the harmless verification prompt VP2 into the language model LM and obtains a harmless verification result VR2 output from the language model LM (S508). The text generation server 10 then transmits the harmless verification result VR2 output from the language model LM to the user terminal 12 (S510).

[0041] The harmlessness verification prompt VP2 is a prompt that instructs the user to verify whether the presumed harmless text ET2 indicates harmless content. Specifically, the harmlessness verification prompt VP2 is a prompt that instructs the user to verify whether the presumed harmless text ET2 explicitly indicates harmless content. More specifically, the harmlessness verification prompt VP2 includes a harmless content specifying phrase VP2a for specifying harmless content, and a directive phrase VP2b for instructing the user to verify whether the presumed harmless text ET2 explicitly indicates the harmless content specified by the harmless content specifying phrase VP2a.

[0042] The harmless verification prompt VP2 is preferably a closed-question question, similar to the harmful verification prompt VP1. The harmless verification prompt VP2 may further include a phrase for instructing the respondent to respond with either a positive or negative phrase.

[0043] (3) Addition of judgment to the training dataset Finally, the training dataset generation system 1 determines whether to add the estimated harmless text ET2 to the training dataset based on the harmlessness verification result VR2, and if it determines to add the estimated harmless text ET2 to the training dataset, it adds the estimated harmless text ET2 to the training dataset as harmless text.

[0044] Specifically, as shown in FIG. 5, the user terminal 12 refers to the harmlessness verification result VR2 and determines whether the estimated harmless text ET2 explicitly indicates harmless content (S512). If the user terminal 12 determines that the estimated harmless text ET2 explicitly indicates harmless content (S512; Y), the user terminal 12 stores the estimated harmless text ET2 in the storage unit 12b as harmless text (S214) and terminates this processing. If the user terminal 12 determines that the estimated harmless text ET2 does not explicitly indicate harmless content (S512; N), the user terminal 12 terminates this processing as is. In S512, if the user terminal 12 determines that the estimated harmless text ET2 does not explicitly indicate harmless content, the user terminal 12 may delete the estimated harmless text ET2.

[0045] Specifically, in S512, the user terminal 12 parses the harmless verification result VR2 and determines whether the harmless verification result VR2 includes positive phrases or negative phrases. If the user terminal 12 determines in S512 that the harmless verification result VR2 includes positive phrases, it determines that the estimated harmless text ET2 explicitly indicates harmless content. If the user terminal 12 determines in S512 that the harmless verification result VR2 includes negative phrases, it determines that the estimated harmless text ET2 does not explicitly indicate harmless content.

[0046] According to the harmless text generation process according to this embodiment described above, harmless text can be generated without human intervention, and therefore a learning dataset can be obtained efficiently and accurately.

[0047] [2-3. Non-explicit harmless text generation process] Fig. 8 is a flow diagram showing an example of non-explicit harmless text generation processing, Fig. 9 is a diagram explaining processing for acquiring estimated non-explicit harmless text ET3, and Fig. 10 is a diagram explaining processing for verifying estimated non-explicit harmless text ET3.

[0048] (1) Acquisition of presumed non-specific harmless text First, as shown in FIG. 9, the learning dataset generation system 1 inputs the non-explicit harmless text generation prompt GP3 ​​into the language model LM, and obtains estimated non-explicit harmless text ET3 output from the language model LM.

[0049] 8, the user terminal 12 transmits a non-explicit harmless text generation prompt GP3 ​​to the text generation server 10 (S800). The text generation server 10, which has received the non-explicit harmless text generation prompt GP3, inputs the non-explicit harmless text generation prompt GP3 ​​into the language model LM and acquires estimated non-explicit harmless text ET3 output from the language model LM (S802). The text generation server 10 then transmits the estimated non-explicit harmless text ET3 output from the language model LM to the user terminal 12 (S804).

[0050] The non-explicit harmless text generation prompt GP3 ​​is a prompt that instructs the user to generate text that non-explicitly indicates harmless content. In this embodiment, in which the harmful content is hate speech, the non-explicit harmless text generation prompt GP3 ​​specifically includes a reference phrase GP3a that instructs the user to reference a specific community and a directive phrase GP3b that instructs the user to generate text that indicates harmless content. This allows the user to generate non-explicit harmless text that mentions a specific community but does not include any offensive, threatening, insulting, or discriminatory statements or behavior.

[0051] (2) Verification of Presumed Non-Explicitly Harmless Text Next, as shown in FIG. 10, the learning dataset generation system 1 inputs the implicit harmlessness verification prompt VP3 to the language model LM, and obtains the implicit harmlessness verification result VR3 output from the language model LM.

[0052] 8, the user terminal 12 transmits the implicit harmless verification prompt VP3 to the text generation server 10 (S806). The text generation server 10, which has received the implicit harmless verification prompt VP3, inputs the implicit harmless verification prompt VP3 into the language model LM and obtains the implicit harmless verification result VR3 output from the language model LM (S808). The text generation server 10 then transmits the implicit harmless verification result VR3 output from the language model LM to the user terminal 12 (S810).

[0053] The implicit harmless verification prompt VP3 is a prompt that instructs to verify whether the presumed implicit harmless text ET3 implicitly indicates harmless content. Specifically, the implicit harmless verification prompt VP3 is a prompt that instructs to verify whether the presumed implicit harmless text ET3 implicitly indicates harmless content. More specifically, the implicit harmless verification prompt VP3 includes a harmless content specifying phrase VP3a for specifying harmless content, and a directive phrase VP3b for instructing to verify whether the presumed implicit harmless text ET3 implicitly indicates the harmless content specified by the harmless content specifying phrase VP3a.

[0054] The implicit harmless verification prompt VP3 is preferably a closed-question question, similar to the harmful verification prompt VP1. The implicit harmless verification prompt VP3 may further include a phrase for instructing the respondent to respond with either a positive or negative phrase.

[0055] (3) Addition of judgment to the training dataset Finally, the training dataset generation system 1 determines whether to add the estimated non-explicitly harmless text ET3 to the training dataset based on the non-explicitly harmless verification result VR3, and if it determines to add the estimated non-explicitly harmless text ET3 to the training dataset, it adds the estimated non-explicitly harmless text ET3 to the training dataset as non-explicitly harmless text.

[0056] Specifically, as shown in FIG. 8, the user terminal 12 refers to the non-explicit harmless verification result VR3 and determines whether the estimated non-explicit harmless text ET3 non-explicitly indicates harmless content (S812). If the user terminal 12 determines that the estimated non-explicit harmless text ET3 non-explicitly indicates harmless content (S812; Y), the user terminal 12 stores the estimated non-explicit harmless text ET3 in the storage unit 12b as harmless text (S214) and terminates this processing. If the user terminal 12 determines that the estimated non-explicit harmless text ET3 does not non-explicitly indicate harmless content (S812; N), the user terminal 12 terminates this processing as is. In S812, if the user terminal 12 determines that the estimated non-explicit harmless text ET3 does not non-explicitly indicate harmless content, the user terminal 12 may delete the estimated non-explicit harmless text ET3.

[0057] Specifically, in S812, the user terminal 12 parses the implicit harmless verification result VR3 and determines whether the implicit harmless verification result VR3 includes a positive phrase or a negative phrase. If the user terminal 12 determines in S812 that the implicit harmless verification result VR3 includes a positive phrase, the user terminal 12 determines that the estimated implicit harmless text ET3 implicitly indicates harmless content. If the user terminal 12 determines in S812 that the implicit harmless verification result VR3 includes a negative phrase, the user terminal 12 determines that the estimated implicit harmless text ET3 does not implicitly indicate harmless content.

[0058] According to the non-explicitly harmless text generation process according to this embodiment described above, non-explicitly harmless text can be generated without human intervention, so that a learning dataset can be obtained efficiently and accurately.

[0059] [2-4. Non-explicit harmful text generation process] Fig. 11 is a flow diagram showing an example of non-explicit harmful text generation processing. Fig. 12 is a diagram explaining processing for acquiring estimated non-explicit harmful text ET4. Fig. 13 is a diagram explaining processing for estimating text type. Fig. 14 is a diagram explaining text type estimation results.

[0060] The inventors of the present application initially considered a method of generating non-explicit harmful text, in which estimated non-explicit harmful text ET4 (see Figure 12) obtained using a language model LM was verified using the language model LM, similar to the above-mentioned harmful text generation process, harmless text generation process, and non-explicit harmless text generation process.

[0061] However, the inventors of the present application have found that there is room for improvement in the quality of the non-explicit harmful text obtained by the above method. Specifically, the inventors of the present application generated multiple non-explicit harmful texts using the above method and manually verified whether these texts actually contained non-explicit harmful content. As a result, the texts obtained by the above method contained a large amount of text that explicitly contained harmful content and text that contained harmless content. This is thought to be partly due to the difficulty of accurately verifying whether the estimated non-explicit harmful text ET4 output from the language model LM contains non-explicit harmful content when using the language model LM.

[0062] Therefore, in this embodiment, it is decided to verify whether the estimated implicit harmful text ET4 implicitly indicates harmful content by a method different from the verification using the language model LM. The details of the implicit harmful text generation process according to this embodiment will be described below.

[0063] (1) Acquisition of Presumed Unspecified Harmful Text First, as shown in FIG. 12, the learning dataset generation system 1 inputs the implicit harmful text generation prompt GP4 into the language model LM, and obtains estimated implicit harmful text ET4 output from the language model LM.

[0064] 11, the user terminal 12 transmits an implicit harmful text generation prompt GP4 to the text generation server 10 (S1100). The text generation server 10, which has received the implicit harmful text generation prompt GP4, inputs the implicit harmful text generation prompt GP4 into the language model LM and acquires estimated implicit harmful text ET4 output from the language model LM (S1102). The text generation server 10 then transmits the estimated implicit harmful text ET4 output from the language model LM to the user terminal 12 (S1104), and ends the process.

[0065] The implicit harmful text generation prompt GP4 is a prompt that instructs the user to generate text that implicitly indicates harmful content. Specifically, the implicit harmful text generation prompt GP4 includes a harmful content designation phrase GP4a for specifying the harmful content of the text to be generated, an implicit phrase GP4b for concealing the content of the text, and a directive phrase GP4c for instructing the user to generate text that implicitly indicates the harmful content designated by the harmful content designation phrase GP4a and the implicit phrase GP4b. The implicit phrase GP4b may include, for example, phrases that impart a harmless impression to the text to be generated, such as "appears harmless at first glance" or "give a harmless impression," as well as adversative phrases such as "but" and "however." Alternatively, the implicit phrase GP4b may be phrases such as "unexplicitly," "ambiguously," "unclearly," "indistinctly," or "indirectly."

[0066] (2) Obtaining text type estimation results Next, as shown in FIGS. 12 and 13, the user terminal 12 (learning dataset generation system 1) inputs the estimated non-explicit harmful text ET4 to the text type estimation model M and obtains the text type estimation result TT output from the text type estimation model M (S1106).

[0067] The text type estimation model M receives the text to be estimated as input and outputs a value belonging to the estimation range ER as the text type estimation result TT for the text to be estimated (see FIG. 14). The estimation range ER is a numerical range from a first value of 0 to a second value of 1. In this specification, the "numerical range from value X to value Y" (X and Y are arbitrary values) refers to the numerical range between value X and value Y, including value X and value Y themselves. When X is greater than Y, the "numerical range from value X to value Y" is a numerical range in which X is the upper limit and Y is the lower limit (i.e., Y or more and X or less). When X is smaller than Y, the "numerical range from value X to value Y" is a numerical range in which X is the lower limit and Y is the upper limit (i.e., X or more and X or less).

[0068] A value of 0 output from the text type estimation model M indicates that the text to be estimated indicates harmless content. That is, the first value is a value indicating that the text to be estimated indicates harmless content. On the other hand, a value of 1 output from the text type estimation model M indicates that the text to be estimated explicitly indicates harmful content. That is, the second value is a value indicating that the text to be estimated explicitly indicates harmful content.

[0069] The text type estimation model M is trained based on explicit harmful training data in which learned explicitly harmful text that explicitly indicates harmful content is associated with a value of 1. The text type estimation model M is trained based on harmless training data in which learned harmless text that indicates harmless content is associated with a value of 0. In other words, the text type estimation model M is trained using learned explicitly harmful text as a positive example and learned harmless text as a negative example. Note that in training the text type estimation model M, learned harmless text may be used as a positive example (learned harmless text may be associated with a value of 1) and learned explicitly harmful text may be used as a negative example (learned explicitly harmful text may be associated with a value of 0).

[0070] (3) Addition of judgment to the training dataset 11 and 14, the user terminal 12 (learning dataset generation system 1) determines whether the value of the text type estimation result TT belongs to a predetermined uncertainty range UCR within the estimation range ER (S1108). If it is determined that the value of the text type estimation result TT belongs to the uncertainty range UCR (S1108; Y), it adds the estimated non-explicit harmful text ET4 to the learning dataset as non-explicit harmful text (S1110), and ends the processing. If the user terminal 12 determines that the value of the text type estimation result TT does not belong to the uncertainty range UCR (S1108; N), it ends the processing as is. In S1108, if the user terminal 12 determines that the value of the text type estimation result TT does not belong to the uncertainty range UCR, it may delete the estimated non-explicit harmful text ET4.

[0071] The uncertainty range UCR is a numerical range from a value greater than 0 (third value) to a value less than 1 (fourth value). The closer the third and fourth values ​​are to the median value of the estimation range ER (0.5 in this embodiment), the more preferable. In this embodiment, the uncertainty range UCR is assumed to be a numerical range of 0.4 to 0.6, for example. Of course, these values ​​are merely examples, and the upper limit value (fourth value) and lower limit value (third value) of the uncertainty range UCR may each be any value greater than 0 and less than 1.

[0072] As shown in Figure 14, when the value of the text type estimation result TT falls within the uncertainty range UCR, there is a high probability that the estimated non-explicit harmful text ET4 does not correspond to either explicitly indicating harmful content or harmless content, i.e., corresponds to implicitly indicating harmful content. Therefore, in the non-explicit harmful text generation process of this embodiment, when it is determined that the value of the text type estimation result TT falls within the uncertainty range UCR, the estimated non-explicit harmful text ET4 is added to the training dataset as non-explicit harmful text. This makes it possible to accurately verify whether the estimated non-explicit harmful text ET4 implicitly indicates harmful content. Furthermore, the non-explicit harmful text generation process of this embodiment can generate non-explicit harmful text without human intervention, thereby enabling efficient and accurate acquisition of training datasets.

[0073] [3. Functions realized by the training dataset generation system] FIG. 15 is a functional block diagram showing an example of functions realized by the training dataset generation system 1 according to an embodiment of the present invention.

[0074] [3-1. Text Generation Server] 15, the text generation server 10 functionally includes a language model LM. The details of the language model LM are as described above, and therefore will not be described here.

[0075] [3-2. User terminal] As shown in Figure 2, the user terminal 12 functionally includes a harmful text generation unit 120, a harmless text generation unit 121, an implicit harmless text generation unit 122, an implicit harmful text generation unit 123, a text type estimation model M, an additional determination unit 124, and a learning dataset storage unit 125.

[0076] The harmful text generation unit 120 includes an estimated harmful text acquisition unit 1200 and a harmful verification unit 1202. The estimated harmful text acquisition unit 1200 inputs a harmful text generation prompt GP1 into a language model LM and acquires estimated harmful text ET1 output from the language model LM (see FIGS. 2 and 3). The harmful verification unit 1202 inputs a harmful verification prompt VP1 into the language model LM and acquires a harmful verification result VR1 output from the language model LM (see FIGS. 2 and 4).

[0077] The harmless text generation unit 121 includes an estimated harmless text acquisition unit 1210 and a harmless verification unit 1212. The estimated harmless text acquisition unit 1210 inputs an implicit harmless text generation prompt GP3 ​​to the language model LM and acquires estimated implicit harmless text ET3 output from the language model LM (see FIGS. 5 and 6). The harmless verification unit 1212 inputs a harmless verification prompt VP2 to the language model LM and acquires an implicit harmless verification result VR3 output from the language model LM.

[0078] The non-explicit harmless text generation unit 122 includes an estimated non-explicit harmless text acquisition unit 1220 and a non-explicit harmless verification unit 1222. The estimated non-explicit harmless text acquisition unit 1220 inputs the non-explicit harmless text generation prompt GP3 ​​to the language model LM and acquires estimated non-explicit harmless text ET3 output from the language model LM (see FIGS. 8 and 9). The non-explicit harmless verification unit 1222 inputs the non-explicit harmless verification prompt VP3 to the language model LM and acquires the non-explicit harmless verification result VR3 output from the language model LM (see FIGS. 8 and 10).

[0079] The implicit harmful text generation unit 123 includes an estimated implicit harmful text acquisition unit 1230 and an estimation result acquisition unit 1232. The estimated implicit harmful text acquisition unit 1230 inputs the implicit harmful text generation prompt GP4 to the language model LM and acquires estimated implicit harmful text ET4 output from the language model LM (see FIGS. 11 and 12). The estimation result acquisition unit 1232 inputs the estimated implicit harmful text ET4 to the text type estimation model M and acquires the text type estimation result TT output from the text type estimation model M (see FIGS. 11 and 13).

[0080] The addition determination unit 124 determines whether or not to add the estimated harmful text ET1 to the learning dataset based on the harmful verification result VR1, and if it determines that the estimated harmful text ET1 should be added to the learning dataset, it adds the estimated harmful text ET1 to the learning dataset as harmful text (see Figure 2).

[0081] In addition, the addition determination unit 124 determines whether or not to add the estimated harmless text ET2 to the training dataset based on the harmlessness verification result VR2, and if it determines that the estimated harmless text ET2 should be added to the training dataset, it adds the estimated harmless text ET2 to the training dataset as harmless text (see Figure 5).

[0082] In addition, the addition determination unit 124 determines whether to add the estimated non-explicitly harmless text ET3 to the training dataset based on the non-explicitly harmless verification result VR3, and if it determines that the estimated non-explicitly harmless text ET3 should be added to the training dataset, it adds the estimated non-explicitly harmless text ET3 to the training dataset as non-explicitly harmless text (see Figure 8).

[0083] In addition, the addition determination unit 124 determines whether the value of the text type estimation result TT belongs to a predetermined uncertainty range UCR within the estimation range ER, and if it determines that the value of the text type estimation result TT belongs to the uncertainty range UCR, it adds the estimated non-explicit harmful text ET4 to the learning dataset as non-explicit harmful text (see Figures 11 and 14).

[0084] 15, the addition determination unit 124 stores harmful text, harmless text, implicit harmless text, and implicit harmful text in the learning dataset storage unit 125. Note that the addition determination unit 124 may store these various types of text in an external device or information storage medium.

[0085] [5. Modifications] The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present disclosure.

[0086] For example, in this embodiment, in the non-explicit harmful text generation process, verification using the language model LM is not performed, but instead verification using the text type estimation model M is performed, but it is also possible to perform both verification using the language model LM and verification using the text type estimation model M.

[0087] In addition, in this embodiment, the same language model LM is used in generating the estimated harmful text ET1, estimated harmless text ET2, estimated implicit harmless text ET3, and estimated implicit harmful text ET4 and verifying these generated texts, but different language models may be used for each. For example, the language model used to obtain the estimated harmful text ET1 may be different from the language model used to verify the estimated harmful text ET1. In other words, the text generation server 10 may include multiple language models.

[0088] In addition, in this embodiment, the harmless text generation process, the implicit harmless text generation process, and the implicit harmful text generation process are described, but at least one of these processes may not be performed. For example, if negative example learning is not performed in learning the determination model, there is no need to prepare harmless text and implicit harmless text.

[0089] Furthermore, the text type estimation model M of this embodiment is trained based on explicit harmful training data in which learned explicit harmful text that explicitly indicates harmful content is associated with a second value (1), and learned harmless training data in which learned harmless text that indicates harmless content is associated with a first value (0). However, the text type estimation model may be trained in a different manner. For example, a model trained based on data in which learned implicit harmful text that implicitly indicates harmful content is associated with a value indicating the correct answer (e.g., 1) may be used as the text type estimation model. In this case, for example, it may be determined whether the value of the text type estimation result output from the text type estimation model is equal to or greater than a predetermined threshold (e.g., 0.8), and if it is determined that the value is equal to or greater than the threshold, the estimated implicit harmful text may be added to the training dataset as implicit harmful text.

[0090] Furthermore, although the text type estimation model M of this embodiment is trained using learned explicitly harmful text as a positive example and learned harmless text as a negative example, a text type estimation model that has only undergone positive example training without negative example training may also be used. In other words, a text type estimation model trained solely on learned explicitly harmful text may also be used. In this modified example, even if the text type estimation result output from the text type estimation model is close to the first value, there is a possibility that the estimated implicit harmful text ET4 may be one that implicitly indicates harmful content. Therefore, in this modified example, the uncertainty range UCR may be a numerical range not exceeding a predetermined upper limit value, with the lower limit value of the estimation range ER (0 in this embodiment) as the lower limit. For example, the uncertainty range UCR may be a numerical range between 0 and 0.6.

[0091] [6. Notes] (1) A learning dataset generation system that receives a text to be judged as an input, is used to train a judgment model that judges whether the text to be judged indicates harmful content, and generates a learning dataset including harmful text that indicates harmful content, a text generation means including at least one language model; an estimated harmful text acquisition means for inputting a harmful text generation prompt to the text generation means to generate text indicating harmful content, and acquiring estimated harmful text output from the text generation means; a harmful verification means for inputting a harmful verification prompt to the text generation means to instruct the text generation means to verify whether the estimated harmful text indicates harmful content, and for acquiring a harmful verification result output from the text generation means; an addition determination means for determining whether or not to add the estimated harmful text to the learning data set based on the harmful verification result, and adding the estimated harmful text to the learning data set as the harmful text when it is determined that the estimated harmful text should be added to the learning data set; A training dataset generation system comprising: (2) the harmful text generation prompt is a prompt that instructs the user to generate text that explicitly indicates harmful content; the harmful verification prompt is a prompt that instructs the user to verify whether the suspected harmful text explicitly indicates harmful content; The training dataset generation system according to (1). (3) The training dataset further includes non-explicit harmful text that non-explicitly indicates harmful content; The training dataset generation system includes: an estimated non-explicit harmful text acquisition means for inputting a non-explicit harmful text generation prompt to the text generation means, the non-explicit harmful text generation prompt instructing the text generation means to generate text that non-explicitly indicates harmful content, and acquiring the estimated non-explicit harmful text output from the text generation means; Further comprising: (2) A training dataset generation system. (4) an estimation result acquisition means for inputting the estimated non-explicit harmful text into a text type estimation model that receives an estimation target text as an input, and outputs one value belonging to an estimation range from a first value indicating that the estimation target text does not explicitly indicate harmful content to a second value indicating that the estimation target text explicitly indicates harmful content as a text type estimation result for the estimation target text, and acquiring the text type estimation result output from the text type estimation model; and The addition determination means further determines whether the value of the text type estimation result falls within a predetermined uncertainty range within the estimation range, and when it determines that the value of the text type estimation result falls within the uncertainty range, adds the estimated non-explicit harmful text to the learning dataset as the non-explicit harmful text. (3) A training dataset generation system according to (3). (5) The text type estimation model is trained based on explicit harmful training data in which training explicit harmful text that explicitly indicates harmful content and the second value are associated. (4) A training dataset generation system according to (4). (6) the first value is a value indicating that the text to be inferred is harmless; the text type estimation model is further trained based on harmless training data in which harmless training text indicating harmless content and the first value are associated with each other. A training dataset generation system according to (4) or (5). (7) The training dataset further includes harmless text that indicates harmless content; an estimated harmless text acquisition means for inputting a harmless text generation prompt to the text generation means to generate text showing harmless content, and acquiring estimated harmless text output from the text generation means; a harmless verification means for inputting a harmless verification prompt to the text generation means to instruct the text generation means to verify whether the content indicated by the presumed harmless text indicates harmless content, and obtaining a harmless verification result output from the text generation means; and The addition determination means further determines whether or not to add the presumed harmless text to the training data set based on the harmlessness verification result, and when it is determined that the presumed harmless text should be added to the training data set, adds the presumed harmless text to the training data set as the harmless text. The training dataset generation system according to any one of (1) to (6). [Explanation of symbols]

[0092] 1 Training dataset generation system, N network, 10 text generation server, 12 user terminal, 10a, 12a control unit, 10b, 12b memory unit, 10c, 12c communication unit, 12d operation unit, 12e display unit, 120 harmful text generation unit, 121 harmless text generation unit, 122 non-explicit harmless text generation unit, 123 non-explicit harmful text generation unit, 124 additional judgment unit, 125 training dataset memory unit, 1200 estimated harmful text acquisition unit, 1202 harmful verification unit, 1210 estimated harmless text acquisition unit, 1212 harmless verification unit, 1220 estimated non-explicit harmless text acquisition unit, 1222 non-explicit harmless verification unit, 1230 estimated non-explicit harmful text acquisition unit, 1232 estimation result acquisition unit, LM language model, ET1 estimated harmful text, ET2 estimated harmless text, ET3 Estimated non-explicit harmless text, ET4 estimated non-explicit harmful text, GP1 harmful text generation prompt, GP1a harmful content specification phrase, GP1b, GP2b, GP3b, GP4c instruction phrase, GP1c explicit phrase, GP2 harmless text generation prompt, GP2a harmless content specification phrase, GP2c non-mention phrase, GP3 non-explicit harmless text generation prompt, GP3a mention phrase, GP4 non-explicit harmful text generation prompt, GP4a harmful content specification phrase, GP4b non-explicit phrase, VP1 harmful verification prompt, VP1a harmful content specification phrase, VP1b, VP2b, VP3b instruction phrase, VP2 harmless verification prompt, VP2a harmless content specification phrase, VP3 non-explicit harmless verification prompt, VP3a harmless content specification phrase, VR1 harmful verification result, VR2 harmless verification result, VR3 non-explicit harmless verification result, M text type estimation model, TT Text type estimation result, ER estimation range, UCR uncertainty range.

Claims

1. A learning dataset generation system that receives a text to be judged as an input, is used to train a judgment model that judges whether the text to be judged indicates harmful content, and generates a learning dataset that includes implicit harmful text that implicitly indicates harmful content, a text generation means including at least one language model; an estimated non-explicit harmful text acquisition means for inputting a non-explicit harmful text generation prompt to the text generation means, the non-explicit harmful text generation prompt instructing the text generation means to generate text that non-explicitly indicates harmful content, and acquiring the estimated non-explicit harmful text output from the text generation means; an estimation result acquisition means for inputting the estimated non-explicit harmful text into a text type estimation model that receives an input of a text to be estimated, and outputs a value belonging to an estimation range from a first value indicating that the text to be estimated does not explicitly indicate harmful content to a second value indicating that the text to be estimated explicitly indicates harmful content as a text type estimation result for the text to be estimated, and acquiring the text type estimation result output from the text type estimation model; an addition determination means for determining whether the value of the text type estimation result falls within a predetermined uncertainty range within the estimation range, and adding the estimated non-declared harmful text to the learning dataset as the non-declared harmful text when it is determined that the value of the text type estimation result falls within the uncertainty range; A training dataset generation system comprising:

2. the text type estimation model is trained based on explicit harmful training data in which training explicit harmful text that explicitly indicates harmful content is associated with the second value; The training dataset generation system according to claim 1 .

3. the first value is a value indicating that the text to be inferred is harmless; the text type estimation model is further trained based on harmless training data in which harmless training text indicating harmless content is associated with the first value; The training dataset generation system according to claim 1 or 2.

4. The training dataset further includes harmless text that indicates harmless content; an estimated harmless text acquisition means for inputting a harmless text generation prompt to the text generation means to generate text showing harmless content, and acquiring estimated harmless text output from the text generation means; a harmless verification means for inputting a harmless verification prompt to the text generation means to instruct the text generation means to verify whether the content indicated by the presumed harmless text indicates harmless content, and obtaining a harmless verification result output from the text generation means; and The addition determination means further determines whether or not to add the presumed harmless text to the training data set based on the harmlessness verification result, and when it is determined that the presumed harmless text should be added to the training data set, adds the presumed harmless text to the training data set as the harmless text. The training dataset generation system according to claim 1 or 2.

5. The training dataset further includes explicit harmful text that explicitly indicates harmful content; an estimated explicit harmful text acquisition means for inputting an explicit harmful text generation prompt to the text generation means, the explicit harmful text generation prompt instructing the text generation means to generate text that explicitly indicates harmful content, and acquiring the estimated explicit harmful text output from the text generation means; an explicit harmful verification means for inputting an explicit harmful verification prompt to the text generation means to verify whether the estimated explicit harmful text explicitly indicates harmful content, and obtaining an explicit harmful verification result output from the text generation means; and The addition determination means further determines whether or not to add the estimated explicit harmful text to the learning dataset based on the explicit harmful verification result, and when it determines that the estimated explicit harmful text should be added to the learning dataset, adds the estimated explicit harmful text to the learning dataset as the explicit harmful text. The training dataset generation system according to claim 1 or 2.

6. A training dataset generation method executed by a training dataset generation system that receives a text to be judged as an input, is used to train a judgment model that judges whether the text to be judged indicates harmful content, and generates a training dataset that includes implicit harmful text that implicitly indicates harmful content, an estimated non-explicit harmful text acquisition step of inputting a non-explicit harmful text generation prompt, which instructs a text generating means including at least one language model to generate text that non-explicitly indicates harmful content, to the text generating means, and acquiring estimated non-explicit harmful text output from the text generating means; an estimation result acquisition step of inputting the estimated non-explicit harmful text into a text type estimation model that receives a text to be estimated as an input, and outputs a value belonging to an estimation range from a first value indicating that the text to be estimated does not explicitly indicate harmful content to a second value indicating that the text to be estimated explicitly indicates harmful content as a text type estimation result for the text to be estimated, and acquiring the text type estimation result output from the text type estimation model; an additional determination step of determining whether the value of the text type estimation result falls within a predetermined uncertainty range within the estimation range, and adding the estimated non-declared harmful text to the training dataset as the non-declared harmful text when it is determined that the value of the text type estimation result falls within the uncertainty range; A training dataset generation method comprising:

7. A program for causing a computer to function to generate a training dataset including non-explicit harmful text that receives a target text as an input, is used to train a determination model that determines whether the target text indicates harmful content, and does not explicitly indicate harmful content, a text generation means including at least one language model; an estimated non-explicit harmful text acquisition means for inputting a non-explicit harmful text generation prompt to the text generation means, the non-explicit harmful text generation prompt instructing the text generation means to generate text that non-explicitly indicates harmful content, and acquiring the estimated non-explicit harmful text output from the text generation means; an estimation result acquisition means for inputting the estimated non-explicit harmful text into a text type estimation model which receives a text to be estimated as an input, and outputs a value belonging to an estimation range from a first value indicating that the text to be estimated does not explicitly indicate harmful content to a second value indicating that the text to be estimated explicitly indicates harmful content as a text type estimation result for the text to be estimated, and acquiring the text type estimation result output from the text type estimation model; an addition determination means for determining whether the value of the text type estimation result falls within a predetermined uncertainty range within the estimation range, and adding the estimated non-declared harmful text to the learning dataset as the non-declared harmful text when it is determined that the value of the text type estimation result falls within the uncertainty range; A program for causing the computer to function as a

Images