Extraction device, extraction method, and extraction program

Abstract

An extraction device (100) receives input character strings which are input to a generative AI for generating information corresponding to inputted character strings. The extraction device (100) determines whether a correction candidate character string is present in generated character strings that have been generated by inputting the received input character strings to the generative AI. If it is determined that a correction candidate character string is present, the extraction device (100) extracts the correction candidate character string from the generated character strings, and extracts supplementary information to be used for correcting the correction candidate character string from the input character strings. The extraction device (100) outputs a correction candidate character string that has been corrected using the extracted correction candidate character string and the extracted supplementary information.

Description

Extraction device, extraction method, and extraction program

[0001] The present invention relates to an extraction device, an extraction method, and an extraction program.

[0002] In recent years, generative AI (Artificial Intelligence) has become known that generates natural language sentences based on commands (prompts) in natural language (see, for example, Non-Patent Document 1).

[0003] ChatGPT (OpenAI), <URL: https: / / openai.com / chatgpt>, <Searched on November 27, 2020>

[0004] However, there are issues with the accuracy of the information output by the generation AI. For example, there is no guarantee that the regular expressions and code output by the generation AI are correct, and they may contain vulnerabilities or errors.

[0005] Therefore, in order to solve the above-mentioned problems and achieve the object, the extraction device of the present invention is characterized by having an input monitoring unit that receives an input string to be input to a generation AI that generates information according to the input string; a correction candidate determination unit that determines whether a correction candidate string exists in a generated string generated by inputting the input string received by the input monitoring unit to the generation AI; an extraction unit that, when the correction candidate determination unit determines that the correction candidate string exists, extracts the correction candidate string from the generated string and extracts supplementary information to be used to correct the correction candidate string from the input string; and an output unit that outputs the correction candidate string corrected using the correction candidate string extracted by the extraction unit and the supplementary information.

[0006] According to the present invention, the effect of improving the accuracy of information output from the generation AI is achieved.

[0007] FIG. 1 is a diagram illustrating an overall view of processing by an extraction device according to this embodiment. FIG. 2 is a diagram illustrating an example of processing by the extraction device according to this embodiment. FIG. 3 is a diagram illustrating an example of the configuration of an extraction device according to this embodiment. FIG. 4 is a table diagram illustrating an example of judgment conditions according to this embodiment. FIG. 5 is a diagram illustrating an example of input monitoring according to this embodiment. FIG. 6 is a diagram illustrating an example of example extraction processing according to this embodiment. FIG. 7 is a diagram illustrating an example of positive example extension processing according to this embodiment. FIG. 8 is a diagram illustrating an example of negative example extension processing according to this embodiment. FIG. 9 is a diagram illustrating an example of correction processing according to this embodiment. FIG. 10 is a diagram illustrating an example of character string output according to this embodiment. FIG. 11 is a flowchart illustrating an example of the procedure of extraction processing according to this embodiment. FIG. 12 is a diagram illustrating an example of a computer that realizes the extraction device according to this embodiment.

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that the embodiments are not limited to the following description.

[0009] <Overall Description of Processing by Extraction Device> Figure 1 is a diagram illustrating an overall view of processing by an extraction device 100 according to this embodiment. The extraction device 100 shown in Figure 1 is an example of a computer that provides technologies related to a process for determining whether or not to correct a string, a process for extracting information for correcting the string, and a process for outputting the corrected string, using an input string input to a string generation tool such as a generation AI (hereinafter, sometimes referred to as a "generation AI") that generates predetermined information according to an input string and a generated string generated by the generation AI.

[0010] Generative AI can generate natural language sentences based on commands (prompts) entered in natural language, but it is known to have issues such as "hallucination," which causes it to output incorrect information that is not based on facts. For example, there is no guarantee that the regular expressions and code output by generative AI are correct, and they may contain vulnerabilities or errors.

[0011] To solve the above-described problems, the extraction device 100 according to this embodiment extracts correction candidate character strings from a generated string related to a generation AI when the generated string includes correction candidate character strings such as regular expressions or SQL (Structured Query Language) statements, and extracts supplemental information to be used in the correction process from an input string related to the generated string. The extraction device 100 then outputs a string corrected using the extracted correction candidate character strings and supplemental information.

[0012] Here, a series of processing steps of the extraction device 100 will be described with reference to Fig. 1. For example, as shown in Fig. 1 (1), the extraction device 100 monitors a window, input field, or the like that is an input area for an input character string, and receives an input character string to be input to the generation AI.

[0013] As shown in FIG. 1(2), the extraction device 100 determines whether or not a correction candidate character string exists in the generated character string generated by inputting the received input character string to the generation AI.

[0014] 1 (3), when it is determined that a correction candidate character string exists, the extraction device 100 extracts the correction candidate character string from the generated character string and extracts supplemental information to be used for correcting the correction candidate character string from the input character string. Note that, when the supplemental information is used to correct a regular expression, it includes, for example, positive examples and negative examples corresponding to the regular expression.

[0015] As shown in (4) of FIG. 1, the extraction device 100 outputs a candidate correction character string corrected using the extracted candidate correction character string and the supplemental information.

[0016] As described above, when a character string generated by the generation AI includes a character string that is a candidate for correction, the extraction device 100 according to this embodiment performs a correction process on the extracted character string that is a candidate for correction using the extracted supplemental information. The extraction device 100 then outputs to the user the character string whose accuracy has been ensured by the above-described process. Therefore, the extraction device 100 has the effect of improving the accuracy of the information output by the generation AI.

[0017] <Description of Extraction Device> Next, as an example of processing by the extraction device 100 according to this embodiment, a flow of extraction processing, correction processing, and output processing using an input string and a generated string related to generation AI will be described. Figure 2 is a diagram illustrating an example of processing by the extraction device 100 according to this embodiment.

[0018] First, the input monitoring unit 131 monitors an input area (hereinafter, sometimes simply referred to as an "input area"), such as an input window or a keyword input field, into which a user inputs a character string such as a prompt to the generation AI. For example, when an input character string to the generation AI is input into a monitored input area, the input monitoring unit 131 receives the input character string ((1-1) in FIG. 2). Next, the input monitoring unit 131 outputs the received input character string to the correction candidate determination unit 132 ((1-2) in FIG. 2).

[0019] The correction candidate determination unit 132 receives a generated string generated by inputting the input string to the generation AI ((2-1) in FIG. 2). The correction candidate determination unit 132 then determines whether the received generated string includes a correction candidate string ((2-2) in FIG. 2).

[0020] If the generated character string does not contain a character string of a correction candidate, the correction candidate determination unit 132 outputs the generated character string as is to the output unit 136 ((3-1) in FIG. 2). On the other hand, if the generated character string contains a character string of a correction candidate, the correction candidate determination unit 132 outputs the generated character string to the correction candidate extraction unit 133 ((3-2) in FIG. 2), and outputs the input character string received from the input monitoring unit 131 to the supplemental information extraction unit 134 ((3-3) in FIG. 2).

[0021] The correction candidate extraction unit 133 extracts correction candidate character strings from the received generated character string ((4) in FIG. 2). For example, the correction candidate extraction unit 133 can extract character strings that match terms related to the correction candidate character strings from among the character strings included in the generated character string.

[0022] The supplemental information extraction unit 134 extracts supplemental information to be used in the correction process from the received input character string ((5) in FIG. 2). For example, the supplemental information extraction unit 134 can extract positive examples and negative examples corresponding to predetermined regular expressions included in the input character string as supplemental information. Note that if the received input character string does not include supplemental information such as positive examples and negative examples, the supplemental information extraction unit 134 can generate positive examples and negative examples based on a predetermined technique for creating positive examples and negative examples for correction.

[0023] The correction unit 135 corrects the character string of the correction candidate extracted by the correction candidate extraction unit 133 and the supplementary information extracted by the supplementary information extraction unit 134 in cooperation with the automatic correction device 200 ((6) in Figure 2).

[0024] If the generated string includes a character string that is a candidate for correction, the output unit 136 outputs the input string ((7-1) in FIG. 2) and the corrected string ((7-2) in FIG. 2). On the other hand, if the generated string does not include a character string that is a candidate for correction, the output unit 136 outputs the input string ((7-1) in FIG. 2) and the generated string that does not require correction and that is generated by the generation AI ((7-3) in FIG. 2) as is.

[0025] As described above, the extraction device 100 according to this embodiment can extract a correction candidate string from a generated string generated by the generation AI and extract supplemental information from an input string. The extraction device 100 then corrects the extracted correction candidate string using the supplemental information and outputs the string to the user. Therefore, the extraction device 100 can output to the user a generated string in which hallucination caused by the generation AI is suppressed.

[0026] (Extraction Device 100) Next, the configuration of the extraction device 100 will be described. FIG. 3 is a diagram showing an example of the configuration of the extraction device 100 according to this embodiment. As shown in FIG. 3, the extraction device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Although not shown in FIG. 3, the extraction device 100 may also include an input unit such as a keyboard or a mouse for receiving input such as operations by a user. The extraction device 100 may also include a display unit such as a display for displaying extracted correction candidate character strings and supplementary information, corrected correction candidate character strings, etc. to the user.

[0027] (Communication unit 110) The communication unit 110 performs data communication related to input of generated character strings generated by the generation AI, output of generated character strings that do not require correction and corrected generated character strings, etc. Specifically, the communication unit 110 mediates the transmission and reception of predetermined data related to the input monitoring unit 131 and output unit 136, which will be described later.

[0028] The communication unit 110 is realized by a network interface card (NIC) or the like, and controls communication via a telecommunication line such as a local area network (LAN), the internet, etc. The communication unit 110 is connected to a network via a wired or wireless connection as necessary, and can transmit and receive information bidirectionally.

[0029] (Storage Unit 120) The storage unit 120 stores data and programs used for various processes by the control unit 130, and various data acquired through operation of the control unit 130. The storage unit 120 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 3 , the storage unit 120 has a judgment condition DB 121.

[0030] (Determination Condition DB 121) The determination condition DB 121 is a database that stores determination conditions used in the determination process by the correction candidate determination unit 132, which will be described later. Specifically, the determination condition DB 121 stores conditions for determining whether a generated character string includes a character string of a correction candidate, in association with identification information for identifying the determination condition.

[0031] Here, the determination conditions stored in the determination condition DB 121 will be described. Fig. 4 is a table diagram showing an example of the determination conditions according to this embodiment. As shown in Fig. 4, the determination condition DB 121 stores "determination conditions" as a table in association with identification information "No." for identifying the determination conditions.

[0032] For example, the judgment condition DB 121 stores a judgment condition such as "matches a regular expression for judgment" identified by No. "1." The above-mentioned "matches a regular expression for judgment" means that a regular expression for judgment is prepared in advance, and if a character string included in the generated character string is contained within the regular expression for judgment, it is determined that the generated character string includes a character string that is a candidate for correction.

[0033] Furthermore, for example, the judgment condition DB 121 stores a judgment condition such as "a character string of 'regular expression' exists in the generated character string" identified by No. "2." The above-mentioned "a character string of 'regular expression' exists in the generated character string" means that a character string such as a "regular expression" is registered in advance as a character string for judgment, and if a character string included in the generated character string matches the character string for judgment (regular expression), it is judged that the generated character string includes a character string of a correction candidate.

[0034] Furthermore, for example, the judgment condition DB 121 stores a judgment condition such as "a character string of 'term related to regular expressions' is present in the generated character string" identified by No. "3." The above-mentioned "a character string of 'term related to regular expressions' is present in the generated character string" means that if a character string included in the generated character string matches a "term related to regular expressions" registered in advance as a character string for judgment, it is determined that the generated character string includes a character string of a correction candidate. Note that terms related to regular expressions include, for example, regex, regular, expression, regexp, regular expression, etc.

[0035] (Control Unit 130) Now, returning to Fig. 3, the explanation will be continued. The control unit 130 has an internal memory for temporarily storing programs that define various processing procedures and the like of the extraction device 100 and processing data, and is realized by electronic circuits such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). As shown in Fig. 3, the control unit 130 has an input monitoring unit 131, a correction candidate determination unit 132, a correction candidate extraction unit 133, a supplemental information extraction unit 134, a correction unit 135, and an output unit 136.

[0036] (Input monitoring unit 131) The input monitoring unit 131 monitors the input of a character string by a user to the generation AI. Specifically, the input monitoring unit 131 monitors an input area on the screen for inputting an input character string to the generation AI, and acquires the input character string when the input character string is input to the input area.

[0037] Here, an example of an input area to be monitored by the input monitoring unit 131 will be described. Fig. 5 is a diagram showing an example of input monitoring according to this embodiment. Fig. 5 shows, as examples of objects to be monitored by the input monitoring unit 131, a pop-up window ((1) in Fig. 5), a command line interface ((2) in Fig. 5), an input form ((3) in Fig. 5), and a web UI (User Interface) of the generated AI ((4) in Fig. 5).

[0038] 5A is a sub-screen in the form of a speech bubble that is displayed on a web browser or a desktop screen, and accepts input of a character string such as a prompt for the generated AI. For example, when the input monitoring unit 131 detects input of an input character string in an input box provided in the pop-up window, it acquires the input character string.

[0039] The command line interface shown in (2) of Fig. 5 is a screen that uses character strings to accept command statements for causing an information processing device or the like to execute commands. For example, the input monitoring unit 131 monitors the entire command line interface, and when it detects input of an input character string to the generation AI within the command line interface, it acquires the input character string.

[0040] 5(3) is a screen that accepts user input for predetermined input items, associates the input items with input information, and collects information. For example, the input monitoring unit 131 monitors the input form as an input area, and when it detects an input of an input string to the generation AI via the input area, it acquires the input string.

[0041] The web UI of the generated AI shown in (4) of Figure 5 is an operation interface screen of the generated AI that is displayed to the user by an information processing device that runs the generated AI. The web UI of the generated AI accepts information generation instructions for the generated AI via an input area that accepts input of input strings such as prompts for the generated AI. For example, the input monitoring unit 131 monitors the input area present in the web UI of the generated AI, and when it detects input of an input string in the input area, it acquires the input string.

[0042] (Correction candidate determination unit 132) Now, returning to Fig. 3, the explanation will be continued. The correction candidate determination unit 132 determines whether or not a character string that is a correction candidate exists in the generated character string generated by the generation AI, based on the input character string input by the user.

[0043] Specifically, the correction candidate determination unit 132 determines whether the generated character string contains a correction candidate character string based on a condition for identifying the correction candidate character string. For example, if the generated character string is a character string related to a "regular expression," the correction candidate determination unit 132 determines whether the generated character string contains a correction candidate character string using the determination condition "matches the determination regular expression" identified by No. "1" in FIG. 4. That is, if the generated character string contains the character string "A regular expression called XX has been generated," the correction candidate determination unit 132 makes a determination based on whether the determination regular expression contains "XX."

[0044] If the correction candidate determination unit 132 determines that the input string contains a correction candidate character string, it outputs the input string and the generated string to extraction units (the correction candidate extraction unit 133 and the supplemental information extraction unit 134) described below. For example, if the correction candidate determination unit 132 determines through the above-mentioned determination process that the generated string contains "XX (regular expression)", it outputs the generated string "A regular expression called XX has been generated" to the correction candidate extraction unit 133. Furthermore, the correction candidate determination unit 132 outputs an input string corresponding to the generated string "A regular expression called XX has been generated" to the supplemental information extraction unit 134, such as "A positive example corresponding to XX (regular expression) is □□ and a negative example is ●●."

[0045] On the other hand, if correction candidate determination unit 132 determines that the generated string does not contain a correction candidate character string, it outputs the input character string and the generated string to output unit 136, which will be described later. For example, if correction candidate determination unit 132 determines that the generated string does not contain a correction candidate character string because the generated string has content that is not related to a regular expression such as "The weather is sunny today," it outputs the generated string and the input character string related to the generated string to output unit 136 as is.

[0046] (Correction Candidate Extraction Unit 133) The correction candidate extraction unit 133 extracts a correction candidate character string output by the correction candidate determination unit 132 from a generated character string in which the correction candidate character string exists.

[0047] Specifically, correction candidate extraction unit 133 extracts a correction candidate string from the generated string if the generated string contains a string that satisfies a condition that specifies an area containing the correction candidate string within the generated string received from correction candidate determination unit 132. For example, if the generated string contains "XX (regular expression)", correction candidate extraction unit 133 extracts a correction candidate string from a code block containing terms related to regular expressions such as "regex, regular, expression, regexp", etc.

[0048] Alternatively, correction candidate extraction unit 133 uses a regular expression that matches the character string of the correction candidate to extract a character string of the correction candidate from the generated character string accepted from correction candidate determination unit 132. For example, if the generated character string includes a character string such as "abc@edf," correction candidate extraction unit 133 extracts the character string as a correction candidate using a regular expression that includes the character string.

[0049] (Supplementary Information Extraction Unit 134) The supplementary information extraction unit 134 extracts supplementary information used to correct a correction candidate character string from the input character string output by the correction candidate determination unit 132. For example, when a character string that satisfies a condition for identifying supplementary information is present in the input character string, the supplementary information extraction unit 134 performs a morphological analysis on the input character string and extracts the obtained character strings as a positive example and a negative example.

[0050] For example, if a predetermined specific character such as "match" or "hit" is present in the received input character string, the supplemental information extraction unit 134 determines that supplemental information exists near the specific character. Then, the supplemental information extraction unit 134 performs a morphological analysis on the input character string including the specific character, and extracts characters around the specific character as positive examples and negative examples corresponding to a predetermined regular expression.

[0051] Furthermore, if the input character string does not contain any supplemental information, the supplemental information extraction unit 134 can generate supplemental information. The generation of supplemental information by the supplemental information extraction unit 134 will now be described with reference to Figures 6 to 8. Note that the following description will use as an example a method of "creating positive examples and negative examples for correcting a regular expression by extending the regular expression through an extension process."

[0052] The supplemental information extraction unit 134 generates examples corresponding to a predetermined regular expression. Specifically, the supplemental information extraction unit 134 selects a generation tool corresponding to the input regular expression and causes the generation tool to generate examples. For example, the supplemental information extraction unit 134 selects a generation AI realized by a generative model or the like according to the input regular expression. Next, the supplemental information extraction unit 134 inputs a prompt to the selected generation AI, such as "Please create examples including positive examples that satisfy (regular expression) and negative examples that do not satisfy (regular expression)." The supplemental information extraction unit 134 then accepts examples including positive examples and negative examples generated based on the input prompt.

[0053] Next, the process of extracting examples from the generation information generated by the generation AI will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the process of extracting examples according to this embodiment. As shown in Fig. 6, the supplemental information extraction unit 134 extracts examples that satisfy predetermined conditions, such as "examples that correspond to regular expressions," from the generation information generated using the generation AI.

[0054] As shown in Fig. 6, the supplemental information extraction unit 134 extracts examples that satisfy a predetermined condition, such as "examples that correspond to an input regular expression," from the output data generated using the generation AI. For example, the supplemental information extraction unit 134 extracts "positive examples" that satisfy the regular expression input by the user and "negative examples" that do not satisfy the regular expression input by the user from the output data from the generation AI ((1) in Fig. 6).

[0055] Next, the supplemental information extraction unit 134 selects positive example candidates and negative example candidates from the examples generated to correspond to the input regular expression. The supplemental information extraction unit 134 then uses, as supplemental information, a plurality of positive examples and negative examples generated using the selected positive example candidates and negative example candidates as base data for the extension process.

[0056] The process of expanding the positive examples and negative examples, which are supplemental information, will now be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the process of expanding the positive examples according to this embodiment.

[0057] The supplemental information extraction unit 134 extracts a matching portion of a positive example candidate, such as "~~~@~~~.~~~" from the positive example candidates shown in (1) of Fig. 7 ((2) of Fig. 7). Note that in this item, "@" and "." represent a matching portion of a positive example candidate, and "~" represents an arbitrary character string to be inserted.

[0058] The supplemental information extraction unit 134 creates an extended regular expression that satisfies the positive example candidate so as to include a portion that matches the extracted positive example candidate. For example, as shown in (3) of FIG. 7, the supplemental information extraction unit 134 creates a regular expression that means "[a-z.]{4,7}[@][a-z]{6,7}[.][a-z]{3,4}". Note that the above-mentioned regular expression indicates a character string that includes "four or seven character strings including a to z and <.>", "@", "six or seven character strings including a to z", and ".", and "three or four character strings including a to z".

[0059] The supplemental information extraction unit 134 randomly selects characters (alphabetical letters, numbers, symbols, etc.) belonging to the character types included in the input pattern other than the matching portion to create one or more positive examples for correction. The length of the output string is between the minimum and maximum lengths of the original input. For example, as shown in (4) of FIG. 7, the supplemental information extraction unit 134 creates examples such as "aa.a.a@aaaaaa.aaaa," "bb.bbb@bbbbbb.bbb," and "cccc@cccccccc.ccc" that satisfy the extended regular expression.

[0060] Next, an example of a method for generating negative example candidates will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a negative example expansion process according to this embodiment.

[0061] The supplemental information extraction unit 134 randomly converts a portion of a character string included in a negative example candidate into a different character to create one or more negative examples for correction. For example, the supplemental information extraction unit 134 converts the character type of the negative example candidate "hxupj242rz2tdw.v" shown in (1) of FIG. 8 into different character types such as "iq63uw.ttuigtrfq," "7der.gjrbcht628r," and "u6ic.n4dggwfpf74," which are negative examples for correction shown in (2) of FIG. 8. In other words, the supplemental information extraction unit 134 uses a negative example selected as a negative example candidate as base data for the expansion process and randomly converts the character type of the negative example candidate to create multiple negative examples for correction.

[0062] (Correction Unit 135) The correction unit 135 corrects the extracted correction candidate character string using the supplemental information. An example of the correction process by the correction unit 135 will now be described with reference to the drawings. FIG. 9 is a diagram showing an example of the correction process according to this embodiment. Note that FIG. 9 explains the correction process by taking the correction of a regular expression as an example.

[0063] The correction unit 135 receives positive examples and negative examples for correction, which are the positive examples and negative examples expanded by the supplemental information extraction unit 134 ((1) in FIG. 9).

[0064] The correction unit 135 starts the correction process ((2-1) in FIG. 9). First, the correction unit 135 assigns a correction tool selection symbol that identifies a correction tool to the regular expression of the correction candidate (hereinafter, this may be referred to as "correction candidate + correction tool selection symbol") ((2-2) in FIG. 9).

[0065] The corrector 135 selects a correction tool to be used to correct the regular expression of the correction candidate based on the correction candidate and the correction tool selection symbol ((3-1) in FIG. 9). Specifically, the corrector 135 checks the status of the automatic corrector 200 in which the correction tool is stored ((3-2) in FIG. 9). Here, the automatic corrector 200 has various correction tools, such as a ReDOS (Regular Expressions DoS) correction tool ((3-3) in FIG. 9), an error correction tool ((3-4) in FIG. 9), and other correction tools ((3-5) in FIG. 9).

[0066] The correction unit 135 links the correction tool to be used, whose status has been checked, based on the information of the correction candidate+correction tool selection symbol, and determines a command to execute correction processing for the correction tool ((3-6) in FIG. 9).

[0067] The correction unit 135 starts correcting the regular expression of the correction candidate ((4-1) in FIG. 9). Specifically, the correction unit 135 executes the correction by inputting a command to execute the correction process to the selected correction tool ((4-2) in FIG. 9).

[0068] The correcting unit 135 outputs the regular expression corrected by performing the above-described processing, and ends the correction processing ((5) in FIG. 9).

[0069] In this way, the corrector 135 can correct the regular expression input by the user using the positive examples for correction and the negative examples for correction created by the expansion process.

[0070] (Output unit 136) For a generated character string generated by the generation AI, output unit 136 outputs at least one of a corrected generated character string and an uncorrected generated character string. Specifically, output unit 136 outputs a correction candidate character string corrected using the correction candidate character string and supplemental information extracted by the extraction unit (correction candidate extraction unit 133 and supplemental information extraction unit 134). In other words, output unit 136 outputs to the user a corrected correction candidate character string corrected by correction unit 135 using the supplemental information extracted by supplemental information extraction unit 134 for the correction candidate character string extracted by correction candidate extraction unit 133.

[0071] On the other hand, if correction candidate determination unit 132 determines that the generated character string does not include a correction candidate character string, output unit 136 outputs the uncorrected generated character string received from correction candidate determination unit 132. In other words, if the generated character string does not include a correction candidate character string, output unit 136 outputs the generated character string to the user as is without performing any correction processing or the like.

[0072] An example of an output format by the output unit 136 will now be described with reference to the drawings. FIG. 10 is a diagram showing an example of a character string output according to this embodiment. FIG. 10 shows an output screen 10, which is an example of a web UI of the generation AI. The output screen 10 displays an uncorrected generated character string generated by the generation AI ((1) in FIG. 10), a generated character string corrected by the extraction device 100 ((2-1) and (2-2) in FIG. 10), and supplemental information representing the extracted correction candidate character string ((2-3) in FIG. 10).

[0073] The character string shown in (1) of Fig. 10 is an uncorrected generated character string generated by the generation AI as described above. That is, the extraction device 100 determines that the character string shown in (1) of Fig. 10 does not contain any character strings that are candidates for correction, and outputs the generated character string generated by the generation AI as is.

[0074] The character string shown in (2-1) of Fig. 10 is a display of the character string corrected by the extraction device 100, which has been replaced with the uncorrected character string. Specifically, the corrected character string shown in (2-1) of Fig. 10 is a display of the character string corrected by the extraction device 100, which has been replaced with the uncorrected generated character string shown in (1) of Fig. 10. Therefore, the extraction device 100 can independently display the corrected character string, which has been corrected to accurate content by the correction process, so that the user can use it.

[0075] The character strings shown in (2-2) and (2-3) of Fig. 10 are character strings corrected by the extraction device 100. Specifically, the corrected character string shown in (2-2) of Fig. 10 is a character string corrected by the extraction device 100, and is displayed after the uncorrected generated character string shown in (1) of Fig. 10. On the other hand, the corrected character string shown in (2-3) of Fig. 10 is displayed as a pop-up window. Therefore, the extraction device 100 can display the corrected character string so that the user can recognize the change before and after the correction.

[0076] (Processing Procedure by Extraction Device 100) Hereinafter, a processing procedure implemented by the extraction device 100 according to this embodiment will be described. Fig. 11 is a flowchart showing an example of the extraction processing procedure according to this embodiment.

[0077] The input monitoring unit 131 receives an input character string (S101).

[0078] The correction candidate determination unit 132 receives a generated string generated by the generation AI using the input string (S102). Next, the correction candidate determination unit 132 determines whether or not a correction candidate string exists in the generated string generated from the input string (S103).

[0079] If a correction candidate character string exists (Yes in S104), the correction candidate extraction unit 133 extracts the correction candidate character string from the generated character string (S105). On the other hand, if a correction candidate character string does not exist (No in S104), the extraction device 100 skips steps S105 to S109.

[0080] If supplemental information is not to be generated (No in S106), the supplemental information extraction unit 134 extracts supplemental information from the input character string (S107). On the other hand, if supplemental information is to be generated (Yes in S106), the supplemental information extraction unit 134 generates supplemental information using the method for generating positive examples and negative examples described above (S108).

[0081] The correction unit 135 corrects the character string of the correction candidate using the supplemental information (S109). Then, the output unit 136 outputs the corrected character string of the correction candidate (S110). Then, the extraction device 100 ends the process.

[0082] On the other hand, if it is determined in step S104 that no correction candidate character string exists (No in S104), the output unit 136 outputs the uncorrected generated character string (S110), and the extraction device 100 ends the process.

[0083] (Effects) The effects of the extraction device 100 according to this embodiment will now be described. The input monitoring unit 131 of the extraction device 100 according to this embodiment receives an input string to be input to a generation AI that generates information corresponding to the input string. The correction candidate determination unit 132 of the extraction device 100 determines whether a correction candidate string is present in a generated string generated by inputting the input string received by the input monitoring unit 131 to the generation AI. The correction candidate extraction unit 133 of the extraction device 100 extracts a correction candidate string from the generated string when the correction candidate determination unit 132 determines that a correction candidate string is present. Furthermore, the supplemental information extraction unit 134 of the extraction device 100 extracts supplemental information to be used to correct the correction candidate string from the input string when the correction candidate determination unit 132 determines that a correction candidate string is present. The output unit 136 of the extraction device 100 then outputs a correction candidate string corrected using the extracted correction candidate string and supplemental information. Therefore, the extraction device 100 of this embodiment has the effect of increasing the accuracy of the information output from the generation AI.

[0084] Specifically, correction candidate determination unit 132 determines whether the generated string contains a correction candidate character string based on the conditions for identifying the correction candidate character string. If correction candidate determination unit 132 determines that the generated string contains a correction candidate character string, it outputs the input string to correction candidate extraction unit 133 and outputs the generated string to supplemental information extraction unit 134.

[0085] On the other hand, if correction candidate determination unit 132 determines that the input string does not include a correction candidate character string, it outputs the input string and the generated string to output unit 136. If correction candidate determination unit 132 determines that the input string does not include a correction candidate character string, output unit 136 outputs the uncorrected generated string received from correction candidate determination unit 132.

[0086] As described above, extraction device 100 determines whether a generated character string generated by generation AI is to be corrected based on a specific character string included in the generated character string, and executes processing according to the determination result. Therefore, extraction device 100 executes the extraction processing of information to be used for correction only for generated character strings to be corrected, and outputs generated character strings that are not to be corrected as is, thereby achieving the effect of reducing unnecessary correction processing and efficiently executing character string correction processing.

[0087] If the generated string contains a character string that satisfies a condition for identifying an area that includes the character string of the correction candidate, correction candidate extraction unit 133 extracts the character string of the correction candidate from the generated string. Alternatively, correction candidate extraction unit 133 extracts the character string of the correction candidate from the generated string using a regular expression that matches the character string of the correction candidate.

[0088] As described above, when a generated character string generated by the generation AI includes a character string of a correction candidate, the extraction device 100 extracts a character string of the correction candidate that matches a predetermined condition. Therefore, the extraction device 100 has the effect of enabling efficient extraction of a character string of a correction candidate from a generated character string.

[0089] If the input character string contains a character string that satisfies the conditions for identifying supplemental information, the supplemental information extraction unit 134 performs a morphological analysis on the input character string and extracts the resulting character strings as positive examples and negative examples.

[0090] As described above, when a generated character string includes a character string that is a candidate for correction, the extraction device 100 extracts, as supplemental information to be used for correction, a character string to be used as supplemental information from the vicinity of a character string that matches a predetermined condition for the input character string related to the generated character string. Therefore, the extraction device 100 has the effect of enabling efficient extraction of supplemental information from the input character string.

[0091] The supplemental information extraction unit 134 selects positive example candidates and negative example candidates from examples generated to correspond to the input regular expression. The supplemental information extraction unit 134 then uses, as supplemental information, a plurality of positive examples and negative examples generated using the selected positive example candidates and negative example candidates as base data for the extension process.

[0092] As described above, when an input character string is related to a regular expression and no supplemental information is included in the input character string, the extraction device 100 creates positive examples and negative examples as supplemental information using the regular expression. Therefore, the extraction device 100 has the effect of enabling correction of a character string that is a correction candidate even when no supplemental information is included.

[0093] As described above, the extraction device 100 according to this embodiment has the effect of extracting the object to be corrected and the supplementary information to be used for the correction from the character string output by the generation AI, and making it possible to correct the incorrect character string based on automatic text correction technology.

[0094] Furthermore, the extraction device 100 according to this embodiment enables automatic correction of generated character strings by using input character strings such as prompts for the generation AI and generated character strings related to the input character strings, whereas information whose accuracy is unknown due to the effects of hallucination must be investigated and verified each time. Therefore, the extraction device 100 can reduce the man-hours required for correcting generated character strings and the computer-based processing required for the correction process.

[0095] <Modifications> Modifications realized by the extraction device 100 according to this embodiment will be described below.

[0096] (Data, etc.) The generated AI, regular expressions, input strings, generated strings, correction candidate strings, supplementary information, names of functional parts of the extraction device 100, steps, processes, names of steps or processes, etc. used in the description of the above embodiment are merely examples and can be changed as desired.

[0097] For example, the judgment condition DB 121 stores "judgment conditions" as a table in association with identification information "No." The judgment condition DB 121 is not limited to this. Also, in FIG. 4, "regular expressions," "terms related to regular expressions," and the like are described as examples of pre-registered character strings for judgment. However, the judgment condition DB 121 is not limited to this. For example, the judgment condition DB 121 may be a character string such as "match" or "hit," or a character string related to an SQL statement.

[0098] (Creating Positive Examples and Negative Examples) In the above embodiment, the supplemental information extraction unit 134 creates positive examples and negative examples to be used for correcting regular expressions. However, this is not limiting. For example, the supplemental information extraction unit 134 may cause an external device for creating positive examples and negative examples to create positive examples and negative examples, and use the created positive examples and negative examples as supplemental information.

[0099] (Regarding the web UI of the generated AI) An example of information output by the extraction device 100 for the web UI of the generated AI has been described using FIG. 10 . However, the extraction device 100 according to this embodiment can output information in a format not limited to the output format shown in FIG. 10 . For example, the extraction device 100 may output information using a pop-up display on the display unit of the information processing device, a notification via a communication tool such as email or chat, or output via a predetermined output device such as a printer.

[0100] (Flowcharts, etc.) The steps in a flowchart, etc. may be interchanged and performed as long as there is no contradiction, and some steps may not be performed. In addition, conjunctions such as "next," "continue," "further," "at this time," and "on this occasion" used in the description of a flowchart do not limit the order or timing of performing the processes in the flowchart.

[0101] (System) The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0102] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown. In other words, all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0103] For example, in the above-described embodiment, the description has been given on the assumption that the generation AI and the automatic correction device 200 are realized by an information processing device different from the extraction device 100. However, this is not limited to this. In other words, the extraction device 100 may be an information processing device that includes the functions realized by the generation AI and the automatic correction device 200.

[0104] <Hardware Configuration> The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0105] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can also be performed manually using known methods. In addition, the information including the processing procedures, control procedures, specific names, various data, and parameters shown in the drawings can be changed as desired unless otherwise specified.

[0106] <Program> In one embodiment, the various devices constituting the extraction device 100 can be implemented by installing an extraction program as package software or online software on a desired computer. For example, by executing the extraction program on an information processing device, the various devices constituting the extraction device 100 can function. The information processing device referred to here includes desktop and notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones and mobile phones, and even slate terminals such as PDAs (Personal Digital Assistants).

[0107] 12 is a diagram showing an example of a computer that realizes the extraction device 100 according to this embodiment. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0108] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0109] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, application programs 1092, program modules 1093, and program data 1094. That is, programs that define the processes of the various devices that make up the extraction device 100 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, on the hard disk drive 1090. For example, program modules 1093 for executing processes similar to those of the functional configurations of the various devices that make up the extraction device 100 are stored on the hard disk drive 1090. The hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0110] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0111] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a LAN or a WAN (Wide Area Network)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0112] REFERENCE SIGNS LIST 100 Extraction device 110 Communication unit 120 Storage unit 121 Determination condition DB 130 Control unit 131 Input monitoring unit 132 Correction candidate determination unit 133 Correction candidate extraction unit 134 Supplementary information extraction unit 135 Correction unit 136 Output unit 200 Automatic correction device

Claims

1. An input monitoring unit that receives an input character string which is an input target for a generation AI that generates information according to the input character string; a correction candidate determination unit that determines whether a correction candidate character string exists in the generated character string generated by inputting the input character string received by the input monitoring unit to the generation AI; an extraction unit that, when it is determined by the correction candidate determination unit that the correction candidate character string exists, extracts the correction candidate character string from the generated character string and extracts supplementary information used to correct the correction candidate character string from the input character string; and an output unit that outputs the corrected correction candidate character string using the correction candidate character string and the supplementary information extracted by the extraction unit. An extraction device characterized by comprising the above.

2. The correction candidate determination unit determines whether the generated character string contains the correction candidate character string based on a condition for specifying the correction candidate character string. When it is determined that the correction candidate character string is included, the input character string and the generated character string are output to the extraction unit. When it is determined that the correction candidate character string is not included, the input character string and the generated character string are output to the output unit. The extraction device according to claim 1, characterized by the above.

3. The extraction unit extracts the correction candidate character string from the generated character string when a character string that satisfies a condition for specifying a region in which the correction candidate character string is included exists in the generated character string, and uses a regular expression that matches the correction candidate character string to extract the correction candidate character string from the generated character string. The extraction device according to claim 1, characterized by performing at least one of the above.

4. The extraction unit, when a character string that satisfies a condition for specifying the supplementary information exists in the input character string, performs morphological analysis on the input character string and extracts the obtained character string as positive and negative examples. The extraction device according to claim 1, characterized by the above.

5. The output unit outputs the uncorrected generated character string received from the correction candidate determination unit when it is determined by the correction candidate determination unit that the correction candidate character string is not included. The extraction device according to claim 1 or 2, characterized by the above.

6. The extraction unit selects candidate positive examples and candidate negative examples from examples generated to correspond to the input regular expression, and uses the selected candidate positive examples and candidate negative examples as base data for an expansion process to generate a plurality of positive examples and negative examples, and uses the generated positive examples and negative examples as supplementary information. The extraction device according to claim 1, characterized in that.

7. An extraction method for causing an extraction device to execute, comprising: an input monitoring step of receiving an input character string that is an input target for a generation AI that generates information corresponding to the input character string; a correction candidate determination step of determining whether a correction candidate character string exists in the generated character string generated by inputting the input character string received in the input monitoring step into the generation AI; an extraction step of, when it is determined in the correction candidate determination step that the correction candidate character string exists, extracting the correction candidate character string from the generated character string and extracting supplementary information used to correct the correction candidate character string from the input character string; and an output step of outputting the corrected correction candidate character string using the correction candidate character string extracted in the extraction step and the supplementary information. An extraction method characterized by including.

8. An input monitoring step of receiving an input character string that is an input target for a generation AI that generates information corresponding to the input character string; a correction candidate determination step of determining whether a correction candidate character string exists in the generated character string generated by inputting the input character string received in the input monitoring step into the generation AI; an extraction step of, when it is determined in the correction candidate determination step that the correction candidate character string exists, extracting the correction candidate character string from the generated character string and extracting supplementary information used to correct the correction candidate character string from the input character string; and an output step of outputting the corrected correction candidate character string using the correction candidate character string extracted in the extraction step and the supplementary information. An extraction program characterized by causing a computer to execute.

Images