Device and method for proofreading sentence and program

Abstract

To improve the convenience of data of character character-recognized.SOLUTION: A system for proofreading a sentence includes a server device having a control unit 300 comprising an OCR processing section that recognizes a character and extracts a text included in an image, a proofreading processing section that inputs a first input sentence including an extracted text and first instruction information instructing to correct a wrong character of the text to a first generative AI functioned to perform outputting matched with an instruction content of the first instruction information and acquires a first output sentence outputted from the first generative AI correspondingly to the first input sentence, an additional-information generation processing section that inputs a second input sentence including the first output sentence acquired by the proofreading processing section and second instruction information instructing to generate additional information representative of a feature of the first output sentence to a second generative AI functioned to perform outputting matched with an instruction content of the second instruction information and acquires a second output sentence outputted from the second generative AI correspondingly to the second input sentence, and a data-provision processing section that outputs the first and second output sentences.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a text proofreading device, a text proofreading method, and a program. [Background technology]

[0002] Conventionally, in order to go paperless, etc., paper documents with printed or handwritten characters have been converted into image data using scanners, cameras, etc. Also, there is a character recognition technology called OCR (Optical Character Recognition / Reader) that recognizes the character portion of the image data document and converts it into character data.

[0003] In the above-mentioned OCR, if a misrecognition occurs during character recognition, the character will be converted into erroneous character data, so the converted character data must be checked and corrected manually. Therefore, a technology for correcting character data using an AI model has been proposed (e.g., Reference 1).

[0004] However, when documents converted into character data are to be sorted or organized into folders according to genre or content, the content of the documents must be manually confirmed. Although the above-mentioned conventional technology can correct misrecognized character data, it does not take into consideration how to handle the documents after conversion, and there is room for improvement in terms of convenience. Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to provide a text proofreading device, a text proofreading method, and a program that can improve the usability of character data obtained by character recognition from a paper document. [Means for solving the problem]

[0006] The text proofreading device of the embodiment comprises a character recognition means for recognizing characters contained in an image and extracting text contained in the image; a first acquisition means for inputting a first input sentence, which includes the text extracted by the character recognition means and first instruction information instructing the correction of typos in the text, into a first generation AI configured to perform output according to the instructions of the first instruction information, and acquiring a first output sentence output from the first generation AI corresponding to the first input sentence; a second acquisition means for inputting a second input sentence, which includes the first output sentence acquired by the first acquisition means and second instruction information instructing the generation of additional information representing the characteristics of the first output sentence, into a second generation AI configured to perform output according to the instructions of the second instruction information, and acquiring a second output sentence output from the second generation AI corresponding to the second input sentence; and an output means for outputting the first output sentence and the second output sentence. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing a schematic configuration of a text proofreading system according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a terminal device according to the embodiment. [Figure 3] FIG. 3 is a block diagram illustrating an example of a hardware configuration of a server device according to the embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of a functional configuration of a terminal device according to the embodiment. [Figure 5] FIG. 5 is a block diagram illustrating an example of a functional configuration of a server device according to the embodiment. [Figure 6] FIG. 6 is a sequence diagram illustrating an example of a control process of the text proofreading system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of a text proofreading system 1 will be described with reference to the drawings. Furthermore, the present invention is not limited to the following embodiment, and the components in the following embodiment include those that a person skilled in the art would easily come up with, those that are substantially the same, and those that are within the scope of what is called equivalents. Furthermore, various omissions, substitutions, modifications, and combinations of the components can be made without departing from the spirit of the following embodiment.

[0009] 1 is a schematic diagram showing a general configuration of a text proofreading system 1 according to an embodiment. As shown in FIG. 1, the text proofreading system 1 includes a terminal device 2, a server device 3, and a text proofreading support device 4.

[0010] The server device 3 and the text proofreading support device 4 are communicably connected to each other via a network Na such as a LAN (Local Area Network). The server device 3 is communicably connected to the terminal device 2 via a network Nb such as the Internet. The server device 3 may be an integrated device with the text proofreading support device 4. The terminal device 2 may be connected to the same network as the server device 3 and the text proofreading support device 4. The number of terminal devices 2 is not limited to the example shown in the figure.

[0011] The terminal device 2 is a terminal device used by a user of the text proofreading system 1. The terminal device 2 executes various processes in response to user operations. The terminal device 2 transmits an image captured by the imaging unit 210 of the terminal device 2 to the server device 3. The terminal device 2 also displays the document received from the server device 3 on the display unit 208 of the terminal device 2. Details of the document will be described later.

[0012] The terminal device 2 can be realized by, for example, a desktop information processing device such as a PC (Personal Computer), or a portable information processing device such as a notebook PC, a tablet terminal, a smartphone, etc. The terminal device 2 may also be a scanner device or a multifunction peripheral / printer (MFP (Multi-Function Peripheral / Printer)) mounted on an image forming device such as a facsimile machine.

[0013] The server device 3 is a server that manages documents and is installed in, for example, an office. When the server device 3 receives an image from the terminal device 2, it performs character recognition processing (hereinafter also referred to as OCR processing) on ​​the image and extracts text contained in the image as character data (hereinafter also referred to as sentences). Note that known technology can be applied to the OCR processing.

[0014] The server device 3 also performs various processes related to text proofreading support in cooperation with the text proofreading support device 4, which will be described later. Specifically, the server device 3 generates a text in which the typos have been corrected (hereinafter also referred to as a text corrected for typos) by sending the first input text, which includes a text extracted by the OCR process and first instruction information instructing the text to be corrected (the instruction information will hereinafter also be referred to as a prompt), to the text proofreading support device 4. The server device 3 also acquires the text corrected for typos generated by the text proofreading support device 4 from the text proofreading support device 4.

[0015] Here, the instruction information is generated in the form of a character string written in a natural language, for example. Specifically, the first instruction information included in the first input sentence includes a character string that instructs a generation AI, which is provided in the text proofreading support device 4 described later and is functionally configured to perform output according to the instruction content of the first instruction information, to correct typos that exist in the sentence extracted by the OCR process.

[0016] Furthermore, the server device 3 causes the text proofreading support device 4 to generate additional information for the text proofreading support device 4 by transmitting to the text proofreading support device 4 a second input text including the text proofreading support device 4 generated by the text proofreading support device 4 and second instruction information instructing the generation of additional information representing the characteristics of the text proofreading support device 4. Furthermore, the server device 3 acquires the additional information generated by the text proofreading support device 4 from the text proofreading support device 4. Details of the additional information will be described later.

[0017] In addition, the second instruction information included in the second input sentence includes a string of characters that instructs a generation AI, which is functionally configured to output according to the instructions of the second instruction information provided in the sentence proofreading support device 4 described later, to generate additional information that represents the characteristics of the typo-corrected sentence.

[0018] The server device 3 also generates a document file converted into a general-purpose file format based on the typographically corrected sentence generated by the proofreading support device 4. For example, the server device 3 generates a file in Portable Document Format (PDF) format (hereinafter also referred to as a PDF file) based on the typographically corrected sentence, and stores the generated PDF file in a PDF file storage unit 306, which will be described later. The server device 3 also generates a document based on the additional information generated by the proofreading support device 4, and stores the generated document in a document DB 307. The PDF file storage unit 306 and the document DB 307 will be described later in detail.

[0019] The server device 3 may be configured with multiple computers. In other words, the functions of the server device 3 may be distributed among multiple computers. The server device 3 may also be a cloud server installed on a cloud.

[0020] The text proofreading support device 4 is, for example, a computer equipped with large language models (LLMs). Here, an LLM is an AI model (hereinafter also referred to as a language model) that has been pre-trained on a large corpus in the field of natural language processing. The LLM is constructed using large amounts of data and deep learning technology, and is functionally configured to generate and output a response sentence that is in line with the meaning of the input sentence when, for example, a question or instruction is input as a sentence. Here, the LLM is an example of a generative AI that can generate new content such as images, sentences, audio, and structured data by learning data patterns and relationships.

[0021] The text proofreading support device 4 of this embodiment receives from the server device 3 a first input text including a text extracted by the OCR process executed by the server device 3 and first instruction information instructing the correction of typos in the text, and then inputs the first input text to the LLM and obtains the typo-corrected text output by the LLM in accordance with the first input text as the first output text. The text proofreading support device 4 then transmits the first output text to the server device 3. Here, the typo-corrected text generated by the text proofreading support device 4 is a text in which the typo-corrected text is corrected based on the first instruction information instructing the correction of typos in a text containing erroneous character data extracted from text included in an image due to, for example, misrecognition in the OCR process.

[0022] Furthermore, when the text proofreading support device 4 receives from the server device 3 a second input text including the typo-corrected text generated by the text proofreading support device 4 and second instruction information instructing the generation of additional information representing the characteristics of the typo-corrected text, the text proofreading support device 4 inputs the second input text to the LLM and obtains the additional information output by the LLM according to the second input text as a second output text. Then, the text proofreading support device 4 transmits the second output text to the server device 3. Here, the additional information generated by the text proofreading support device 4 is a text including information representing the characteristics of the typo-corrected text, such as a summary, title, and genre of the typo-corrected text.

[0023] The LLM may be generated in advance using a known or publicly used method. The LLM may also be pre-trained using training data that includes various sentences and character data related to the generation of typo-corrected sentences and additional information. For example, if character data that is likely to cause misrecognition in OCR processing is known in advance, the LLM may be pre-trained with the character data that is likely to cause misrecognition.

[0024] Next, the configuration of the above-mentioned terminal device 2 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the terminal device 2 according to the embodiment. As shown in Fig. 2, the terminal device 2 includes a CPU (Central Processing Unit) 201, which is an example of a processor, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a memory unit 204, etc.

[0025] The CPU 201 comprehensively controls each unit of the terminal device 2. The ROM 202 stores various programs. The RAM 203 is a workspace for developing programs and various data. The memory unit 204 stores various programs.

[0026] The CPU 201, ROM 202, RAM 203, and memory unit 204 are connected to one another via a bus 211. The CPU 201, ROM 202, and RAM 203 constitute a control unit 200 having a computer configuration. That is, the control unit 200 executes control processing of the terminal device 2, which will be described later, by the CPU 201 operating in accordance with a control program unit 205 stored in the ROM 202 and memory unit 204 and expanded in the RAM 203.

[0027] The memory unit 204 is a non-volatile memory such as a hard disk drive (HDD) or flash memory that retains stored information even when the power is turned off. The memory unit 204 includes a control program unit 205.

[0028] The control unit 200 is connected to a communication unit 207, a display unit 208, an operation unit 209, and an imaging unit 210 via a controller 206 and a bus 211. Here, the communication unit 207 is a communication interface for connecting to a network Nb.

[0029] The controller 206 is connected to a communication unit 207, a display unit 208, an operation unit 209, and an imaging unit 210. Based on instructions from the control unit 200, the controller 206 controls each of the connected units.

[0030] The display unit 208 is a display device such as an LCD (Liquid Crystal Display). The display unit 208 displays various types of information under the control of the CPU 201. The operation unit 209 is an input device such as a keyboard or a pointing device. The operation unit 209 outputs operation details input via the input device to the CPU 201. Note that the operation unit 209 may be a touch panel provided on the display unit 208. The imaging unit 210 is an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

[0031] Next, the configuration of the above-mentioned server device 3 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the hardware configuration of the server device 3 according to the embodiment. As shown in Fig. 3, the server device 3 includes a CPU 301, which is an example of a processor, a ROM 302, a RAM 303, a memory unit 304, etc.

[0032] The CPU 301 comprehensively controls each unit of the server device 3. The ROM 302 stores various programs. The RAM 303 is a workspace for developing programs and various data. The memory unit 304 stores various programs.

[0033] The CPU 301, ROM 302, RAM 303, and memory unit 304 are connected to one another via a bus 311. The CPU 301, ROM 302, and RAM 303 constitute a control unit 300 having a computer configuration. That is, the control unit 300 executes control processing of the server device 3, which will be described later, by the CPU 301 operating in accordance with a control program unit 305 stored in the ROM 302 and memory unit 304 and loaded into the RAM 303.

[0034] The memory unit 304 is a non-volatile memory such as a HDD or flash memory that retains stored information even when the power is turned off. The memory unit 304 includes a control program unit 305, a PDF file storage unit 306, and a document DB 307.

[0035] The PDF file storage unit 306 is an example of a storage device. The PDF file storage unit 306 stores PDF files. For example, the PDF file storage unit 306 stores a PDF file generated based on a typo-corrected sentence. The PDF file storage unit 306 may store the PDF file in association with additional information (e.g., genre, etc.) of the typo-corrected sentence, which will be described later.

[0036] The document DB 307 is a database that stores documents generated based on additional information. The document DB 307 stores, for example, documents including a typo-corrected sentence, additional information (summary, title, genre) of the typo-corrected sentence, and a link to a PDF file generated based on the typo-corrected sentence. Here, the PDF file link is address information such as a URL (Uniform Resource Locator) for accessing the PDF file of the corresponding typo-corrected sentence stored in the PDF file storage unit 306.

[0037] The control unit 300 is connected to a first communication unit 309 via a controller 308 and a bus 311. Here, the first communication unit 309 is a communication interface for connecting to a network Na. Also, the second communication unit 310 is a communication interface for connecting to a network Nb.

[0038] The controller 308 is connected to the first communication unit 309 and the second communication unit 310. Based on instructions from the control unit 300, the controller 308 controls each of the connected units.

[0039] Next, the control unit 200 of the terminal device 2 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the functional configuration of the terminal device 2 according to the embodiment. As shown in Fig. 4, the control unit 200 includes, as functional components, an image acquisition unit 2001, a transmission / reception unit 2002, and a display control unit 2003. Note that the functional configuration of the terminal device 2 is not limited to this.

[0040] Specifically, the control unit 200 (CPU 201) of the terminal device 2 realizes the above-mentioned functional configuration by executing the control program unit 205 stored in the memory unit 204. In this embodiment, the above-mentioned functional configuration is a software configuration realized by cooperation between the processor and program of the terminal device 2, but is not limited to this, and a hardware configuration in which part or all of the functional configuration is realized by a dedicated circuit or the like may also be used.

[0041] The image acquisition unit 2001 acquires an image via the imaging unit 210. Specifically, when the image acquisition unit 2001 receives an instruction to capture an image from a user operating the terminal device 2 via the operation unit 209, the image acquisition unit 2001 starts the imaging unit 210 to capture an image. Then, the image acquisition unit 2001 acquires the captured image.

[0042] The transmitting / receiving unit 2002 transmits and receives various information to and from external devices via the communication unit 207. Specifically, the transmitting / receiving unit 2002 transmits the image acquired by the image acquisition unit 2001 to the server device 3. The transmitting / receiving unit 2002 also receives the document generated by the server device 3 from the server device 3.

[0043] The display control unit 2003 displays various types of information on the display unit 208. Specifically, the display control unit 2003 displays a document received from the server device 3 on the display unit 208 in response to a user operation or the like.

[0044] Next, the control unit 300 of the server device 3 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the functional configuration of the server device 3 according to the embodiment. As shown in Fig. 5, the control unit 300 includes, as functional components, an OCR processing unit 3001, a proofreading processing unit 3002, an additional information generation processing unit 3003, a data generation processing unit 3004, and a data provision processing unit 3005. Note that the functional configuration of the server device 3 is not limited to this.

[0045] Specifically, the control unit 300 (CPU 301) of the server device 3 realizes the above-mentioned functional configuration by executing the control program unit 305 stored in the memory unit 304. Note that in this embodiment, the above-mentioned functional configuration is a software configuration realized by cooperation between the processor and program of the server device 3, but is not limited to this, and a hardware configuration in which part or all of the functional configuration is realized by a dedicated circuit or the like may also be used.

[0046] The OCR processing unit 3001 is an example of a character recognition unit. The OCR processing unit 3001 executes OCR processing to recognize characters included in an image and extract character data representing the recognized characters. Specifically, the OCR processing unit 3001 executes OCR processing on an image transmitted from the terminal device 2 and extracts the character data included in the image as a sentence.

[0047] The proofreading processor 3002 is an example of a first acquisition means. The proofreading processor 3002 performs processing related to proofreading of a sentence. Specifically, the proofreading processor 3002 transmits a first input sentence including the sentence extracted by the OCR processor 3001 and a prompt instructing to correct typos in the sentence to the proofreading support device 4. The proofreading processor 3002 also acquires, from the proofreading support device 4, a typo-corrected sentence generated by the proofreading support device 4 in response to the first input sentence.

[0048] The method for generating the prompt is not particularly limited, but may be based on a predetermined template, which may include additional instruction information, such as highlighting the corrections made to typos.

[0049] Furthermore, the proofreading processor 3002 may determine the situation in which the sentence to be corrected was written based on the words and meaning contained in the sentence, and may input a prompt to the LLM to instruct the LLM to correct the typos in accordance with the situation by selecting a template according to the determined situation. In this case, the situation can be determined using publicly known and used techniques such as text mining.

[0050] The additional information generation processing unit 3003 is an example of a second acquisition means. The additional information generation processing unit 3003 performs processing related to the generation of additional information for a sentence. Specifically, the additional information generation processing unit 3003 transmits to the sentence proofreading support device 4 a second input sentence including the typo-corrected sentence acquired by the proofreading processing unit 3002 and a prompt instructing the generation of additional information related to the typo-corrected sentence. Here, the prompt instructing the generation of additional information related to the typo-corrected sentence includes content instructing the output of a summary, a title, a genre, etc. Next, the additional information generation processing unit 3003 acquires, from the sentence proofreading support device 4, the additional information generated by the sentence proofreading support device 4 in correspondence with the second input sentence.

[0051] The method for generating the prompt is not particularly limited, but a method for generating the prompt based on a predetermined template may be adopted, for example. For example, the prompt may specify a limit on the number of characters in the summary of the error-corrected sentence, which is an example of additional information. Furthermore, the prompt may specify a genre of the error-corrected sentence, which is an example of additional information, to select a genre that matches the meaning of the error-corrected sentence from registered genres stored in a database (not shown) of the server device 3.

[0052] The additional information generation processor 3003 may also determine the situation in which the typo-corrected sentence was written from the meaning of the sentence, and select a template according to the determined situation, thereby inputting a prompt to the LLM to instruct the generation of additional information such as a summary, title, and genre of the typo-corrected sentence that is appropriate for the situation. In this case, the situation can be determined using publicly known and used techniques such as text mining.

[0053] The data generation processing unit 3004 is an example of a conversion means, a storage control means, and a third acquisition means. The data generation processing unit 3004 generates various data by working together with other functional components. Specifically, the data generation processing unit 3004 generates a character-searchable PDF file from the typo-corrected text acquired by the proofreading processing unit 3002. After generating the PDF file, the data generation processing unit 3004 associates it with the additional information acquired by the additional information generation processing unit 3003 and stores the PDF file in the PDF file storage unit 306.

[0054] The data generation processing unit 3004 also acquires a URL link (hereinafter also referred to as a link) of the PDF file stored in the PDF file storage unit 306. Next, the data generation processing unit 3004 generates a document including the additional information acquired by the additional information generation processing unit 3003 and the link of the PDF file acquired by the data generation processing unit 3004, and stores the document in the document DB 307. Here, the document generated by the data generation processing unit 3004 can also be said to be index data that describes characteristic information such as a summary, title, and genre of the typo-corrected text.

[0055] The PDF file and document may be generated by the control unit 300 of the server device 3 executing a predetermined program to embed the information contained in the typo-corrected sentence and the additional information into a registered template stored in a database (not shown). Alternatively, the template to be used may be switched depending on the content of the typo-corrected sentence and the additional information. Furthermore, when generating a document, the document may be generated in a format that can be displayed by an application (not shown) installed on the terminal device 2.

[0056] The data providing processing unit 3005 is an example of an output unit. The data providing processing unit 3005 outputs data to an external device. Specifically, the data providing processing unit 3005 transmits a document stored in the document DB 307 to the terminal device 2.

[0057] Here, the data providing processing unit 3005 transmits a document related to the image to the terminal device 2 that transmitted the image via the transmitting / receiving unit 2002 of the terminal device 2. Furthermore, in response to an instruction from a user, the data providing processing unit 3005 may read the specified document from the document DB 307 and transmit it to the terminal device 2. In this case, the data providing processing unit 3005 may transmit a list of documents stored in the document DB 307 and accept an instruction from the user. Furthermore, the terminal device 2 that has received the list of documents may sort the list of documents based on the genre included in the additional information and display the list of documents on the display unit 208, or may display a list of summaries and titles of each document on the display unit 208.

[0058] In addition, access to documents stored in the document DB 307 may be restricted depending on the authority of the terminal device 2.

[0059] As described above, the server device 3, with the above-mentioned configuration, generates a typo-corrected sentence in which the typo has been corrected by inputting the sentence extracted by the OCR processing unit 3001 and a prompt including instructions to correct the typo to the LLM provided in the sentence proofreading support device 4. The server device 3 also generates additional information about the typo-corrected sentence by inputting the typo-corrected sentence and a prompt including instructions to generate additional information to the LLM provided in the sentence proofreading support device 4. The server device 3 then transmits a document including the additional information generated by the data generation processing unit 3004 to the terminal device 2.

[0060] Next, the control processing of the text proofreading system 1 will be described with reference to Fig. 6. Fig. 6 is a sequence diagram showing an example of the control processing of the text proofreading system 1 according to the embodiment. The sequence diagram shown in Fig. 6 shows an example of processing when the terminal device 2 transmits an image to the server device 3, and then the server device 3 transmits (returns) a document to the terminal device 2.

[0061] First, the user operating the terminal device 2 instructs the capture of an image of a paper document to be subjected to OCR processing via the operation unit 209. When the image capture instruction is received via the operation unit 209, the image acquisition unit 2001 of the terminal device 2 activates the image capture unit 210 to capture the image. Next, the image acquisition unit 2001 acquires the captured image (step S101). Next, the transmission / reception unit 2002 of the terminal device 2 transmits the image acquired by the image acquisition unit 2001 to the server device 3 (step S102).

[0062] When the OCR processing unit 3001 of the server device 3 receives the image from the terminal device 2, it performs OCR processing on the received image and extracts character data contained in the image as a sentence (step S103). Next, the proofreading processing unit 3002 of the server device 3 transmits the first input sentence including the sentence extracted by the OCR processing unit 3001 and a prompt instructing the user to correct typos in the sentence to the proofreading support device 4 (step S104).

[0063] When the text proofreading support device 4 receives the first input text from the server device 3, it generates a typo-corrected text by inputting the first input text into the LLM included in the text proofreading support device 4 (step S105). Then, the text proofreading support device 4 transmits the generated typo-corrected text to the server device 3 (step S106).

[0064] When the proofreading processing unit 3002 acquires the typo-corrected sentence from the sentence proofreading support device 4, the additional information generation processing unit 3003 of the server device 3 sends the second input sentence including the typo-corrected sentence and a prompt instructing the generation of additional information to the sentence proofreading support device 4 (step S107).

[0065] Next, when the text proofreading support device 4 receives the second input sentence from the server device 3, it generates additional information by inputting the second input sentence into the LLM included in the text proofreading support device 4 (step S108).Then, the text proofreading support device 4 transmits the generated additional information to the server device 3 (step S109).

[0066] The additional information generation processing unit 3003 of the server device 3 acquires the additional information from the text proofreading support device 4. Next, the data generation processing unit 3004 of the server device 3 generates a character-searchable PDF file based on the typographically corrected text acquired by the proofreading processing unit 3002 (step S110). Subsequently, the data generation processing unit 3004 associates the generated PDF file with the additional information acquired by the additional information generation processing unit 3003 and stores it in the PDF file storage unit 306 (step S111). Furthermore, the data generation processing unit 3004 acquires a link that allows access to the stored PDF file (step S112).

[0067] Next, the data generation processing unit 3004 of the server device 3 generates a document including the additional information acquired by the additional information generation processing unit 3003 and a link that enables access to the PDF file acquired by the data generation processing unit 3004 (step S113), and stores the document in the document DB 307 (step S114). Next, the data provision processing unit 3005 of the server device 3 transmits the document stored in the document DB 307 to the terminal device 2 (step S115).

[0068] On the other hand, the display control unit 2003 of the terminal device 2 causes the display unit 208 to display the document received from the server device 3 (step S116).

[0069] As described above, in the text proofreading system 1 of this embodiment, when the server device 3 receives an image from the terminal device 2, it performs OCR processing on the received image. That is, the server device 3 recognizes characters contained in the received image and extracts the character data contained in the image as a sentence. Next, the server device 3 transmits a first input sentence including the extracted sentence and a prompt instructing the user to correct typos in the sentence to the text proofreading support device 4. Next, the text proofreading support device 4 inputs the first input sentence received from the server device 3 to the LLM included in the text proofreading support device 4. Then, the server device 3 acquires, from the text proofreading support device 4, a typo-corrected sentence generated by the LLM corresponding to the first input sentence. Next, the server device 3 transmits a second input sentence including the acquired typo-corrected sentence and a prompt instructing the user to generate additional information related to the typo-corrected sentence to the text proofreading support device 4. Next, the text proofreading support device 4 inputs the second input sentence received from the server device 3 to the LLM included in the text proofreading support device 4. Then, the server device 3 acquires the additional information generated by the LLM based on the second input sentence from the sentence proofreading support device 4. Furthermore, the server device 3 generates a document including the additional information from the additional information acquired from the sentence proofreading support device 4. Then, the server device 3 transmits the generated document to the terminal device 2. Upon receiving the document from the server device 3, the terminal device 2 displays the document on the display unit 208.

[0070] As a result, the text proofreading system 1 extracts the text contained in the image, corrects any typos, and outputs it by converting it into a general-purpose file format. The text proofreading system 1 also generates additional information that describes the characteristics of the text after the typos have been corrected, and outputs the information in association with the corrected text. Therefore, the text proofreading system 1 can improve the accuracy of the text read by OCR processing and make it easier to understand the content of the text, thereby improving convenience.

[0071] Furthermore, the text proofreading system 1 of this embodiment is configured to use a text proofreading support device 4 including a server device 3 and an LLM, which are arranged in a local environment.

[0072] This can contribute to reducing the risk of confidential information being leaked when confidential information is included in the first input sentence and the second input sentence input to the LLM provided in the sentence proofreading support device 4.

[0073] The above-described embodiment can be modified as needed by partially changing the configuration or functions of each of the above-described devices. Therefore, several modifications of the above-described embodiment will be described below as other embodiments. The following mainly focuses on differences from the above-described embodiment, and detailed descriptions of commonalities with the content already described will be omitted. The modifications described below may be implemented individually or in appropriate combination.

[0074] (Variation 1) In the above-described embodiment, the first input sentence and the second input sentence are both input to the same LLM provided in the sentence proofreading support device 4, but this is not limited to this, and the first input sentence and the second input sentence may also be input to LLMs of different models.

[0075] (Variation 2) In the above-described embodiment, when an instruction to capture an image is given via the operation unit 209, the image acquisition unit 2001 of the terminal device 2 activates the image capture unit 210 to capture the image. Subsequently, the image acquisition unit 2001 is configured to acquire the captured image (step S101). However, this is not limiting, and when an image stored in advance in the terminal device 2 is to be transmitted to the server device 3, this process may be skipped.

[0076] (Variation 3) In the above-described embodiment, the OCR process is executed by the server device 3, but this is not limiting, and the OCR process may be executed by the terminal device 2. In this case, the control unit 200 of the terminal device 2 has a functional configuration equivalent to the OCR processing unit 3001 of the server device 3 and executes the OCR process.

[0077] (Variation 4) In the above-described embodiment, the proofreading processing unit 3002 of the server device 3 is configured to send a first input sentence including the sentence extracted by the OCR processing unit 3001 and a prompt instructing the correction of typos in the sentence to the proofreading support device 4 (step S104). However, this is not limited to this, and in step S104, the proofreading processing unit 3002 of the server device 3 may be configured to send a second input sentence including the typo-corrected sentence and a prompt instructing the generation of additional information together with the first input sentence to the proofreading support device 4.

[0078] As a result, information is sent and received only once between the server device 3 and the text proofreading support device 4, which is expected to reduce the time required for a series of processes.

[0079] (Variation 5) In the above-described embodiment, the data generation processing unit 3004 of the server device 3 is configured to generate a document including the additional information acquired by the additional information generation processing unit 3003 and a link that allows access to the PDF file (step S113), but this is not limited to this, and the data generation processing unit 3004 may also generate a document that does not include a link that allows access to the PDF file.

[0080] This makes it possible to skip the processes relating to the generation of a PDF file from step S110 to step S112, which is expected to reduce the time required for the series of processes.

[0081] (Variation 6) In the above embodiment, the data providing processing unit 3005 of the server device 3 is configured to transmit the document stored in the document DB 307 to the terminal device 2 (step S115), but this is not limiting. Immediately after the processing of step S109, the first output sentence and the second output sentence may be transmitted to the terminal device 2, thereby generating content corresponding to the document on the terminal device 2 side and displaying it on the display unit 208. In this case, the control unit 200 of the terminal device 2 has a functional configuration corresponding to the data generation processing unit 3004 of the server device 3.

[0082] This makes it possible to skip the processes from step S110 to step S114, which is expected to reduce the time required for the series of processes. Also, since the server device 3 does not generate the document, it is expected to reduce the risk of information leakage.

[0083] (Variation 7) In the above-described embodiment, the text proofreading support device 4 is configured to be placed and used in a local environment, but this is not limited to this. If the first input sentence and the second input sentence do not contain confidential information, the text proofreading support device 4 may be configured to be installed on the cloud.

[0084] The programs executed by the text proofreading system 1 of the embodiment and the modified example may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by the text proofreading system 1 of the embodiment and the modified example may be provided or distributed via a network such as the Internet.

[0085] The programs executed by each device in the above-described embodiments are provided in a state where they are pre-installed in a ROM, a storage unit, etc. The programs executed by each device in the above-described embodiments may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD).

[0086] Furthermore, the programs executed by each device in the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by each device in the above-described embodiments may be provided or distributed via a network such as the Internet.

[0087] Although the embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and their modifications can be embodied in various other forms, and various omissions, substitutions, changes, and combinations can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0088] 1. Grammar proofreading system 2. Terminal Device 3. Server equipment 4 Grammar proofreading support device 2001 Image Acquisition Unit 2002 Transmitter / Receiver 2003 Display control unit 3001 OCR processing unit 3002 Calibration processing section 3003 Additional information generation processing unit 3004 Data generation processing unit 3005 Data provision processing unit [Prior art documents] [Patent documents]

[0089] [Patent Document 1] Patent Publication No. 2021-039694

Claims

1. character recognition means for recognizing characters included in an image and extracting text included in the image; a first acquisition means for inputting a first input sentence including the text extracted by the character recognition means and first instruction information instructing correction of typos in the text to a first generation AI that is functionally configured to perform output according to the instruction content of the first instruction information, and acquiring a first output sentence output from the first generation AI corresponding to the first input sentence; a second acquisition means for inputting the first output sentence acquired by the first acquisition means and a second input sentence including second instruction information instructing the generation of additional information representing the characteristics of the first output sentence into a second generation AI that is functionally configured to perform output according to the instruction content of the second instruction information, and acquiring a second output sentence output from the second generation AI corresponding to the second input sentence; an output means for outputting the first output sentence and the second output sentence; A grammar checker comprising:

2. the second acquisition means acquires the second output sentence including, as a feature of the first output sentence, any one of a summary, a title, and a genre of the first output sentence; The grammar checker of claim 1 .

3. a storage control means for storing the first output sentence and the second output sentence in a storage device in association with each other; a third acquiring means for acquiring address information for accessing a file stored in the storage device; Further provided with the output means outputs the second output sentence and the address information. The grammar checker of claim 1 .

4. a conversion means for converting the first output document into a general-purpose file format; Further provided with the storage control means stores the first output sentence converted by the conversion means and the second output sentence in a storage device in association with each other; The text proofreader according to claim 3 .

5. The first generated AI and the second generated AI are the same generated AI. The grammar checker of claim 1 .

6. A grammar proofreading method executed by a grammar proofreading device, comprising: a character recognition step of recognizing characters included in an image and extracting text included in the image; a first acquisition step of inputting a first input sentence including the text extracted in the character recognition step and first instruction information instructing correction of typos in the text into a first generation AI that is functionally configured to perform output according to the instructions of the first instruction information, and acquiring a first output sentence output from the first generation AI corresponding to the first input sentence; a second acquisition step of inputting the first output sentence acquired in the first acquisition step and a second input sentence including second instruction information instructing the generation of additional information representing the characteristics of the first output sentence into a second generation AI that is functionally configured to perform output according to the instruction content of the second instruction information, and acquiring a second output sentence output from the second generation AI corresponding to the second input sentence; an output step of outputting the first output sentence and the second output sentence; Grammar checking methods, including:

7. The grammar checker computer, character recognition means for recognizing characters included in an image and extracting text included in the image; a first acquisition means for inputting a first input sentence including the text extracted by the character recognition means and first instruction information instructing correction of typos in the text to a first generation AI that is functionally configured to perform output according to the instruction content of the first instruction information, and acquiring a first output sentence output from the first generation AI corresponding to the first input sentence; a second acquisition means for inputting the first output sentence acquired by the first acquisition means and a second input sentence including second instruction information instructing the generation of additional information representing the characteristics of the first output sentence into a second generation AI that is functionally configured to perform output according to the instruction content of the second instruction information, and acquiring a second output sentence output from the second generation AI corresponding to the second input sentence; an output means for outputting the first output sentence and the second output sentence; A program to function as a

Images