Electronic data generation apparatus and electronic data generation method

Abstract

We provide application tools that can be effectively used even by users with low IT literacy. [Solution] The system includes a text information generation unit 11 that generates item text information to be entered into the information items of electronic data from an image of handwritten information contained in a scanned image of a paper medium that has handwritten input items similar to the information items of the electronic data to be generated in a predetermined format, and an electronic data generation unit 12 that inputs the generated item text information into the information items to generate electronic data in a predetermined format. By having the user write the necessary information in the handwritten input items of the paper medium and then scanning the paper medium, electronic data in a predetermined format is automatically generated, making it possible for even users with low IT literacy to easily generate electronic data in a predetermined format.

Description

Technical Field

[0005] , ,

[0006]

[0001] The present invention relates to an electronic data generation device and an electronic data generation method, and is particularly suitable for use in an apparatus and method for generating electronic data in a predetermined format from a description on a paper medium in which printed information and handwritten information are mixed.

Background Art

[0002] In recent years, for the purpose of improving business efficiency, quality, cost reduction, etc., the use of IT (Information Technology) has been promoted. For example, the abolition of handwritten documents by using applications installed on information processing terminals and appropriate information management using data collected from information processing terminals to servers through ICT (Information and Communication Technology) are cited as examples.

[0003] However, due to reasons such as the barrier of IT literacy of users who use application tools at the site, the business efficiency may be reduced by the introduction of IT, or stress may be given to the users who use the tools. Therefore, the introduction of a system that can be effectively used by users with low IT literacy is desired. <​​​​​​​​​The electronic document creation devices described in Patent Documents 2 and 3 create electronic documents from paper documents containing a mixture of printed and handwritten characters. In the device described in Patent Document 2, the paper document is scanned to obtain scan data, the obtained scan data is analyzed to extract printed and handwritten characters as objects. Then, the arrangement information of the extracted objects is obtained, and character recognition processing is performed on each object according to a character recognition method determined based on the characteristic information of the layout configuration corresponding to the obtained arrangement information, and an electronic document is created based on the character recognition results.

[0007] In the electronic document creation device described in Patent Document 3, arrangement information indicating the placement of characters in a document and acquisition area information indicating the character acquisition area to be used as a property when the document is filed are linked to the document format and managed for each type of document. A scanned image of a paper document is acquired, and based on the acquisition area information associated with the document format identified based on the character arrangement information in the scanned image, it is determined whether the characters contained in the character area extracted by analyzing the scanned image are handwritten or printed characters. Then, a predetermined character area on the scanned image is determined using different methods for handwritten and printed characters, and character recognition is performed on the predetermined character area to acquire character information.

[0008] The system described in Patent Document 4 automatically recognizes scanned document text and formats it for display to the user. That is, it automatically recognizes the text of a scanned document based on the format and context of the text in the document. Furthermore, it automatically creates data entries representing the scanned text and organizes these data entries for display to the user. Here, the recognition process of the information contained in the document is guided according to input templates corresponding to different types of documents. The templates provide information about the expected nature, structure, and content of the document that the system guides. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2015-2448 [Patent Document 2] Japanese Patent Publication No. 2019-8697 [Patent Document 3] Japanese Patent Publication No. 2022-45011 [Patent Document 4] Japanese Patent Publication No. 2001-229340 [Overview of the Initiative] [Problems that the invention aims to solve]

[0010] This invention was made to solve the problems described above, and aims to provide an application tool that can be effectively used even by users with low IT literacy. [Means for solving the problem]

[0011] To solve the above-mentioned problems, the present invention generates item text information to be entered into the information items of electronic data from an image of handwritten information contained in a scanned image of a paper medium that has handwritten input items similar to the information items of the electronic data to be generated in a predetermined format, and then inputs the generated item text information into the information items to generate electronic data in a predetermined format. [Effects of the Invention]

[0012] According to the present invention configured as described above, after a user writes the necessary information in the handwritten input fields on a paper document and then scans the document, electronic data in a predetermined format is automatically generated. This allows even users with low IT literacy to easily generate electronic data in a predetermined format. [Brief explanation of the drawing]

[0013] [Figure 1] This is a diagram illustrating the processing details of the electronic data generation device according to this embodiment. [Figure 2] This is a diagram illustrating the processing details of the electronic data generation device according to this embodiment. [Figure 3] This is a block diagram showing an example of the functional configuration of the electronic data generation device according to this embodiment. [Figure 4] This figure shows free-text input areas and selection-based handwriting areas identified from scanned images. [Modes for carrying out the invention]

[0014] Hereinafter, one embodiment of the present invention will be described based on the drawings. Figures 1 and 2 are diagrams illustrating the processing content of the electronic data generation device according to this embodiment. Figure 1(a) shows a paper medium containing a mixture of printed and handwritten information. Figure 1(b) shows the paper medium shown in Figure 1(a) being scanned using the camera of an information processing terminal. Figure 1(c) shows electronic data in a predetermined format generated from the scanned image. Figures 2(a) and 2(b) show two electronic data generated by scanning two types of paper mediums. The electronic data in Figure 2(a) is the same as the electronic data in Figure 1(c). Figure 2(c) shows electronic data in a predetermined format (hereinafter referred to as integrated electronic data) generated by integrating multiple electronic data (hereinafter referred to as individual electronic data).

[0015] The paper medium shown in Figure 1(a) has handwritten input items similar to the information items in the individual electronic data of a predetermined format shown in Figure 1(c). The electronic data generation device of this embodiment generates item text information to be entered into the information items of the individual electronic data by analyzing the scanned image obtained by scanning the paper medium on which the information necessary for the handwritten input items is written, and generates individual electronic data of a predetermined format by inputting the generated item text information into the information items. The individual electronic data shown in Figure 1(c) is data in which item text information has been entered into the information items on the information input screen provided by the application installed on the information processing terminal.

[0016] As described above, the individual electronic data shown in Fig. 2(a) is the same as the individual electronic data in Fig. 1(c) generated as shown in Fig. 1. Fig. 2(b) shows different types of individual electronic data generated in the same procedure as Fig. 1 using a different type of paper medium from the paper medium shown in Fig. 1(a). In Fig. 2, only two pieces of individual electronic data are shown for simplicity of illustration, but actually more individual electronic data are generated from multiple types of paper media. And by integrating these multiple pieces of individual electronic data, integrated electronic data in a predetermined format shown in Fig. 2(c) is generated. The information items included in the integrated electronic data shown in Fig. 2(c) include item text information generated from the handwritten information described in the handwritten input items of multiple types of paper media including the paper medium shown in Fig. 1(a).

[0017] In Figs. 1 and 2, an example is shown in which after the staff of a nursery school handwrites various information on the care of children (including information such as children's behavior, physical condition, and the content of care by the staff) on a dedicated paper medium, the integrated electronic data of the contact book for the guardians is generated by scanning the paper medium.

[0018] Fig. 1(a) is a paper medium for describing information on the meal records of multiple children. The individual electronic data shown in Figs. 1(c) and 2(a) is generated from this paper medium. This individual electronic data is data in a state where item text information for each child is input for the information items for each child that the meal record input screen of the application has. Also, the individual electronic data shown in Fig. 2(b) is generated from a paper medium for describing information on the stool conditions of multiple children. This individual electronic data is data in a state where item text information for each child is input for the information items for each child that the stool record input screen of the application has.

[0019] In addition to the individual electronic data shown in FIGS. 2(a) and 2(b), for example, individual electronic data is generated from paper media for describing information on the body temperatures of a plurality of children, paper media for describing information on the moods of a plurality of children, paper media for describing information on the naps of a plurality of children, paper media for describing information on the snacks of a plurality of children, and the like. Then, the item text information for each child included in these plurality of individual electronic data is input into the corresponding information items of the integrated electronic data of the contact book generated individually for each child. FIG. 2(c) shows the integrated electronic data of the contact book generated for a certain child.

[0020] FIG. 3 is a block diagram showing an example of the functional configuration of the electronic data generation device 1 according to the present embodiment. As shown in FIG. 3, the electronic data generation device 1 of the present embodiment includes a text information generation unit 11 and an electronic data generation unit 12 as functional configurations. The text information generation unit 11 includes a prompt generation unit 111 and an LLM processing unit 112 as specific functional configurations.

[0021] The text information generation unit 11 and the electronic data generation unit 12 execute the processes described below through the cooperation of hardware and software. For example, the processes of the text information generation unit 11 and the electronic data generation unit 12 are executed by the operation of a program stored in a storage medium such as a RAM, a ROM, a hard disk, or a semiconductor memory under the control of a processor of a microcomputer configured to include a CPU, a RAM, a ROM, and the like. ]]

[0022] The text information generation unit 11 generates item text information to be input into the information items of the electronic data from the handwritten information image included in the scanned image of the paper medium. The item text information generated here is also the information to be input into the information items of the individual electronic data such as FIGS. 2(a) and 2(b), and is also the information to be input into the information items of the integrated electronic data of FIG. 2(c) generated by integrating a plurality of individual electronic data.

[0023] Figure 4 is a diagram illustrating the details of the process by which the text information generation unit 11 generates item text information from a scanned image. The text information generation unit 11 identifies handwritten regions 41 containing free-text input items from the scanned image shown in Figure 4, replaces the images of handwritten characters contained in the identified handwritten regions 41 with character codes using OCR (Optical Character Recognition / Reader), and generates item text information using the replaced character codes.

[0024] Furthermore, the text information generation unit 11 identifies handwriting areas 42 containing multiple-choice handwriting input items from the scanned image shown in Figure 4, determines which option has been selected by hand in the identified handwriting area 42, and generates item text information using the information of the determined option. The multiple-choice handwriting area 42 is, for example, an area where characters related to multiple options are printed, as shown in Figure 4, and the user marks the appropriate option with a circle. The multiple-choice handwriting area 42 may also be an area where checkboxes are printed along with characters related to multiple options, and the user marks the checkbox of the appropriate option with a check. The text information generation unit 11 identifies the option selected by the user by determining where the circle or check is written on the option, and generates item text information corresponding to that option.

[0025] The free-text input handwriting area 41 and the multiple-choice handwriting area 42 are located at predetermined positions on the paper medium. Therefore, the text information generation unit 11 can identify the free-text input handwriting area 41 and the multiple-choice handwriting area 42, respectively, based on these predetermined positions.

[0026] The text information generation unit 11 executes the processes of the prompt generation unit 111 and the LLM processing unit 112 when generating item text information from a scanned image. The prompt generation unit 111 replaces handwritten character images contained in the scanned image with character codes using OCR, and uses the replaced character codes to generate prompts for input to the Large Language Models (LLM) 100. The prompts include character information recognized by OCR and command statements that instruct the output of item text information based on that character information.

[0027] The LLM processing unit 112 generates item text information by inputting the prompt generated by the prompt generation unit 111 to the LLM 100. At this time, the LLM 100 generates item text information based on the character information described in the prompt, according to the command statement described in the prompt input from the LLM processing unit 112, and responds to the LLM processing unit 112.

[0028] The prompt generation unit 111 may generate prompts by adding reference information to the character information and command statements recognized by OCR, for use by the LLM 100 when it processes the acquisition of item text information based on the character information. For example, a list of children's names can be used as reference information. By generating prompts with a list of children's names added as reference information, even if a child's name is misrecognized by OCR, the LLM 100 can obtain item text information linked to the correct name. Alternatively, example answers to command statements may be added as reference information.

[0029] Furthermore, the prompt generation unit 111 may be equipped with a character correction function that corrects the recognized characters indicated by the character codes replaced by OCR, and may generate prompts using the corrected character information. For example, if the name recognized by OCR is not included in the list of children's names, a screen showing the recognition results of the scanned image may be displayed, allowing the user to correct the names of children not included in the name list. In this case, the part of the name not included in the name list may be highlighted on the screen showing the recognition results to make it easier to understand the part that needs to be corrected.

[0030] The electronic data generation unit 12 inputs the item text information generated by the text information generation unit 11 (item text information acquired by the LLM processing unit 112 from the LLM 100) into the corresponding information items to generate electronic data (individual electronic data and integrated electronic data) in a predetermined format. Here, the generation of both individual electronic data and integrated electronic data may be performed on the information processing terminal. Alternatively, the generation of individual electronic data may be performed on the information processing terminal, the generated individual electronic data may be sent to the server, and the generation of integrated electronic data may be performed on the server.

[0031] Furthermore, when the text information generation unit 11 generates item text information from a scanned image of a paper document containing handwritten input items in a table format, it may analyze the scanned image to determine whether or not handwritten information is included in each description field of the recognized table area, add predetermined symbolic information to the description fields that do not contain handwritten information, and then generate item text information from the images of the handwritten information in each description field of the table area with the added symbolic information. The predetermined symbolic information used is symbolic information that is not handwritten in the handwritten input items in that table format. In this way, even when scanning a paper document in which there is a mixture of fields with handwriting and fields that are blank within the table, the possibility of misrecognition can be reduced.

[0032] Furthermore, the above embodiments are merely examples of how the present invention may be implemented, and the technical scope of the invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its gist or its main features. [Explanation of Symbols]

[0033] 1. Electronic data generation device 11 Text Information Generation Unit 12 Electronic Data Generation Unit 111 Prompt generation unit 112 LLM Processing Unit

Claims

1. An electronic data generation device that generates electronic data in a predetermined format from a paper medium containing a mixture of printed and handwritten information, A text information generation unit generates item text information to be entered into the information items of the electronic data from an image of the handwritten information contained in a scanned image of the paper medium having handwritten input items similar to the information items of the electronic data in the predetermined format described above. The system comprises an electronic data generation unit that inputs the item text information generated by the text information generation unit into the information items and generates electronic data in the predetermined format. An electronic data generation device characterized by the following features.

2. The above text information generation unit, A prompt generation unit that replaces handwritten character images contained in the scanned image with character codes by optical character recognition, and generates prompts for input into a large-scale language model using the replaced character codes, The system includes an LLM processing unit that generates the item text information by inputting the prompts generated by the prompt generation unit into the large-scale language model. The electronic data generation apparatus according to feature 1.

3. The electronic data generation apparatus according to claim 2, characterized in that the prompt generation unit generates the prompt by adding reference information to the character information indicated by the replaced character code, which is used by the large-scale language model when it performs the process of acquiring the item text information based on the character information.

4. The electronic data generation apparatus according to claim 2 or 3, characterized in that when the text information generation unit generates the item text information from the scanned image of the paper medium containing the handwritten input items in a table format, it analyzes the scanned image to determine whether or not the handwritten information is included in each description field of the recognized table area, adds predetermined symbol information to the description fields that do not contain the handwritten information, and generates the item text information from the images of the handwritten information in each description field of the table area with the symbol information added.

5. The electronic data generation apparatus according to claim 2 or 3, wherein the prompt generation unit has a character correction function for correcting the recognized characters indicated by the character code replaced by the optical character recognition, and generates the prompt using the corrected character information.

6. An electronic data generation method for generating electronic data in a predetermined format from a paper document containing a mixture of printed and handwritten information, The computer's text information generation unit generates item text information to be entered into the information items of the electronic data from an image of the handwritten information contained in a scanned image of the paper medium having handwritten input items similar to the information items of the electronic data in the predetermined format, The electronic data generation unit of the above computer has the step of inputting the item text information generated by the text information generation unit into the information item and generating electronic data in the predetermined format. A method for generating electronic data characterized by the following features.

Images