Information processing device

Abstract

To provide an information processing device that can properly reproduce data of a scanned document even if the read document contains defects in sentences.SOLUTION: An information processing device 100 comprises a detection unit 2, a removal unit 3, an estimation unit 4, a reception unit 7, and a reproduction unit 8. The detection unit 2 detects defects in sentences included in scanned data. The removal unit 3 removes the detected defects from the scanned data to generate defect-removed data. The estimation unit 4 estimates a plurality of candidates for correct character strings for the defects based on a content of the sentences excluding the defects. The reception unit 7 accepts one of the estimated candidates or an input character string entered by a user. The reproduction unit 8 adds the one candidate or the input character string accepted by the reception unit 7 to a portion of the defect-removed data from which the defects were removed to generate reproduced data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device. [Background technology]

[0002] An information processing apparatus reads an image from a document, and the data of the read document may contain defects due to damage such as stains or bending of the document. As a method for detecting defects contained in data of a scanned document, Patent Document 1 describes a method that uses deep learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Chinese Patent Application Publication No. 112070716 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method described in Patent Document 1 merely detects defects contained in the data of the scanned document. Therefore, even if defects are detected by the method described in Patent Document 1, the data of the scanned document is incomplete due to the defects and therefore needs to be reproduced.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an information processing device that can properly reproduce data of a read document even if the text of the document to be read has defects. [Means for solving the problem]

[0006] According to a first aspect of the present invention, an information processing device includes a detection unit, a removal unit, an estimation unit, a reception unit, and a reproduction unit. The detection unit detects defects in sentences included in read data. The removal unit removes the detected defects from the read data to generate defect-removed data. The estimation unit estimates multiple candidates for a correct string of characters for the defect based on the content of the sentence other than the defect. The reception unit accepts one of the multiple estimated candidates or an input string entered by a user. The reproduction unit adds one of the candidates or the input string accepted by the reception unit to a portion of the defect-removed data from which the defect has been removed to generate reproduced data. [Effects of the Invention]

[0007] According to the information processing apparatus of the present invention, even if the document to be read has defects in the text, the data of the read document can be properly reproduced. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of an information processing device according to a first embodiment. [Figure 2] FIG. 2 is an enlarged view of a touch screen of the information processing device. [Figure 3] FIG. 10 is a block diagram of an information processing device according to a second embodiment. [Figure 4] FIG. 10 is a diagram displayed on a touch screen of an information processing device. [Figure 5A] FIG. 10 is a diagram of the touch screen when the first option is selected. [Figure 5B] FIG. 10 is a view of the touch screen after the first option has been selected. [Figure 6A] FIG. 10 is a diagram of the touch screen when the second option is selected. [Figure 6B] FIG. 10 is a view of the touch screen after the second option has been selected. [Figure 7A] FIG. 10 is a diagram of the touch screen when the third option is selected. [Figure 7B] FIG. 10 is a view of the touch screen after the third option has been selected. [Figure 8]FIG. 10 is a block diagram of an information processing device according to a third embodiment. [Figure 9] FIG. 10 is a block diagram of an image forming apparatus including an information processing apparatus according to a fourth embodiment. [Figure 10] 10 is a flowchart illustrating an operation of the information processing device. [Figure 11] 10 is a flowchart detailing the process for detecting a sentence defect. [Figure 12] 10 is a flowchart illustrating in detail the flow from estimation of candidates to generation of playback data. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals, and the description will not be repeated. Furthermore, in the following description, terms meaning specific positions and directions, such as "upper," "lower," "left," or "right," may be used, but these terms are used for convenience to facilitate understanding of the contents of the embodiments, and do not relate to the directions when actually implemented. <Embodiment>

[0010] An information processing device 100 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram of the information processing device 100 according to the first embodiment.

[0011] The information processing device 100 according to this embodiment is, for example, a multifunction peripheral having multiple functions. The multiple functions include, for example, a scanner function, a printer function, a copy function, a fax receiving function, a fax sending function, a finisher function, and an optional paper feeding function. The multifunction peripheral has at least a scanner function.

[0012] As shown in FIG. 1, the information processing device 100 includes a detection unit 2, a removal unit 3, an estimation unit 4, a reception unit 7, and a reproduction unit 8.

[0013] The detection unit 2 detects defects in sentences contained in the read data, and the removal unit 3 removes the detected defects from the read data to produce defect-free data.

[0014] The estimation unit 4 estimates multiple candidates for the correct character string for the defect based on the content of the sentence other than the defect. The reception unit 7 receives one of the multiple estimated candidates or an input character string input by the user.

[0015] The reproducing unit 8 adds one of the candidates accepted by the accepting unit 7 or the input character string to the part of the defect elimination data from which the defect has been removed, to generate reproduced data.

[0016] The above-described configuration makes it easier to detect and remove defects in the text of the original document, and then add correct character strings to the data. Therefore, even if the text of the original document has defects, the information processing device 100 can properly reproduce the data of the original document.

[0017] Since the data of the original is properly reproduced, the burden on the user for reproducing the data can be reduced. For example, the user does not need to prepare an original without defects for data reproduction.

[0018] As shown in FIG. 1, the information processing device 100 further includes an image reading unit 1, a control device 10, and a storage unit 11.

[0019] The image reading unit 1 reads an image from a document as read data. The image reading unit 1 is, for example, a reading slit (not shown). The reading slit ensures a scanner function. The reading slit optically reads the document as it passes through, thereby generating read data.

[0020] The control device 10 is a processor such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The control device 10 includes a detection unit 2, a removal unit 3, an estimation unit 4, a reception unit 7, and a reproduction unit 8.

[0021] The storage unit 11 stores data and computer programs. The storage unit 11 includes storage devices (main storage device and auxiliary storage device). The storage unit 11 includes, for example, a memory and a hard disk drive. The memory is a ROM (Read Only Memory) and a RAM (Random Access Memory), etc. The storage unit 11 is connected to the control device 10. The storage unit 11 may include removable media.

[0022] The character string may include not only letters but also numbers and symbols. The character string is not limited to multiple characters and may be a single character.

[0023] A sentence defect is one that cannot be identified as a character string that constitutes a sentence. For example, a sentence defect is one in which a character string is missing or overwritten to the extent that it cannot be identified as a character string.

[0024] The correct character string for a defect is a character string that would have been distinguishable if there had been no defect. For example, if the character string "Sun" written in a sentence is indistinguishable and therefore corresponds to a defect, then the written "Sun" is the correct character string for the defect.

[0025] The estimation unit 4 estimates the most suitable candidate from among the multiple estimated candidates as the optimum candidate. By estimating the optimum candidate, the estimation unit 4 makes it easier to add a correct character string to the portion from which the defect has been removed. Therefore, the information processing device 100 can more appropriately reproduce the data of the read manuscript even if there is a defect in the sentence of the manuscript to be read.

[0026] The display and operation configuration of the information processing device 100 will be described below with reference to Fig. 2. Fig. 2 is an enlarged view of the touch screen 50 of the information processing device 100.

[0027] 2, the information processing device 100 has a touch screen 50. The touch screen 50 displays information required for operation and is operated by the user.

[0028] The touch screen 50 displays, for example, "Clean Scan" indicated by the reference numeral 51. By tapping on "Clean Scan" indicated by the reference numeral 51, the user can switch ON / OFF the function for reproducing the data of the scanned document (hereinafter referred to as the reproduction function).

[0029] When the playback function is switched ON, defects are detected and removed, and correct character strings are added to the text of the scanned original. When the playback function is switched OFF, defects are not detected and removed, and correct character strings are not added to the text of the scanned original. Note that even if the playback function is switched ON, if there are no defects in the text of the scanned original, defects are not detected and removed, and correct character strings are not added. <Embodiment 2>

[0030] An information processing device 100 according to the second embodiment will be described below with reference to Fig. 3 and Fig. 4. Fig. 3 is a block diagram of the information processing device 100 according to the second embodiment. Fig. 4 is a diagram displayed on the touch screen 50 of the information processing device 100.

[0031] 3, the information processing device 100 further includes an option display unit 5. The option display unit 5 displays options that the user can select.

[0032] The options include a first option, a second option, and a third option. The first option is an option for the user to select one from multiple estimated candidates. The second option is an option for the user to input an input string. The third option is an option for the best candidate to be automatically selected.

[0033] The option display unit 5 allows the user to select one of the first to third options, making it easier to add the correct character string to the part where the defect has been removed. Therefore, the information processing device 100 can more appropriately reproduce the data of the read original even if there is a defect in the text of the original to be read.

[0034] The information processing device 100 further includes a character number proposal unit 6. The character number proposal unit 6 proposes the maximum number of characters for an input string to be entered by the user. By having the character number proposal unit 6 propose the maximum number of characters, it becomes easier for the user to input a string of characters with an appropriate number of characters. Therefore, even if there are defects in the sentences of the original document to be read, the information processing device 100 can more appropriately reproduce the data of the original document to be read.

[0035] The control device 10 further includes an option display unit 5 and a character number suggestion unit 6. The option display unit 5 displays, for example, first to third options on a touch screen 50. As shown in FIG. 4, defect removal data indicated by reference numeral 30 is displayed on the left side of the touch screen 50. The first to third options indicated by reference numeral 70 are displayed on the right side of the touch screen 50.

[0036] The defect removal data indicated by the symbol 30 displayed on the left side of the touch screen 50 is assigned a circled number for each removed defect. In the example shown in FIG. 4, two defects were removed, so the first removed defect is assigned a circled 1 (circled number 1), and the second removed defect is assigned a circled 2 (circled number 2). Therefore, if N defects were removed, they would be assigned numbers 1 to N (circled number N).

[0037] The first to third options displayed on the right side of the touch screen 50 are three vertical rows of buttons 70 that can be tapped. The top button 71, i.e., the button 71 labeled "Suggested Words," corresponds to the first option. The middle button 72, i.e., the button 72 labeled "Manual Edit," corresponds to the second option. The bottom button 73, i.e., the button 73 labeled "Autofill," corresponds to the third option.

[0038] The user can select one of the first to third options by tapping any of the three vertically arranged buttons 70. Below, the cases where each of the first to third options is selected will be described.

[0039] First, the case where the first option is selected will be described with reference to Figures 5A and 5B. Figure 5A is a diagram of the touch screen 50 when the first option is selected. Figure 5B is a diagram of the touch screen 50 after the first option has been selected.

[0040] As shown in Fig. 5A, the first option is selected by tapping the button 71 labeled "Suggested Words" corresponding to the first option. When the first option is selected, the image on the touch screen 50 changes from Fig. 5A to Fig. 5B.

[0041] As shown in Figure 5B, defect removal data, designated by the reference numeral 30, is displayed on the left side of touch screen 50. A circled number, designated by the reference numeral 31 (a circled 1 in Figure 5B), is displayed in the center of touch screen 50. Four vertical rows of buttons 54 are displayed on the right side of touch screen 50.

[0042] The defect removal data 30 displayed on the left side of the touch screen 50 is the same as the defect removal data shown in FIGS. 4 and 5A.

[0043] The circled number of the reference numeral 31 (circled 1 in FIG. 5B ) displayed in the center of the touch screen 50 corresponds to the number of the removed defect to which a character string will be added in the defect removal data of the reference numeral 30. In the example shown in FIG. 5B , the circled number of the reference numeral 31 is a circled 1. Therefore, a character string will be added to the removed defect corresponding to the circled 1 in the defect removal data of the reference numeral 30.

[0044] Of the four vertical rows of buttons 54 displayed on the right side of the touch screen 50, the topmost button 41 through the third row of buttons 43 correspond to the multiple (three) estimated candidates. Radio buttons are arranged to the right of each of the topmost button 41 through the third row of buttons 43. Of the four vertical rows of buttons 54, the bottommost button 49 is a button for confirming the selected candidate. The radio buttons and the bottommost button 49 can be operated by tapping.

[0045] The topmost button 41 is labeled "Sun (Closest)" and corresponds to the best candidate. The tagging of "(Closest)" in "Sun (Closest)" indicates that it is the best candidate. The second row of buttons 42 and the third row of buttons 43 are labeled "Son" and "Sin," respectively, and correspond to candidates other than the best candidate. That is, in the example shown in FIG. 5B, "Sun," "Son," and "Sin" are multiple estimated candidates, with "Sun" being the best candidate. The bottommost button 49 is labeled "Fill Text," which is used to determine the selected candidate.

[0046] The user taps one radio button corresponding to the candidate they wish to select from among the buttons 43 in the third row from the top, starting with the top button 41. Then, the user confirms the selected candidate by tapping button 49 labeled "Fill Text."

[0047] 5B, three candidates are displayed on the touch screen 50, but the number may be two, four, or more. Regardless of the number of candidates displayed on the touch screen 50, the best candidate is tagged with a tag such as "(Closest)" to indicate that it is the best candidate.

[0048] Next, the case where the second option is selected will be described with reference to Figures 6A and 6B. Figure 6A is a diagram of the touch screen 50 when the second option is selected. Figure 6B is a diagram of the touch screen 50 after the second option has been selected.

[0049] As shown in Fig. 6A, the second option is selected by tapping the button 72 labeled "Manual Edit" corresponding to the second option. When the second option is selected, the image on the touch screen 50 changes from Fig. 6A to Fig. 6B.

[0050] As shown in Fig. 6B, defect removal data denoted by reference numeral 30 is displayed on the left side of the touch screen 50. A circled number denoted by reference numeral 31 (a circled 1 in Fig. 6B) is displayed in the center of the touch screen 50. A character string input field 61 and a panel keyboard 63 are displayed on the right side of the touch screen 50.

[0051] The defect removal data indicated by the reference numeral 30 on the left side of the touch screen 50 is the same as the defect removal data shown in Figures 4, 5A, and 6A. The circled number indicated by the reference numeral 31 on the center of the touch screen 50 (circled 1 in Figure 6B) is the same as the circled number shown in Figure 5B.

[0052] A character string is input into a character string input field 61 displayed on the right side of the touch screen 50 by the user operating a panel keyboard 63. The character number suggestion unit 6 suggests the maximum number of characters for the input character string in the character string input field 61. In the example shown in FIG. 6B , three characters, indicated by the reference symbol 62, are suggested as the maximum number of characters in the character string input field 61.

[0053] Next, the case where the third option is selected will be described with reference to Figures 7A and 7B. Figure 7A is a diagram of the touch screen 50 when the third option is selected. Figure 7B is a diagram of the touch screen 50 after the third option has been selected.

[0054] As shown in Fig. 7A, the third option is selected by tapping button 73 labeled "Autofill," which corresponds to the third option. When the third option is selected, the image on touch screen 50 changes from Fig. 7A to Fig. 7B.

[0055] 7B, defect removal data indicated by the reference numeral 30 is displayed on the left side of the touch screen 50. Two vertical rows of buttons 52 and a page input field 56 are displayed on the right side of the touch screen 50.

[0056] The defect removal data 30 displayed on the left side of the touch screen 50 is the same as the defect removal data shown in FIGS. 4, 5A, 6A and 7A.

[0057] Two vertical rows of buttons 52 displayed on the right side of the touch screen 50 can be operated by tapping. Of the two vertical rows of buttons 52, the top button 53 is a button for specifying all pages of the defect removal data as the target from which the optimum candidate for the removed defect will be automatically selected. The page input field 56 is a field for specifying any page of the defect removal data as the target from which the optimum candidate for the removed defect will be automatically selected. Any page is input into the page input field 56. Of the two vertical rows of buttons 52, the bottom button 59 is a button for confirming the page input into the page input field 56.

[0058] The upper button 53 is labeled "Autofill (All)." The lower button 59 is labeled "Autofill Selected Pages." The page input field 56 is located between the upper button 53 and the lower button 59.

[0059] If the user wants the best candidate for the removed defect to be automatically selected for all pages, the user taps the upper button 53 labeled "Autofill (All)." On the other hand, if the user wants the best candidate for the removed defect to be automatically selected for any page, the user inputs the desired page in the page input field 56. Then, the user taps the lower button 59 labeled "Autofill Selected Pages," and the input page is determined. <Embodiment 3>

[0060] Hereinafter, an information processing device 100 according to the third embodiment will be described with reference to Fig. 8. In the information processing device 100 according to the third embodiment, the detection unit 2 and the estimation unit 4 use AI (Artificial Intelligence). Fig. 8 is a block diagram of the information processing device 100 according to the third embodiment.

[0061] 8, the information processing device 100 further includes a first learning unit 20. The first learning unit 20 generates a first trained model by machine learning defects in sentences. The detection unit 2 detects defects in sentences included in the read data using the first trained model.

[0062] The detection unit 2 uses the first trained model to detect defects in sentences with high accuracy. Therefore, even if the sentences in the scanned document have defects, the information processing device 100 can more appropriately reproduce the data of the scanned document.

[0063] The information processing device 100 further includes a second learning unit 40. The second learning unit 40 generates a second trained model by machine learning a character string of a sentence. The estimation unit 4 estimates a plurality of candidates using the second trained model.

[0064] The estimation unit 4 uses the second trained model, which allows multiple candidates to be estimated with high accuracy. Therefore, even if the sentences in the read manuscript have defects, the information processing device 100 can more appropriately reproduce the data of the read manuscript.

[0065] The estimation unit 4 uses the second trained model to estimate multiple candidates for the correct character string of the defect based on the content other than the defect, such as the context other than the defect, character spacing, and font style.

[0066] The control device 10 further includes a first learning unit 20 and a second learning unit 40. The first learning unit 20 and the second learning unit 40 perform machine learning using a teacher dataset to generate a first trained model and a second trained model, respectively. Hereinafter, the first trained model and the second trained model may be collectively referred to as trained models. The generated trained models are stored in the storage unit 11.

[0067] The machine learning algorithm for generating the trained model is not particularly limited as long as it is supervised learning, and may be, for example, a decision tree, a nearest neighbor method, a naive Bayes classifier, a support vector machine, or a neural network. Therefore, the trained model includes a decision tree, a nearest neighbor method, a naive Bayes classifier, a support vector machine, or a neural network. In the machine learning for generating the trained model, backpropagation may be used.

[0068] The machine learning algorithm that generates the first trained model is, for example, a convolutional neural network (CNN). That is, the first trained model includes a convolutional neural network. The convolutional neural network includes an input layer, multiple convolutional layers, multiple pooling layers, a fully connected layer, and an output layer. In the convolutional neural network, convolutional layers and pooling layers are alternately repeated between the input layer and the fully connected layer.

[0069] The teacher dataset of the first trained model used by the first learning unit 20 is, for example, a dataset including a plurality of image data of sentences containing defects, a plurality of image data of sentences not containing defects, information on whether or not the sentences in these image data contain defects, and position information of the defects in the image data of the sentences containing defects among these image data. The teacher dataset may be a dataset prepared in advance, or may be a dataset read by the image reading unit 1 before the first learning unit 20 performs machine learning.

[0070] The machine learning algorithm that generates the second trained model is, for example, natural language processing (NLP) and a recurrent neural network (RNN) after undergoing optical character recognition (OCR). That is, the second trained model includes natural language processing and a recurrent neural network.

[0071] Optical character recognition is the process of converting a string of characters into a machine-readable text format. Therefore, the defect removal data undergoes optical character recognition, which makes the string of characters into a machine-readable text format. Optical character recognition also makes the font style and character spacing of the string of characters machine-readable.

[0072] Natural language processing is a set of techniques that allow computers to process natural language. Natural language is the language used by humans to communicate with each other. Natural language processing includes morphological analysis, syntactic analysis, semantic analysis, and contextual analysis.

[0073] Morphological analysis is the process of dividing text into morphemes, the smallest units that have meaning. Syntactic analysis is performed based on the results of morphological analysis. Syntactic analysis is the process of analyzing the relationship between morphemes. Semantic analysis is the process of determining a syntax tree based on the results of syntactic analysis. A syntax tree shows the progress and results of syntactic analysis in a tree structure. Contextual analysis is performed based on the results of semantic analysis. Contextual analysis is the process of analyzing the relationship between sentences. A sentence contains a subject and a predicate and represents a complete statement.

[0074] A recurrent neural network has an input layer, a hidden layer, and an output layer. The hidden layer may be one layer or two or more layers.

[0075] The teacher dataset of the second trained model used by the second learning unit 40 is, for example, a dataset of a large number of sentences. The teacher dataset may be a dataset prepared in advance, or may be a dataset read by the image reading unit 1 before the second learning unit 40 performs machine learning. <Embodiment 4>

[0076] An information processing device 100 according to the fourth embodiment and an image forming device 101 including the information processing device 100 will be described below with reference to Fig. 9. Fig. 9 is a block diagram of the image forming device 101 including the information processing device 100 according to the fourth embodiment.

[0077] As shown in FIG. 9, the information processing device 100 further includes a processing unit 9. The processing unit 9 transmits, saves, or deletes the reproduced data. By transmitting, saving, or deleting the reproduced data, the reproduced data can be applied. Therefore, even if the text of the read document has defects, the information processing device 100 can properly reproduce the data of the read document and apply the reproduced data.

[0078] The information processing device 100 further includes a transmission unit 91. The transmission unit 91 transmits data to the outside via a network (not shown). For example, when the processing unit 9 transmits playback data, the playback data is transmitted to the outside via the network by the transmission unit 91.

[0079] When the processing unit 9 saves the playback data, the playback data is saved in the storage unit 11. Specifically, the processing unit 9 stores the playback data stored in the memory of the storage unit 11 in the hard disk drive of the storage unit 11.

[0080] When the processing unit 9 deletes the playback data, the playback data is deleted from the storage unit 11. Specifically, the processing unit 9 erases the playback data stored in the memory or hard disk drive of the storage unit 11 from the memory or hard disk drive.

[0081] The image forming apparatus 101 includes an information processing device 100. A processing unit 9 of the information processing device 100 prints the reproduced data. By printing the reproduced data, the reproduced data can be used as a printed matter. Therefore, even if the text of the original document read by the information processing device 100 has defects, the image forming apparatus 101 can properly reproduce the data of the read original document and use the reproduced data as a printed matter.

[0082] The control device 10 further includes a processing unit 9. The image forming device 101 further includes an image forming unit 92. The image forming unit 92 receives reproduction data from the processing unit 9. The image forming unit 92 forms an image on a sheet (not shown) based on the received reproduction data.

[0083] The operation of the information processing device 100 will be described below with reference to Fig. 10. Fig. 10 is a flowchart illustrating the operation of the information processing device 100.

[0084] 10, in step S1, the image reading unit 1 reads an image from a document as read data. In step S2, the control unit 10 and the storage unit 11 perform optical character recognition on the read data. In step S3, the detection unit 2 detects defects in sentences included in the read data that has undergone optical character recognition.

[0085] In step S4, if the sentence contained in the read data has a defect, the process proceeds to step S6 (Yes in step S4), and if the sentence contained in the read data has no defect, the process proceeds to step S5 (No in step S4).

[0086] In step S5, if the document is to be read again, the process returns to step S1 (Yes in step S5), and if the document is not to be read again, the process ends (No in step S5).

[0087] In step S6, the removal unit 3 removes the defects. In step S7, the estimation unit 4 performs optical character recognition on the defect-removed data, which is data from which the defects have been removed. In step S8, the estimation unit 4 performs natural language processing on the defect-removed data that has been subjected to optical character recognition. In step S9, the estimation unit 4 estimates multiple candidates from the defect-removed data that has been subjected to natural language processing using a recurrent neural network.

[0088] In step S10, the accepting unit 7 accepts one of the candidates or an input character string. In step S11, if the playback data is complete, the process ends (Yes in step S11), and if the playback data is not complete, the process returns to step S10 (No in step S11).

[0089] The flow up to the detection of a sentence defect and the flow from the estimation of candidates to the generation of reproduced data will be described in detail below with reference to Figures 11 and 12. Figure 11 is a flowchart that explains in detail the flow up to the detection of a sentence defect. Figure 12 is a flowchart that explains in detail the flow from the estimation of candidates to the generation of reproduced data.

[0090] 11, in step S21, the detection unit 2 acquires read data. In step S22, the detection unit 2 performs a process of highlighting defects. In step S23, the detection unit 2 converts the read data into grayscale.

[0091] In step S24, the detection unit 2 detects edges. In step S25, the detection unit 2 performs binarization processing. In step S26, the detection unit 2 marks the image of the defect.

[0092] In step S27, the detection unit 2 performs region proposal. In step S28, the detection unit 2 identifies a region of interest based on the region proposal.

[0093] 12, in step S41, the estimation unit 4 performs context analysis using natural language processing based on tokens. In step S42, the estimation unit 4 embeds character strings. In step S43, the estimation unit 4 generates a second trained model of a recurrent neural network.

[0094] In step S44, the estimation unit 4 classifies the character string. In step S45, the estimation unit 4 estimates a plurality of candidates. After step S45, the process proceeds to step S46 or step S47.

[0095] In step S46, the user selects either the first option or the second option. If the first option is selected, the user selects one of the multiple predicted candidates. If the second option is selected, the user inputs an input string.

[0096] In step S47, the user selects the third option. When the third option is selected, the best candidate is automatically selected.

[0097] The embodiments of the present invention have been described above with reference to the drawings. However, the present invention is not limited to the above embodiments and can be embodied in various forms without departing from the spirit and scope of the present invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above embodiments. For example, some components may be omitted from all components shown in the embodiments. The drawings mainly show each component in a schematic manner for ease of understanding, and the thickness, length, number, spacing, etc. of each illustrated component may differ from the actual components due to the convenience of drawing. Furthermore, the shapes, etc. of each component shown in the above embodiments are merely examples and are not particularly limited. Various modifications are possible within a scope that does not substantially deviate from the configuration of the present invention.

[0098] (1) The diagrams shown in FIGS. 4 to 7B do not necessarily need to be displayed on the touch screen 50 of the information processing device 100, but may be displayed on a computer display or smartphone (not shown). Operations based on the displayed content may also be performed on the computer or smartphone. In this case, the option display unit 5 displays the first to third options on the computer display or smartphone, not on the touch screen 50. The computer or smartphone is connected to the information processing device 100 via a network (not shown).

[0099] (2) In the above explanation, an example was given in which the text of the document to be read was in English, but it may be in other languages ​​such as Japanese. [Industrial Applicability]

[0100] The present invention relates to an information processing device and has industrial applicability. [Explanation of symbols]

[0101] 1 Image reading unit 2. Detection unit 3 Removal section 4 Estimation part 5 Option display area 6 Character count suggestion part 7 Reception 8 Playback section 9 Processing section 20 First Study Section 40 Second Learning Section 92 Image forming unit 100 Information processing device 101 Image forming device

Claims

1. a detection unit that detects defects in sentences included in the read data; a removal unit that removes the detected defect from the read data to generate defect-removed data; an estimation unit that estimates a plurality of candidates for a correct character string for the defect based on content other than the defect in the sentence; a receiving unit that receives one of the plurality of estimated candidates or an input character string input by a user; a reproduction unit that adds one of the candidates or the input character string received by the reception unit to a portion of the defect removal data from which the defect has been removed, to generate reproduced data; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the estimation unit estimates an optimal one of the multiple estimated candidates as the optimal candidate.

3. further comprising an option display unit that displays options that can be selected by the user; The options are: a first option for a user to select one of the plurality of estimated candidates; a second option for the user to enter the input string; A third option in which the best candidate is automatically selected; The information processing device according to claim 2 , further comprising:

4. The information processing device according to claim 1 , further comprising a character number suggestion unit that suggests a maximum number of characters for the input character string input by the user.

5. A first learning unit is further provided, The first learning unit generates a first trained model by machine learning a defect in a sentence; The information processing device according to claim 1 , wherein the detection unit detects the defect in the sentence included in the read data using the first trained model.

6. Further provided with a second learning unit, The second learning unit generates a second trained model by machine learning a character string of a sentence; The information processing device according to claim 1 or 2, wherein the estimation unit estimates the plurality of candidates using the second trained model.

7. Further comprising a processing unit, The information processing device according to claim 1 , wherein the processing unit stores, transmits, or deletes the playback data.

Images