Information processing device, information processing system, information processing method, and program

Abstract

To use both character information and image information extracted from image data in a document in retrieving the document.SOLUTION: An information processing device for managing a digitized document stored in a database includes an area discrimination part for discriminating areas of character information and image information included in image data in the document, a character string extraction part for extracting a character string from the area of the character information of the image data, a sentence acquisition part for using information included in the character string to acquire a sentence from a preset range of the document, a word selection part for selecting at least one or more input words from the sentence in accordance with the frequency of appearance, a classification tag generation part for outputting a classification tag that classifies the image data by inputting the input words and the image information to a model learned in advance by machine learning, an information update part for storing the classification tag in the database, and a retrieval part for using the classification tag to retrieve a document in the database including relevant image data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] When digitizing internal documents and managing them as assets in a database, linking related documents using classification tags is important for traceability, which is used to search the database and track document update histories. Furthermore, generating classification tags that classify image data such as block diagrams and flow charts contained in documents such as design documents and specifications during development is effective for searching documents that contain related images.

[0003] As a related art technique, Patent Document 1 discloses a technique for improving search accuracy by using character strings extracted from image data such as log images by OCR (Optical Character Recognition) to search documents. Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, there is a problem in that only character information extracted from image data is used in document retrieval, and image information of the image data is not used.

[0005] In view of the above-mentioned problems, an embodiment of the present invention has an object to use both character information and image information extracted from image data in a document when searching for the document. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the present invention is an information processing device that manages digitized documents stored in a database, comprising: an area identification unit that identifies areas of character information and image information contained in image data in the document; a string extraction unit that extracts a string of characters from an area of ​​character information in the image data; a sentence acquisition unit that acquires sentences from a predetermined range of the document using information contained in the string of characters; a word selection unit that selects at least one input word from the sentence according to its frequency of appearance; a classification tag generation unit that outputs a classification tag that classifies the image data by inputting the input words and the image information into a model that has been trained in advance by machine learning; an information update unit that stores the classification tag in the database; and a search unit that uses the classification tag to search for documents in the database that contain related image data. Effect of the Invention

[0007] According to an embodiment of the present invention, in searching a document, both character information and image information extracted from image data in the document can be used. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a schematic diagram of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device and a terminal device according to an embodiment of the present invention. [Diagram 3] FIG. 1 is a diagram showing an example of a functional block diagram of an information processing system according to an embodiment of the present invention; [Figure 4] FIG. 11 is a diagram showing an example of a processing flow for generating classification tags for images within a document according to an embodiment of the present invention. [Diagram 5] 11A and 11B are diagrams illustrating a process of extracting a sentence from a document in an asset management DB according to an embodiment of the present invention. [Figure 6] 5A to 5C are diagrams illustrating an example of input / output data of a classification tag generation unit according to an embodiment of the present invention. [Figure 7]FIG. 2 is a diagram illustrating an example of a network model of a classification tag generation unit according to an embodiment of the present invention. [Figure 8] FIG. 2 is a diagram showing an example of traceability information according to an embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing another example of traceability information according to an embodiment of the present invention. [Figure 10] 11A and 11B are diagrams illustrating a method for specifying a range of documents to be acquired by a text acquisition unit according to an embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of a processing flow for generating classification tags for images in a document using a co-occurrence relationship list according to an embodiment of the present invention. [Figure 12] 1 is a diagram illustrating a method for generating and using a co-occurrence relationship list according to an embodiment of the present invention. [Figure 13] FIG. 11 is a diagram showing an example of a processing flow for generating a classification tag for an image read from outside according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an information processing device, an information processing system, an information processing method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.

[0010] [First embodiment] <System Overview> FIG. 1 is a diagram showing an example of a schematic diagram of an information processing system according to an embodiment of the present invention. The information processing system 4 is configured in such a manner that a terminal device 5, an image forming device 6, and an information processing device 2 are connected to a communication network 1. The administrator can operate the terminal device 5 to access the information processing device 2, generate classification tags for image data in a digitized document stored in the management asset DB3, and perform management such as updating traceability information. The user can operate the terminal device 5 to search for documents containing related images in the management asset DB3 using classification tags. It is also possible to register documents containing image data captured using a scanner function of the image forming device 6 in the asset management DB3. In the following description, image data may be simply expressed as an image.

[0011] <Hardware configuration example> Fig. 2 is a diagram showing an example of the hardware configuration of the information processing device 2 and the terminal device 5 according to the embodiment of the present invention. As shown in Fig. 2, the information processing device 2 and the terminal device 5 are constructed by a computer, and include a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0012] Among these, the CPU 501 controls the overall operation of the information processing device 2 and the terminal device 5. The ROM 502 stores programs such as an IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data from the HD 504 under the control of the CPU 501. The display 506 displays various information such as a cursor, a menu, a window, a character, or an image. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the communication network 1. The data bus 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG. 5.

[0013] Moreover, the keyboard 511 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513 as an example of a removable recording medium. Note that this is not limited to DVD-RW, and may be DVD-R, etc. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515 such as a flash memory.

[0014] <About the function> 3 is a diagram showing an example of a functional block diagram of an information processing system 4 according to an embodiment of the present invention. As shown in FIG. 3, the information processing system 4 includes an information processing device 2 and a terminal device 5 as devices having main functional blocks.

[0015] <Functions of information processing device> The information processing device 2 includes an image acquisition unit 101, an area identification unit 102, a string extraction unit 103, a first analysis unit 104, a sentence acquisition unit 105, a second analysis unit 106, a word selection unit 107, a classification tag generation unit 108, an information update unit 109, a first communication unit 110, and a search unit 111.

[0016] The image acquisition unit 101 acquires image data that does not have classification tag information from among image data included in documents registered in the managed asset DB 3, and transmits the acquired image data to the area identification unit 102. Alternatively, the image acquisition unit 101 transmits image data scanned by the scan function of the image forming device 6 or image data captured by a camera to the area identification unit 102.

[0017] The area identification unit 102 identifies and divides the image data received from the image acquisition unit 101 into character areas and non-character areas, transmits information about the character areas (character information in the image data) to the string extraction unit 103, and transmits information about the non-character areas (image information in the image data) to the classification tag generation unit 108.

[0018] The character string extraction unit 103 extracts a character string from the information on the character region received from the region identification unit 102, and transmits the character string to the first analysis unit 104. This extraction process can be realized by a process similar to that performed in general OCR (Optical Character Recognition).

[0019] The first analysis unit 104 transmits to the text acquisition unit 105 a list of words and parts of speech of each word (word list) obtained by performing a morphological analysis on the character string received from the character string extraction unit 103. Alternatively, the first analysis unit 104 transmits to the text acquisition unit 105 a list of words and the occurrence frequency of each word (word list) obtained by performing an N-gram analysis on the received character string.

[0020] The text acquisition unit 105 acquires text that includes words included in the word list received from the first analysis unit 104 from a preset range of the asset management DB 3, and transmits the acquired text to the second analysis unit .

[0021] The second analysis unit 106 transmits to the word selection unit 107 a list of words and parts of speech of each word obtained by performing a morphological analysis on the sentence received from the sentence acquisition unit 105. Alternatively, the second analysis unit 106 transmits to the sentence acquisition unit 105 a co-occurrence relationship list 221 created by dividing the received character string into specified character strings.

[0022] The word selection unit 107 transmits at least one or more words selected from the list received from the second analysis unit 106 to the classification tag generation unit 108 as input words.

[0023] The classification tag generation unit 108 takes as input the image information received from the area identification unit 102 and the words received from the classification tag generation unit 108, generates classification tags using a model previously trained through machine learning (deep learning), and outputs them to the information update unit 109.

[0024] The information update unit 109 updates the traceability information stored in the asset management DB 3 by storing the classification tag received from the classification tag generation unit 108 as a classification tag for the image data of the document acquired by the image acquisition unit 101.

[0025] The first communication unit 110 transmits and receives data to and from the second communication unit 123 of the terminal device 5 via the communication network 1. Information such as screen information of the document search screen and document search results is transmitted from the information processing device 2 to the terminal device 5.

[0026] Based on a request from a user operating the terminal device 5, the search unit 111 uses classification tags to search for documents including related image data in the asset management DB 3.

[0027] <Terminal device functions> The terminal device 5 includes a display control unit 121, an operation receiving unit 122, and a second communication unit 123.

[0028] The display control unit 121 displays screen information such as a document search screen received from the information processing device 2, document search results, and the like on the display of the terminal device 5.

[0029] The operation reception unit 122 receives input operations related to document search by the user of the terminal device 5 and the like.

[0030] The second communication unit 123 transmits and receives data to and from the first communication unit 110 of the information processing device 2 via the communication network 1. Information on an input operation by a user accepted by the operation acceptance unit 122 and the like is transmitted from the terminal device 5 to the information processing device 2.

[0031] <Process of generating classification tags to classify images in a document> 4 is a diagram showing an example of a process flow for generating classification tags for images in a document according to an embodiment of the present invention. Below, an overall flow of processes performed by each functional block of the information processing system 4 will be described. Details of each process will be described with reference to the following drawings.

[0032] In the image acquisition step S201, the administrator accesses the management screen of the information processing device 2 by performing an operation from the operation reception unit 122 of the terminal device 5. The first communication unit 110 of the information processing device 2 transmits information related to the management screen to the second communication unit 123 of the terminal device 5. The display control unit 121 of the terminal device 5 displays the received screen information on the display of the terminal device 5. On the management screen, the administrator causes the information processing device 2 to execute a process of adding classification tags to images of documents that do not have classification tag information, such as documents newly registered in the managed asset DB3. The image acquisition unit 101 acquires image data that does not have classification tag information from image data included in documents registered in the managed asset DB3, and transmits the acquired image data to the area identification unit 102.

[0033] In area selection step S202, the area identification unit 102 divides the image data received from the image acquisition unit 101 into character areas and non-character areas, transmits information about the character areas (character information in the image data) to the string extraction unit 103, and transmits information about the non-character areas (image information in the image data) to the classification tag generation unit 108.

[0034] In character string extraction step S203, character string extraction unit 103 extracts a character string from the information on the character region received from region identification unit 102, and transmits the character string to first analysis unit 104. This extraction process can be realized by a process similar to that performed in general OCR.

[0035] In the first morphological analysis step S204, the first analysis unit 104 breaks down the received character string into morphemes based on the morphological analysis dictionary 210, and transmits to the sentence acquisition unit 105 a list of words obtained by the break-down and their parts of speech (word list).

[0036] In the text acquisition step S205, the text acquisition unit 105 selects a word of a preset part of speech from the received word list, acquires a text including the selected word from a preset range of the asset management DB3, and transmits the acquired text to the second analysis unit 106. Here, the preset part of speech is set to a part of speech including a meaning related to the content, such as a noun, verb, adjective, or adverb. The preset range of the asset management DB3 for acquiring a text is specified, for example, as a paragraph (or a range called a clause, section, or the like) including an image, a range located before and after an image, or a range including a text with a link to an image. If the area identification unit 102 cannot identify a character area, or if the character string extraction unit 103 cannot extract a character string from a character area, or if the text acquisition unit 105 cannot acquire a text from a preset range of the asset management DB3, all of the text in the preset range of the asset management DB3 is transmitted to the second analysis unit 106.

[0037] In the second morphological analysis step S206, the second analysis unit 106 breaks down the received sentence into morphemes based on the morphological analysis dictionary 210, and transmits a list of the words obtained by the break-down and their parts of speech (word list) to the word selection unit 107.

[0038] In word selection step S207, the word selection unit 107 selects words of predetermined parts of speech with a high occurrence rate (words included in the list the most frequently) from the received word list, and outputs the selected words to the classification tag generation unit 108. For example, the top N words with the highest occurrence rates are selected.

[0039] In classification tag generation step S208, the classification tag generation unit 108 receives as input the image information received from the area identification unit 102 and the words received from the word selection unit 107, and generates classification tags using a model previously trained by machine learning (deep learning). The generated classification tags are output to the information update unit 109.

[0040] In traceability information update step S209, the information update unit 109 updates the traceability information stored in the asset management DB3 by storing the received classification tag as a classification tag for the image data of the document acquired in image acquisition step S201.

[0041] By executing the above process for all image data of documents registered in the asset management DB 3, the information processing system 4 becomes able to search for documents having images with the same classification tag as related documents.

[0042] FIG. 5 is a diagram for explaining a process of extracting text from a document in the asset management DB 3 according to an embodiment of the present invention. The document 300 registered in the asset management DB 3 shown in FIG. 5 has a text portion 301 and an image portion 302. In the image acquisition step S201 of FIG. 4, the image acquisition unit 101 acquires the image portion 302 as image data, and in the region selection step S202, the region identification unit 102 selects the character region 304 from the image portion 302. Furthermore, in the character string extraction step S203, the character string extraction unit 103 extracts "Figure X ###◎◎◎+++" as a character string from the character region 304, and in the first morphological analysis step S204, the first analysis unit 104 performs morphological analysis on the extracted character string to classify it into words and parts of speech, and generate a word list 305. Meanwhile, in the text acquisition step S205, the text acquisition unit 105 acquires a text 303 from the text portion 301 as a text containing words of parts of speech preset in the word list 305.

[0043] FIG. 6 is a diagram for explaining an example of input / output data of the classification tag generating unit 108 according to the embodiment of the present invention. The classification tag generating unit 108 is pre-trained by machine learning (deep learning) so as to output classification tags corresponding to character strings (words) 311 and image data 312, which are input data 310, as output data 313. The character strings (words) 311 can be obtained by vectorizing n words (maximum N words) selected by the word selecting unit 107 using a known technique such as Word2Vec. The image data 312 is an array in pixel units (for example, m pixels in the main scanning line and k pixels in the sub-scanning line), and preprocessing such as noise removal is performed as necessary. The luminance of one pixel value may be, for example, a monochrome image expressed in 8 bits (values ​​from 0 to 255), or may be a color pixel using the three primary colors of RGB (Red, Green, Blue). The size of the input image is, for example, a maximum of 1024x1024 (m=1024, k=1024), and if the input image is smaller than the maximum size, for example, it is moved to the upper left and the remaining pixel values ​​are set to 0. The output data 313 is a classification tag that is the result of classifying the image, and is, for example, a Onehot representation in which only one bit is 1 and the rest are 0. In this case, the length of the output bits is the number of classification types, and is, for example, 10 types (p=10).

[0044] FIG. 7 is a diagram for explaining an example of a network model of the classification tag generating unit according to an embodiment of the present invention. Details of the words 321 and images 322, which are input data of the neural network 323, and the classification tag 327, which is output data, are as described in FIG. 6. The image 322 is in a form in which three types of pixel values ​​of color (RGB) are input. The neural network 323 has a word CNN (Convolutional Neural Network) 324, an image CNN 325, and an output layer 326. In the final layers of the word CNN 324 and the image CNN 325, a one-dimensional vector value having α elements is output, and an input value having an α×2 array obtained by combining the two vectors is input to the output layer 326, and the output from the output layer 326 becomes a value indicating the classification tag 327. Here, a value larger than the number of elements of the output value (=types of classification tags) is used for the value of α. Since the input data is composed of multiple elements (words and image data), it is effective to use, for example, a multimodal deep learning technique in training the neural network 323.

[0045] FIG. 8 is a diagram showing an example of traceability information according to an embodiment of the present invention. The information update unit 109 updates the traceability information 330 stored in the asset management DB 3 based on information received from the classification tag generation unit 108. In the images (image A, image B, image C, . . .) included in document A of the traceability information 330 and the images (image D, image E, image F, . . .) included in document B, the locations to which classification tags are assigned are 1 (True), and the other locations are 0 (False). For example, classification tags of classification C, classification E, and classification F, each of which has a value of 1 (True), are assigned to image A of document A. During document search, it is possible to present, for example, image E of document B, to which classification tag A is assigned, as an image of a document related to image C of document A, using the traceability information.

[0046] FIG. 9 is a diagram showing another example of traceability information according to an embodiment of the present invention. The difference from the traceability information 330 shown in FIG. 8 is that the information update unit 109 further stores the date on which the classification tag was updated. In the traceability information 340, the dates of 2020 / 12 / 1 and 2021 / 1 / 2 are stored as the update dates and times on which the classification tags of image A and image B of document A were updated. By recording the update dates and times of the classification tags in this way, for example, when a document or image is updated by another system or manually, it is possible to perform management such as generating and updating classification tags again using the updated document or image by comparing the update dates and times.

[0047] FIG. 10 is a diagram showing a method for specifying the range of documents to be acquired by the text acquisition unit 105 according to the embodiment of the present invention. In the asset management DB 3, titles 351 and 353 for document A 350 and document B 352, and information on paragraphs and images in each page are stored. For these documents, a document acquisition range 354, which is set in advance when the text acquisition unit 105 acquires text, is stored in the asset management DB 3. In the document acquisition range 354, a range for acquiring text in a document is set for a document having a specified document type (document type) and keyword. Also, a number is assigned to the set range in each row. Here, it is assumed that document A 350 is a specification and document B 352 is a report with regard to the document type. Information on the document type may be managed in a property or the like in the document, or may be managed in a separate database. Also, by specifying a character string contained in the title of a document in the keyword field, only documents containing that character string in the title are set as the target of the setting. When no keyword is specified, it is also possible to set a value indicating that no range is specified, for example, "none", so that documents are not limited by title. For example, since title 351 of document A 350 contains the character string "AAA", the text acquisition range of number 355 (paragraph A, paragraph B, ..., paragraph Q) is applied. Meanwhile, title 353 of document B 352 also contains the character string "AAA", but since it is a different document type (report), number 355 is not applied, and text acquisition range of number 357, which has no keyword specification, is applied. It is possible to specify N ranges within the document to be specified, but if after specifying a certain number of ranges there are no more ranges to specify, "none" can be set.

[0048] [Second embodiment] A second embodiment of the present invention will be described with reference to FIG. 11. FIG. 11 is a diagram showing an example of a process flow for generating classification tags for images in a document using a co-occurrence relationship list according to an embodiment of the present invention. Here, only differences from the process for generating classification tags described in FIG. 4 will be described. Co-occurrence means that another word appears frequently in the same sentence as a certain word, and a list of these words is called a co-occurrence relationship list. In addition, although it may be more appropriate to call the state after the character string is divided and generated by N-gram analysis a character string rather than a word, the state generated after the character string is divided is called a word, as in the case of the morphological analysis in FIG. 4.

[0049] In the first N-gram analysis step S224, the first analysis unit 104 divides the character string received from the character string extraction unit 203 into words of a preset number of characters, and transmits to the text acquisition unit 105 a list created based on the divided words and their occurrence frequencies.

[0050] In a sentence acquisition step S225, the sentence acquisition unit 105 acquires sentences including the received word from a preset range of the asset management DB3, and transmits the acquired sentences to the second analysis unit 106. Here, it is possible to use only words selected from the received words in descending order of appearance frequency so that the number of words is within a preset maximum number.

[0051] In the second N-gram analysis step S226, the second analysis unit 106 divides the received sentence into words of a predetermined number of characters, and creates a co-occurrence relationship list 221 by analyzing the co-occurrence relationships of those words, and transmits the co-occurrence relationship list 221 to the word selection unit 107.

[0052] In word selection step S227, the word selection unit 107 selects a predetermined number or less of words having co-occurrence relationships from the received co-occurrence relationship list 221 using the words in the list created by the first analysis unit 104, and transmits them to the classification tag generation unit 108.

[0053] 4, in the above-described process, N-gram analysis is performed in S224 and S226 instead of morphological analysis using the dictionary 210, making it unnecessary to prepare the dictionary 210. There is no difference in the data input to and output from the classification tag generation unit 108, but the data used when learning in advance the neural network, which is an internal process, is a combination of words and images generated in each process.

[0054] FIG. 12 is a diagram for explaining an example of a method for generating and using a co-occurrence relationship list according to an embodiment of the present invention. First, assuming that the character string extracted by the text acquisition unit 105 is a list 360, in the second N-gram analysis step S226 in FIG. 11, the second analysis unit 106 generates a list 361 by shifting, for example, three-character words from the list 360 one character at a time. Next, a list 362 is generated by dividing, for example, every three words while shifting the words in the list 361 one word at a time. Next, the rows are sorted in order of the first characters of the words in the middle (second) column of the list 362, and further, rows that appear in duplicate are integrated to generate a list 363 in which the occurrence frequency is written on the right. The list 363 is an example of the co-occurrence relationship list 221 shown in FIG. 11. Therefore, if the word in the list created by the first analysis unit 104 is, for example, "Iroha", the word selection unit 107 selects from the co-occurrence relationship list 221 the words "toiro", "Iroha", and "rohani" which are in the row in which the word in the middle (second) column is "Iroha". Here, if the co-occurrence relationship list 221 contains the row ""wa-ka-yo", "iroha", "tsu-ne-na", 2", the words in the row with the highest frequency of occurrence, "wa-ka-yo", "iroha", and "tsu-ne-na", are selected. Also, if there are multiple pieces of data with the same frequency of occurrence, the top row may be selected first within the upper limit of the number of selectable words.

[0055] [Third embodiment] A third embodiment of the present invention will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of a process flow for generating a classification tag for an image read from outside according to an embodiment of the present invention. Here, only differences from the process for generating a classification tag explained in Fig. 4 will be explained.

[0056] In an image acquisition step S231, the administrator accesses the management screen of the information processing device 2 by performing an operation from the operation reception unit 122 of the terminal device 5. The first communication unit 110 of the information processing device 2 transmits information related to the management screen to the second communication unit 123 of the terminal device 5. The display control unit 121 of the terminal device 5 displays the received screen information on the display of the terminal device 5. On the management screen, the administrator adds image data imported from an external device to a document registered in the managed asset DB3, and further causes the information processing device 2 to execute a process of adding a classification tag to the image data. The image acquisition unit 101 acquires image data scanned by the scanner function of the image forming device 6 or image data captured by a camera, and transmits the acquired image to the area identification unit 102.

[0057] 4 differs from the present embodiment in that image data is acquired from the managed asset DB 3 in image acquisition step S201. Also, in the present embodiment, the functions of the area identification unit 102 and the character string extraction unit 103 are the same as the processes shown in FIG. 4, but the processes may be executed in the image forming device 6 instead of the information processing device 2, and the output results may be transmitted to the information processing device 2. These processes enable the information processing system 4 to generate classification tags for image data that is not registered in the managed asset DB 3.

[0058] Although several forms for carrying out the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0059] For example, the example of the functional block configuration diagram in FIG. 3 is divided according to main functions to facilitate understanding of the processing by the information processing device 2 and the information processing system 4. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the information processing device 2 can be further divided into more processing units depending on the processing contents. Also, it can be divided so that one processing unit includes more processes.

[0060] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0061] Moreover, the described group of devices merely represents one of a plurality of computing environments for implementing the embodiments disclosed herein. In one embodiment, the information processing device 2 or the information processing system 4 includes a plurality of computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network, a shared memory, and the like, and perform the processes disclosed herein. Furthermore, the asset management DB 3 may store the database using a technology such as distributed storage or distributed database. [Explanation of symbols]

[0062] 1. Communication Network 2. Information processing equipment 3. Asset Management Database 4. Information Processing Systems 5 Terminal Equipment 6. Image forming equipment 101 Image acquisition unit 102 Area Identification Unit 103 String extraction part 104 1st Analysis Department 105 Text acquisition part 106 2nd Analysis Department 107 Word Selection Section 108 Classification tag generator 109 Information update department 110 First Communications Department 111 Search Department 121 Display control unit 122 Operation reception section 123 2nd Communications Department [Prior art documents] [Patent documents]

[0063] [Patent Document 1] JP 2008-311828 A

Claims

1. An information processing device for managing electronic documents stored in a database, an area identification unit that identifies areas of character information and image information included in image data in the document; a character string extraction unit that extracts a character string from a character information area of ​​the image data; a text acquisition unit that acquires text from a preset range of documents using information included in the character string; a word selection unit for selecting at least one input word from the sentence according to a frequency of occurrence of the input word; a classification tag generation unit that outputs a classification tag that classifies the image data by inputting the input word and the image information into a model that has been trained in advance by machine learning; an information update unit that stores the classification tag in the database; a search unit that uses the classification tags to search for documents in the database that contain related image data; 13. An information processing device comprising:

2. A first analysis unit that generates a word list including words included in the character string and parts of speech of the words, 2 . The information processing apparatus according to claim 1 , wherein the sentence acquisition unit acquires sentences including words of a predetermined part of speech included in the word list from a predetermined range of documents in the database.

3. a second analysis unit that generates a co-occurrence relationship list by analyzing co-occurrence relationships of the sentences acquired by the sentence acquisition unit; 3. The information processing apparatus according to claim 2, wherein the word selection unit selects a predetermined number or less of input words from the co-occurrence relationship list by using words of a predetermined part of speech included in the word list.

4. 4. The information processing apparatus according to claim 1, wherein the information update unit stores a date and time when the classification tag is stored in the database together with the classification tag.

5. 5. The information processing apparatus according to claim 4, wherein a range of documents to be acquired by said document acquisition section is preset for each type of document.

6. 6. The information processing apparatus according to claim 5, wherein image data acquired from an external device is registered in the database and used as image data within the document.

7. An information processing system having an information processing device for managing electronic documents stored in a database, The information processing device includes: an area identification unit that identifies areas of character information and image information included in image data in the document; a character string extraction unit that extracts a character string from a character information area of ​​the image data; a text acquisition unit that acquires a text including the character string from a preset range of documents; a word selection unit for selecting at least one input word from the sentence according to its frequency of occurrence; a classification tag generation unit that outputs a classification tag that classifies the image data by inputting the input word and an area of ​​image information of the image data into a model that has been trained in advance by machine learning; an information update unit that stores the classification tag in the database; a search unit that searches for documents in the database that contain related image data using the classification tag in response to a request from a terminal device; a first communication unit that transmits screen information related to the search result to the terminal device; An information processing system comprising:

8. An information processing method performed by an information processing device that manages electronic documents stored in a database, comprising: identifying areas of text information and image information contained in image data within the document; extracting a character string from a character information area of ​​the image data; acquiring a sentence including the character string from a preset range of documents; selecting at least one input word from said sentence according to its frequency of occurrence; a step of inputting the input word and an area of ​​image information of the image data into a model previously trained by machine learning, and outputting a classification tag that classifies the image data; storing said classification tags in said database; using the classification tags to search for documents in the database that contain relevant image data; An information processing method comprising the steps of:

9. An information processing device for managing electronic documents stored in a database, identifying areas of text information and image information contained in image data within the document; extracting a character string from a character information area of ​​the image data; acquiring a sentence including the character string from a preset range of documents; selecting at least one input word from said sentence according to its frequency of occurrence; a step of inputting the input word and an area of ​​image information of the image data into a model previously trained by machine learning, and outputting a classification tag that classifies the image data; storing said classification tags in said database; using the classification tags to search for documents in the database that contain relevant image data; A program for executing the above.

Images