Information processing device

Abstract

Classify document data appropriately. [Solution] The information processing device includes an acquisition means for acquiring document data, a first extraction means for extracting a first keyword from the entire document data, a second extraction means for extracting a second keyword from a text box contained in the document data, a determination means for determining the difference set between the first keyword and the second keyword and determining non-repeating keywords based on the difference set, and a classification means for classifying the document data by assigning classification tags. The classification means assigns classification tags to the document data after excluding classification tags related to the non-repeating keywords from the candidates for classification tags to be assigned to the document data.

Description

Technical Field

[0001] This disclosure relates to the technical field of information processing apparatuses.

Background Art

[0002] As an apparatus of this kind, for example, a system has been proposed in which a large language model (LLM) generates query data based on a document, and a pair of the document and the query data is used for learning a search model for a chatbot (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When classifying document data using a model such as a machine-learned LLM, there is a risk of making an incorrect classification. However, it is difficult to automatically detect that an incorrect classification has been made.

[0005] This disclosure has been made in view of the above problems, and an object thereof is to provide an information processing apparatus capable of appropriately classifying document data.

Means for Solving the Problems

[0006] An information processing device according to one aspect of this disclosure includes: an acquisition means for acquiring document data; a first extraction means for extracting a first keyword from the entire document data; a second extraction means for extracting a second keyword from a text box contained in the document data; a determination means for determining the difference set between the first keyword and the second keyword and determining non-repeating keywords based on the difference set; and a classification means for classifying the document data by assigning classification tags, wherein the classification means assigns the classification tags after excluding classification tags related to the non-repeating keywords from the candidates for classification tags to be assigned to the document data. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing the hardware configuration of an information processing device according to an embodiment. [Figure 2] This is a block diagram showing the functional configuration of an information processing apparatus according to an embodiment. [Figure 3] This is a flowchart showing the operation flow of the information processing device according to the embodiment. [Figure 4] This is a schematic diagram illustrating an example of the keyword extraction process. [Modes for carrying out the invention]

[0008] The following describes an embodiment of the information processing device with reference to the drawings.

[0009] (Hardware configuration) First, the hardware configuration of the information processing device according to the embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the hardware configuration of the information processing device according to the embodiment.

[0010] In Figure 1, the information processing device 10 according to this embodiment comprises an arithmetic unit 110, a storage device 120, a communication device 130, an input device 140, and an output device 150. The arithmetic unit 110, the storage device 120, the communication device 130, the input device 140, and the output device 150 are connected to each other via a data bus.

[0011] The arithmetic unit 110 is configured to perform various arithmetic operations in the information processing device 10. The arithmetic unit 110 may have a processor. The arithmetic unit 110 may have a single processor or may have multiple processors. In other words, the arithmetic unit 110 may have one or more processors. The processor may be a multi-core processor. If the arithmetic unit 110 has a single processor that is a multi-core processor, then logically, the arithmetic unit 110 can be said to have multiple processors.

[0012] The processor in the arithmetic unit 110 may be at least one of the following: CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and TPU (Tensor Processing Unit).

[0013] The storage device 120 may be at least one of the following: RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, magneto-optical disk drive, SSD (Solid State Drive), and optical disk array. In other words, the storage device 120 may be implemented by a single device or by multiple devices.

[0014] The storage device 120 is capable of storing desired data. The storage device 120 may store the computer program CP that the arithmetic unit 110 will execute. The storage device 120 may temporarily store data that the arithmetic unit 110 will use temporarily when the arithmetic unit 110 is executing the computer program CP.

[0015] The computer program CP may be recorded on a non-temporary recording medium that is readable by a computer. In this case, the computer program CP may be stored in the storage device 120 by reading the recording medium using a recording medium reading device (not shown) provided by the information processing device 10. At least one of the following may be used as the recording medium: an optical disc, a magnetic medium, a magneto-optical disc, a semiconductor memory, and any other medium capable of storing a program. The computer program CP may also be obtained from an external device (not shown) of the information processing device 10 via a communication device 130. In other words, the computer program CP may be downloaded from an external device to the storage device 120 of the information processing device 10.

[0016] The arithmetic unit 110 (for example, a processor) may execute the processing that the information processing device 10 should perform together with the memory device 120 in which the computer program CP is stored (in other words, together with the memory device 120 and the computer program CP stored in the memory device 120). For example, by the arithmetic unit 110 executing the computer program CP, a logical functional block for executing the processing that the information processing device 10 should perform may be realized within the arithmetic unit 110 (for example, within the processor).

[0017] The communication device 130 is configured to communicate with devices outside the information processing device 10. The communication device 130 may use wired communication or wireless communication.

[0018] The input device 140 is a device capable of receiving input of information to the information processing device 10 from the outside. The input device 140 may include an operating device (for example, a keyboard, a mouse, a touch panel, etc.) that can be operated by the user of the information processing device 10. The input device 140 may include a recording medium reader that can read information recorded on a detachable recording medium of the information processing device 10, such as a USB (Universal Serial Bus) memory. When information is input to the information processing device 10 via the communication device 130 (in other words, when the information processing device 10 acquires information via the communication device 130), the communication device 130 may function as an input device.

[0019] The output device 150 is a device capable of outputting information to the outside of the information processing device 10. The output device 150 may have a display device capable of outputting visual information such as characters and images as the above information. The output device 150 may also have a speaker capable of outputting auditory information such as sound as the above information. The output device 150 may be configured to output the above information (for example, control information of other devices, etc.) to other devices. The output device 150 may be able to output information to a detachable recording medium of the information processing device 10, such as a USB memory. When the information processing device 10 outputs information via the communication device 130, the communication device 130 may function as an output device.

[0020] <Functional Configuration> Next, the functional configuration of the information processing device 10 according to the embodiment will be described while referring to FIG. 2. FIG. 2 is a block diagram showing the functional configuration of the information processing device according to the embodiment.

[0021] In FIG. 2, the information processing apparatus 10 is configured as a device that classifies document data and stores it in the document data DB 300. The information processing apparatus 10 includes, as components for realizing its functions, a document data acquisition unit 210, a first extraction unit 220, a second extraction unit 230, a third extraction unit 240, a determination unit 250, an alert output unit 260, and a classification unit 270. Each of the document data acquisition unit 210, the first extraction unit 220, the second extraction unit 230, the third extraction unit 240, the determination unit 250, the alert output unit 260, and the classification unit 270 may be a processing block realized by the arithmetic unit 110 described above.

[0022] The document data acquisition unit 210 is configured to be able to acquire document data. The document data is data including text, and examples thereof include technical documents, specifications, etc. The document data may be data including images in addition to text. The document data acquisition unit 210 may acquire, for example, document data input by a user. Alternatively, the document data acquisition unit 210 may acquire document data stored in advance from a storage or the like.

[0023] The first extraction unit 220 is configured to be able to extract a first keyword from the entire document data. The first keyword may be a keyword with a high degree of importance when viewing the entire document data. The first keyword may be, for example, a keyword that frequently appears in the entire document data. The first extraction unit 220 may extract a plurality of first keywords from one piece of document data.

[0024] The second extraction unit 230 is configured to be able to extract a second keyword from a text box included in the document data. The second keyword may be a keyword with a high degree of importance in the text block. The second keyword may be, for example, a keyword used in the heading of the text block. Here, the text block is a group of texts (a collection of texts) in the document data and is not limited to being separated in a block shape. For example, the text block may be one paragraph. The second extraction unit 230 may extract a plurality of second keywords from one text block.

[0025] The third extraction unit 240 is configured to extract a third keyword from an image contained in the document data. The third keyword may be extracted from text contained in the image. Alternatively, the third keyword may be extracted from an image summary. Alternatively, the third keyword may be extracted by recognizing an object in the image. The third extraction unit 240 may extract multiple third keywords from a single image.

[0026] The determination unit 250 determines which keywords are not duplicated among the first keyword extracted by the first extraction unit 220, the second keyword extracted by the second extraction unit 230, and the third keyword extracted by the third extraction unit 240. More specifically, the determination unit 250 calculates the set difference between the first keyword, the second keyword, and the third keyword, and determines which keywords are not duplicated based on that set difference.

[0027] The alert output unit 260 calculates the percentage (hereinafter referred to as the "non-overlap percentage") of all keywords extracted by the first extraction unit 220, the second extraction unit 230, and the third extraction unit 240 that are non-overlap keywords determined by the determination unit 250. The alert output unit 260 is configured to output alert information when the non-overlap percentage exceeds a predetermined threshold. The alert information may be information output to the user. The alert information may be displayed as text or an image on a display, for example. Alternatively, the alert information may be output as audio from a speaker, for example. The alert information may be information that notifies the user that classification errors are likely to occur in the document data being processed. The alert information may be information that encourages manual classification checks, for example.

[0028] The classification unit 270 is configured to perform classification by assigning classification tags to document data. For example, the classification unit 270 selects and assigns a classification tag appropriate to the document data from among several pre-prepared classification tags. The classification unit 270 may also perform classification by methods other than assigning classification tags. The classification unit 270 may classify document data using a machine learning model. This model may take document data as input and output classification tags to be assigned to the document data. The classification unit 270 may be configured to store the classified document data in a document data database.

[0029] Furthermore, the classification unit 270 specifically excludes classification tags related to non-duplicate keywords determined by the determination unit 250 from the candidates for classification tags to be assigned to the document data, and then assigns the classification tags. Therefore, the document data will not be assigned classification tags related to non-duplicate keywords.

[0030] The document data DB 300 is configured to store document data classified by the classification unit 270. The document data stored in the document data DB 300 may be made readable as needed. For example, the document data stored in the document data DB 300 may be configured to be searchable using classification tags as a condition.

[0031] Although Figure 2 shows an example where the document data DB 300 is located outside the information processing device 10, the information processing device 10 may also be configured to include the document data DB 300.

[0032] (Flow of operations) Next, the operation flow of the information processing device 10 according to the embodiment will be described with reference to Figures 3 and 4. Figure 3 is a flowchart showing the operation flow of the information processing device according to the embodiment. Figure 4 is a schematic diagram showing an example of keyword extraction.

[0033] As shown in Figure 3, when the operation of the information processing device 10 according to the embodiment is started, first the document data acquisition unit 210 acquires document data (step S101). Then, the first extraction unit 220 extracts the first keyword (step S102). The second extraction unit extracts the second keyword (step S103). The third extraction unit extracts the third keyword (step S104).

[0034] As shown in Figure 4, the first extraction unit 220 extracts a first keyword from the entire document data. The second extraction unit 230 extracts a second keyword from the text blocks contained in the document data. The third extraction unit 240 extracts a third keyword from the images contained in the document data. The order in which the first, second, and third keywords are extracted is not particularly limited. That is, the processes in steps S102, S103, and S104 may be executed sequentially or simultaneously in parallel.

[0035] Returning to Figure 3, once each keyword has been extracted, the determination unit 250 calculates the set difference between the first keyword, the second keyword, and the third keyword, and determines whether or not there are any non-repeating keywords (step S105). If there are no non-repeating keywords (step S105: NO), the document data is classified as is (step S109), and the series of operations ends.

[0036] On the other hand, if there are duplicate keywords (step S105: YES), the classification unit 270 excludes classification tags related to non-duplicate keywords from the candidates for classification tags to be assigned to the document data (step S106).

[0037] Furthermore, the alert output unit 260 determines whether the proportion of non-repeating keywords exceeds a predetermined threshold (step S107). If the proportion of non-repeating keywords exceeds the predetermined threshold (step S107: YES), the alert output unit 260 outputs alert information (step S108). On the other hand, if the proportion of non-repeating keywords does not exceed the predetermined threshold (step S107: NO), step S108 is omitted. In other words, the alert output unit 260 does not output alert information.

[0038] Subsequently, the classification unit 270 classifies the document data (step S109). In step S106, classification tags related to non-duplicate keywords were excluded from the candidates for classification tags to be assigned to the document data. Therefore, the classification unit 270 selects the classification tags to be assigned to the document data from among the classification tags that were not excluded.

[0039] (Technical effects) Next, the technical effects obtained by the information processing device 10 according to this embodiment will be described.

[0040] As explained in Figures 1 to 4, in the information processing device 10 according to this embodiment, non-repeating keywords are determined from keywords extracted from the entire document data, text blocks, and images. Then, classification tags related to non-repeating keywords are excluded from the candidates, and the document data is classified. In this way, it is possible to suppress the assignment of classification tags related to keywords that are not relevant to the document data. In other words, it is possible to suppress incorrect classification. In particular, in this embodiment, since keywords extracted from the entire document data, text blocks, and images are used, the importance of keywords can be judged from various perspectives, and inappropriate classification tags can be excluded in advance.

[0041] The embodiments of the invention derived from the above-described embodiments are described below.

[0042] An information processing device according to one aspect of this disclosure includes: an acquisition means for acquiring document data; a first extraction means for extracting a first keyword from the entire document data; a second extraction means for extracting a second keyword from text boxes included in the document data; a determination means for determining the difference set between the first keyword and the second keyword and determining non-repeating keywords based on the difference set; and a classification means for classifying the document data by assigning classification tags. The classification means assigns the classification tags after excluding classification tags related to the non-repeating keywords from the candidates for classification tags to be assigned to the document data. In the above embodiment, the "document data acquisition unit 210" corresponds to an example of the "acquisition means," the "first extraction unit 220" corresponds to an example of the "first extraction means," the "second extraction unit 230" corresponds to an example of the "second extraction means," the "determination unit 250" corresponds to an example of the "determination means," and the "classification unit 270" corresponds to an example of the "classification means."

[0043] In the information processing apparatus according to the above embodiment, a third extraction means is further provided for extracting a third keyword from an image contained in the document data, and the determination means may calculate the difference set of the first keyword, the second keyword, and the third keyword, and determine non-repeating keywords based on the difference set. In this way, it becomes possible to determine non-repeating keywords while also considering the images contained in the document data. In the above embodiment, the "third extraction unit 240" corresponds to an example of the "third extraction means".

[0044] The information processing device according to the above embodiment may further include an alert means that outputs alert information when the ratio of non-overlapping keywords to all extracted keywords exceeds a predetermined threshold. In this way, it is possible to notify the user that there is a high proportion of non-overlapping keywords. In this case, for example, manual checking may be performed. In the above embodiment, the "alert output unit 260" corresponds to an example of the "alert means".

[0045] This disclosure is not limited to the embodiments described above and can be modified as appropriate without contradicting the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole. Information processing devices with such modifications are also included within the technical scope of the present invention. [Explanation of symbols]

[0046] 10...Information processing unit, 110...Calculation unit, 120...Storage device, 130...Communication device, 140...Input device, 150...Output device, 210...Document data acquisition unit, 220...First extraction unit, 230...Second extraction unit, 240...Third extraction unit, 250...Determination unit, 260...Alert output unit, 270...Classification unit, 300...Document data DB

Claims

1. means of acquiring document data, A first extraction means for extracting a first keyword from the entire document data, A second extraction means for extracting a second keyword from a text box contained in the document data, A determination means that calculates the set difference between the first keyword and the second keyword, and determines keywords that do not overlap based on the set difference, A classification means that classifies the document data by assigning classification tags, Equipped with, The classification means excludes classification tags related to the non-duplicate keywords from the candidates for classification tags to be assigned to the document data, and then assigns the classification tags. Information processing device.

2. The system further comprises a third extraction means for extracting a third keyword from an image contained in the document data, The determination means calculates the set difference between the first keyword, the second keyword, and the third keyword, and determines the keywords that do not overlap based on the set difference. The information processing apparatus according to claim 1.

3. The system further includes an alert means that outputs alert information when the ratio of non-overlapping keywords to all extracted keywords exceeds a predetermined threshold. The information processing apparatus according to claim 1 or 2.

Images