Text classification apparatus, text classification system, text classification method, and text classification program

Abstract

To automatically generate a label corresponding to each cluster obtained by clustering each sentence of a model document, and to automatically create sentence classification information which is teacher data.SOLUTION: The cluster configuration unit 130 clusters a plurality of sentence units 511 obtained as a result of decomposing the model document 52 into a plurality of clusters. The label setting unit 140 acquires a label setting result in which a label is set to each cluster based on the clustering result. The model learning unit 150 acquires the text classification information 53 in which the text units included in each of the plurality of clusters are associated with the labels based on the label setting result. Here, the text classification information 53 is training data used for learning of the classification model 60 that classifies text units included in the classification target document.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a text classification device, a text classification system, a text classification method, and a text classification program. [Background technology]

[0002] As the accuracy of text classification technology using machine learning models improves, it is now being used in many applications. Technology using text classification technology using machine learning models has also been disclosed for supporting specification review. To improve the classification accuracy of machine learning models, it is necessary to prepare a sufficient amount of training data. Because manually labeling training data is a heavy burden, there is a need for technology to automate the labeling of training data.

[0003] Patent Document 1 discloses a technology for efficiently classifying information contained in semi-structured documents written in natural language, such as RFPs (Request for Proposal). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-113035 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, text classification information is used in which the requirement specification text for learning, converted into a word vector, is associated with the classification result. Machine learning is performed using this text classification information as training data to generate a classification model for classifying the requirement specification text. Therefore, the technology in Patent Document 1 has a problem in that the classification model cannot be trained unless the text classification information, which is a pair of a sentence and its classification label, is manually prepared.

[0006] The present disclosure aims to automatically create sentence classification information, which is training data, by clustering each sentence in a model document and automatically generating labels corresponding to each cluster. [Means for solving the problem]

[0007] The text classification device according to the present disclosure includes: a clustering unit that clusters a plurality of sentence units obtained as a result of decomposing a model document that is a model of a document to be classified into predetermined units into a plurality of clusters and obtains a clustering result; a label setting unit that obtains a label setting result in which a label is set for each of the plurality of clusters based on the clustering result; a model learning unit that acquires sentence classification information in which sentence units included in each of the plurality of clusters are associated with labels based on the label setting result, the sentence classification information being training data used for learning a classification model that classifies sentence units included in the document to be classified; Equipped with. [Effects of the Invention]

[0008] The text classification device according to the present disclosure clusters multiple text units included in a model document and automatically assigns a label to each cluster, thereby achieving the effect of automatically generating text classification information, which is training data for a classification model. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a sentence classification device according to a first embodiment. [Figure 2] FIG. 1 is a schematic diagram showing an outline of the functions of a sentence classification device according to a first embodiment. [Figure 3] 3 is a flowchart showing an example of classification processing in the sentence classification device according to the first embodiment. [Figure 4]FIG. 2 is a diagram showing an example of a document to be classified according to the first embodiment. [Figure 5] 4 is a flowchart showing an example of a learning process in the sentence classification device according to the first embodiment. [Figure 6] FIG. 1 is a diagram showing an example of the overall configuration of a text classification system according to a first embodiment. [Figure 7] 4 is a flowchart showing an example of a label setting process in the sentence classification device according to the first embodiment. [Figure 8] FIG. 10 is a diagram showing an example of the configuration of a sentence classification device according to a modified example of the first embodiment. [Figure 9] FIG. 10 is a diagram showing an example of the configuration of a sentence classification device according to a second embodiment. [Figure 10] FIG. 10 is a schematic diagram showing an outline of the functions of a sentence classification device according to a second embodiment. [Figure 11] FIG. 10 is a flowchart showing an example of a learning process in the sentence classification device according to the second embodiment. [Figure 12] FIG. 10 is a diagram showing an example of the configuration of a sentence classification device according to a third embodiment. [Figure 13] FIG. 11 is a schematic diagram showing an outline of the functions of a sentence classification device according to a third embodiment. [Figure 14] FIG. 11 is a diagram showing an example of the overall configuration of a text classification system according to a third embodiment. [Figure 15] FIG. 11 is a flowchart showing an example of a learning process in the sentence classification device according to the third embodiment. [Figure 16] FIG. 10 is a diagram showing an example of the configuration of a sentence classification device according to a fourth embodiment. [Figure 17] FIG. 10 is a schematic diagram showing an outline of the functions of a sentence classification device according to a fourth embodiment. [Figure 18] FIG. 11 is a flowchart showing an example of a learning process in the sentence classification device according to the fourth embodiment. [Figure 19] FIG. 13 is a diagram showing an example of the configuration of a sentence classification device according to a fifth embodiment. [Figure 20] FIG. 12 is a schematic diagram showing an outline of the functions of a sentence classification device according to a fifth embodiment. [Figure 21] FIG. 13 is a flowchart showing an example of a learning process in the sentence classification device according to the fifth embodiment. [Figure 22]FIG. 13 is a flowchart showing an example of a cluster configuration process in the text classification device according to the fifth embodiment. [Figure 23] FIG. 1 is a diagram showing an example of the configuration of a text classification system to which the text classification devices according to the first to fifth embodiments are applied. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present embodiment will be described below with reference to the drawings. In each drawing, the same or corresponding parts are assigned the same reference numerals. In the description of the embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate. The arrows in the drawings mainly indicate the flow of data or the flow of processing.

[0011] Embodiment 1 ***Configuration Description*** FIG. 1 is a diagram showing an example of the configuration of a sentence classification device 100 according to this embodiment. The sentence classification device 100 is a computer. The sentence classification device 100 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls the other hardware.

[0012] The text classification device 100 includes, as functional elements, an acquisition unit 110, a decomposition unit 120, a cluster construction unit 130, a label setting unit 140, a model learning unit 150, a classification unit 160, a result output unit 170, and a memory unit 105. The memory unit 105 stores a classification model 60, a document to be classified 51, a classification result 602, a model document 52, a document to be classified 51, a model document 52, and text classification information 53. The acquisition unit 110, decomposition unit 120, cluster construction unit 130, label setting unit 140, model learning unit 150, classification unit 160, and result output unit 170 may be referred to as each unit of the sentence classification device 100.

[0013] The functions of each unit of the sentence classification device 100 are realized by software. The storage unit 105 is provided in the memory 921. The storage unit 105 may be provided in the auxiliary storage device 922, or may be provided separately in the memory 921 and the auxiliary storage device 922.

[0014] The processor 910 is a device that executes a text classification program. The text classification program is a program that realizes the functions of each unit of the text classification device 100. The processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, a DSP, and a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0015] The memory 921 is a storage device that temporarily stores data. Specific examples of the memory 921 are SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The auxiliary storage device 922 is a storage device that stores data. A specific example of the auxiliary storage device 922 is a HDD. The auxiliary storage device 922 may also be a portable storage medium such as an SD (registered trademark) memory card, CF, NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. Note that HDD is an abbreviation for Hard Disk Drive. SD (registered trademark) is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash (registered trademark). DVD is an abbreviation for Digital Versatile Disk.

[0016] The input interface 930 is a port connected to an input device such as a mouse, keyboard, or touch panel. Specifically, the input interface 930 is a USB terminal. The input interface 930 may also be a port connected to a LAN. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network.

[0017] The output interface 940 is a port to which a cable of an output device such as a display is connected. Specifically, the output interface 940 is a USB terminal or an HDMI (registered trademark) terminal. Specifically, the display is an LCD. The output interface 940 is also called a display interface. HDMI (registered trademark) is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display.

[0018] The communication device 950 has a receiver and a transmitter. The communication device 950 is connected to a communication network such as a LAN, the Internet, a telephone line, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. NIC is an abbreviation for Network Interface Card.

[0019] The sentence classification program is executed in the sentence classification device 100. The sentence classification program is read into the processor 910 and executed by the processor 910. The memory 921 stores not only the sentence classification program but also an OS. OS is an abbreviation for Operating System. The processor 910 executes the sentence classification program while running the OS. The sentence classification program and the OS may be stored in an auxiliary storage device 922. The sentence classification program and the OS stored in the auxiliary storage device 922 are loaded into the memory 921 and executed by the processor 910. Note that part or all of the sentence classification program may be incorporated into the OS.

[0020] The sentence classification device 100 may include multiple processors that replace the processor 910. These multiple processors share the task of executing the sentence classification program. Each processor is a device that executes the sentence classification program, just like the processor 910.

[0021] Data, information, signal values ​​and variable values ​​used, processed or output by the sentence classification program are stored in memory 921, secondary storage device 922, or registers or cache memory within processor 910.

[0022] The "part" of each part of the sentence classification device 100 may be read as a "circuit," "process," "procedure," "process," or "circuitry." The sentence classification program causes a computer to execute an acquisition process, a decomposition process, a cluster configuration process, a label setting process, a model learning process, a classification process, and a result output process. The acquisition process, the decomposition process, the cluster configuration process, the label setting process, the model learning process, the classification process, and the result output process may be referred to as each process of the sentence classification program. The "process" of each process of the sentence classification program may be read as a "program," "program product," "computer-readable storage medium storing a program," or "computer-readable recording medium recording a program." Furthermore, the sentence classification method is a method performed by the sentence classification device 100 executing the sentence classification program. The sentence classification program may be provided by being stored in a computer-readable recording medium, or may be provided as a program product.

[0023] *** Functionality Overview *** FIG. 2 is a schematic diagram showing an outline of the functions of the sentence classification device 100 according to this embodiment. The text classification device 100 clusters each text unit 521 of a model document 52 that serves as a model for a classification target document 51. Then, the text classification device 100 automatically creates text classification information 53 by using artificial intelligence to generate labels, which are cluster names corresponding to each cluster. The main functions are as follows:

[0024] The cluster construction unit 130 clusters a plurality of sentence units 521 contained in a model document 52, which serves as a model for the classification target document 51, into a plurality of clusters. The plurality of sentence units 521 are obtained as a result of breaking down the model document 52 into predetermined units. The cluster construction unit 130 obtains a clustering result 55.

[0025] The label setting unit 140 obtains a label setting result 82 in which a label is set for each of the multiple clusters based on the clustering result 55. The label setting unit 140 creates a prompt 81 requesting that a label be set for each of the multiple clusters, and inputs the prompt 81 to the artificial intelligence 200. Then, the label setting unit 140 obtains the label setting result 82 output from the artificial intelligence 200.

[0026] The model learning unit 150 acquires sentence classification information 53 in which the sentence units included in each of the multiple clusters are associated with labels based on the label setting result 82. The model learning unit 150 uses the sentence classification information 53 as training data to learn a classification model 60 that classifies sentence units 511 included in the document 51 to be classified.

[0027] ***Explanation of Operation*** Next, we will explain the text classification process, which is the operation of the text classification device 100 according to this embodiment. The operating procedure of the text classification device 100 corresponds to a text classification method. Furthermore, the program that realizes the text classification process, which is the operation of the text classification device 100, corresponds to a text classification program. The text classification process includes a classification process in which the text units 511 of the document to be classified 51 are classified using the classification model 60, and a learning process in which the classification model 60 is trained using the text units 521 of the model document 52. The classification process is a process of classifying sentence units 511 by setting labels to sentence units 511 of a document 51 to be classified using a trained classification model 60. The learning process is a process of automatically generating sentence classification information 53, which is training data, using sentence units 521 of a sample document 52, and learning a classification model 60.

[0028] <Classification process: Classifying sentence units in documents to be classified> FIG. 3 is a flow diagram showing an example of classification processing in the sentence classification device 100 according to this embodiment.

[0029] In step S110, the acquisition unit 110 acquires the classification target document 51. The classification target document 51 is electronic data of a document to be classified, such as a requirements specification or a system design document.

[0030] FIG. 4 is a diagram showing an example of a document 51 to be classified according to this embodiment. The classification target document 51 is a collection of sentences with a clause structure and is recorded as electronic data. The file format of the classification target document 51 may be a common document storage format such as DOCX, XLSX, PPTX, PDF, or TXT, or may be any other storage format.

[0031] In step S120, the decomposing unit 120 decomposes the sentences included in the classification target document 51 into predetermined units, and stores the plurality of sentence units 511 obtained as a result of the decomposition in the storage unit 105. A sentence unit may be a single sentence. Alternatively, a sentence unit may include multiple sentences. Alternatively, a sentence unit may be a section into which a sentence is broken down. A sentence unit may be a single section, or multiple sections. Alternatively, a sentence may be broken down into any other unit to generate a sentence unit.

[0032] In step S130, the classification unit 160 assigns a label to each sentence unit 511 of the document to be classified 51 using the trained classification model 60. The classification unit 160 outputs the label assignment results for all sentence units 511 as classification results 602. Specifically, the classification unit 160 inputs each sentence unit 511 of the document 51 to be classified into the trained classification model 60. The classification model 60 receives the sentence units as input, assigns labels to the sentence units, and outputs a classification result 602 that classifies the sentence units. The classification unit 160 acquires the classification result 602 output from the classification model 60.

[0033] In step S140, the result output unit 170 outputs the classification result 602. The result output unit 170 displays the classification result 602 on an output device such as a display via the output interface 940, for example.

[0034] <Learning process: A process of learning the classification model 60 using the text classification information 53, which is training data> FIG. 5 is a flowchart showing an example of the learning process in the sentence classification device 100 according to this embodiment.

[0035] In step S210, the acquisition unit 110 acquires the model document 52. The model document 52 is a document that serves as a model when creating a document to be classified. The model document 52 is electronic data of one or more documents that serve as models for the document to be classified, such as a past version or format of the document to be classified. The model document 52 may be in the same file format as the document to be classified 51, or it may be in a different file format. In step S220, the decomposition unit 120 decomposes the sentences included in the model document 52 into predetermined units, and stores the resulting sentence units 521 in the storage unit 105.

[0036] In step S230, the cluster constructing unit 130 clusters the plurality of sentence units 521 of the model document 52. The cluster constructing unit 130 stores the clustering result 55, which is the result of the clustering, in the storage unit 105.

[0037] As a clustering method, a hierarchical clustering method such as the group average method or Ward's method may be used, or a non-hierarchical clustering method such as the k-means method may be used. Furthermore, the distance between sentences when clustering may be measured using edit distance, or by vectorizing sentences and using Euclidean distance or cosine similarity between vectors. In the example of FIG. 2, a clustering result 55 in which sentence units 521 of a sample document 52 are classified into clusters 1, 2, and 3 is shown.

[0038] <<Label Setting Process: Step S240>> In step S240, the label setting unit 140 performs a label setting process to set a label to each cluster in the clustering result 55 obtained by the clustering.

[0039] FIG. 6 is a diagram showing an example of the overall configuration of a text classification system 500 according to this embodiment. In the label setting process, the label setting unit 140 sets a label to each cluster using an artificial intelligence 200, such as a generative AI. AI is an abbreviation for Artificial Intelligence. The label setting unit 140 obtains a label setting result 82 output from the artificial intelligence 200. The artificial intelligence 200 is also referred to as a generative AI service. The label setting unit 140 is also referred to as a generative AI connection unit.

[0040] The label setting unit 140 generates a prompt 81 using the clustering result 55 of the sentence units 521 created by the cluster configuration unit 130 as input, and sends it to the artificial intelligence 200. The label setting unit 140 receives the calculation result of the artificial intelligence 200 as a label setting result 82. The artificial intelligence 200 generates a label for each cluster based on the prompt 81 input from the label setting unit 140, thereby generating a label setting result 82. The artificial intelligence 200 outputs the label setting result 82 to the label setting unit 140. The label is also called a classification label or a cluster name.

[0041] Here, the overall configuration of a text classification system 500 including the artificial intelligence 200 and the text classification device 100 will be described with reference to FIG. The text classification system 500 according to this embodiment includes a text classification device 100 and an artificial intelligence 200. The text classification system 500 is a communication system in which the text classification device 100 and the artificial intelligence 200 communicate with each other. The text classification device 100 and the artificial intelligence 200 are capable of inputting and outputting data or transmitting and receiving data. In the text classification device 100 of FIG. 6, only the label setting unit 140, which is a functional element that transmits and receives data to and from the artificial intelligence 200, is shown, and other functional elements are not shown.

[0042] The artificial intelligence 200 includes an inference unit that performs inference using a trained model. The inference unit receives a prompt 81 from the label setting unit 140 as input, and outputs a label setting result 82 corresponding to the prompt 81 based on the trained model. The label setting result 82 is information in which a label is associated with each cluster. The artificial intelligence 200 may be a generative AI. The label setting unit 140 inputs a prompt 81 to the artificial intelligence 200, which is an external component, and instructs it to generate a label setting result 82, and causes it to output the label setting result 82. In this way, it is possible to use various external artificial intelligence services to output the label setting result 82.

[0043] In FIG. 6, the artificial intelligence 200 is an external component of the sentence classification device 100, but it may be an internal component. The generation AI service connected by the generation AI connection unit may be a service published on the Internet or a service published in an on-premise environment.In addition, instead of a generation AI service, a trained generation model, which is a machine learning model that takes text as input and outputs text information, may be called.

[0044] The artificial intelligence 200 may be configured using algorithms such as Transformer, BERT, and GPT. The artificial intelligence 200 may also be configured by combining multiple algorithms including these algorithms. BERT is an abbreviation for Bidirectional Encoder Representations from Transformers. GPT is an abbreviation for Generative Pre-Training.

[0045] FIG. 7 is a flowchart showing an example of the label setting process in the sentence classification device 100 according to this embodiment. In step S241, the label setting unit 140 obtains the clustering result 55. In step S242, the label setting unit 140 creates a prompt 81 for each cluster to be input to the artificial intelligence 200 based on the clustering result 55. In step S 243 , the label setting unit 140 sends the prompt 81 to the artificial intelligence 200 . In step S244, the label setting unit 140 receives, from the artificial intelligence 200, a label setting result 82 corresponding to the prompt 81. Here, the label setting result 82 is information in which a label is set for each cluster. That is, the label setting result 82 is a pair of a cluster name, which is a label, and each sentence unit of the cluster. In step S245, the label setting unit 140 stores the label setting result 82 in the storage unit 105 as the text classification information 53.

[0046] Through the above process, the sentence classification information 53, which is the training data for the classification model 60, is automatically generated using the artificial intelligence 200.

[0047] Artificial intelligence, or generative AI services, refers to services such as ChatGPT, which provides external parties with trained generative models that output relevant text information in response to prompts. One reason for using generative AI services is to improve the quality of classification labels. Furthermore, from the perspective of checking for omissions in documents, it is desirable for classification labels to summarize each sentence. However, when using a method that extracts the most frequently occurring words within a cluster and uses them as the cluster name, it is not possible to generate classification labels correctly if such words are not included in the sentence. By using a generation AI service like ChatGPT, it is possible to extract a general concept common to multiple sentences and use it as the cluster name, even if the sentence does not contain the words that would serve as classification labels.

[0048] Below is an example of a prompt for creating a label such as a cluster name. The prompt for creating this label takes as input the sentence units contained in the cluster. For example, if you use ChatGPT as the generation AI service, create and input the following prompt: Example prompt: "Summarize the following three sentence units into one word: 'Sentence unit 1,' 'Sentence unit 2,' and 'Sentence unit 3.'"

[0049] Returning to FIG. In step S250, the model learning unit 150 inputs the sentence classification information 53 as training data, thereby learning the classification model 60. The model learning unit 150 stores the learned classification model 60 in the storage unit 105. As described above, the classification model 60 is a machine learning model that receives sentence units 511 of the document 51 to be classified as input, and classifies the sentence units 511 by setting a label for each sentence unit 511.

[0050] ***Other Configurations*** In this embodiment, the functions of the respective units of the sentence classification device 100 are realized by software. As a variation, the functions of the respective units of the sentence classification device 100 may be realized by hardware. Specifically, the text classification device 100 includes an electronic circuit 909 instead of the processor 910 .

[0051] FIG. 8 is a diagram showing an example of the configuration of a sentence classification device 100 according to a modified example of this embodiment. The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of each part of the sentence classification device 100. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.

[0052] The functions of each unit of the sentence classification device 100 may be realized by one electronic circuit, or may be distributed and realized by multiple electronic circuits.

[0053] As another variation, some of the functions of each unit of the sentence classification device 100 may be implemented by electronic circuits, and the remaining functions may be implemented by software. Also, some or all of the functions of each unit of the sentence classification device 100 may be implemented by firmware.

[0054] Each of the processor and electronic circuit is also called a processing circuitry. That is, the function of each part of the sentence classification device 100 is realized by the processing circuitry.

[0055] ***Explanation of the effect of this embodiment*** As described above, the text classification device according to this embodiment automatically creates text classification information by clustering each sentence unit of a model document that serves as a model for a document to be classified and generating labels corresponding to each cluster using artificial intelligence. Therefore, the text classification device according to this embodiment makes it possible to train a classification model without manually creating text classification information, which serves as training data.

[0056] If the text classification information contains text from the document to be classified, the text from the document to be classified, i.e., the correct answer data, will be referenced as training data when training the classification model. This could result in overfitting. In the text classification device according to this embodiment, model documents, rather than documents to be classified, are input to the generation AI, so there is no risk of overfitting.

[0057] One of the purposes of the text classification device according to this embodiment is to support the detection of missing or incomplete information in documents to be classified by displaying the classification results of the text. To support the detection of missing or incomplete information, information is needed about what content, i.e., the classification label names, should be included in the documents to be classified. Therefore, it is not sufficient to simply cluster the text of the documents to be classified and provide the clustered text to a generation AI for classification. Therefore, it is necessary to extract the content that should be included in the documents to be classified from the model documents, as in the text classification device according to this embodiment.

[0058] Embodiment 2 In this embodiment, differences from and additions to the first embodiment will be mainly described. In this embodiment, components having the same functions as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.

[0059] ***Configuration Description*** FIG. 9 is a diagram showing an example of the configuration of the sentence classification device 100 according to this embodiment. The sentence classification device 100 according to this embodiment includes a sentence selection unit 180 in addition to the configuration of the first embodiment. The other configurations are the same as those of the first embodiment.

[0060] *** Functionality Overview *** FIG. 10 is a schematic diagram showing an outline of the functions of the sentence classification device 100 according to this embodiment. In this embodiment, a sentence classification device 100 will be described that, when creating a prompt 81 including clustered sentence units, creates the prompt 81 by limiting it to some sentence units. FIG. 10 shows an example in which only sentence units 1 and 3 out of sentence units 1, 2, 3, and 4 are transmitted to the label setting unit 140.

[0061] The sentence selection unit 180 selects a predetermined number of sentence units from the sentence units included in each of the plurality of clusters. The label setting unit 140 sets a label to each cluster, with each cluster being a sentence unit selected from each of the plurality of clusters.

[0062] ***Explanation of Operation*** In this embodiment, part of the learning process is different from that in the first embodiment.

[0063] <Learning process> FIG. 11 is a flowchart showing an example of the learning process in the sentence classification device 100 according to this embodiment. The processes from step S210 to step S230 and step S250 are the same as those in the first embodiment.

[0064] In this embodiment, step S231 is carried out after step S230. In step S231, the sentence selection unit 180 selects a predetermined number of sentence units for each cluster created in step S230. The predetermined number, which is the number of sentence units to select, may be a predetermined fixed value. Alternatively, the predetermined number may be determined by calculating the maximum number of sentence units that can be input based on information on the maximum number of characters that can be input to the API and information on the number of characters of each sentence included in the cluster. API is an abbreviation for Application Programming Interface. The selection of each sentence unit may be performed by randomly selecting sentence units within a cluster, or by further clustering the sentence units within a cluster into sub-clusters and randomly selecting one sentence unit from each sub-cluster in turn until a predetermined number has been selected.

[0065] Step S240a is basically the same as the process of step S240 in the first embodiment, except for the following points. In step S240a, the label setting unit 140 generates a prompt 81 for each cluster using the sentence units selected in step S231 as input, and sends it to the artificial intelligence 200. Then, the label setting unit 140 receives the calculation results of the artificial intelligence 200, and stores pairs of sentence units and cluster names for each cluster in the storage unit 105 as sentence classification information 53.

[0066] ***Explanation of the effect of this embodiment*** As described above, the sentence classification device according to this embodiment can limit the sentence units included in each cluster to a portion of the prompt, rather than including all of them in the prompt. Therefore, the sentence classification device according to this embodiment can generate labels such as cluster names even when there is an upper limit to the length of the prompt that can be input due to API restrictions of the generation AI service.

[0067] Embodiment 3 In this embodiment, differences from and additions to the first embodiment will be mainly described. In this embodiment, components having the same functions as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.

[0068] ***Configuration Description*** FIG. 12 is a diagram showing an example of the configuration of the sentence classification device 100 according to this embodiment. The sentence classification device 100 according to this embodiment includes a sentence generation unit 181 in addition to the configuration of the first embodiment. The storage unit 105 also includes a first threshold value 71. The other configurations are the same as those of the first embodiment.

[0069] *** Functionality Overview *** FIG. 13 is a schematic diagram showing an outline of the functions of the sentence classification device 100 according to this embodiment. FIG. 14 is a diagram showing an example of the overall configuration of a text classification system 500 according to this embodiment. In this embodiment, when the number of sentence units contained in each cluster generated by the cluster construction unit 130 is small, new sentence units are generated by the artificial intelligence 200 using the sentence units contained in the cluster as input, thereby increasing the number of data.

[0070] If there is a cluster among the plurality of clusters in which the number of sentence units is equal to or less than the first threshold, the sentence generation unit 181 generates a new sentence unit to be included in the cluster. 13, the first threshold is set to 20 sentences. Since cluster 1 has 10 sentence units, the sentence generation unit 181 generates new sentence units to be included in the cluster. At this time, as shown in FIG. 14, the sentence generation unit 181 may generate a new sentence unit 83 using an artificial intelligence 200.

[0071] ***Explanation of Operation*** In this embodiment, part of the learning process is different from that in the first embodiment.

[0072] <Learning process> FIG. 15 is a flowchart showing an example of the learning process in the sentence classification device 100 according to this embodiment. The processes from step S210 to step S240 and step S250 are the same as those in the first embodiment.

[0073] In this embodiment, step S248 is carried out after step S240. In step S248, for each cluster in which the number of sentence units is equal to or less than the first threshold, the sentence generation unit 181 generates a new sentence using the sentence units of each cluster as input. Then, the sentence generation unit 181 stores the generated pairs of sentence units and cluster names as sentence classification information 53 in the storage unit 105.

[0074] The sentence generation method of the sentence generation unit 181 may be to edit the original sentence units, such as by replacing words or changing the word order, to generate a new sentence that is different from the original sentence. Alternatively, a generation AI service such as ChatGPT may be used to generate a new sentence that is different from the original sentence. The first threshold 71 may be a predetermined fixed value. Alternatively, it may be set by calculating the average value or maximum value from the number of sentence units included in each cluster.

[0075] The main reason for using a generative AI service in a text generation method is to improve the quality of the training data. Also, from the perspective of improving the accuracy of the classification model, it is desirable for the content of the text in the text classification information to have a variety of patterns. On the other hand, when generating new text by editing the original text, the generated text will have content similar to the original text. This can lead to bias in the training data. By using a generative AI service, it is possible to generate text with content different from the original text and include it in the training data.

[0076] Below is an example of a prompt for generating sentences. The sentence unit included in the cluster or the cluster name is input to the prompt for generating sentences. As an example, if you use ChatGPT as the generation AI service, create and input the following prompt. Example prompt 1: "Generate 10 sentences that are similar to the following three sentences: 'Sentence 1,' 'Sentence 2,' and 'Sentence 3.'" Example prompt 2: "Generate 10 example sentences about 'Cluster name'." Example prompt 3: "Generate 10 example sentences related to 'Cluster name' using 'Sentence 1' as a reference."

[0077] ***Explanation of the effect of this embodiment*** As described above, the text classification device according to this embodiment can eliminate bias in the amount of data by generating new sentences from sentences included in clusters with a small amount of data. The text classification device according to this embodiment can improve the accuracy of the classification model by reducing bias in the amount of data during training of the classification model.

[0078] Embodiment 4 In this embodiment, differences from and additions to the third embodiment will be mainly described. In this embodiment, components having the same functions as those in the third embodiment are given the same reference numerals, and the description thereof will be omitted.

[0079] ***Configuration Description*** FIG. 16 is a diagram showing an example of the configuration of the sentence classification device 100 according to this embodiment. The text classification device 100 according to this embodiment includes a similarity verification unit 182 in addition to the configuration of the third embodiment. The storage unit 105 also includes a second threshold value 72. The other configurations are the same as those of the third embodiment.

[0080] *** Functionality Overview *** FIG. 17 is a schematic diagram showing an outline of the functions of the sentence classification device 100 according to this embodiment. In this embodiment, when a sentence unit generated by the sentence generation unit 181 is added to a cluster, the distance between the sentence unit and the sentence unit included in the cluster is calculated. Then, an embodiment will be described in which it is determined to add only sentence units whose distance is equal to or less than a certain threshold.

[0081] The similarity verification unit 182 calculates the distance between the sentence units included in the cluster and the new sentence unit, and includes sentence units whose distance is equal to or less than a second threshold in the cluster. Here, including in the cluster means adding a sentence unit whose distance is equal to or less than the second threshold and the label of the corresponding cluster as a pair to the sentence classification information 53. 17, the distances between sentence units 1, 2, and 3 generated by the sentence generation unit 181 and the sentence units of cluster 1 are 1.0, 0.2, and 0.1. Since the second threshold is 0.15, only sentence unit 3, whose distance is 0.15 or less, is included in cluster 1.

[0082] ***Explanation of Operation*** In this embodiment, part of the learning process is different from that in the first embodiment.

[0083] <Learning process> FIG. 18 is a flowchart showing an example of the learning process in the sentence classification device 100 according to this embodiment. The processes from step S210 to step S248 and step S250 are the same as those in the third embodiment.

[0084] In this embodiment, step S249 is carried out after step S248. In step S249, the similarity verification unit 182 calculates the distance between each sentence unit generated by the sentence generation unit 181 and each sentence unit included in the corresponding cluster. The similarity verification unit 182 compares the distance between each sentence unit and a second threshold value, and determines whether to add it to a cluster. The similarity verification unit 182 adds newly generated sentence units that satisfy the conditions to the corresponding cluster. Then, the similarity verification unit 182 stores pairs of sentence units and cluster names included in each cluster as sentence classification information 53 in the storage unit 105.

[0085] The distance between sentence units may be measured using edit distance, or the sentence units may be converted into vectors and the Euclidean distance or cosine similarity between the vectors may be used. The criteria for determining whether to add a sentence to a cluster may be the average distance between all sentences in the cluster, or the maximum or minimum distance. Alternatively, multiple types of distances may be calculated independently, and a decision may be made based on a combination of multiple criteria using each distance.

[0086] ***Explanation of the effect of this embodiment*** As described above, the text classification device according to this embodiment determines the distance between the generated text and the text in the original cluster, and verifies whether the distance is equal to or less than a certain threshold. Therefore, the text classification device according to this embodiment can prevent text that is significantly different from the text included in a cluster from being mixed into the cluster, and can prevent a decrease in the learning accuracy of the classification model.

[0087] Embodiment 5. In this embodiment, differences from and additions to the first embodiment will be mainly described. In this embodiment, components having the same functions as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.

[0088] ***Configuration Description*** FIG. 19 is a diagram showing an example of the configuration of the sentence classification device 100 according to this embodiment. The text classification device 100 according to this embodiment includes a pre-cluster construction unit 190 in addition to the configuration of the first embodiment. The other configurations are the same as those of the first embodiment.

[0089] *** Functionality Overview *** FIG. 20 is a schematic diagram showing an outline of the functions of the sentence classification device 100 according to this embodiment. The pre-cluster constructing unit 190 creates clusters from a small number of sample sentence classification information. The pre-cluster constructing unit 190 performs clustering of sentence units of the model document 52, including clusters made up of sample sentence units generated from the small number of sample sentence classification information. Hereinafter, the small number of sample sentence classification information will be referred to as sample sentence classification information. Furthermore, the clusters made up of sample sentence units generated from the sample sentence classification information will be referred to as sample clusters. The sample sentence classification information is information generated in advance as sentence classification information for sentence units with the same label. That is, the pre-cluster construction unit 190 clusters sentence units with the same label in advance as sample clusters. Then, the clustering unit 130 clusters a plurality of sentence units obtained from the model document 52 assuming that a sample cluster exists.

[0090] In FIG. 20, sample sentence classification information is given in which a sentence unit "XXX" which is text is paired with a classification "functional requirement." The pre-cluster construction unit 190 creates cluster 1 from this sample sentence classification information. At this time, the label of cluster 1 is "functional requirement." The cluster construction unit 130 clusters multiple sentence units 521 obtained from the model document 52, assuming that cluster 1 (labeled "functional requirement"), which is a sample cluster, exists.

[0091] ***Explanation of Operation*** In this embodiment, part of the learning process is different from that in the first embodiment.

[0092] <Learning process> FIG. 21 is a flowchart showing an example of the learning process in the sentence classification device 100 according to this embodiment. The process of step S210a is basically the same as the process of step S210 in the first embodiment, except for the following points. The acquisition unit 110 acquires the model document 52 and sample sentence classification information. The process in step S220 is the same as in the first embodiment.

[0093] In this embodiment, step S221 is carried out after step S220. In step S221, the pre-cluster forming unit 190 clusters the sentence units of the sample sentence classification information so that all sentence units with the same label are included in the same cluster.

[0094] The process of step S230a is basically the same as the process of step S230 in the first embodiment, except for the following points. In step S230a, the cluster constructing unit 130 receives the sample clusters created in step S221 as input and performs additional clustering on the sentence units decomposed in step S220.

[0095] FIG. 22 is a flowchart showing an example of the cluster configuration process in the sentence classification device 100 according to this embodiment. FIG. 22 shows the detailed process of step S230a in FIG. The difference from step S230 in FIG. 5 described in the first embodiment is that when clustering is performed on sentence units of the model document 52, some sample clusters are formed in advance.

[0096] In step S301, the cluster constructing unit 130 obtains sentence units of the model document 52 that have been decomposed into predetermined units, and sentence units of the sample clusters that have been clustered by the pre-cluster constructing unit 190. In step S302, the cluster constructing unit 130 sets the sample cluster, that is, the sentence-unit cluster of the sample sentence classification information, to the initial state of clustering. In step S303, the cluster constructing unit 130 clusters the sentence units 521 of the model document 52, assuming that there exists a cluster made up of sentences in the sample sentence classification information at the start of clustering.

[0097] Returning to FIG. 21, the explanation will be continued. In step S232, the cluster construction unit 130 determines whether each cluster contains a sentence unit of the sample sentence classification information. The cluster construction unit 130 sets the cluster name of each cluster that contains a sentence unit of the sample sentence classification information as a label corresponding to the sentence unit of the sample sentence classification information.

[0098] The process of step S240b is basically the same as the process of step S240 in the first embodiment, except for the following points. In step S240b, the label setting unit 140 generates labels, which are cluster names, from the sentence units of each cluster that do not include sentence units in the sample sentence classification information.The label setting unit 140 then stores pairs of clusters and labels, which are cluster names, as sentence classification information 53 in the storage unit 105. The label setting method is the same as in the first embodiment.

[0099] The process in step S250 is the same as in the first embodiment.

[0100] ***Explanation of the effect of this embodiment*** As described above, the sentence classification device according to this embodiment has sample sentence classification information, which is a pair of sentence units and their classification labels, in advance. Clustering is then performed based on the data of the sample sentence classification information, and the cluster names are set to the classification label names of the sentence classification information. In this way, by providing classification label information as input to the pre-cluster construction unit, a classification model capable of classifying sentences using predetermined classification labels can be created.

[0101] In the above first to fifth embodiments, each unit of the sentence classification device has been described as an independent functional block. However, the configuration of the sentence classification device does not have to be like the configurations of the above-mentioned embodiments. The functional blocks of the sentence classification device may have any configuration as long as they can realize the functions described in the above-mentioned embodiments.

[0102] FIG. 23 is a diagram showing an example of the configuration of a sentence classification system 500 to which the sentence classification device 100 according to this embodiment is applied. The text classification system 500 includes, for example, a text classification server 400, an input terminal 401, and an output terminal 402. The text classification server 400 includes the text classification device 100 . The input terminal 401 transmits, for example, a classification target document 51 and a model document 52 to the text classification device 100 . The text classification device 100 transmits the classification result 602 to the output terminal 402 . In this way, the text classification device may not be a single device, but may be a system made up of multiple devices.

[0103] Furthermore, it is possible to combine two or more parts of the first to fifth embodiments. Alternatively, it is possible to implement only one part of these embodiments. In addition, it is possible to implement any combination of these embodiments, either as a whole or in part. That is, in the first to fifth embodiments, the embodiments can be freely combined, or any of the components in each embodiment can be modified, or any of the components in each embodiment can be omitted.

[0104] The above-described embodiments are essentially preferred examples and are not intended to limit the scope of the present disclosure, the scope of application of the present disclosure, or the scope of use of the present disclosure. The above-described embodiments can be modified in various ways as needed. For example, the procedures described using flow charts or sequence diagrams may be modified as appropriate.

[0105] Various aspects of the present disclosure are summarized below as appendices.

[0106] (Appendix 1) a clustering unit that clusters a plurality of sentence units obtained as a result of decomposing a model document that is a model of a document to be classified into predetermined units into a plurality of clusters and obtains a clustering result; a label setting unit that acquires a label setting result in which a label is set for each of the plurality of clusters based on the clustering result; a model learning unit that acquires sentence classification information in which sentence units included in each of the plurality of clusters are associated with labels based on the label setting result, the sentence classification information being training data used for learning a classification model that classifies sentence units included in the document to be classified; A text classification device comprising: (Appendix 2) The label setting unit A text classification device as described in Appendix 1, which creates a prompt requesting that a label be set for each of the plurality of clusters and inputs it to an artificial intelligence, and obtains a label setting result from the artificial intelligence. (Appendix 3) The text classification device a sentence selection unit that selects a predetermined number of sentence units from sentence units included in each of the plurality of clusters, The label setting unit The text classification device of claim 2, wherein a text unit selected from each of the plurality of clusters is treated as each cluster, and a prompt is generated requesting the user to set a label for each cluster. (Appendix 4) The text classification device A text classification device according to any one of Supplementary Note 1 to Supplementary Note 3, further comprising a text generation unit that generates new text units to be included in a cluster if the number of text units among the plurality of clusters is equal to or less than a first threshold. (Appendix 5) The text classification device A text classification device as described in Appendix 4, comprising a similarity verification unit that calculates the distance between a text unit included in the cluster and the new text unit, and includes in the cluster a text unit whose distance is less than or equal to a second threshold. (Appendix 6) The text classification device A pre-clustering unit is provided which pre-clusters sentence units with the same label as sample clusters, The cluster configuration unit 6. A text classification device according to any one of appendices 1 to 5, which clusters a plurality of text units obtained from the model document assuming that the sample cluster exists. (Appendix 7) a clustering unit that clusters a plurality of sentence units obtained as a result of decomposing a model document that is a model of a document to be classified into predetermined units into a plurality of clusters and obtains a clustering result; a label setting unit that inputs the clustering result into an artificial intelligence and obtains from the artificial intelligence a label setting result in which a label is set for each of the plurality of clusters; a model learning unit that acquires sentence classification information in which sentence units included in each of the plurality of clusters are associated with labels based on the label setting result, the sentence classification information being training data used for learning a classification model that classifies sentence units included in the document to be classified; A text classification system comprising: (Appendix 8) The computer decomposes the example document serving as the example of the document to be classified into predetermined units, and then clusters the resulting plurality of sentence units into a plurality of clusters, thereby obtaining the clustering results; a computer obtains a labeling result in which a label is set for each of the plurality of clusters based on the clustering result; A text classification method in which a computer acquires text classification information, based on the label setting results, in which the text units contained in each of the multiple clusters are associated with labels, and which serves as training data used to train a classification model that classifies the text units contained in the document to be classified. (Appendix 9) a cluster configuration process for clustering a plurality of sentence units obtained as a result of decomposing a model document serving as a model for a document to be classified into predetermined units into a plurality of clusters to obtain a clustering result; a labeling process for obtaining a labeling result in which a label is set for each of the plurality of clusters based on the clustering result; a model learning process for acquiring sentence classification information in which sentence units included in each of the plurality of clusters are associated with labels based on the label setting result, the sentence classification information being training data used for learning a classification model that classifies sentence units included in the document to be classified; A text classification program that runs on a computer. [Explanation of symbols]

[0107] 51 Document to be classified, 511, 521 Sentence unit, 52 Example document, 53 Sentence classification information, 55 Clustering result, 60 Classification model, 71 First threshold, 72 Second threshold, 81 Prompt, 82 Label setting result, 83 New sentence unit, 100 Sentence classification device, 105 Memory unit, 110 Acquisition unit, 120 Decomposition unit, 130 Cluster construction unit, 140 Label setting unit, 150 Model learning unit, 160 Classification unit, 170 Result output unit, 180 Sentence selection unit, 181 Sentence generation unit, 182 Similarity verification unit, 190 Pre-cluster construction unit, 200 Artificial intelligence, 400 Sentence classification server, 401 Input terminal, 402 Output terminal, 500 Sentence classification system, 602 Classification result, 909 Electronic circuit, 910 Processor, 921 Memory, 922 Auxiliary storage device, 930 input interface, 940 output interface, 950 communication device.

Claims

1. a clustering unit that clusters a plurality of sentence units obtained as a result of decomposing a model document that is a model of a document to be classified into predetermined units into a plurality of clusters and obtains a clustering result; a label setting unit that acquires a label setting result in which a label is set for each of the plurality of clusters based on the clustering result; a model learning unit that acquires sentence classification information in which sentence units included in each of the plurality of clusters are associated with labels based on the label setting result, the sentence classification information being training data used for learning a classification model that classifies sentence units included in the document to be classified; A text classification device comprising:

2. The label setting unit The text classification device according to claim 1 , wherein a prompt requesting setting of a label for each of the plurality of clusters is created and input to an artificial intelligence, and a label setting result is obtained from the artificial intelligence.

3. The text classification device a sentence selection unit that selects a predetermined number of sentence units from sentence units included in each of the plurality of clusters, The label setting unit The text classification device according to claim 2 , wherein a sentence unit selected from each of the plurality of clusters is treated as a cluster, and a prompt is generated requesting setting of a label for each cluster.

4. The text classification device The text classification device according to claim 1 , further comprising: a text generation unit that generates a new text unit to be included in a cluster if the number of text units in the cluster is equal to or less than a first threshold value among the plurality of clusters.

5. The text classification device The text classification device according to claim 4 , further comprising a similarity verification unit that calculates a distance between a text unit included in the cluster and the new text unit, and includes in the cluster a text unit for which the distance is equal to or less than a second threshold.

6. The text classification device A pre-clustering unit is provided which pre-clusters sentence units with the same label as sample clusters, The cluster configuration unit 4. The text classification device according to claim 1, wherein a plurality of text units obtained from the model document are clustered assuming that the sample cluster exists.

7. a clustering unit that clusters a plurality of sentence units obtained as a result of decomposing a model document that is a model of a document to be classified into predetermined units into a plurality of clusters and obtains a clustering result; a label setting unit that inputs the clustering result into an artificial intelligence and obtains from the artificial intelligence a label setting result in which a label is set for each of the plurality of clusters; a model learning unit that acquires sentence classification information in which sentence units included in each of the plurality of clusters are associated with labels based on the label setting result, the sentence classification information being training data used for learning a classification model that classifies sentence units included in the document to be classified; A text classification system comprising:

8. The computer decomposes the example document serving as the example of the document to be classified into predetermined units, and then clusters the resulting plurality of sentence units into a plurality of clusters, thereby obtaining the clustering results; a computer obtains a labeling result in which a label is set for each of the plurality of clusters based on the clustering result; A text classification method in which a computer acquires text classification information, based on the label setting results, in which the text units contained in each of the multiple clusters are associated with labels, and which serves as training data used to train a classification model that classifies the text units contained in the document to be classified.

9. a cluster configuration process for clustering a plurality of sentence units obtained as a result of decomposing a model document serving as a model for a document to be classified into predetermined units into a plurality of clusters to obtain a clustering result; a labeling process for obtaining a labeling result in which a label is set for each of the plurality of clusters based on the clustering result; a model learning process for acquiring sentence classification information in which sentence units included in each of the plurality of clusters are associated with labels based on the label setting result, the sentence classification information being training data used for learning a classification model that classifies sentence units included in the document to be classified; A text classification program that runs on a computer.

Images