Data extraction system and data extraction method

Abstract

An input of a data group in which tag ID data and time information of generation of the tag ID data are associated with each other is accepted, the number of pieces of the tag ID data is counted for each time slice, and in a case where the counted number of pieces is greater than a predetermined piece threshold and a proportion of time slices in which the number of pieces does not satisfy a prescribed criterion is greater than a prescribed proportion threshold, the tag is extracted as a minority tag that is active and beneficial for a minority, and data in which a score obtained from two kinds of occurrence rates satisfies a prescribed criterion is determined as data represented in the tag ID data of the minority tag, the two kinds of occurrence rates being an occurrence rate of a word used in the tag ID data of the minority tag obtained by analyzing the tag ID data of the minority tag for a peak time period of the minority tag ID, and an occurrence rate of a word used in the tag ID data of the minority tag obtained by analyzing the tag ID data of the minority tag in a time slice in which the number of pieces of the tag ID data of the minority tag becomes a peak in a time line for the minority tag ID.

Description

TECHNICAL FIELD

[0001] The present application relates to a data extraction system and a data extraction method. BACKGROUND

[0002] Communication using social media such as blogs, social networking services, and the like is becoming widespread, and a large amount of text data is accumulating. In addition, within each organization such as a company, accumulation of text data using an intranet and the like is also being performed. In recent years, the demand for utilizing the thus accumulated large amount of text data to find new values and contribute to business activities is increasing. At the same time, it is desired to efficiently obtain desired text data from a large amount of text data.

[0003] One of the methods of obtaining desired text data from a large amount of text data generally uses a search method such as full-text search. In such a search method, a keyword indicating a feature of the desired text data is generally specified to perform a search. However, since the data collected by keyword search becomes large, sometimes the desired text is efficiently obtained by classifying the text. There are various technologies in the classification technology, but there is a method of classifying the text according to each feature thereof by assigning a label indicating the feature of the text. Here, generally, the label is defined as a keyword or metadata of a theme of the text itself. Which data is extracted from the text data classified by attaching the label depends on the utilization scenario of the user. For example, depending on the tendency of the attached label, sometimes a label assigned to a plurality of texts as a majority is utilized.

[0004] On the other hand, sometimes a minority label assigned to a minority of text data other than the majority is also utilized. However, it is difficult to determine which one is selected from the minority label, and in addition, the number of minority labels is larger than that of majority labels, and confirming all of them damages the advantage of assigning the label. In the mixed minority label, sometimes there is data of a minority opinion, a minority active but beneficial. Therefore, a method of obtaining beneficial information from the minority label is required.

[0005] As a method of extracting data of a minority theme, Patent Literature 1 is known.

[0006] PRIOR ART DOCUMENT

[0007] PATENT LITERATURE

[0008] Patent Literature 1: Japanese Patent Application Publication No. 2016-99868 SUMMARY

[0009] PROBLEMS TO BE SOLVED BY THE INVENTION

[0010] In Patent Literature 1, in order to extract a minority cluster that should be extracted, classification is performed even if the number of occurrences of a topic has an extreme deviation. Specifically, patterning is performed using the modification relationship of a text document. By processing the pattern using a specific threshold, a minority cluster is output. Here, a cluster means a group divided by a text classification method.

[0011] A minority cluster can also be extracted by the method of Patent Literature 1. However, it cannot be guaranteed that a cluster based on word and syntax classification relates to the same topic. For example, even if a cluster of "worry" is observed, what is worried about differs from user to user. In addition, there is no description of a means of extracting data representing a cluster from an extracted minority cluster.

[0012] An aspect of the present application aims to provide a technology capable of extracting a small number of active and useful minority tags from among a plurality of minority tags in a data group to which tags are assigned. In addition, the present application aims to provide a technology capable of extracting data representing a data group to which the same tag is assigned from among extracted tags.

[0013] Means for solving the problem

[0014] A data extraction system of one embodiment of the present application includes an input unit that accepts input of a data group in which tag ID data to which a tag has been assigned and time information of generation of the tag ID data are associated with each other; a tag extraction unit that counts the number of pieces of the tag ID data to which the tag identified by a tag ID included in the tag ID data has been assigned in each time slice obtained by dividing a time line including the time information into a predetermined time width, extracts the tag as a small number of active and useful minority tags in a case where the counted number of pieces is greater than a predetermined piece threshold and a proportion of the time slice in which the counted number of pieces does not satisfy a predetermined criterion is greater than a predetermined proportion threshold; and a data extraction unit that determines data in which a score obtained from two kinds of occurrence rates satisfies a predetermined criterion as data representing a data group to which a minority tag ID data has been assigned, the two kinds of occurrence rates being an occurrence rate of a word used in the minority tag ID data to which a minority tag identified by the minority tag ID has been assigned, which is obtained by analyzing the minority tag ID data to which the minority tag identified by the minority tag ID has been assigned, in a peak time period of the minority tag ID, and an occurrence rate of a word used in the minority tag ID data to which the minority tag identified by the minority tag ID has been assigned, which is obtained by analyzing the minority tag ID data in the time slice in which the number of pieces of the minority tag ID data including the minority tag identified by the minority tag ID becomes a peak in the time line with respect to the minority tag ID.

[0015] Inventive Effects

[0016] According to one embodiment of the present application, in a data set to which a tag is assigned, a minority active and beneficial minority tag can be extracted from a data set in which a plurality of minority tags exist. Also, data representing a data set to which the same tag is assigned can be extracted from the extracted tag.

[0017] The above-mentioned problems, configurations, and effects become apparent from the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a diagram showing an example of a hardware configuration of the data extraction device of Embodiment 1.

[0019] Figure 2 is a diagram showing an example of a functional configuration of the data extraction device of Embodiment 1.

[0020] Figure 3 is a diagram showing an example of a tag ID text list to which a tag has been assigned in Embodiment 1.

[0021] Figure 4 is a flowchart showing an example of the operation of the text input unit of the data extraction device of Embodiment 1.

[0022] Figure 5 is a diagram showing an example of a tag extraction parameter in Embodiment 1.

[0023] Figure 6 is a diagram showing an example of a minority tag ID list in Embodiment 1.

[0024] Figure 7 is a diagram showing an example of a timeline for each tag ID in Embodiment 1.

[0025] Figure 8 is a flowchart showing an example of the operation of the tag extraction unit of the data extraction device of Embodiment 1.

[0026] Figure 9 is a diagram showing an example of a representative data list in Embodiment 1.

[0027] Figure 10 is a diagram showing an example of a minority tag ID assignment text list in Embodiment 1.

[0028] Figure 11 is a diagram showing an example of a word occurrence rate list for each tag ID in Embodiment 1.

[0029] Figure 12 is a diagram showing an example of a peak time period list for each tag ID in Embodiment 1.

[0030] Figure 13 This is a diagram showing an example of a peak time period text list for each peak time period of a tag ID in the first embodiment.

[0031] Figure 14 This is a diagram showing an example of a word appearance rate list for each peak time period of a tag ID in Example 1.

[0032] Figure 15 This is a diagram showing an example of a word score list in Example 1.

[0033] Figure 16 This is a diagram showing an example of a text score list for each peak time period of a tag ID in Example 1.

[0034] Figure 17A This is a flowchart illustrating an example of the operation of the data extraction unit of the data extraction device of the first embodiment.

[0035] Figure 17B This is a flowchart illustrating an example of the operation of the data extraction unit of the data extraction device of the first embodiment.

[0036] Figure 18 This is a diagram showing an example of the functional structure of the data extraction device of Example 2.

[0037] Figure 19 This is a diagram showing an example of a query in Example 2.

[0038] Figure 20 This is a diagram showing an example of a text list according to the second embodiment.

[0039] Figure 21 This is a diagram showing an example of a tag ID list in Example 2.

[0040] Figure 22 This is a flowchart illustrating an example of the operation of the text data acquisition unit of the data extraction device of the second embodiment.

[0041] Figure 23 This is a flowchart illustrating an example of the operation of the text data classification unit of the data extraction device of the second embodiment.

[0042] Figure 24 This is a diagram showing an example of label extraction parameters in Example 3.

[0043] Figure 25A This is a flowchart illustrating an example of the operation of the label extraction unit of the data extraction device of the third embodiment.

[0044] Figure 25B This is a flowchart illustrating an example of the operation of the label extraction unit of the data extraction device of the third embodiment.

[0045] Figure 26 FIG. 6 is a flowchart illustrating an example of the action of the label extraction section of the data extraction apparatus of Embodiment 4.

[0046] Figure 27 FIG. 7 is a diagram of an example of the label similarity matrix of Embodiment 4.

[0047] Figure 28 FIG. 8 is a diagram of an example of the minority label ID score list of Embodiment 4.

[0048] Figure 29 FIG. 9 is a flowchart illustrating an example of the action of the minority label ID list rearrangement processing in the label extraction section of the data extraction apparatus of Embodiment 4. DETAILED DESCRIPTION

[0049] Hereinafter, the embodiments will be described using the drawings. Note that the embodiments described below do not limit the scope of the invention of the range of the patent to be requested. Also, each element and combination thereof described in the embodiments is not necessarily essential to the solution means of the invention.

[0050] Embodiment 1

[0051] Figure 1 FIG. 1 is a structural diagram showing the hardware structure of the data extraction apparatus of Embodiment 1. Figure 1 The data extraction apparatus 10 shown in FIG. 1 is, for example, an information processing apparatus. The data extraction apparatus 10 can also be implemented as a data extraction system such as a cloud server provided by a cloud system. The data extraction apparatus 10 can also be used for development, maintenance, and the like of a software system.

[0052] Figure 1 The data extraction apparatus 10 shown in FIG. 1 includes a processor 11, a main storage device 12, an auxiliary storage device 13, an input device 14, an output device 15, and a communication device 16. These are connected to each other in a manner capable of communicating with each other via a communication unit such as a bus, which is not shown.

[0053] The processor 11 is configured using, for example, a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and the like. The processor 11 reads and executes a program stored in the main storage device 12 to realize various functions of the data extraction apparatus 10. The main storage device 12 is a device that stores a program and data, and is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile semiconductor memory (NVRAM (Non Volatile RAM)), and the like.

[0054] The auxiliary storage device 13 is constituted by, for example, a hard disk drive, an SSD (Solid State Drive), an optical storage device (for example, a CD (Compact Disc), a DVD (Digital Versatile Disc), or the like), an IC card, an SD memory card, or the like. In addition, as the auxiliary storage device 13, a storage system or a cloud server, or the like can be used. The auxiliary storage device 13 stores programs and data. The programs and data stored in the auxiliary storage device 13 are loaded into the main storage device 12 as necessary.

[0055] The input device 14 is constituted by, for example, a keyboard, a mouse, a touch panel, a card reader, a voice input device, or the like. The input device 14 receives various information from a user who utilizes the data extraction device 10. The output device 15 provides various information such as a process history and a process result to the user. The output device 15 is constituted by, for example, a screen display device (a liquid crystal monitor, an LCD (Liquid Crystal Display), a graphics card, or the like), a voice output device (a speaker, or the like), a printing device, or the like.

[0056] The communication device 16 is a communication interface of a wired or wireless type that realizes communication with other devices via a communication unit such as a LAN, the Internet, or the like, and is constituted by, for example, a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, a serial communication module, or the like.

[0057] Furthermore, input and output of information can also be performed between other devices not illustrated via the communication device 16. In addition, the data extraction device 10 can be provided with a hardware such as an ASIC (Application Specific Integrated Circuit) different from the above-described structure.

[0058] Figure 2 is a diagram that shows an example of a functional structure of the data extraction device 10 of Embodiment 1. As shown in FIG. 1, the data extraction device 10 of Embodiment 1 is constituted by, for example, a CPU (Central Processing Unit) 11, a main storage device 12, an auxiliary storage device 13, an input device 14, an output device 15, a communication device 16, and the like. Figure 2As shown, the data extraction device 10 is provided with a text input section 101, a label extraction section 102, a data extraction section 103, and an information storage section 104. In addition, the information storage section 104 is provided with a tagged text list storage section 111, an extracted label storage section 112, and a representative data storage section 113. The tagged text list storage section 111 stores a tagged ID text list 121, the extracted label storage section 112 stores a minority label ID list 123, and the representative data storage section 113 stores a representative data list 124. In addition, the information storage section 104 can also store information referred to and generated by the text input section 101, the label extraction section 102, and the data extraction section 103, and the like. For example, the label extraction parameters 122 Figure 5 ), the minority label ID list 123 Figure 6 ), the timeline 70 for each label ID Figure 7 ), the representative data list 124 Figure 9 ), the tagged ID text list 100 Figure 10 ), the word occurrence rate list 110 for each label ID Figure 11 ), the peak time period list 120 for each label ID Figure 12 ), the peak time period text list 130 for each peak time period of a label ID Figure 13 ), the word occurrence rate list 140 for each peak time period of a label ID Figure 14 ), the word score list 150 Figure 15 ), the text score list 160 for each peak time period of a label ID Figure 16 ), and the like. In this regard, Embodiments 2 to 4 are the same.

[0059] Figure 2 Each section of the data extraction device 10 shown can be implemented by any one or more of the structures of the device 11 to the device 16 shown. For example, at least one of each section can also be implemented by the processor 11 reading out and executing a program stored in the main storage device 12 or the auxiliary storage device 13. In addition, at least one of each section can also be implemented using hardware such as an ASIC. Figure 1

[0060] Figure 2 The text input section 101 shown is an input section that receives the tagged ID text list 121 as an object of data extraction. The text input section 101 stores the received tagged ID text list 121 in the tagged text list storage section 111 of the information storage section 104.

[0061] Figure 3 is a diagram showing an example of the tagged ID text list 121 of Embodiment 1. ​

[0062] Figure 3 The illustrated assigned-tag-ID text list 121 has an assigned-tag-ID text 301 as a record, which contains a text 302, time information 303, and a tag ID 304. In the illustrated example, the text 1 indicates a text that was assigned a tag identified by a tag ID of "Tag ID 001" at a time "2020 / 07 / 23 17:58:36". Figure 3

[0063] Figure 4 is a flowchart for illustrating an example of the action of the text input section 101 of Embodiment 1.

[0064] First, the text input section 101 receives the assigned-tag-ID text list 121 (step S401). At this time, the text input section 101 can receive the assigned-tag-ID text list 121 that the user directly inputs to the input device 14, or can access a storage location designated by the user and receive the assigned-tag-ID text list 121 from the storage location. In the latter case, for example, the assigned-tag-ID text list 121 is stored in advance in a storage location that the data extraction device 10 can access, and the user inputs information designating the storage location to the input device 14. In this case, the text input section 101 accesses the storage location based on the input information and receives the assigned-tag-ID text list 121 from the storage location.

[0065] Next, the text input section 101 stores the assigned-tag-ID text list 121 in the assigned-tag-text list storage section 111 (step S402).

[0066] Figure 2 The illustrated tag extraction section 102 reads the assigned-tag-ID text list 121 from the assigned-tag-text list storage section 111 of the information storage section 104 and reads the tag extraction parameter 122 from the information storage section 104. The tag extraction parameter 122 is directly input to the information storage section 104 by the user via the input device 14 of the data extraction device 10 in advance. The tag extraction section 102 generates the minority-tag-ID list 123 based on the assigned-tag-ID text list 121 and the tag extraction parameter 122 and stores it in the extracted-tag storage section 112 of the information storage section 104.

[0067] Figure 5 is a diagram showing an example of the tag extraction parameter 122 of Embodiment 1. Figure 5 ​The illustrated label extraction parameter 122 has a label extraction parameter 501 as a record, which contains a parameter name 502 of the extraction parameter, and a setting value 503 as a setting value of the parameter. The parameter name 502 of the label extraction parameter 501 includes a timeline piece number threshold value 510 and a time slice ratio threshold value 511.

[0068] The timeline piece number threshold value 510 indicates a threshold value used when labels are acquired. For example, the setting value 503 of the timeline piece number threshold value 510 is set to a value of "30" or the like.

[0069] The time slice ratio threshold value 511 indicates a threshold value used when labels are acquired. For example, the setting value 503 of the time slice ratio threshold value 511 is set to a value of "0.7" or the like.

[0070] A time slice indicates a time period obtained by dividing a time axis to which the time information 303 of the label ID text list 121 is assigned into a predetermined time width. In addition, a time line is a time series list containing a time slice and the number of pieces of text generated in the time period.

[0071] Figure 6 is a diagram indicating an example of the minority label ID list 123 of Embodiment 1. Figure 6 The illustrated minority label ID list 123 contains a list of the minority label IDs 601 extracted by the label extraction unit 102. In Figure 6 For example, as the minority label IDs 601, it is indicated that the labels identified by "label ID 001", "label ID 033", and "label ID 091" are extracted.

[0072] Figure 7 is a diagram indicating an example of a time line 70 of each label ID generated and used in the action of the label extraction unit of Embodiment 1. Figure 7 The illustrated time line 70 of each label ID has a time line 701 as a record, which contains a time slice 702 and the number of pieces of text 703 generated in the time period. In addition, Figure 7 The time width of the time slice 702 in is 1 hour, but the time width is adjusted according to the amount of data to be processed, the acquisition period of data, or the like. In Figure 7In this case, for example, with respect to a tag identified by a certain tag ID, in the time slice "2020 / 07 / 23 16:00~16:59:59" constituting a certain timeline, the number of texts containing the tag identified by the tag ID is 0, in the time slice "2020 / 07 / 23 17:00~17:59:59", the number of texts containing the tag identified by the tag ID is 34, and in the time slice "2020 / 07 / 23 18:00~18:59:59", the number of texts containing the tag identified by the tag ID is 8.

[0073] Figure 8 is a flowchart for explaining an example of the action of the tag extraction section 102 of the data extraction apparatus 10 of Embodiment 1. By performing the action shown below, in a case where tagged data to which a tag has been assigned and time information in which the data has been generated are input, it is possible to extract a minority tag corresponding to the number of data pieces and the data occurrence frequency on the time series based on the time series information that takes data groups to which the same tag has been assigned in the data group as a target.

[0074] First, the tag extraction section 102 initializes the list S of the minority tag ID list 123 (step S801).

[0075] Next, the tag extraction section 102 receives the tagged ID text list 121 and the tag extraction parameter 122 (step S802). For example, the setting value 503 of the timeline piece threshold value 510 of the tag extraction parameter 122 of "30" is set as the variable U of the piece threshold value of the timeline. The setting value 503 of the time slice ratio threshold value 511 of the tag extraction parameter 122 of "0.7" is set as the variable V of the ratio threshold value of the timeline. Figure 5

[0076] Next, the tag extraction section 102 generates the time slice 702 in the timeline 70 from the tagged ID text list 121 (step S803). For example, Figure 7 The time slice 702 of the timeline 70 of Figure 3 The earliest time information and the latest time information of the time information 303 in the tagged ID text 301 of the tagged ID text list 121 of

[0077] Next, the tag extraction section 102 repeatedly performs the processing of steps S805 to S809 for each of all the tag IDs as a loop processing T (step S804). In addition, all the tag IDs are all the tag IDs 304 included in the tagged ID text list 121 of Figure 3 Next, the tag extraction section 102 repeatedly performs the processing of steps S805 to S809 for each of all the tag IDs as a loop processing T (step S804). In addition, all the tag IDs are all the tag IDs 304 included in the tagged ID text list 121 of​

[0078] In the loop processing T, the label extraction section 102 generates the number of pieces 703 of the timeline 70 of the label ID T that is the object (step S805). A copy of the timeline 70 generated in step S803 is generated. Next, based on the time information 303 in the texts 302 to which the same label has been assigned in the label ID text list 121 to which the label ID has been assigned 304, the number of pieces 703 of the timeline 70 of the label ID T is generated. For example, Figure 3 Text 1, text 2, and text 3 of the texts 302 of the label ID text list 121 to which the label ID has been assigned 121 are generated in the time period of "2020 / 07 / 23 17:00~17:59:59", and with respect to a certain label (for example, the label of "label ID 001"), Figure 7 The number of pieces 703 of "2020 / 07 / 23 17:00:00~17:59:59" of the time slice 702 of the timeline 70 becomes the number of pieces "34" including texts 1 to 3.

[0079] Next, in the loop processing T, the label extraction section 102 judges based on the maximum value of the number of pieces of the timeline of the label ID T that is the object and the timeline piece threshold value U (step S806). In the case where the maximum value of the number of pieces of the timeline of the label ID T that is the object is not greater than the timeline piece threshold value U (step S806; false), the processing of the label ID T is ended, and the next processing is entered. When the maximum value of the number of pieces of the timeline of the label ID T that is the object is greater than the timeline piece threshold value U (step S806; true), the next processing of step S807 is entered. For example, Figure 7 The maximum number of pieces "34" in the number of pieces 703 of the timeline 70 of the label ID T in the above example is greater than the timeline piece threshold value U "30". At this time, the next processing of step S807 is entered.

[0080] Next, in the loop processing T, the label extraction section 102 judges based on the proportion of the time slice that is 0 pieces in the time slice of the label ID T that is the object and the time slice proportion threshold value V (step S807).

[0081] In the case where the proportion of the time slice that is 0 pieces in the time slice of the label ID T that is the object is not greater than the time slice proportion threshold value V (step S807; false), the processing of the label ID T is ended, and the next processing is entered. When the proportion of the time slice that is 0 pieces in the time slice of the label ID T that is the object is greater than the time slice proportion threshold value V (step S807; true), the next processing of step S808 is entered. For example, when Figure 7The number of time lines 701 included in the time line 70 of the tag ID T in the tag ID T is 1000, and the number of records with the number of items 703 being "0" is 800. The time slice ratio is 0.8, which is greater than the set value 503 of the time slice ratio threshold 511 of the tag extraction parameter 122, which is "0.7". At this time, the process proceeds to step S808 of the next process. In addition, Figure 8 In step S807, the time slice with 0 events is used as the determination target, but the time slice with 5 or less events may be used as the determination target depending on the number of data to be processed, the acquisition period, etc.

[0082] Next, in the loop process T, the label extraction unit 102 adds the label ID T to the list S of the minority label ID list 123 (step S808 ).

[0083] When the processes of steps S805 to S808 are executed for all tag IDs T, the tag extraction unit 102 exits the loop process T (step S809).

[0084] Next, the label extraction unit 102 stores the list S of the minority label ID list 123 in the extracted label storage unit 112 (step S810 ).

[0085] In this way, by Figure 8 By performing the processing of step S807, if a certain time line including multiple time slices has zero items and no labels are assigned for a certain time slice or more of a certain proportion, the entire time line is judged to be inactive. Furthermore, by performing the processing of step S806, if the maximum value of the number of items in the time line is greater than the threshold, it is judged that only the time period of a certain time slice in the time line is active. Therefore, it is possible to extract labels that are inactive as a whole in a certain time line but active in a certain time period as minority labels. That is, in a data group that has been assigned a label, a small number of active and beneficial minority labels can be extracted from a data group with multiple minority labels.

[0086] Figure 2 The data extraction unit 103 shown reads the labeled text list 121 from the labeled text list storage unit 111 of the information storage unit 104 and reads the minority tag ID list 123 from the extracted tag storage unit of the information storage unit 104.

[0087] 121 and the minority tag ID list 123, generate a representative data list 124, and store it in the information storage unit

[0088] 104 is stored in the representative data storage unit 113.

[0089] Figure 9is a drawing showing an example of a representative data list 124 of Embodiment 1. Figure 9 The representative data list 124 shown includes, as a record, representative data 901 including a tag ID 902, a peak time slice 903, a text 904, and time information 905. In the representative data 901, for example, it is shown that a tag identified by the tag ID "tag ID 001" as a minority tag ID, becomes a peak time slice that is a time slice of a peak time, is a time period of "2020 / 07 / 23 17:00:00~17:59:59", and the tag is attached to a text 1 at a time of "2020 / 7 / 23 17:58:36". Figure 9

[0090] Figure 10 is a drawing showing an example of a minority tag ID-attached text list 100 generated and used in the operation of the data extraction unit of Embodiment 1. Figure 10 The minority tag ID-attached text list 100 shown includes, as a record, a minority tag ID-attached text 1001 including a text 1002, time information 1003, and a used word 1004. As will be described later in detail, the minority tag ID-attached text list 100 is generated from the tag ID-attached text list 121. In the minority tag ID-attached text list 100, for example, it is shown that the word 1 and the word 2 are used at the time of "2020 / 07 / 23 17:58:36" in the text 1 to which the tag identified by the tag ID "tag ID 001" as a minority tag is attached. Figure 10

[0091] Figure 11 is a drawing showing an example of a word occurrence rate list 110 per tag ID generated and used in the operation of the data extraction unit of Embodiment 1. Figure 11 The word occurrence rate list 110 shown includes, as a record, a word occurrence rate 1101 including a word 1102 and an occurrence rate 1103. As will be described later in detail, the word occurrence rate list 110 per tag ID is generated in the processing shown. Figure 17A 17B In the processing shown, for example, it is shown that the occurrence rate of the word "word 1" used in a data group to which a tag identified by a certain tag ID is attached is "0.8". Figure 11

[0092] Figure 12 is a drawing showing an example of a peak time period list 120 per tag ID generated and used in the operation of the data extraction unit of Embodiment 1. Figure 12 The peak time period list 120 shown includes a list of time slices 1201 of the extracted peak time period. As will be described later in detail, the peak time period list 120 per tag ID is generated in the processing shown.​​​​Figure 17A , 17B is generated in the processing shown in Figure 12 , for example, a peak time slice that is a label identified by a certain label ID is shown, and the time period of "2020 / 07 / 23 17:00:00~17:59:59" is extracted.

[0093] Figure 13 is a graph showing an example of a peak time period text list 130 that represents each peak time period of a label ID generated and utilized in the action of the data extraction section of Embodiment 1. Figure 13 The peak time period text list 130 shown in Figure 17A , 17B is generated in the processing shown in Figure 13 , for example, it is shown that there is a text 1 in the peak time slice that is a label ID, and in the text 1, a word 1, a word 2 are used at the time "2020 / 7 / 23 17:58:36".

[0094] Figure 14 is a graph showing an example of a word occurrence rate list 140 that represents each peak time period of a label ID generated and utilized in the action of the data extraction section of Embodiment 1. Figure 14 The word occurrence rate list 140 shown in

[0095] 140 has a word occurrence rate 1401 as a record, and the word occurrence rate 1401 includes a word 1402, an occurrence rate 1403. As will be described later, the word occurrence rate list 140 of each peak time period of a label ID is generated in the processing shown in

[0096] 1403. As will be described later, the word occurrence rate list 140 of each peak time period of a label ID is generated in the processing shown in Figure 17A , 17B is generated in the processing shown in Figure 14 , for example, it is shown that the occurrence rate of the word "word 1" used in the data group to which the label identified by a certain label ID is assigned, which is included in the peak time slice, is "0.9".

[0097] Figure 15 is a graph showing an example of a word score list 150 that represents each peak time period of a label ID generated and utilized in the action of the data extraction section of Embodiment 1. Figure 15 The word score list 150 shown in Figure 17A , 17B is generated in the processing shown in Figure 15For example, for a tag identified by a certain tag ID, the calculation Figure 14 The peak time slice in the time period shown has a word 1 occurrence rate of 0.9 and Figure 11 The difference "0.1" from the appearance rate of word 1 0.8 in the time period other than the peak time slice is shown as the score.

[0098] Figure 16 This is a diagram showing an example of a text score list 160 for each peak time period of a tag ID generated and used in the operation of the data extraction unit of the first embodiment. Figure 16 The text score list 160 shown has a text score 1601 as a record, and the text score 1601 includes text 1602, time information 1603, and score 1604. The details will be described later, but the text score list 160 for each peak time period of the tag ID is shown in FIG. Figure 17A 、 17B Generated in the process shown. Figure 16 For example, in Figure 13 In the peak time period text list 130 for each peak time period of the tag ID shown, the words used in text 1 are word 1 and word 2. Therefore, Figure 15 The score of the text 1 containing these words is expressed as "0.5". Figure 13 As shown, the time when these words are used is "2020 / 7 / 23 17:58:36".

[0099] Figure 17A and 17B This is a flowchart for explaining an example of the operation of the data extraction unit 103 in the first embodiment.

[0100] First, the data extraction unit 103 initializes the representative data list 124 (step S1701 ).

[0101] Next, the data extraction unit 103 reads the tagged ID-added text list 121 and the minority tag ID list 123 (step S1702 ).

[0102] Next, the data extraction unit 103 repeats the processes of steps S1704 to S1715 for each of all minority tag IDs as a loop process T (step S1703 ). Specifically, the process is repeated for the number of records of the minority tag ID 601 in the minority tag ID list 123 .

[0103] In the loop processing T, the data extraction section 103 generates a minority label ID assigned text list 100 from the label ID assigned text list 121 with respect to the minority label ID T as the object (step S1704). The minority label ID assigned text list 100 with respect to the minority label ID T is set as U ALL(T). For example, if only the record to which the label ID 304 "label ID001" is assigned is extracted from the label ID assigned text list 121 of Figure 3 the text 1002, the time information 1003 in the minority label ID assigned text 1001 of the minority label ID assigned text list 100 of Figure 10 is generated. Further, the word 1004 is kept empty in step S1704.

[0104] Next, in the loop processing T, the data extraction section 103 generates a word occurrence rate list 110 in the minority label ID assigned text list 100 with respect to the minority label ID T as the object (step S1705). The word occurrence rate list 110 with respect to the minority label ID T is set as W ALL(T). For example, the word 1004 is generated by taking the word used in each text 1002 as the object from all the minority label ID assigned texts 1001 in the minority label ID assigned text list 100 of Figure 10 This time, the processing of taking the word can use a general morphological analysis technique or the like. Next, the word 1102, the occurrence rate 1103 of the word occurrence rate list 110 is generated from the minority label ID assigned text list 100 of Figure 10 Figure 11 The word 1102 of the word occurrence rate list 110 is all the words included in the word 1004 of the minority label ID assigned text list 100. The occurrence rate 1103 of the word occurrence rate list 110 is the proportion of the specific word used in the text 1002 in the record of all the minority label ID assigned texts 1001 of the minority label ID assigned text list 100.

[0105] Next, in the loop processing T, the data extraction section 103 generates a timeline 70 with respect to the minority label ID T as the object (step S1706). In addition, the timeline 70 with respect to the minority label ID T as the object is generated in the same order as in step S804 of the label extraction section 102.

[0106] ​Next, in the loop processing T, the data extraction section 103 generates the peak time period list 120 of the timeline 70 of the minority label ID T as an object (step S1707). For example, the time slice 702 in which the number of pieces temporarily sharply increases is acquired from the timeline 70 of the minority label ID T, and the time slice 1201 of the peak time period list 120 is generated. Further, the processing of acquiring the peak time period can use a general abnormality detection technique or the like. For example, when the deviation detection technique, which is one of the abnormality detection techniques, is applied to the timeline 70 shown in FIG. 7, the time slice in "2020 / 7 / 23 17:00:00~17:59:59" is detected as the peak time period, and thus it is sufficient to record the time slice as the time slice 1201 of the peak time period list 120. Figure 7

[0107] Next, in the loop processing T, the data extraction section 103 repeatedly performs the processing of steps S1710 to S1714 for each of all the peak time periods acquired in step S1707 as the loop processing H (step S1708). Specifically, the processing is repeatedly performed in accordance with the number of records of the peak time period list 120. Figure 12

[0108] In the loop processing H, the data extraction section 103 generates the peak time period text list 130 for the peak time period H of the minority label ID T as an object (step S1709). The peak time period text list 130 for the peak time period H of the minority label ID T is set to U_Peak(T, H). For example, from the time slice 1201 "2020 / 7 / 23 17:00:00~17:59:59" of the peak time period list 120, the records of the minority label ID assigned text 1001 of the texts 1 and 3 that coincide with the time point information 1003 of the text list 100 of the minority label ID are acquired, and the peak time period text list 130 is generated. Figure 12 Figure 10

[0109] In the loop processing H, the data extraction section 103 generates the word occurrence rate list 140 in the peak time period text list U_Peak(T, H) 130 for the peak time period H of the minority label ID T as an object (step S1710). The word occurrence rate list 140 for U_Peak(T, H) is set to W_Peak(T, H). Based on the peak time period text list 130, the word occurrence rate list 140 is generated. Figure 13 Figure 14 ​​​​​Words 1402 and occurrence rates 1403 of word occurrence rate list 140 are shown. Words 1402 of word occurrence rate list 140 are all words included in used words 1304 of peak time period text list 130. Occurrence rate 1403 of word occurrence rate list 140 is the ratio of the use of a specific word in text 1302 among all peak time period texts 1301 in peak time period text list 130.

[0110] In the loop process H, the data extraction unit 103 generates a word score list 150 for the peak time period H of the minority tag ID T as the target, based on the word appearance rate list W_ALL(T) 110 for the minority tag ID T and the word appearance rate list W_Peak(T, H) 140 for the peak time period H (step S1711). The word score list 150 for the peak time period H of the minority tag ID T is set to W_Score(T, H). For example, Figure 11 The word occurrence list W_ALL(T)110 and Figure 14 When the word occurrence rate list W_Peak(T, H) 140 for the same word 1 is compared with the word occurrence rate list W_Peak(T, H) 140, the occurrence rate 1103 "0.8" of the word occurrence rate list W_ALL(T, H) 110 is subtracted from the occurrence rate 1403 "0.9" of the word occurrence rate list W_Peak(T, H) 140. Figure 15 In word score list 150, word 1502 in word score 1501 has a score 1503 of "0.1" for word 1. Thus, the peak time period is compared with all time periods to extract words that are characteristic of the peak time period. Alternatively, score 1503 in word score list 150 can be calculated using other calculation steps.

[0111] In the loop process H, the data extraction unit 103 generates a text score list 160 for the peak time period H of the target minority tag ID T based on the peak time period text list U_Peak(T, H) 130 and the word score list W_Score(T, H) 150 (step S1712). The text score list 160 for the peak time period H of the minority tag ID T is set to U_Score(T, H). For example, Figure 13 The text 1302 and time information 1303 in the peak time period text 1301 of the peak time period text list U_Peak(T,H)130 are generated. Figure 16 The text score list 160 includes the text score 1601 and the time information 1603. Figure 13the usage word 1304 of "word 1" and "word 2" for "text 1" of the peak time period text list U_Peak(T, H) 1301 of the peak time period text 130 of the peak time period H of the minority label ID T is "word 1" and "word 2", if the score 1503 "0.1" of the word 1502 "word 1" and the score 1503 "0.4" of the word 1502 "word 2" are added in the word score 1501 of the word score list W_Score(T, H) 150, Figure 15 Figure 16 the score 1604 "0.5" of the text 1602 "text 1" in the text score 1601 of the text score list 160. Thus, the text having a characteristic word in the peak time period is extracted. In addition, the score 1604 of the text score list 160 can also be calculated by other operation steps.

[0112] In the loop processing H, the data extraction unit 103 acquires the record having the largest score 1604 in the text score list U_Score(T, H) 160 for the peak time period H of the minority label ID T as the object, and appends it to the representative data list 124 (step S1713). For example, the text 1602 "text 1" having the largest score 1604 "0.5" in the text score list U_Score(T, H) 160 is acquired in the record of the text score 1601 of the text 1602 "text 1", the time information 1603 "2020 / 7 / 23 17:58:36", and is appended to the representative data 901 of the representative data list 124. Figure 16 Figure 9 Figure 9 In the representative data 901 of the representative data list 124 at this time, the label ID 902 is given to the minority label ID T as the object in the processing, and the peak time slice 903 is given to the peak time period H of the minority label ID T as the object in the processing.

[0113] When the processing of steps S1709 to S1713 is performed for all the peak time periods H of the minority label ID T, the data extraction unit 103 exits the loop processing H (step S1714).

[0114] When the processing of steps S1704 to S1714 is performed for all the minority label IDs T, the data extraction unit 103 exits the loop processing T (step S1715).

[0115] Next, the data extraction unit 103 stores the representative data list 124 in the representative data storage unit 113 (step S1716).

[0116] In this way, by performing Figure 17A 17B ​​​​The processing shown can extract data of a data group to which the representative of the minority label extracted in Figure 8 is assigned. That is, it can determine and provide data of a data group to which the representative of the minority label extracted in Figure 8 corresponds based on the score of the word in the time series of the frequency of occurrence of the word in the data group to which the representative of the minority label extracted in

[0117] As explained above, in the present embodiment, there are provided: an input section that accepts input of a data group in which data to which one or more label IDs have been assigned (e.g., the assigned label ID text list 121) and time information (e.g., the time information 303) in which the data to which one or more label IDs have been assigned have been generated are associated (e.g., the text input section 101); a label extraction section that, in a case where the number of pieces of the data to which the label identified by the label ID is assigned in each time slice (e.g., the number of pieces 703 of the time line 70) is greater than a prescribed number of pieces threshold (e.g., the maximum value of the number of pieces of the time line) (e.g., step S806; true), and the proportion of the time slice in which the number of pieces does not satisfy a prescribed reference (e.g., the proportion of the time slice in which the number of pieces is 0) is greater than a prescribed proportion threshold (step S807; true), extracts the label as a minority label that is active and beneficial in a small number; and a data extraction section that determines data (e.g., the text 1602 of the text score list 160) in which the score obtained from two kinds of occurrence rates satisfies a prescribed reference (e.g., the maximum score 1604) as data that represents the data represented in the assigned minority label ID data, the two kinds of occurrence rates being: an occurrence rate (e.g., the occurrence rate 1103 of the word occurrence rate list 110) of a word used in the assigned minority label ID data obtained by analyzing the assigned minority label ID data to which the minority label identified by the minority label ID is assigned in the peak time period of the minority label ID, with respect to the minority label ID for identifying the extracted minority label; and an occurrence rate (e.g., the occurrence rate 1403 of the word occurrence rate list 140) of a word used in the assigned minority label ID data obtained by analyzing the assigned minority label ID data that contains the minority label identified by the minority label ID in the time slice in which the number of pieces of the assigned minority label ID data becomes a peak (each peak in the case where there are a plurality of peaks) in the time line with respect to the minority label ID. Thus, in the case of processing data to which a label is assigned, in the minority label, it is possible to extract a minority label assigned to a data group that particularly mentions the same topic, and in addition, it is possible to extract representative data from a data group to which the same label is assigned among the extracted labels.

[0118] Embodiment 2

[0119] In Embodiment 2, an example in which the text list 121 to which a tag ID has been assigned is acquired by assigning a tag to a text list to which a tag has not been assigned instead of receiving the text list 121 to which a tag ID has been assigned as input is explained. The processes of the tag extraction section 102 and the data extraction section 103 are the same as in Embodiment 1. Hereinafter, mainly the structure and the operation different from Embodiment 1 are explained.

[0120] Figure 18 is a diagram showing an example of the functional structure of the data extraction apparatus 10 of Embodiment 2. As shown in the diagram, the data extraction apparatus 10 of Embodiment 2 has the text data acquisition section 105 and the text data classification section 106 in addition to the structure of the data extraction apparatus 10 of Embodiment 1. In addition, the information storage section 104 of this embodiment has the text list storage section 114 in addition to the structure of the information storage section 104 of Embodiment 1. In addition, the data extraction apparatus 10 is communicably connected to the text medium 107 that stores text data. The text medium 107 is, for example, a web server that stores web information of a website such as a microblog. Figure 18

[0121] In addition, the information storage section 104 can store information and the like referred to and generated by the text data acquisition section 105 and the text data classification section 106 in addition to the text input section 101, the tag extraction section 102, and the data extraction section 103. For example, a query 125 (described later), a text list 126 (described later), a tag ID list 127 (described later), and the like. Figure 19 Figure 20 Figure 21

[0122] When the query 125 is read from the information storage section 104, the text data acquisition section 105 transmits the query 125 to the text medium 107, receives the text list 126 corresponding to the query 125, and stores it in the text list storage section 114.

[0123] Figure 19 is a diagram showing an example of the query 125 of Embodiment 2. Figure 19 The query 125 shown in the diagram is a conditional expression for acquiring a desired text list 126 from text data stored in the text medium 107, and is described in a format determined by the text medium 107. The query 125 can be, for example, a search word or a logical expression of a search word used in a general text medium 107, and can include attribute information of text data such as a search period in addition to the search word. In addition, in a case where the text medium 107 is prepared with a provision means of text data that does not depend on the query 125, the text list 126 can be acquired by the provision means.

[0124] Figure 20 ​​​​is a diagram showing an example of the text list 126 of Example 2. The text list 126 has a text record 2101 as a record, and contains a text 2002 and time information 2003. In addition, the text record 2101 can contain information other than the text 2002 and the time information 2003. In the example shown in FIG. 10, for example, the text 1 is a text generated at the time "2020 / 07 / 23 17:58:36". Figure 20 Figure 20

[0125] The text data classification section 106 generates the text list 121 to which the tag ID has been assigned, based on the text list 126 read from the text list storage section 114, while referring to the tag ID list 127 read from the information storage section 104, and stores it in the text list storage section 111 to which the tag ID has been assigned.

[0126] Figure 21 is a diagram showing an example of the tag ID list 127. Figure 21 The tag ID list 127 shown in FIG. 11 has a tag ID record 2101 as a record, and contains a tag ID 2102 and a tag name 2103. In the example shown in FIG. 11, for example, it is shown that the tag name of the tag identified by the tag ID "tag ID001" is "tag name 1". Figure 21

[0127] The text list 126 received by the text data acquisition section 105 is stored in the text list storage section 114. In addition, the information storage section 104 can also store information and the like referred to and generated by the text data acquisition section 105 and the text data classification section 106.

[0128] Figure 22 is a flowchart for explaining an example of the operation of the text data acquisition section 105 of Example 2. First, the text data acquisition section 105 reads the query 125 from the information storage section 104 (step S2201). Next, the query 125 is transmitted to the text medium 107 (step S2202). Next, the text list 126 corresponding to the query 125 is received from the text medium 107, and stored in the text list storage section 114 (step S2203).

[0129] Figure 23 ​​​is a flowchart for explaining an example of the action of the text data classification section 106 of Embodiment 2. First, the text data classification section 106 reads in the text list 126 from the text list storage section 114 and the label ID list 127 from the information storage section 104 (step S2301). Next, for all the text records 2001 of the text list 126, the label ID record 2101 having the label name 2103 associated with the text 2002 of the text record 2001 is extracted from the label ID records 2101 of the label ID list 127, and the label ID 2102 of the label ID record 2101 is assigned to the text record 2001, thereby obtaining the label ID assigned text list 121 (step S2302). Here, whether the label name 2103 is associated with the text 2002 can be determined by whether the label name 2103 is included as a string in the text 2002, or can be determined by a method other than this. In addition, assigning the label ID 2102 to the text record 2001 means that a column related to the label ID 2102 is added to the text list 126, and the label ID 2102 assigned to each text record 2001 is stored as a value thereof, and the table obtained as a result thereof is the label ID assigned text list 121. In addition, there can be one or none of the label ID records 2101 extracted for one text record 2001. Next, the generated label ID assigned text list 121 is stored in the label ID assigned text list storage section 111 (step S2303).

[0130] As explained above, in the present embodiment, the above input section has a retrieval section (e.g., the text data retrieval section 105) that retrieves a non-ID assigned data group in which the above time information is associated with non-ID assigned data (e.g., the text list 126) to which the above label ID is not assigned, from a text medium (e.g., the text medium 107) connected to the above data extraction system, and a classification section (e.g., the text data classification section 106) that generates the above data group from the above non-ID assigned data group, based on the above non-ID assigned data group and a label ID list (e.g., the label ID list 127) in which a label ID and a label name are associated in advance. Therefore, even a text to which a label ID has not been assigned in advance can obtain a label ID assigned data (e.g., the label ID assigned text list 121).

[0131] Embodiment 3

[0132] In Embodiment 3, an example of extracting a minority tag ID list 123 having a number of records, i.e., a number of elements, close to a value specified by a user by the tag extraction section 102 of Embodiment 1 or Embodiment 2 is explained. The structure is the same as that of Embodiment 1 or Embodiment 2. In addition, the processing other than the tag extraction section 102 is the same as that of Embodiment 1 or Embodiment 2. Hereinafter, the operation different from that of Embodiment 1 or Embodiment 2 is mainly explained.

[0133] The tag extraction section 102 of Embodiment 3 stores a minority tag ID list 123 having a number of elements close to the set value 503 of the target extraction tag number 2411 described in the tag extraction parameter 122 in the extraction tag storage section 112 when reading in the assigned tag ID text list 121 from the assigned tag text list storage section 111 and reading in the tag extraction parameter 122 from the information storage section 104.

[0134] In addition, the information storage section 104 can store information referred to and generated by the text input section 101, the tag extraction section 102, the data extraction section 103, and the like, in addition to the text data acquisition section 105, the text data classification section 106. For example, it is the tag extraction parameter 122 (501) of Embodiment 3 explained later and the like. Figure 24

[0135] Figure 24 is a diagram showing an example of the tag extraction parameter 122 of Embodiment 3. Figure 24 The tag extraction parameter 501 shown includes a timeline element number threshold initial value 2410, a time slice ratio threshold 511, and a target extraction tag number 2411. In Embodiment 3, while changing the value of the timeline element number threshold 510 in Embodiment 1 or Embodiment 2 to repeatedly attempt extraction of a minority tag ID list 123, the timeline element number threshold 510 of the number of elements of the minority tag ID list 123 close to the set value 503 of the target extraction tag number 2411 is searched. The timeline element number threshold initial value 2410 indicates the initial timeline element number threshold 510 for starting such a search. For example, the set value 503 of the timeline element number threshold initial value 2410 is set to a value such as "30". The target extraction tag number 2411 indicates the number of elements of the minority tag ID list 123 that is intended to be extracted. For example, the set value 503 of the target extraction tag number 2411 is set to a value such as "10".

[0136] Figure 25A , Figure 25B ​is a flowchart for explaining an example of the action of the label extraction section 102 of Embodiment 3. First, the label extraction section 102 reads in the label ID text list 121 to which labels have been assigned and the label extraction parameters 122, sets the setting value 503 of the timeline piece number threshold value initial value 2410 to U init, sets the setting value 503 of the time slice ratio threshold value 511 to V, and sets the setting value 503 of the target extraction label number 2411 to W goal (step S2501). Next, the variable U that represents the timeline piece number threshold value 510 is initialized with U init (step S2502).

[0137] Next, the extraction label number determination index F used for the condition determination of the processing loop from step S2504 to step S2511 after this step is initialized to an empty set (step S2503). Next, the processing of steps S2504 to S2511 is repeated as the loop processing L until the number of elements of the extraction label number determination index F becomes two or more (step S2504). The case where the number of elements is two or more means, for example, if there are both a case where a label of a number less than the number of target extraction labels "10" is obtained and a case where a label of a number more than the number of target extraction labels "10" is obtained, the case where the number of elements is two or more is determined.

[0138] In the loop processing L, first, the variable S current that represents the minority label ID list 123 is initialized with an empty list (step S2505). Next, the same processing as the processing of replacing the variable S corresponding to the minority label ID list 123 with the variable S current is performed in the loop processing T of steps S803 and S804 to S809 of Embodiment 1 or Embodiment 2 (steps S803 to S809). The value of S current is the minority label ID list 123 extracted in correspondence with the value of the variable U used in this loop processing.

[0139] In the loop processing L, next, the number of elements of the variable S_current indicating the minority label ID list 123 is set to W_current (step S2506). Next, the values of W_current and W_goal are compared (step S2507). If W_current is larger (step S2507; Yes), the value 1 is added to the extracted label number determination index F, and the value of the variable U is increased by 1, and the next step S2510 is entered (step S2508). Here, if the value 1 is already included in the extracted label number determination index F, the value is not added to the extracted label number determination index F. In addition, here, the value 1 is added to the extracted label number determination index F, which means that the number of elements of the minority label ID list 123 extracted in the previous loop processing L takes a value larger than the value of W_goal. In addition, the value of the variable U is increased by 1 because the extraction of the minority label ID list 123 is attempted with a stricter condition in the next loop processing L, and thus the minority label ID list 123 with a smaller number of elements is expected to be extracted.

[0140] On the other hand, if W_current is not larger (step S2507; No), the value -1 is added to the extracted label number determination index F, and the value of the variable U is decreased by 1, and the next step S2510 is entered (step S2509). Here, if the value -1 is already included in the extracted label number determination index F, the value is not added to the extracted label number determination index F. In addition, here, the value -1 is added to the extracted label number determination index F, which means that the number of elements of the minority label ID list 123 extracted in the previous loop processing L takes a value not larger than the value of W_goal. In addition, the value of the variable U is decreased because the extraction of the minority label ID list 123 is attempted with a looser condition in the next loop processing L, and thus the minority label ID list 123 with a larger number of elements is expected to be extracted.

[0141] In the loop processing L, next, in order to maintain the values of the variables this time in the next loop processing L, the value of S_current is set to the variable S_prev, and the value of W_current is set to the variable W_prev (step S2510). When the processing of step S2505 to step S2510 is performed until the number of elements of the extracted label number determination index F becomes two or more, the label extraction section 102 exits the loop processing L (step S2511). In addition, in the loop processing L, in order to ensure that it does not become an infinite loop, an upper limit of the number of loops or the like can be set.

[0142] Next, the absolute value of the difference between W_prev and W_goal is compared with the absolute value of the difference between W_current and W_goal (step S2512). If the absolute value of the difference between W_prev and W_goal is large (step S2512: YES), the value of S_current is set to the variable S indicating the finally extracted minority label ID list 123, and the processing proceeds to step S810 (step S2513). If the absolute value of the difference between W_prev and W_goal is small (step S2512; NO), the value of S_prev is set to the variable S indicating the finally extracted minority label ID list 123, and the processing proceeds to step S810 (step S2514).

[0143] The processing of steps S2513 and S2514 corresponds to processing of selecting, from S_prev and S_current, the number of elements closer to W_goal as the finally extracted minority label ID list 123. Next, the value of the variable S indicating the minority label ID list 123 is stored in the extracted label storage section 112 (step S810).

[0144] As explained above, in the present embodiment, the above-described label extraction section repeatedly performs the processing of extracting the above-described minority label with the above-described prescribed number of elements threshold value being larger than the current value in the case where the current number of the above-described minority label is larger than the above-described target number (for example, step S2507; YES and step S2508), and the processing of extracting the above-described minority label with the above-described prescribed number of elements threshold value being smaller than the current value in the case where the current number of the above-described minority label is not larger than the above-described target number (for example, step S2507; NO and step S2509) using the number of the extracted above-described minority label (for example, the number of elements of the minority label ID list 123) and the target number of the number of the extracted above-described minority label (for example, the target number of extracted labels 2411), thereby extracting the above-described minority label of the number close to the above-described target number to some extent (for example, steps S2512 to S2514). Therefore, it is possible to extract the minority label of the number desired by the user while searching the timeline number of elements threshold value.

[0145] Embodiment 4

[0146] In Embodiment 4, an example in which the label extraction section 102 of Embodiment 1, Embodiment 2, or Embodiment 3 rearranges in the order of priority before storing the extracted minority label ID list 123 is explained. The structure and the processing other than the label extraction section 102 are the same as those of Embodiment 1, Embodiment 2, and Embodiment 3. Hereinafter, the actions different from those of Embodiment 1, Embodiment 2, or Embodiment 3 are mainly explained.

[0147] When the tag extraction section 102 of Embodiment 4 reads in the assigned tag ID text list 121 from the assigned tag text list storage section 111 and reads in the tag extraction parameter 122 from the information storage section 104, it extracts the minority tag ID list 123 rearranged in the order of priority and stores it in the extracted tag storage section 112.

[0148] In addition, the information storage section 104 can store information referred to and generated by the text input section 101, the tag extraction section 102, the data extraction section 103, and the like. For example, the label similarity matrix 270 Figure 27 ), the minority tag ID score list 280 Figure 28 ), and the like described later.

[0149] Figure 26 is a flowchart for explaining an example of the operation of the tag extraction section 102 of Embodiment 4. The processing up to Step S809 is the same as that of Figure 8 . Next, the tag extraction section 102 rearranges the elements of the variable S corresponding to the minority tag ID list 123 in the order of priority (Step S2601). An example of the operation of Step S2601 is explained in the flowchart of Figure 29 . Next, the same processing as Step S810 of Figure 8 is performed.

[0150] Figure 27 is a diagram showing an example of the label similarity matrix 270 used for scoring the elements of the variable S corresponding to the minority tag ID list 123 in Step S2601. Figure 27 The label similarity matrix 270 shown in the diagram includes an abscissa axis 2701, an ordinate axis 2702, and a cell 2703. The elements of the variable S representing the minority tag ID list 123 enter the abscissa axis 2701 and the ordinate axis 2702. In addition, the value of the cell 2703 located at the intersection of a column C corresponding to a certain tag ID 2102 of the abscissa axis 2701 and a row R corresponding to a certain tag ID 2102 of the ordinate axis 2702 represents the similarity of the tag name 2103 corresponding to the tag ID 2102 of the column C and the tag name 2103 corresponding to the tag ID 2102 of the row R. Here, in the calculation of the similarity of the tag name 2103, a generally known cosine similarity, a lexicon information in which concepts are hierarchically defined, or the like can be used. Figure 27 In the diagram, for example, the similarity between the tag identified by "tag ID 001" and the representative tag that is the tag identified by "tag ID 033" is "0.36".

[0151] Figure 28 is a diagram showing an example of the minority tag ID score list 280. Figure 28The minority tag ID score list 280 shown has a minority tag ID score 2801 as a record, and contains a tag ID 2802 and a score 2803.

[0152] Figure 29 is a flowchart for explaining an example of an action of the process of rearranging the minority tag ID list in the tag extraction section 102 of Embodiment 4 (step S2601). First, the tag extraction section 102 reads in the tag similarity matrix 270 (step S2901). Next, the tag extraction section 102 extracts the tag ID 2102 included in the assigned tag ID text 301 most frequently in the tag ID text list 121 as the representative tag T (step S2902). Note that the representative tag T can also be extracted by a method other than this. For example, in the case where there is a tag name 2103 having the tag ID 2102 of the query 125, the query 125 can also be taken as the representative tag T.

[0153] Next, the tag extraction section 102 extracts, for each minority tag ID 601 of the minority tag ID list 123, the value of the cell 2703 at the intersection of the column having the value of the representative tag T of the horizontal axis 2701 and the row having the value of the minority tag ID 601 of the vertical axis 2702 on the tag similarity matrix 270, takes the value as the score 2803 of the minority tag ID score 2801 corresponding to the minority tag ID 601 of the minority tag ID score list 280, and thereby generates the minority tag ID score list 280 (step S2903). Next, the elements of the variable S corresponding to the minority tag ID list 123 are rearranged in order from high to low of the scores 2803 of the minority tag ID score list 280 (step S2904).

[0154] As explained above, in the present embodiment, the above-described tag extraction section takes the tag included in the above-described minority tag ID data most frequently in the above-described minority tag ID for identifying the extracted above-described minority tag as the representative tag, and rearranges the extracted above-described minority tag in order from high to low of the similarity to the extracted representative tag (for example, in the order shown by the minority tag ID score list 280). Thus, the visibility of the list of the extracted minority tag can be improved.

[0155] Symbol Explanation

[0156] 10 Data extraction device

[0157] 11 Processor

[0158] 12 Main storage device

[0159] 13 Auxiliary storage device

[0160] 14 input device

[0161] 15 output device

[0162] 16 communication device

[0163] 101 text input section

[0164] 102 label extraction section

[0165] 103 data extraction section

[0166] 104 information storage section

[0167] 105 text data acquisition section

[0168] 106 text data classification section

Claims

1. A data extraction system, characterized in that: The data extraction system has: an input unit that receives input of a data set in which tag-assigned ID data is associated with time information at which the tag-assigned ID data is generated; a tag extraction unit that counts the number of pieces of tagged ID data assigned with a tag identified by the tag ID, generated in each time slice, according to the tag ID included in the tagged ID data and the time slices obtained by dividing the time line including the moment information into time slices of a predetermined time width, and extracts the tag as a minority active and beneficial minority tag if the counted number is greater than a predetermined number threshold and the proportion of the time slices in which the count does not meet a predetermined criterion is greater than a predetermined proportion threshold; as well as A data extraction unit determines data whose scores satisfy a prescribed benchmark based on the following two occurrence rates as data representative of the minority label ID data, the two occurrence rates being: the occurrence rate of words used in the minority label ID data obtained by parsing the minority label ID data assigned with the minority label identified by the minority label ID according to the peak time period of the minority label ID for identifying the extracted minority label; and the occurrence rate of words used in the minority label ID data obtained by parsing the minority label ID data in the time slice in which the number of pieces of minority label ID data including the minority label identified by the minority label ID reaches a peak in the timeline for the minority label ID.

2. The data extraction system according to claim 1, characterized in that The input unit has: an acquisition unit that acquires, from a text medium connected to the data extraction system, a non-ID-assigned data group in which the time information is associated with non-ID-assigned data to which the tag ID is not assigned; as well as The classification unit generates the data group from the un-ID-assigned data group based on the un-ID-assigned data group and a tag ID list that preliminarily associates tag IDs with tag names.

3. The data extraction system according to claim 1, wherein: The label extraction unit uses the number of the extracted minority labels and the target number of the extracted minority labels to repeatedly perform the process of extracting the minority labels by making the specified number threshold greater than the current value when the current number of the minority labels is greater than the target number, and extracting the minority labels by making the specified number threshold less than the current value when the current number of the minority labels is not greater than the target number, thereby extracting the minority labels in a number close to the target number to a certain extent.

4. The data extraction system according to claim 1, wherein: The label extraction unit extracts the label with the largest number of minority label IDs included in the label ID-assigned data from the minority label IDs used to identify the extracted minority labels as representative labels, and rearranges the extracted minority labels in descending order of similarity with the extracted representative labels.

5. A data extraction method performed by a computer, characterized in that: The input unit receives input of a data set in which tag ID data is associated with time information of when the tag ID data was generated. The tag extraction unit counts the number of pieces of tagged ID data and the tags identified by the tag IDs generated in each time slice according to the tag IDs included in the tagged ID data and the time slices obtained by dividing the time line including the moment information into time slices of a predetermined time width, and extracts the tags as the minority active and beneficial minority tags when the counted number is greater than a predetermined number threshold and the proportion of the time slices in which the count does not meet a predetermined criterion is greater than a predetermined proportion threshold; as well as The data extraction unit determines data whose scores satisfy a prescribed benchmark based on the following two occurrence rates as data representative of the minority label ID data, the two occurrence rates being: the occurrence rate of words used in the minority label ID data obtained by parsing the minority label ID data assigned with the minority label identified by the minority label ID according to the peak time period of the minority label ID for identifying the extracted minority label; and the occurrence rate of words used in the minority label ID data obtained by parsing the minority label ID data in the time slice in which the number of pieces of minority label ID data containing the minority label identified by the minority label ID reaches a peak in the time line for the minority label ID.

6. The data extraction method according to claim 5, characterized in that: The acquisition unit of the input unit acquires, from a text medium connected to the computer, a non-ID-assigned data group that associates the time information with non-ID-assigned data to which the tag ID is not assigned, The classification unit of the input unit generates the data group from the un-ID-assigned data group based on the un-ID-assigned data group and a tag ID list that preliminarily associates tag IDs with tag names.

7. The data extraction method according to claim 5, characterized in that: The label extraction unit uses the number of the extracted minority labels and the target number of the extracted minority labels to repeatedly perform the process of extracting the minority labels by making the specified number threshold greater than the current value when the current number of the minority labels is greater than the target number, and extracting the minority labels by making the specified number threshold less than the current value when the current number of the minority labels is not greater than the target number, thereby extracting the minority labels in a number close to the target number to a certain extent.

8. The data extraction method according to claim 5, characterized in that: The label extraction unit extracts the label with the largest number of minority label IDs included in the label ID-assigned data from the minority label IDs used to identify the extracted minority labels as representative labels, and rearranges the extracted minority labels in descending order of similarity with the extracted representative labels.

Images