DATA EXTRACTION SYSTEM AND DATA EXTRACTION METHOD

Abstract

To provide a data extraction system and an extraction method of the same for extracting a small number of exciting and beneficial minority tags in a data group to which tags are attached.SOLUTION: A method comprises: receiving input of a data group in which tag ID assigned data is associated with time information when the tag ID assigned data was created; and determining data whose score satisfies a predetermined criterion as representative data among minority tag ID assigned data, wherein the score is obtained from an appearance rate of words used in the minority tag ID assigned data and an appearance rate of words in a time slice where the number of the minority tag ID assigned data is peaked for each peak time zone of the minority tag ID if the number of the tag ID assigned data is greater than a predetermined number threshold, and the ratio of the time slices in which the number of the tag ID assigned data does not satisfy a predetermined standard is greater than a predetermined ratio threshold for each tag ID contained in the tag ID assigned data and for each time slice obtained by dividing the timeline containing the time information by a predetermined time width.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a data extraction system and a data extraction method. [Background technology]

[0002] With the widespread use of social media such as blogs and social networking services, large amounts of text data have been accumulated. In addition, text data is increasingly being accumulated within individual organizations such as companies using intranets. In recent years, there has been an increasing need to analyze such accumulated large amounts of text data to discover new value and utilize it in corporate activities. At the same time, there is a demand for efficiently acquiring desired text data from large amounts of text data.

[0003] One of the methods for obtaining desired text data from a large amount of text data is generally to use a search method such as a full-text search. In such a search method, a search is usually performed by specifying a keyword that represents the characteristics of the desired text data. However, since the amount of data collected by keyword search is huge, there are cases where the desired text can be obtained efficiently by classifying the text. There are various classification techniques, and there is a method of classifying text by its characteristics by attaching a tag that represents the characteristics of the text. Here, a tag is generally defined as metadata such as keywords or topics of the text itself. The user's usage scenario determines which data to extract from the text data classified by tagging. For example, there are cases where a tag that is the majority that is attached to many texts due to the tendency of tagging is used.

[0004] On the other hand, there are also cases where minority tags are used, which are assigned to a small number of text data other than the majority. However, it is difficult to decide which minority tags to select, and the number of minority tags will be greater than the majority tags, so checking all of them will undermine the benefits of tagging. Among the mixed minority tags, there may be a minority of opinions or a small number of trends, but they may contain useful data. Therefore, a method is needed to obtain useful information from minority tags.

[0005] Regarding a method for extracting data on minority topics, Patent Document 1 is available. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-99868 A Summary of the Invention [Problem to be solved by the invention]

[0007] In Patent Document 1, in order to extract minority clusters that should be extracted, classification is possible even if there is an extreme bias in the number of topics that appear. Specifically, dependency relationships in text documents are used to create patterns. The patterns are processed using a specific threshold to output minority clusters. Here, clusters refer to groups divided by a text classification method.

[0008] The method of Patent Document 1 also makes it possible to extract minority clusters. However, there is no guarantee that clusters classified based on words and grammar refer to the same topic. For example, even if we look at a cluster called "worry," what each user is worried about varies from one user to another. In addition, there is no description of a means for extracting data representative of a cluster from an extracted minority cluster.

[0009] An object of the present invention is to provide a technique that can extract a small number of useful minority tags from a tagged data group in which multiple minority tags exist, and also to provide a technique that can extract data representative of a data group to which the same tag is assigned from among the extracted tags. [Means for solving the problem]

[0010] A data extraction system according to one embodiment of the present invention includes an input unit that accepts input of tag ID-assigned data and a data group associated with time information for creating the tag ID-assigned data; a tag extraction unit that counts the number of tag ID-assigned data items to which tags identified by the tag IDs occurring in each time slice are assigned for each tag ID included in the tag ID-assigned data and for each time slice obtained by dividing a timeline including the time information into a predetermined time interval; and, if the counted number is greater than a predetermined number threshold and the proportion of the time slices in which the number does not meet a predetermined criterion is greater than a predetermined proportion threshold, extracts the tag as a minority tag with a small number of trends and is useful; and and a data extraction unit that determines, as representative data among the minority tag ID-assigned data, data whose score satisfies a predetermined criterion obtained from: an occurrence rate of words used in the minority tag ID-assigned data obtained by analyzing the minority tag ID-assigned data to which a minority tag identified by the minority tag ID has been assigned, for each peak time period of the minority tag ID; and an occurrence rate of words used in the minority tag ID-assigned data obtained by analyzing the minority tag ID-assigned data in the time slice in which the number of minority tag ID-assigned data including the minority tag identified by the minority tag ID in the timeline for the minority tag ID peaks. Effect of the Invention

[0011] According to one aspect of the present invention, it is possible to extract a small number of useful minority tags from a tagged data group in which a plurality of minority tags exist. Furthermore, it is possible to extract data representative of the data group to which the same tag is assigned from among the extracted tags.

[0012] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2 illustrates an example of a hardware configuration of a data extraction device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of a data extraction device according to a first embodiment. [Diagram 3] FIG. 11 is a diagram showing an example of a tag ID-added text list according to the first embodiment. [Figure 4] 10 is a flowchart illustrating an example of an operation of a text input unit of the data extraction device according to the first embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of tag extraction parameters according to the first embodiment. [Figure 6] FIG. 4 is a diagram illustrating an example of a minority tag ID list according to the first embodiment. [Figure 7] FIG. 4 is a diagram illustrating an example of a timeline for each tag ID according to the first embodiment. [Figure 8] 11 is a flowchart illustrating an example of an operation of a tag extraction unit of the data extraction device according to the first embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a representative data list according to the first embodiment; [Figure 10] FIG. 11 is a diagram showing an example of a minority tag ID-added text list according to the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a word appearance rate list for each tag ID according to the first embodiment. [Figure 12] FIG. 11 is a diagram illustrating an example of a peak time period list for each tag ID according to the first embodiment. [Figure 13] FIG. 11 is a diagram illustrating an example of a peak time period text list for each peak time period of a tag ID according to the first embodiment. [Figure 14] FIG. 13 is a diagram illustrating an example of a word appearance rate list for each peak time period of a tag ID according to the first embodiment. [Figure 15] FIG. 4 is a diagram showing an example of a word score list according to the first embodiment. [Figure 16] FIG. 13 is a diagram showing an example of a text score list for each peak time period of a tag ID according to the first embodiment. [Figure 17A] 11 is a flowchart illustrating an example of an operation of a data extraction unit of the data extraction device according to the first embodiment. [Figure 17B] 11 is a flowchart illustrating an example of an operation of a data extraction unit of the data extraction device according to the first embodiment. [Figure 18] FIG. 11 is a diagram illustrating an example of a functional configuration of a data extraction device according to a second embodiment. [Figure 19] FIG. 11 is a diagram illustrating an example of a query according to the second embodiment. [Figure 20] FIG. 11 is a diagram showing an example of a text list according to the second embodiment. [Figure 21] FIG. 11 is a diagram illustrating an example of a tag ID list according to the second embodiment. [Figure 22] 11 is a flowchart illustrating an example of an operation of a text data acquiring unit of the data extracting device according to the second embodiment. [Diagram 23] 11 is a flowchart illustrating an example of an operation of a text data classification unit of the data extraction device according to the second embodiment. [Figure 24] FIG. 13 is a diagram illustrating an example of tag extraction parameters according to the third embodiment. [Figure 25A] 13 is a flowchart illustrating an example of an operation of a tag extraction unit of the data extraction device according to the third embodiment. [Figure 25B] 13 is a flowchart illustrating an example of an operation of a tag extraction unit of the data extraction device according to the third embodiment. [Figure 26] 13 is a flowchart illustrating an example of an operation of a tag extraction unit of the data extraction device according to the fourth embodiment. [Figure 27] FIG. 13 is a diagram illustrating an example of a tag similarity matrix according to the fourth embodiment. [Figure 28] FIG. 13 is a diagram illustrating an example of a minority tag ID score list according to the fourth embodiment. [Figure 29] 13 is a flowchart illustrating an example of an operation of a minority tag ID list rearrangement process in a tag extraction unit of a data extraction device according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The following describes the embodiments with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention. Furthermore, not all of the elements and combinations described in the embodiments are essential to the solution of the invention. EXAMPLES

[0015] Fig. 1 is a configuration diagram showing a hardware configuration of a data extraction device according to a first embodiment. The data extraction device 10 shown in Fig. 1 is, for example, an information processing device. The data extraction device 10 may be realized as a data extraction system such as a cloud server provided by a cloud system. The data extraction device 10 may be used for development and maintenance of a software system.

[0016] 1 includes a processor 11, a main memory device 12, an auxiliary memory device 13, an input device 14, an output device 15, and a communication device 16. These are connected to each other so as to be able to communicate with each other via a communication means such as a bus (not shown).

[0017] The processor 11 is configured using, for example, a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The processor 11 realizes various functions of the text data collection device 10 by reading and executing programs stored in the main memory device 12. The main memory device 12 is a device that stores programs and data, and is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and a non-volatile semiconductor memory (NVRAM (Non Volatile RAM)).

[0018] The auxiliary storage device 13 is configured, for example, with a hard disk drive, an SSD (Solid State Drive), an optical storage device (for example, a CD (Compact Disc) or a DVD (Digital Versatile Disc)), an IC card, an SD memory card, or the like. A storage system or a cloud server may also be used as the auxiliary storage device 13. The auxiliary storage device 13 stores programs and data. The programs and data stored in the auxiliary storage device 13 are loaded into the main storage device 12 as necessary.

[0019] The input device 14 is configured, for example, using a keyboard, a mouse, a touch panel, a card reader, and a voice input device. The input device 14 accepts various information from a user who uses the text data collection device 10. The output device 15 provides the user with various information such as the progress and results of processing. The output device 15 is configured, for example, using a screen display device (such as a liquid crystal monitor, LCD (Liquid Crystal Display), and a graphics card), a voice output device (such as a speaker), and a printer.

[0020] The communication device 16 is a wired or wireless communication interface that realizes communication with other devices via communication means such as a LAN or the Internet, and is configured using, for example, a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, and a serial communication module.

[0021] The input and output of information may be performed between the data extracting device 10 and another device (not shown) via the communication device 16. The data extracting device 10 may also include hardware such as an ASIC (Application Specific Integrated Circuit) in addition to the above configuration.

[0022] FIG. 2 is a diagram showing an example of a functional configuration of the data extraction device 10 according to the first embodiment. As shown in FIG. 2, the data extraction device 10 includes a text input unit 101, a tag extraction unit 102, a data extraction unit 103, and an information storage unit 104. The information storage unit 104 includes a tagged text list storage unit 111, an extracted tag storage unit 112, and a representative data storage unit 113. The tagged text list storage unit 111 stores an ID-added text list 121, the extracted tag storage unit 112 stores a minority tag ID list 123, and the representative data storage unit 113 stores a representative data list 124. The information storage unit 104 may further store information referred to and generated by the text input unit 101, the tag extraction unit 102, and the data extraction unit 103. For example, the tag extraction parameters 122 (FIG. 5), minority tag ID list 123 (FIG. 6), timeline 70 for each tag ID (FIG. 7), representative data list 124 (FIG. 9), minority tag ID assigned text list 100 (FIG. 10), word appearance rate list 110 for each tag ID (FIG. 11), peak time period list 120 for each tag ID (FIG. 12), peak time period text list 130 for each peak time period of tag ID (FIG. 13), word appearance rate list 140 for each peak time period of tag ID (FIG. 14), word score list 150 (FIG. 15), text score list 160 for each peak time period of tag ID (FIG. 16), etc. This point is the same for the second to fourth embodiments.

[0023] Each unit of the data extraction device 10 shown in Fig. 2 is realized by one or more of the devices 11 to 16 shown in Fig. 1. For example, at least one of the units may be realized by the processor 11 reading and executing a program stored in the main storage device 12 or the auxiliary storage device 13. Also, at least one of the units may be realized by using hardware such as an ASIC.

[0024] 2 is an input unit that accepts the tag ID-assigned text list 121 that is the target of data extraction. The text input unit 101 stores the accepted tag ID-assigned text list 121 in the tagged text list storage unit 111 of the information storage unit 104.

[0025] FIG. 3 is a diagram illustrating an example of the tag ID-added text list 121 according to the first embodiment. The tag ID-assigned text list 121 shown in Fig. 3 has tag ID-assigned text 301 as a record, and tag ID-assigned text 301 includes text 302, time information 303, and tag ID 304. Fig. 3 shows, for example, that text 1 is text to which tags identified by tag IDs "tag ID001" and "tag ID002" were assigned at time "2020 / 07 / 23 17:58:36".

[0026] FIG. 4 is a flowchart for explaining an example of the operation of the text input unit 101 according to the first embodiment. First, the text input unit 101 accepts the tag ID-assigned text list 121 (step S401). At this time, the text input unit 101 may accept the tag ID-assigned text list 121 directly input by the user to the input device 14, or may access a storage location designated by the user and accept the tag ID-assigned text list 121 from the storage location. In the latter case, for example, the tag ID-assigned text list 121 is stored in advance in a storage location accessible by the data extracting device 10, and the user inputs information specifying the storage location to the input device 14. In this case, the text input unit 101 accesses the storage location based on the input information and accepts the tag ID-assigned text list 121 from the storage location.

[0027] Next, the text input unit 101 stores the tag ID-added text list 121 in the tagged text list storage unit 111 (step S402).

[0028] 2 reads tag ID-assigned text list 121 from tagged text list storage section 111 of information storage section 104, and reads tag extraction parameters 122 from information storage section 104. The tag extraction parameters 122 are input directly to information storage section 104 in advance by a user via input device 14 of data extraction device 10. Tag extraction section 102 creates minority tag ID list 123 based on tag ID-assigned text list 121 and tag extraction parameters 122, and stores it in extracted tag storage section 112 of information storage section 104.

[0029] Fig. 5 is a diagram illustrating an example of the tag extraction parameter 122 according to the first embodiment. The tag extraction parameter 122 illustrated in Fig. 5 has a tag extraction parameter 501 as a record, and the tag extraction parameter 501 includes a parameter name 502 of the extraction parameter and a setting value 503 which is a setting value of the parameter. The parameter name 502 of the tag extraction parameter 501 includes a timeline count threshold 510 and a time slice ratio threshold 511.

[0030] The timeline number threshold 510 indicates a threshold used when acquiring tags. For example, a value such as “30” is set as the setting value 503 of the timeline number threshold 510.

[0031] The time slice ratio threshold 511 indicates a threshold used when acquiring a tag. For example, a value such as "0.7" is set as the setting value 503 of the time slice ratio threshold 511.

[0032] A time slice refers to a time period obtained by dividing the time axis of the time information 303 in the tag ID assigned text list 121 into a predetermined time interval. A time line is a time series list including a time slice and the number of texts occurring in that time period.

[0033] Fig. 6 is a diagram illustrating an example of the minority tag ID list 123 according to the first embodiment. The minority tag ID list 123 illustrated in Fig. 6 includes a list of minority tag IDs 601 extracted by the tag extraction unit 102. Fig. 6 illustrates that, for example, tags identified by the tag IDs "tag ID001", "tag ID033", and "tag ID091" are extracted as the minority tag IDs 601.

[0034] Fig. 7 is a diagram showing an example of a timeline 70 for each tag ID that is created and used during the operation of the tag extraction unit according to the first embodiment. The timeline 70 for each tag ID shown in Fig. 7 has a timeline 701 as a record, and includes a time slice 702 and the number of texts occurring in that time period 703. Note that the time width of the time slice 702 in Fig. 7 is set to one hour, but the time width is adjusted depending on the number of data to be handled, the data acquisition period, and the like. Figure 7 shows, for example, that for a tag identified by a certain tag ID, in the time slice "2020 / 07 / 23 16:00~16:59:59" that makes up a certain timeline, there are 0 texts containing the tag identified by that tag ID, in the time slice "2020 / 07 / 23 17:00~17:59:59" there are 34 texts containing the tag identified by that tag ID, and in the time slice "2020 / 07 / 23 18:00~18:59:59" there are 8 texts containing the tag identified by that tag ID.

[0035] 8 is a flowchart for explaining an example of the operation of the tag extraction unit 102 of the data extraction device 10 according to the first embodiment. By performing the operation described below, when tagged data to which a tag has been added and time information on the creation of the data are input, it becomes possible to extract minority tags according to the number of data items in the time series and the frequency of data appearance based on time series information on a data group to which the same tag has been added from the data group.

[0036] First, the tag extraction unit 102 initializes the list S of the minority tag ID list 123 (step S801).

[0037] Next, the tag extraction unit 102 receives the tag ID assigned text list 121 and the tag extraction parameters 122 (step S802). For example, the setting value 503 of the timeline count threshold 510 of the tag extraction parameters 122 in Fig. 5, "30", is set to the timeline count threshold variable U. The setting value 503 of the time slice ratio threshold 511 of the tag extraction parameters 122, "0.7", is set to the timeline ratio threshold variable V.

[0038] Next, the tag extraction unit 102 creates a time slice 702 in the timeline 70 from the tag ID-added text list 121 (step S803). For example, the time slice 702 in the timeline 70 in Fig. 7 is created by treating the oldest and newest time information of the time information 303 of the tagged text 301 in the tag ID-added text list 121 in Fig. 3 as the start and end, and creating the time slice 702 with a time span of one hour.

[0039] Next, the tag extraction unit 102 repeats the processes of steps S805 to S809 as a loop process T for each of all tag IDs (step S804). Note that all tag IDs are all tag IDs included in the tag IDs 304 assigned in the tag ID assigned text list 121 in FIG.

[0040] In the loop process T, the tag extraction unit 102 creates the number 703 of the timeline 70 of the target tag ID T (step S805). A copy of the timeline 70 created in step S803 is created. Next, the number 703 of the timeline 70 of the tag IDT is created based on the time information 303 of the text 302 to which the same tag is assigned in the tag ID 304 of the tag ID assigned text list 121. For example, text 1, text 2, and text 3 of text 302 in tag ID-assigned text list 121 in Figure 3 were all generated during the time period "2020 / 07 / 23 17:00-17:59:59," and for a certain tag (for example, a tag with "tag ID 001"), the number 703 of times in time slice 702 of timeline 70 in Figure 7 for the period "2020 / 07 / 23 17:00:00-17:59:59" is "34," including texts 1 to 3.

[0041] Next, in loop processing T, the tag extraction unit 102 makes a judgment based on the maximum number of timelines of the target tag IDT and the timeline number threshold U (step S806). If the maximum number of timelines of the target tag IDT is not greater than the timeline number threshold U (step S806; False), the processing of the tag IDT is terminated and the process proceeds to the next process. If the maximum number of timelines of the target tag IDT is greater than the timeline number threshold U (step S806; True), the process proceeds to step S807 of the next process. For example, the maximum number "34" of the number 703 of the timelines 70 of the tag IDT in FIG. 7 is greater than the timeline number threshold U "30". In this case, the process proceeds to step S807 of the next process.

[0042] Next, in the loop process T, the tag extraction unit 102 makes a determination based on the ratio of the time slices of the target tag IDT that are 0 and the time slice ratio threshold V (step S807).

[0043] If the ratio of the time slices of the target tag IDT that are 0 is not greater than the time slice ratio threshold V (step S807; False), the processing of the tag IDT is terminated and the processing proceeds to the next processing. If the ratio of the time slices of the target tag IDT that are 0 is greater than the time slice ratio threshold V (step S807; True), the processing proceeds to step S808 of the next processing. For example, if the number of time lines 701 included in the time line 70 of the tag IDT in FIG. 7 is 1000 records, and the number of records in which the number of records 703 is "0" is 800 records, the time slice ratio is 0.8, which is greater than the setting value 503 of the time slice ratio threshold 511 of the tag extraction parameter 122, which is "0.7". In this case, the processing proceeds to step S808 of the next processing. Note that in step S807 of FIG. 8, the time slices that are 0 are the object of the judgment, but the time slices that are 5 or less may be the object of the judgment depending on the number of data to be handled, the acquisition period, etc.

[0044] Next, in loop processing T, the tag extraction unit 102 adds the tag IDT to list S of the minority tag ID list 123 (step S808).

[0045] When the processing of steps S805 to S808 is executed for all tag IDT, the tag extraction unit 102 exits from the loop processing T (step S809).

[0046] Next, the tag extraction unit 102 stores the list S of the minority tag ID list 123 in the extracted tag storage unit 112 (step S810).

[0047] In this way, by performing the process of step S807 in Fig. 8, if a certain percentage or more of a certain time slice in a certain timeline including multiple time slices is zero and not tagged, it is determined that the timeline as a whole is not trending. Furthermore, by performing the process of step S806, if the maximum number of items in the timeline is equal to or greater than a threshold, it is determined that only a certain time slice in the timeline is trending. Therefore, it is possible to extract a tag that is not trending as a whole in a certain timeline but is trending in a certain time period as a minority tag. In other words, it is possible to extract a useful minority tag with a small number of trends from a data group in which multiple minority tags exist in a tagged data group.

[0048] 2 reads tag ID-assigned text list 121 from tagged text list storage section 111 of information storage section 104, and minority tag ID list 123 from extracted tag storage section 104. Data extraction section 103 creates representative data list 124 based on tag ID-assigned text list 121 and minority tag ID list 123, and stores it in representative data storage section 113 of information storage section 104.

[0049] Fig. 9 is a diagram showing an example of the representative data list 124 according to the first embodiment. The representative data list 124 shown in Fig. 9 has representative data 901 as a record, and the representative data 901 includes a tag ID 902, a peak time slice 903, text 904, and time information 905. Fig. 9 shows, for example, that a tag whose tag ID, which is a minority tag ID, is identified by "tag ID 001" has a peak time slice, which is a time slice that became a peak time, in the time period "2020 / 07 / 23 17:00:00 to 17:59:59", and that the time when the tag was tagged to text 1 was "2020 / 7 / 23 17:58:36".

[0050] FIG. 10 is a diagram showing an example of a minority tag ID-assigned text list 100 created and used during the operation of the data extraction unit according to the first embodiment. The minority tag ID-assigned text list 100 shown in FIG. 10 has a minority tag ID-assigned text 1001 as a record, and the minority tag ID-assigned text 1001 includes text 1002, time information 1003, and used words 1004. As will be described later in detail, the minority tag ID-assigned text list 100 is created from the tag ID-assigned text list 121. FIG. 10 shows, for example, that in text 1 to which a tag identified by "tag ID001", which is a minority tag, was assigned, words 1 and 2 were used at the time "2020 / 7 / 23 17:58:36".

[0051] Fig. 11 is a diagram showing an example of a word appearance rate list 110 for each tag ID that is created and used during the operation of the data extraction unit according to the first embodiment. The word appearance rate list 110 shown in Fig. 11 has a word appearance rate 1101 as a record, and the word appearance rate 1101 includes a word 1102 and an appearance rate 1103. As will be described later in detail, the word appearance rate list 110 for each tag ID is created in the process shown in Figs. 17A and 17B. Fig. 11 shows, for example, that the appearance rate of the used word "word 1" in a data group to which a tag identified by a certain tag ID is added is "0.8".

[0052] FIG. 12 is a diagram illustrating an example of a peak time period list 120 for each tag ID that is created and used during operation of the data extraction unit according to the first embodiment. The peak time period list 120 illustrated in FIG. 12 includes a list of time slices 1201 of the extracted peak time periods. As will be described in detail later, the peak time period list 120 for each tag ID is created in the process illustrated in FIGS. 17A and 17B. FIG. 12 illustrates, for example, that the time period "2020 / 07 / 23 17:00:00 to 17:59:59" is extracted as a peak time slice of a tag identified by a certain tag ID.

[0053] FIG. 13 is a diagram showing an example of a peak time period text list 130 for each peak time period of a tag ID that is created and used during the operation of the data extraction unit according to the first embodiment. The peak time period text list 130 shown in FIG. 13 has a peak time period text 1301 as a record, and the peak time period text 1301 includes text 1302, time information 1303, and used words 1304. As will be described later in detail, the peak time period text list 130 for each peak time period of a tag ID is created in the process shown in FIGS. 17A and 17B. In FIG. 13, for example, text 1 is included in the peak time slice of the tag ID, and in the text 1, words 1 and 2 are used at the time "2020 / 7 / 23 17:58:36".

[0054] Fig. 14 is a diagram showing an example of a word appearance rate list 140 for each peak time period of a tag ID, which is created and used during the operation of the data extraction unit according to the first embodiment. The word appearance rate list 140 shown in Fig. 14 has a word appearance rate 1401 as a record, and the word appearance rate 1401 includes a word 1402 and an appearance rate 1403. As will be described later in detail, the word appearance rate list 140 for each peak time period of a tag ID is created in the process shown in Figs. 17A and 17B. Fig. 14 shows, for example, that the appearance rate of the used word "word 1" in a data group to which a tag identified by a certain tag ID included in a peak time slice is assigned is "0.9".

[0055] FIG. 15 is a diagram showing an example of a word score list 150 created and used during the operation of the data extracting unit according to the first embodiment. The word score list 150 shown in FIG. 15 has a word score 1501 as a record, and the word score 1501 includes a word 1502 and a score 1503. As will be described later in detail, the word score list 150 is created in the process shown in FIGS. 17A and 17B. FIG. 15 shows that, for example, for a tag identified by a certain tag ID, the difference "0.1" between the occurrence rate of word 1 in the peak time slice time period shown in FIG. 14 (0.9) and the occurrence rate of word 1 in the time period other than the peak time slice time period shown in FIG. 11 (0.8) is calculated as a score.

[0056] FIG. 16 is a diagram showing an example of a text score list 160 for each peak time period of a tag ID that is created and used during the operation of the data extraction unit according to the first embodiment. The text score list 160 shown in FIG. 16 has a text score 1601 as a record, and the text score 1601 includes a text 1602, time information 1603, and a score 1604. Specifically, the text score list 160 for each peak time period of a tag ID is created in the process shown in FIGS. 17A and 17B, which will be described later. In FIG. 16, for example, in the peak time period text list 130 for each peak time period of a tag ID shown in FIG. 13, the words used in text 1 are word 1 and word 2. Therefore, it is shown that the score of text 1 including these words in FIG. 15 is expressed as "0.5". In addition, it is shown that the time at which these words are used is the time "2020 / 7 / 23 17:58:36" as shown in FIG. 13.

[0057] 17A and 17B are flowcharts illustrating an example of the operation of the data extraction unit 103 according to the first embodiment. First, the data extracting unit 103 initializes the representative data list 124 (step S1701). Next, the data extraction unit 103 reads the tag ID assigned text list 121 and the minority tag ID list 123 (step S1702). Next, the data extraction unit 103 repeats the processes of steps S1704 to S1715 as a loop process T for each of the minority tag IDs (step S1703). Specifically, the processes are repeated the number of times corresponding to the number of records of the minority tag IDs 601 in the minority tag ID list 123.

[0058] In loop processing T, data extraction unit 103 creates minority tag ID-assigned text list 100 from tag ID-assigned text list 121 for T, which is a target minority tag ID (step S1704). The minority tag ID-assigned text list 100 for minority tag IDT is U_ALL(T). For example, when only records with tag ID 304 "tag ID001" are extracted from tag ID-assigned text list 121 in FIG. 3, text 1002 and time information 1003 are created from minority tag ID-assigned text 1001 in minority tag ID-assigned text list 100 in FIG. 10. Note that used words 1004 remains empty in step S1704.

[0059] Next, in the loop process T, the data extraction unit 103 creates a word appearance rate list 110 in the minority tag ID assigned text list 100 of the target minority tag IDT (step S1705). The word appearance rate list 110 for the minority tag IDT is W_ALL(T). For example, all minority tag ID assigned texts 1001 included in the minority tag ID assigned text list 100 in FIG. 10 are targeted, and words used from each text 1002 are obtained to create used words 1004. At this time, the process of obtaining words may use a general morphological analysis technique or the like. Next, based on the minority tag ID assigned text list 100 in FIG. 10, the words 1102 and appearance rates 1103 of the word appearance rate list 110 in FIG. 11 are created. The words 1102 of the word appearance rate list 110 are all words included in the used words 1004 of the minority tag ID assigned text list 100. The occurrence rate 1103 of the word occurrence rate list 110 is the rate at which a particular word is used in the text 1002 among all records of the minority tag ID assigned text 1001 in the minority tag ID assigned text list 100.

[0060] Next, in the loop process T, the data extraction unit 103 creates a timeline 70 of the target minority tag IDT (step S1706). Note that the timeline 70 of the target minority tag IDT is created in the same procedure as in step S804 of the tag extraction unit 102.

[0061] Next, in the loop process T, the data extraction unit 103 creates a peak time period list 120 of the timeline 70 of the target minority tag IDT (step S1707). For example, a time slice 702 in which the number of cases temporarily increases sharply is acquired from the timeline 70 of the minority tag IDT, and a time slice 1201 of the peak time period list 120 is created. Note that the process of acquiring the peak time period may use a general anomaly detection technique. For example, when an out-of-range detection technique, which is a type of anomaly detection technique, is applied to the timeline 70 shown in FIG. 7, the time slice "2020 / 7 / 23 17:00:00~17:59:59" is detected as a peak time period, and the time slice is recorded as the time slice 1201 of the peak time period list 120.

[0062] Next, in loop process T, the data extracting unit 103 repeats the processes of steps S1710 to S1714 as loop process H for each of the peak time periods acquired in step S1707 (step S1708). Specifically, the processes are repeated the number of times equal to the number of records in the peak time period list 120 in FIG.

[0063] In loop process H, data extraction unit 103 creates peak time zone text list 130 for peak time zone H of target minority tag IDT (step S1709). Peak time zone text list 130 for peak time zone H of minority tag IDT is U_Peak(T,H). For example, based on time slice 1201 "2020 / 7 / 23 17:00:00~17:59:59" of peak time zone list 120 in FIG. 12, records of minority tag ID assigned text 1001 of text 1 and text 3 corresponding to time information 1003 of minority tag ID assigned text list 100 in FIG. 10 are obtained, and peak time zone text list 130 is created.

[0064] In the loop process H, the data extraction unit 103 creates a word appearance list 140 in the peak time period text list U_Peak(T,H) 130 for the peak time period H of the target minority tag IDT (step S1710). The word appearance rate list 140 for U_Peak(T,H) is set as W_Peak(T,H). Based on the peak time period text list 130 in FIG. 13, the words 1402 and appearance rates 1403 in the word appearance rate list 140 in FIG. 14 are created. The words 1402 in the word appearance rate list 140 are all words included in the used words 1304 in the peak time period text list 130. The appearance rates 1403 in the word appearance rate list 140 are the proportion of the use of a specific word in the text 1302 among all the records of the peak time period text 131 in the peak time period text list 130.

[0065] In loop process H, data extraction unit 103 creates word score list 150 for peak time zone H of the target minority tag IDT from word occurrence rate list W_ALL(T) 110 for minority tag IDT and word occurrence rate list W_Peak(T,H) 140 for peak time zone H (step S1711). Word score list 150 for peak time zone H of minority tag IDT is designated as W_Score(T,H). For example, comparing the word appearance rate list W_ALL(T) 110 in FIG. 11 with the word appearance rate list W_Peak(T,H) 140 in FIG. 14, subtracting the appearance rate 1103 "0.8" in the word appearance rate list W_ALL(T,H) 110 from the appearance rate 1403 "0.9" in the word appearance rate list W_Peak(T,H) 140 for the same word 1, the score 1503 of "word 1" in the word score 1501 in the word score list 150 in FIG. 15 is "0.1". This allows the peak time period to be compared with the entire time period, and words characteristic of the peak time period to be extracted. Note that the score 1503 in the word score list 150 may be calculated using other calculation procedures.

[0066] In loop process H, data extraction unit 103 creates text score list 160 for peak time period H of target minority tag IDT based on peak time period text list U_Peak(T,H) 130 and word score list W_Score(T,H) 150 (step S1712). Text score list 160 for peak time period H of minority tag IDT is U_Score(T,H). For example, text 1302 and time information 1303 are obtained from peak time period text 1301 in peak time period text list U_Peak(T,H) 130 in FIG. 13, and text 1602 and time information 1603 are created from text score 1601 in text score list 160 in FIG. 16. In addition, in the peak time period text 1301 "Text 1" in the peak time period text list U_Peak(T,H) 130 in FIG. 13, the used words 1304 are "Word 1" and "Word 2". When the score 1503 "0.1" of the word 1502 "Word 1" and the score 1503 "0.4" of the word 1502 "Word 2" in the word score 1501 in the word score list W_Score(T,H) 150 in FIG. 15 are added, the score 1604 "0.5" of the text 1602 "Text 1" in the text score 1601 in the text score list 160 in FIG. 16 is obtained. In this way, texts having words characteristic of the peak time period are extracted. The score 1604 in the text score list 160 may be calculated by other calculation procedures.

[0067] In loop process H, the data extraction unit 103 acquires a record with the maximum score 1604 from the text score list U_Score(T,H) 160 for the peak time period H of the target minority tag IDT, and adds it to the representative data list 124 (step S1713). For example, from the text score list U_Score(T,H) 160 in FIG. 16, from the record of the text score 1601 of the text 1602 "Text 1" with the maximum score 1604 of "0.5", acquires the text 1602 "Text 1" and the time information 1603 "2020 / 7 / 23 17:58:36", and adds them to the text 904 and the time information 905 of the representative data 901 in the representative data list 124 in FIG. 9. At this time, in representative data 901 of representative data list 124 in FIG. 9, tag ID 902 passes the minority tag IDT being processed, and peak time slice 903 passes the peak time period H of the minority tag IDT being processed.

[0068] When the processes of steps S1709 to S1713 are executed for the peak time periods H of all the minority tag IDTs, the data extraction unit 103 exits from the loop process H (step S1714).

[0069] When the processing of steps S1704 to S1714 is executed for all minority tags IDT, the data extraction unit 103 exits from the loop processing T (step S1715).

[0070] Next, the data extracting unit 103 stores the representative data list 124 in the representative data storage unit 113 (step S1716).

[0071] In this way, by performing the processes shown in Figures 17A and 17B, it is possible to extract data representative of a data group to which the same tag is assigned from among the minority tags extracted in Figure 8. That is, based on the word scores according to the frequency of occurrence in a time series of words appearing in a data group to which the same tag is assigned from among the minority tags extracted in Figure 8, data representative of a data group to which the same tag is assigned can be determined and provided.

[0072] As described above, in this embodiment, an input unit (e.g., text input unit 101) that accepts input of tag ID-assigned data to which one or more tag IDs have been assigned (e.g., tag ID-assigned text list 121) and a data group in which the tag ID-assigned data is associated with time information (e.g., time information 303) at which the tag ID-assigned data was created is included, and for each tag ID included in the tag ID-assigned data and for each time slice (e.g., time slice 702 in timeline 70) that is a time period obtained by dividing a timeline (e.g., timeline 70 for each tag ID) that is a time period including the time information by a predetermined time width, counts the number of tag ID-assigned data to which a tag identified by the tag ID that occurs in each time slice is assigned (e.g., number of items in timeline 70) and if the counted number is greater than a predetermined number threshold (e.g., maximum number of items in the timeline) (e.g., step S806; True) and the proportion of the time slices in which the number does not meet a predetermined criterion (e.g., proportion of time slices with 0 items) is greater than a predetermined proportion threshold (step S807;a tag extraction unit that extracts the tag as a useful minority tag having a minority trend, and for a minority tag ID for identifying the extracted minority tag, for each peak time period of the minority tag ID, an appearance rate (e.g., appearance rate 1103 in word appearance rate list 110) of a word (e.g., used word 1004 in minority tag ID assigned text list 100) used in the minority tag ID assigned data obtained by analyzing the minority tag ID assigned data to which the minority tag identified by the minority tag ID is assigned, and and a data extraction unit that determines, as representative data among the minority tag ID-assigned data, data (e.g., text 1602 in text score list 160) whose score satisfies a predetermined standard (e.g., score 1604 is maximum) and whose appearance rate (e.g., appearance rate 1403 in word appearance rate list 140) of words (e.g., used words 1304 in peak time period text list 130) used in the minority tag ID-assigned data obtained by analyzing the minority tag ID-assigned data in the time slice in which the number of the minority tag ID-assigned data reaches a peak (or each peak if there are multiple peaks). Thus, when dealing with tagged data, it is possible to extract minority tags that are particularly assigned to data groups that mention the same topic from among the minority tags, and also to extract representative data from a data group to which the same tag is assigned among the extracted tags. EXAMPLES

[0073] In the second embodiment, an example will be described in which, instead of receiving the tag ID-added text list 121 as an input, a tag is added to a text list to which no tag is added, thereby acquiring the tagged text list 121. The processing of the tag extraction unit 102 and the data extraction unit 103 is similar to that of the first embodiment. The following mainly describes the configuration and operation that are different from the first embodiment.

[0074] FIG. 18 is a diagram illustrating an example of a functional configuration of a data extraction device 10 according to a second embodiment. As illustrated in FIG. 18, the data extraction device 10 according to the second embodiment further includes a text data acquisition unit 105 and a text data classification unit 106 in addition to the configuration of the data extraction device 10 of the first embodiment. The information storage unit 104 of the second embodiment further includes a text list storage unit 114 in addition to the configuration of the information storage unit 104 of the first embodiment. The data extraction device 10 is communicably connected to a text medium 107 that stores text data. The text medium 107 is, for example, a web server that stores web information indicating a website such as a microblog.

[0075] The information storage unit 104 may further store information referred to and generated by the text data acquisition unit 105 and the text data classification unit 106 in addition to the text input unit 101, the tag extraction unit 102, and the data extraction unit 103. For example, these include a query 125 (FIG. 19), a text list 126 (FIG. 20), and a tag ID list 127 (FIG. 21), which will be described later.

[0076] When the text data acquisition unit 105 reads the query 125 from the information storage unit 104 , it transmits the query 125 to the text medium 107 , receives the text list 126 corresponding to the query 125 , and stores it in the text list storage unit 114 .

[0077] Fig. 19 is a diagram showing an example of a query 125 according to the second embodiment. The query 125 shown in Fig. 19 is a conditional expression for acquiring a desired text list 126 from text data stored in the text medium 107, and is written according to a format determined by the text medium 107. The query 125 may be, for example, a search term or a logical expression of a search term used in a general text medium 107, or may include attribute information of the text data such as a search period in addition to the search term. In addition, if the text medium 107 has a means for providing text data other than the query 125, the text list 126 may be acquired by the providing means.

[0078] Fig. 20 is a diagram illustrating an example of the text list 126 according to the second embodiment. The text record 2001 illustrated in Fig. 20 is included as a record, and the text record 2001 includes text 2002 and time information 2003. Note that the text record 2001 may include information other than the text 2002 and the time information 2003. Fig. 20 illustrates, for example, that text 1 is a text created at the time "2020 / 07 / 23 17:58:36".

[0079] The text data classification unit 106 creates a tag ID-assigned text list 121 based on the text list 126 read from the text list storage unit 114 and while referring to the tag ID list 127 read from the information storage unit 104, and stores the tagged text list 121 in the tagged text list storage unit 111.

[0080] Fig. 21 is a diagram showing an example of tag ID list 127. Tag ID list 127 shown in Fig. 21 has a tag ID record 2101 as a record, and includes a tag ID 2102 and a tag name 2103. Fig. 21 shows, for example, that the tag name of a tag identified by the tag ID "tag ID001" is "tag name 1."

[0081] Text list storage section 114 stores text list 126 received from text data acquisition section 105. Information storage section 104 may further store information referenced and generated by text data acquisition section 105 and text data classification section 106, and the like.

[0082] 22 is a flowchart for explaining an example of the operation of the text data acquisition unit 105 according to the second embodiment. First, the text data acquisition unit 105 reads the query 125 from the information storage unit 104 (step S2201). Then, the text data acquisition unit 105 transmits the query 125 to the text medium 107 (step S2202). Then, the text data acquisition unit 105 receives the text list 126 corresponding to the query 125 from the text medium 107 and stores it in the text list storage unit 114 (step S2203).

[0083] 23 is a flowchart for explaining an example of the operation of the text data classification unit 106 according to the second embodiment. First, the text data classification unit 106 reads the text list 126 from the text list storage unit 114 and the tag ID list 127 from the information storage unit 104 (step S2301). Next, for all text records 2001 in the text list 126, the tag ID records 2101 having the tag name 2103 related to the text 2002 in the text record 2001 are extracted from the tag ID records 2101 in the tag ID list 127, and the tag ID 2102 of the tag ID records 2101 is assigned to the text record 2001 to obtain the tag ID assigned text list 121 (step S2302). Here, whether the tag name 2103 is related to the text 2002 may be determined based on whether the tag name 2103 is included as a character string in the text 2002, or may be determined by other methods. Also, assigning tag ID 2102 to text record 2001 means adding a column for tag ID 2102 to text list 126 and storing the tag ID 2102 assigned to each text record 2001 as its value, and the table obtained as a result is tag ID-assigned text list 121. Note that there may or may not be multiple tag ID records 2101 extracted for one text record 2001. Next, the created tag ID-assigned text list 121 is stored in tag ID-assigned text list storage unit 111 (step S2303).

[0084] As described above, in this embodiment, the input unit includes an acquisition unit (e.g., text data acquisition unit 105) that acquires an ID-unassigned data group in which the time information and the ID-unassigned data (e.g., text list 126) to which the tag ID is not assigned are associated from a text medium (e.g., text medium 107) connected to the data extraction system, and a classification unit (e.g., text data classification unit 106) that creates the data group from the ID-unassigned data group based on the ID-unassigned data group and a tag ID list (e.g., tag ID list 127) in which tag IDs and tag names are previously associated. Therefore, even if a tag ID is not previously assigned to text, tag ID-assigned data (e.g., tag ID-assigned text list 121) can be obtained. EXAMPLES

[0085] In the third embodiment, an example will be described in which the tag extraction unit 102 of the first or second embodiment extracts a minority tag ID list 123 having an element number that is a record number close to a value specified by a user. The configuration is the same as that of the first or second embodiment. Moreover, the processing other than that of the tag extraction unit 102 is the same as that of the first or second embodiment. The following mainly describes operations that differ from the first or second embodiment.

[0086] The tag extraction unit 102 of Example 3 reads the tag ID-assigned text list 121 from the tagged text list storage unit 111 and the tag extraction parameters 122 from the information storage unit 104, and stores in the extracted tag storage unit 112 a minority tag ID list 123 having a number of elements close to the setting value 503 of the target number of extracted tags 2411 described in the tag extraction parameters 122.

[0087] The information storage unit 104 may further store information referred to and generated by the text data acquisition unit 105 and the text data classification unit 106 in addition to the text input unit 101, the tag extraction unit 102, and the data extraction unit 103. For example, this is tag extraction parameters 122 (FIG. 24) in the third embodiment described later.

[0088] FIG. 24 is a diagram showing an example of the tag extraction parameters 122 according to the third embodiment. The tag extraction parameters 501 shown in FIG. 24 include a timeline number threshold initial value 2410, a time slice ratio threshold value 511, and a target number of tags to be extracted 2411. In the third embodiment, the timeline number threshold value 510 is searched for so that the number of elements in the minority tag ID list 123 is close to the setting value 503 of the target number of tags to be extracted 2411 while repeatedly trying to extract the minority tag ID list 123 by changing the value of the timeline number threshold value 510 in the first or second embodiment. The timeline number threshold initial value 2410 indicates the initial timeline number threshold value 510 for starting such a search. For example, the setting value 503 of the timeline number threshold value 2410 is set to a value such as "30". The target number of tags to be extracted 2411 indicates the number of elements in the minority tag ID list 123 to be extracted. For example, the setting value 503 of the target number of tags to be extracted 2411 is set to a value such as "10."

[0089] 25A and 25B are flowcharts for explaining an example of the operation of the tag extraction unit 102 according to the third embodiment. First, the tag extraction unit 102 reads the tag ID-added text list 121 and the tag extraction parameters 122, and sets the setting value 503 of the timeline number threshold initial value 2410 of the tag extraction parameters 122 to U_init, the setting value 503 of the time slice ratio threshold 511 to V, and the setting value 503 of the target number of extracted tags 2411 to W_goal (step S2501). Next, the variable U representing the timeline number threshold 510 is initialized with U_init (step S2502).

[0090] Next, an extracted tag number determination index F to be used for determining the condition to exit the processing loop from step S2504 to step S2511 that starts immediately after this step is initialized to an empty set (step S2503). Next, steps S2504 to S2511 are repeated as loop processing L until the number of elements of extracted tag number determination index F becomes 2 or more (step S2504). The number of elements is determined to be 2 or more if, for example, there are both a case where a small number of tags are obtained for the target number of extracted tags "10" and a case where a large number of tags are obtained for the target number of extracted tags "10".

[0091] In the loop process L, first, the variable S_current representing the minority tag ID list 123 is initialized to an empty list (step S2505). Then, in the loop process T of steps S803 and S804 to S809 in the first or second embodiment, the same process is performed in which the variable S corresponding to the minority tag ID list 123 is replaced with the variable S_current (steps S803 to S809). The value of S_current is the minority tag ID list 123 extracted corresponding to the value of the variable U used in the current loop process.

[0092] In the loop process L, the number of elements of the variable S_current representing the minority tag ID list 123 is set to W_current (step S2506). Then, the values ​​of W_current and W_goal are compared (step S2507). If W_current is larger (step S2507; Yes), the value 1 is added to the extracted tag number judgment index F, the value of the variable U is incremented by 1, and the process proceeds to the next step S2510 (step S2508). Here, if the extracted tag number judgment index F already contains the value 1, no value is added to the extracted tag number judgment index F. Here, the addition of the value 1 to the extracted tag number judgment index F means that the number of elements of the minority tag ID list 123 extracted in the previous loop process L has been greater than the value of W_goal. The reason why the value of the variable U is incremented by 1 is because it is desired to extract the minority tag ID list 123 with a smaller number of elements by trying to extract the minority tag ID list 123 under stricter conditions in the next loop process L.

[0093] On the other hand, if W_current is not greater (step S2507; No), add the value -1 to the extracted tag number determination index F, decrement the value of variable U by 1, and proceed to the next step S2510 (step S2509). Here, if the extracted tag number determination index F already contains the value -1, no value is added to the extracted tag number determination index F. Also, the addition of the value -1 to the extracted tag number determination index F means that the number of elements in the minority tag ID list 123 extracted in the previous loop processing L has taken a value not greater than the value of W_goal. Also, the value of variable U is decremented because we want to extract a minority tag ID list 123 with a greater number of elements by attempting to extract the minority tag ID list 123 under looser conditions in the next loop processing L.

[0094] In the loop process L, the variable S_prev is set to the value of S_current and the variable W_prev is set to the value of W_current in order to hold the current variable values ​​in the next loop process L (step S2510). When the processes of steps S2505 to S2510 are executed until the number of elements of the extracted tag number determination index F becomes 2 or more, the tag extraction unit 102 exits the loop process L (step S2511). Note that in the loop process L, an upper limit on the number of loops may be set to ensure that an infinite loop does not occur.

[0095] Next, the absolute value of the difference between W_prev and W_goal is compared with the absolute value of the difference between W_current and W_goal (step S2512). If the absolute value of the difference between W_prev and W_goal is greater (step S2512; Yes), the value of S_current is set to the variable S representing the finally extracted minority tag ID list 123, and the process proceeds to step S810 (step S2513). If the absolute value of the difference between W_prev and W_goal is smaller (step S2512; No), the value of S_prev is set to the variable S representing the finally extracted minority tag ID list 123, and the process proceeds to step S810 (step S2514).

[0096] The processing in steps S2513 and S2514 corresponds to a process of selecting, from S_prev and S_current, the one whose number of elements is closer to W_goal, as the finally extracted minority tag ID list 123. Next, the value of the variable S representing the minority tag ID list 123 is stored in the tag extraction storage unit 112 (step S810).

[0097] As described above, in this embodiment, the tag extraction unit uses the number of extracted minority tags (for example, the number of elements in the minority tag ID list 123) and the target number of extracted minority tags (for example, the target number of extracted tags 2411) to extract the minority tags by increasing the predetermined number threshold value to a value greater than the current value if the current number of minority tags is greater than the target number (for example, step S2507; Yes and step S2508), and by repeating the process of extracting the minority tags by decreasing the predetermined number threshold value to a value less than the current value if the current number of minority tags is not greater than the target number (for example, step S2507; No and step S2509), extracts the minority tags in a number close to the target number by a certain amount (for example, step S2512 to step S2514). Therefore, it is possible to extract a user-desired number of minority tags while searching for the timeline number threshold value. EXAMPLES

[0098] In the fourth embodiment, an example will be described in which the tag extraction unit 102 of the first, second or third embodiment rearranges the extracted minority tag ID list 123 in order of priority before storing it. The configuration and processing other than the tag extraction unit 102 are the same as those of the first, second and third embodiments. The following mainly describes operations that differ from those of the first, second or third embodiment.

[0099] The tag extraction unit 102 according to the fourth embodiment reads the tag ID assigned text list 121 from the tagged text list storage unit 111 and the tag extraction parameters 122 from the information storage unit 104, extracts the minority tag ID list 123 sorted in order of priority, and stores it in the extracted tag storage unit 112.

[0100] The information storage unit 104 may further store information referred to and generated by the text data acquisition unit 105 and the text data classification unit 106 in addition to the text input unit 101, the tag extraction unit 102, and the data extraction unit 103. For example, the information includes a tag similarity matrix 270 (FIG. 27) and a minority tag ID score list 280 (FIG. 28) described later.

[0101] Fig. 26 is a flowchart for explaining an example of the operation of the tag extraction unit 102 according to the fourth embodiment. The process up to step S809 is the same as that of Fig. 8. Next, the tag extraction unit 102 rearranges the elements of the variable S corresponding to the minority tag ID list 123 in order of priority (step S2601). An example of the operation of step S2601 will be described with reference to the flowchart of Fig. 29. Next, the same process as step S810 of Fig. 8 is performed.

[0102] FIG. 27 is a diagram showing an example of a tag similarity matrix 270 for scoring elements of the variable S corresponding to the minority tag ID list 123 in step S2601. The tag similarity matrix 270 shown in FIG. 27 includes a horizontal axis 2701, a vertical axis 2702, and a cell 2703. The horizontal axis 2701 and the vertical axis 2702 contain elements of the variable S representing the minority tag ID list 123. The value of a cell 2703 located at the intersection of a column C corresponding to a certain tag ID 2102 on the horizontal axis 2701 and a row R corresponding to a certain tag ID 2102 on the vertical axis 2702 represents the similarity between the tag name 2103 corresponding to the tag ID 2102 in the column C and the tag name 2103 corresponding to the tag ID 2102 in the row R. Here, the similarity of the tag name 2103 may be calculated using a commonly known cosine similarity or thesaurus information in which concepts are hierarchically defined. FIG. 27 shows that the similarity between a tag identified by "tag ID 001" and a representative tag identified by "tag ID 033" is "0.36", for example.

[0103] 28 is a diagram showing an example of minority tag ID score list 280. Minority tag ID score list 280 shown in FIG.

[0104] FIG. 29 is a flowchart for explaining an example of the operation of the minority tag ID list rearrangement process (step S2601) in the tag extraction unit 102 according to the fourth embodiment. First, the tag extraction unit 102 reads the tag similarity matrix 270 (step S2901). Next, the tag extraction unit 102 extracts the tag ID 2102 included in the most tagged texts 301 from the tag ID-assigned text list 121 as the representative tag T (step S2902). Note that the representative tag T may be extracted by a method other than the above. For example, if there is a tag ID 2102 having a query 125 in the tag name 2103, the query 125 may be the representative tag T.

[0105] Next, the tag extraction unit 102 extracts the value of the cell 2703 at the intersection of the column whose horizontal axis 2701 value is the representative tag T and the row whose vertical axis 2702 value is the minority tag ID 601 in the tag similarity matrix 270, and sets this value as the score 2803 of the minority tag ID score 2801 corresponding to the minority tag ID 601 in the minority tag ID score list 280, thereby creating the minority tag ID score list 280 (step S2903). Next, the elements of the variable S corresponding to the minority tag ID list 123 are sorted in descending order of the score 2803 in the minority tag ID score list 280 (step S2904).

[0106] As described above, in this embodiment, the tag extraction unit extracts the tag of the minority tag ID included in the largest number of the tag ID-added data from among the minority tag IDs for identifying the extracted minority tags as a representative tag, and rearranges the extracted minority tags in order of high similarity to the extracted representative tag (for example, in the order shown in the minority tag ID score list 280). Therefore, the visibility of the list of extracted minority tags can be improved. [Explanation of symbols]

[0107] 10 Data Extraction Device 11 Processors 12 Main storage 13 Auxiliary storage device 14 Input Devices 15 Output Devices 16 Communication Equipment 101 Text input section 102 Tag Extraction Unit 103 Data Extraction Unit 104 Information storage section 105 Text data acquisition unit 106 Text Data Classification Unit

Claims

1. an input unit that receives an input of a data group in which tag ID-added data and time information of the tag ID-added data are associated with each other; a tag extraction unit that counts the number of tag ID-assigned data items to which a tag identified by the tag ID that occurs in each time slice is assigned for each tag ID included in the tag ID-assigned data and for each time slice obtained by dividing a timeline including the time information into a predetermined time interval, and extracts the tag as a minority tag that is useful and has a small number of trends if the counted number is greater than a predetermined number threshold and the proportion of the time slices in which the number does not meet a predetermined standard is greater than a predetermined proportion threshold; a data extraction unit that determines, for a minority tag ID for identifying the extracted minority tag, data as representative data among the minority tag ID-assigned data, the score obtained from: an appearance rate of words used in the minority tag ID-assigned data obtained by analyzing the minority tag ID-assigned data to which the minority tag identified by the minority tag ID is assigned for each peak time period of the minority tag ID; and an appearance rate of words used in the minority tag ID-assigned data obtained by analyzing the minority tag ID-assigned data in the time slice in which the number of minority tag ID-assigned data including the minority tag identified by the minority tag ID peaks in the timeline for the minority tag ID, and satisfies a predetermined criterion; A data extraction system comprising:

2. 2. The data extraction system according to claim 1, The input unit includes an acquisition unit that acquires an un-ID-assigned data group in which the time information is associated with the un-ID-assigned data to which the tag ID is not assigned from a text medium connected to the data extraction system; a classification unit that creates the data group from the unid-assigned data group based on the unid-assigned data group and a tag ID list in which tag IDs are previously associated with tag names; A data extraction system comprising:

3. 2. The data extraction system according to claim 1, the tag extraction unit uses the number of extracted minority tags and a target number of extracted minority tags to repeat a process of extracting the minority tags by making the predetermined number threshold larger than the current value if the current number of minority tags is larger than the target number, and a process of extracting the minority tags by making the predetermined number threshold smaller than the current value if the current number of minority tags is not larger than the target number, thereby extracting the minority tags in a number that is closer to the target number by a certain amount. A data extraction system comprising:

4. 2. The data extraction system according to claim 1, the tag extraction unit extracts, as a representative tag, a tag having a minority tag ID included in the largest number of the tag ID-assigned data from among the minority tag IDs for identifying the extracted minority tags, and rearranges the extracted minority tags in order of similarity to the extracted representative tag. A data extraction system comprising:

5. 1. A computer-implemented method for data extraction, comprising: The input unit receives an input of a data group in which the tag ID-added data and time information of the creation of the tag ID-added data are associated with each other, a tag extraction unit counts the number of tag ID-assigned data items to which a tag identified by the tag ID occurring in each time slice is assigned, for each tag ID included in the tag ID-assigned data and for each time slice obtained by dividing a timeline including the time information into a predetermined time width, and if the counted number is greater than a predetermined number threshold and the proportion of the time slices in which the number does not meet a predetermined criterion is greater than a predetermined proportion threshold, extracts the tag as a minority tag that is useful and has a small number of trends; the data extraction unit determines, for a minority tag ID for identifying the extracted minority tag, data that satisfies a predetermined criterion as representative data among the minority tag ID-assigned data, the score obtained from: an appearance rate of words used in the minority tag ID-assigned data obtained by analyzing the minority tag ID-assigned data to which the minority tag identified by the minority tag ID is assigned for each peak time period of the minority tag ID; and an appearance rate of words used in the minority tag ID-assigned data obtained by analyzing the minority tag ID-assigned data in the time slice in which the number of minority tag ID-assigned data including the minority tag identified by the minority tag ID peaks in the timeline for the minority tag ID. A data extraction method comprising:

6. 6. The data extraction method according to claim 5, an acquisition unit of the input unit acquires, from a text medium connected to the computer, a group of un-ID-assigned data in which the time information is associated with un-ID-assigned data to which the tag ID is not assigned; a classification unit of the input unit creates the data group from the ID-unassigned data group based on the ID-unassigned data group and a tag ID list in which tag IDs are previously associated with tag names; A data extraction method comprising:

7. 6. The data extraction method according to claim 5, the tag extraction unit uses the number of extracted minority tags and a target number of extracted minority tags to repeat a process of extracting the minority tags by making the predetermined number threshold larger than the current value if the current number of minority tags is larger than the target number, and a process of extracting the minority tags by making the predetermined number threshold smaller than the current value if the current number of minority tags is not larger than the target number, thereby extracting the minority tags in a number that is closer to the target number by a certain amount. A data extraction method comprising:

8. 6. The data extraction method according to claim 5, the tag extraction unit extracts, as a representative tag, a tag having a minority tag ID included in the largest number of the tag ID-assigned data from among the minority tag IDs for identifying the extracted minority tags, and rearranges the extracted minority tags in order of similarity to the extracted representative tag. A data extraction method comprising:

Images