Information processing system, information processing method, and information processing program
The information processing system optimizes language analysis by using multiple accuracy levels to efficiently process location and text information during disasters, addressing resource and time constraints.
Patent Information
- Application Number
- JP2021061285
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-31
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2041-03-31
AI Technical Summary
Existing language analysis processing based on deep learning during disasters requires significant computational resources and time, especially when using general-purpose hardware, and there is a need to quickly process large amounts of text to take timely action.
An information processing system that includes a processing result database, area determination means, and control means to efficiently analyze text using multiple analysis processes with varying accuracy levels, prioritizing faster but less accurate processes when resources are limited, and switching to more accurate processes when feasible.
Enables efficient processing of location and text information during disasters, balancing speed and accuracy to meet the demands of rapid information dissemination and analysis.
Smart Images

Figure 0007766901000001 
Figure 0007766901000002 
Figure 0007766901000003
Abstract
Description
[Technical Field]
[0001] The present invention relates to an information processing system, an information processing method, and an information processing program. [Background technology]
[0002] With the rapid spread of mobile devices such as smartphones, social networking services (SNS) are being used for various purposes as a means for anyone to easily disseminate information. One of the uses of SNS that has attracted attention is information gathering during disasters.
[0003] The information that should be collected during a disaster includes not only the details of the event that has occurred, but also the location where the event is occurring. However, in order to protect the privacy and personal information of the sender, the location information held by the sender's mobile device is not used, and the method used is to identify the target location solely by analyzing the location expression explicitly sent by the user.
[0004] Natural language processing is used to analyze the text contained in the message and extract the content of the event. For example, by utilizing a language model based on deep learning as disclosed in Non-Patent Document 1, higher accuracy can be achieved. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," Proceedings of NAACL-HLT 2019, pages 4171-4186 Summary of the Invention [Problem to be solved by the invention]
[0006] To realize language analysis processing based on deep learning, it is necessary to perform a huge amount of calculations both during training and execution, and dedicated hardware resources such as a GPU (Graphics Processing Unit) or a neurochip are often used. Therefore, when language analysis processing based on deep learning is adopted, costs increase compared to when it is realized using general-purpose hardware resources such as a CPU (Central Processing Unit).
[0007] In particular, during disasters, a large amount of information is disseminated and a huge amount of text needs to be processed, but at the same time, there is a need to quickly take action and take countermeasures, so there is also a demand to complete the analysis process within a limited processing time.
[0008] An object of the present invention is to provide a technique for efficiently processing an object containing location information and text. [Means for solving the problem]
[0009] An information processing system according to one embodiment includes a processing result database that stores processing results of a processing object including location information and text, an area determination means that determines a unit area corresponding to the location information included in the processing object, an analysis means that analyzes the text using a specified analysis process from among a plurality of analysis processes including a first analysis process and a second analysis process that is more accurate than the first analysis process, and outputs the processing result, and a control means that, after analyzing the text using the first analysis process to obtain the first processing result, if a processing result of another processing object has already been stored for the determined unit area, adds the first processing result to the processing result database, and if a processing result of another processing object has not been stored for the determined unit area, analyzes the text using the second analysis process to obtain the second processing result and stores it in the processing result database.
[0010] The plurality of analysis processes may further include a third analysis process that is more accurate than the first analysis process but less accurate than the second analysis process. If the load state is such that the second analysis process cannot be executed, the control means may analyze the text using the third analysis process instead of the second analysis process.
[0011] The processing result may include a phrase, which is a character string indicating the information to be extracted, and semantic information indicating the meaning of the phrase.
[0012] The information management system may further include a processing status database that stores the processing status of the analysis processing for each unit area. The processing status may be set to one of a state in which the analysis processing has been executed and the processing result has been obtained, a state in which the analysis processing has been executed and the processing result has not been obtained, and a state in which the analysis processing has not yet been executed.
[0013] The processing status may include information specifying the type of analysis processing that has been executed. The control means may additionally execute an analysis processing that is more accurate than the analysis processing that has been used to obtain the processing result for the arbitrary unit area.
[0014] If the processing status for the determined unit area does not exist in the processing status database, the control means may search for the processing status for other unit areas that exist within a predetermined range from the determined unit area.
[0015] Even when the control means is unable to obtain the first processing result through the first analysis process, the control means may analyze the text through the second analysis process and obtain the second processing result.
[0016] The location information may include at least one of location information indicated by latitude and longitude and location information indicated in the UTM (Universal Transverse Mercator) coordinate system.
[0017] An information processing method according to another embodiment includes the steps of receiving a processing object including location information and text, determining a unit area corresponding to the location information included in the processing object, analyzing the text using a first analysis process to obtain a first processing result, determining whether a processing result of another processing object for the determined unit area has already been stored in the processing result database, and adding the first processing result to the processing result database if a processing result of another processing object for the determined unit area has already been stored in the processing result database, and if a processing result of another processing object for the determined unit area has not been stored in the processing result database, analyzing the text using a second analysis process that is more accurate than the first analysis process to obtain a second processing result and storing the second processing result in the processing result database.
[0018] According to yet another aspect, there is provided an information processing program for causing a computer to execute the above information processing method. [Effects of the Invention]
[0019] According to the present invention, it is possible to efficiently process a processing object including position information and text. [Brief explanation of the drawings]
[0020] [Figure 1] 1 is a schematic diagram showing an example of a system configuration of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing an example of a dialogue between a user of the mobile terminal shown in FIG. 1 and a chatbot. [Figure 3] FIG. 2 is a schematic diagram showing an example of a hardware configuration of an analysis device that constitutes the information processing system according to the present embodiment. [Figure 4] FIG. 10 is a schematic diagram illustrating an analysis process provided by the information processing system according to the present embodiment. [Figure 5] FIG. 2 is a schematic diagram showing an example of a processing result database generated by the information processing system according to the present embodiment. [Figure 6]FIG. 10 is a schematic diagram showing another example of the processing result database generated by the information processing system according to the present embodiment. [Figure 7] FIG. 2 is a schematic diagram showing an example of a processing state database generated by the information processing system according to the present embodiment. [Figure 8] 10A to 10C are diagrams illustrating an example of differences in processing results due to differences in analysis processing programs in the information processing system according to the present embodiment. [Figure 9] 10 is a flowchart showing an example of a processing procedure for analysis processing provided by the information processing system according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION
[0021] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings, in which the same or corresponding parts in the drawings are designated by the same reference numerals and the description thereof will not be repeated.
[0022] [A. System Configuration] Fig. 1 is a schematic diagram showing an example of a system configuration of an information processing system 1 according to the present embodiment. Referring to Fig. 1, information processing system 1 includes an analysis device 100 and an SNS server 200. For ease of explanation, the following example shows a configuration in which analysis device 100 and SNS server 200 are separated, but the two may also be integrated.
[0023] The SNS server 200 collects messages exchanged with the mobile terminal 300. Note that the messages may include any content such as still images and moving images in addition to text.
[0024] The SNS server 200 has a chatbot 250. The chatbot 250 behaves as a virtual user of the SNS, interacts with the user of the mobile terminal 300, and collects messages from the user. Typically, the chatbot 250 engages in an interaction to collect information (hereinafter also referred to as "report content") in the event of a disaster.
[0025] The analysis device 100 analyzes the report content 50 generated from one or more messages collected in the SNS server 200 by the chatbot 250 or the like, and outputs a processing result (processing result 71 described below). For example, the processing result includes information such as an event that has occurred and the location where the event has occurred.
[0026] Fig. 2 is a schematic diagram showing an example of a dialogue between the user of the mobile terminal 300 shown in Fig. 1 and the chatbot 250. Referring to Fig. 2, in response to a message from the user of the mobile terminal 300, the chatbot 250 transmits a message prompting input or a message for making an additional inquiry.
[0027] The user of the mobile terminal 300 reports the situation and damage caused by the disaster. When making this report, any text indicating the details of the situation and damage, location information (e.g., latitude and longitude information) related to the report, and still images and videos indicating the situation and damage are transmitted. Note that still images and videos do not have to be attached. Through the exchanges shown in FIG. 2, report contents 50 are collected in the SNS server 200.
[0028] The information processing system 1 according to the present embodiment processes report content 50 including location information and text. That is, as a whole process executed by the information processing system 1, the processing results of processing the location information and text are stored in a database or the like (a processing result database 70 and a processing status database 80 described below). More specifically, as shown in FIGS. 1 and 2, the information processing system 1 analyzes report content 50 (typically, reports of damage in the event of a disaster) collected by a chatbot 250 on an SNS, for example, to extract information, and stores the extracted information in a database.
[0029] Typically, the result of processing the text is a character string (phrase 74 described below) that describes the required information. Note that the character string that is output may include not only the information contained in the text, but also information that represents some meaning determined by referring to a dictionary or the like, predetermined symbols, etc.
[0030] Based on the information contained in the output strings, a procedure for determining the semantic distance between strings or whether they are synonymous can be defined. Furthermore, the analysis device 100 extracts substrings used to determine the semantic distance within the strings as semantic flags (corresponding to phrase types 75 and semantic categories 76 described below). The semantic flags correspond to semantic information indicating the meaning of the strings (phrases 74).
[0031] [B. Hardware configuration example] 3 is a schematic diagram showing an example of a hardware configuration of analysis device 100 constituting the information processing system according to the present embodiment. Typically, analysis device 100 can be realized using a general-purpose computer.
[0032] 3, analysis device 100 includes, as its main hardware components, a CPU 102, a GPU 104, a main memory 106, a display 108, a network interface (I / F) 110, an input device 112, an optical drive 114, and a secondary storage device 120. These components are connected to one another via an internal bus 118.
[0033] The CPU 102 and / or the GPU 104 are processors that execute the information processing method according to the present embodiment. A plurality of CPUs 102 and GPUs 104 may be provided, and each may have a plurality of cores.
[0034] The main memory 106 is a storage area that temporarily stores (or caches) program code, work data, etc. when the processor (CPU 102 and / or GPU 104) executes processing, and is composed of volatile memory devices such as DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory).
[0035] The display 108 is a display unit that outputs a user interface related to processing, processing results, etc., and is configured, for example, by an LCD (liquid crystal display) or an organic EL (electroluminescence) display.
[0036] Network interface 110 exchanges data with any information processing device on the Internet or an intranet. In this embodiment, network interface 110 is responsible for transmitting report content 50 from SNS server 200 to analysis device 100. Any communication method such as Ethernet (registered trademark), wireless LAN (local area network), or Bluetooth (registered trademark) can be used as network interface 110.
[0037] The input device 112 is a device that accepts instructions and operations from the user, and is configured by, for example, a keyboard, a mouse, a touch panel, a pen, and the like.
[0038] The optical drive 114 reads information stored on an optical disk 116, such as a CD-ROM (compact disc read only memory) or a DVD (digital versatile disc), and outputs the information to other components via the internal bus 118. The optical disk 116 is an example of a non-transitory recording medium, and is distributed with any program stored therein in a non-volatile manner. The optical drive 114 reads the program from the optical disk 116 and installs it in a secondary storage device 120 or the like, causing the computer to function as the analysis device 100. Therefore, the subject matter of the present invention may also be the program itself installed in a secondary storage device 120 or the like, or a recording medium such as the optical disk 116 that stores a program for implementing the functions and processing according to the present embodiment.
[0039] FIG. 3 shows an optical recording medium such as an optical disk 116 as an example of a non-transitory recording medium, but the present invention is not limited to this. Alternatively, a semiconductor recording medium such as a flash memory, a magnetic recording medium such as a hard disk or storage tape, or a magneto-optical recording medium such as an MO (magneto-optical disk) may be used.
[0040] Secondary storage device 120 stores programs and data necessary for a computer to function as analysis device 100. For example, it is configured by a non-volatile storage device such as a hard disk or SSD (solid state drive).
[0041] More specifically, the secondary storage device 120 stores an OS (operating system) (not shown), as well as a process management program 122 for managing the execution of the analysis process, a reverse geocoding program 124 for determining a corresponding address from a geocode, and a first analysis process program 126, a second analysis process program 128, and a third analysis process program 130 for implementing the analysis process. The secondary storage device 120 may also have a process result database 70 and a process status database 80 formed therein.
[0042] FIG. 3 shows an example in which analysis device 100 is configured using a single computer, but this is not limiting; multiple computers connected via a computer network may work together explicitly or implicitly to realize the information processing method according to this embodiment.
[0043] All or part of the functions realized by the processor (CPU 102 and / or GPU 104) executing the program may be realized using a hard-wired circuit such as an integrated circuit, for example, an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).
[0044] A person skilled in the art would be able to realize analysis device 100 according to this embodiment by appropriately using technology appropriate for the era in which the present invention is implemented.
[0045] [C. Overview of analysis process] Next, an outline of the analysis process provided by the information processing system 1 according to the present embodiment will be given.
[0046] Fig. 4 is a schematic diagram showing the analysis process provided by information processing system 1 according to the present embodiment. Referring to Fig. 4, analysis device 100 analyzes report content 50 acquired from SNS server 200 and stores the processing results in processing result database 70 and processing status database 80. Analysis device 100 analyzes report content 50 including location information (e.g., latitude and longitude information) and text at higher speed.
[0047] The SNS server 200 may provide the report content 50 to the analysis device 100 in, for example, a JSON format or an XML format. The report content 50 may be provided to the process management program 122 via an API (Application Programming Interface) implemented in the analysis device 100. The report content 50 includes, for example, identification information 51, user information 52, latitude and longitude information 53, text 54, and photo attachment attributes 55.
[0048] The processing management program 122 selectively executes the first analysis processing program 126, the second analysis processing program 128, and the third analysis processing program 130 for each of the report contents 50 to generate processing results 71. Note that for the same report contents 50, one or more of the first analysis processing program 126, the second analysis processing program 128, and the third analysis processing program 130 may be executed.
[0049] The first analysis processing program 126, the second analysis processing program 128, and the third analysis processing program 130 differ in processing speed and accuracy. For example, the first analysis processing program 126 is capable of relatively high-speed processing, but has relatively low accuracy. The third analysis processing program 130 has a relatively low processing speed, but has relatively high accuracy. The processing speed and accuracy of the second analysis processing program 128 are both between those of the first analysis processing program 126 and the third analysis processing program 130.
[0050] In this way, the information processing system 1 (analysis device 100) has analysis means that analyzes the text 54 included in the report content 50 by a specified analysis process among a plurality of analysis processes including an analysis process by the first analysis process program 126 and an analysis process by the third analysis process program 130, which has higher accuracy than the analysis process by the first analysis process program 126, and outputs the processing result 71. Furthermore, the plurality of analysis processes to be selected may also include an analysis process by the second analysis process program 128, which has higher accuracy than the analysis process by the first analysis process program 126 but lower accuracy than the analysis process by the third analysis process program 130.
[0051] In terms of language analysis processing, the first analysis processing program 126, the second analysis processing program 128, and the third analysis processing program 130 all execute processing to analyze the meaning of text and output processing results 71. Although the content of the analysis processing executed by each analysis processing program differs, the format of the output processing results 71 is configured to be the same for all of them.
[0052] For example, the first analysis processing program 126 extracts information based on a rule base, the second analysis processing program 128 extracts information using an SVM (Support Vector Machine), and the third analysis processing program 130 extracts information using deep learning.
[0053] It is not necessary to prepare three types of analysis processes as shown in Fig. 4, and only two types of analysis processes may be prepared, or more types of analysis processes may be prepared. In other words, multiple types of analysis processes with different processing speeds and accuracies are prepared, and these analysis processes are ordered according to a predetermined criterion.
[0054] Each of the processing results 71 stored in the processing result database 70 includes, for example, a unit area 72 corresponding to the report content 50, latitude and longitude information 73 corresponding to the report content 50, an extracted phrase 74, a phrase type 75 of the extracted phrase 74, a semantic category 76 of the extracted phrase 74, a photo attachment attribute 77 of the corresponding report content 50, and identification information 78 of the corresponding report content 50. The processing result 71 may be output as a string that summarizes all of the information.
[0055] Here, a "phrase" refers to a character string that corresponds to information to be collected from the text 54 included in the report content 50 (for example, an expression of the situation or damage caused by the disaster).
[0056] The value of the phrase type 75 may be, for example, one of "request / problem," "contradiction," or "countermeasure." The value of the semantic category 76 may be text that specifies the situation or damage caused by the disaster, such as "damage" or "fire." The phrase type 75 and semantic category 76 correspond to the semantic flag of the report content 50.
[0057] Thus, the processing result 71 includes a character string (phrase 74) indicating the information to be extracted, and semantic flags (phrase type 75 and semantic category 76) that are semantic information indicating the meaning of the phrase 74.
[0058] When multiple phrases 74 are extracted from one report content 50, a character string summarizing the unit area 72, phrase type 75, and semantic category 76 may be output for each extracted phrase 74. Furthermore, if no phrase 74 is extracted from the text 54 included in the report content 50, a character string indicating that no phrase was extracted may be output.
[0059] The reverse geocoding program 124 corresponds to an area determination means for determining a unit area corresponding to the location information included in the report content 50 to be processed. More specifically, the reverse geocoding program 124 outputs an address representation of the unit area 72 corresponding to the location information (latitude and longitude information 53) included in the report content 50. The unit area 72 output by the reverse geocoding program 124 may have any granularity. For example, the unit area may be an address representation in block units, or an area mesh such as a half area mesh (fourth-order mesh). The location information may be expressed in the Universal Transverse Mercator (UTM) coordinate system in addition to the latitude and longitude representation format. That is, the location information may include at least one of location information expressed in latitude and longitude and location information expressed in the UTM coordinate system. Thus, the location information may be expressed in any format as long as it can identify an arbitrary location on the Earth's surface.
[0060] The processing status database 80 stores a processing status 81 of the analysis process at least for each unit area. Each processing status 81 stored in the processing status database 80 indicates the processing status of the analysis process for the report content 50. More specifically, the processing status database 80 stores an execution status value 82 as the processing status 81 for each combination of unit area 72, phrase type 75, and semantic category 76.
[0061] The processing status 81 can be set to one of the following execution status values 82: "a state in which the analysis processing has been executed and the processing results have been obtained," "a state in which the analysis processing has been executed and the processing results have not been obtained," and "a state in which the analysis processing has not yet been executed."
[0062] More specifically, the execution status value 82 may store any one of "Processed with results (pn:id)," "Processed with no results (pn:id)," and "Unprocessed." "Processed with results (pn:id)" and "Processed with no results (pn:id)" mean that analysis processing has been performed on one or more report contents 50. On the other hand, "Unprocessed" means that analysis processing on the report contents 50 has not yet been performed.
[0063] The processing status 81 also includes information for identifying the type of analysis processing that has been executed. More specifically, the "pn" set in the execution status value 82 of the processing status 81 stores information for identifying the executed analysis processing program, such as "p1," "p2," or "p3."
[0064] The processing status 81 further includes information for identifying the processing target of the analysis process. More specifically, the "id" set in the execution status value 82 of the processing status 81 stores identification information 51 for identifying the target report content 50.
[0065] The execution status value 82 can be used in a process of extracting report contents 50 for which analysis processing should be prioritized, or in a process of extracting report contents 50 for which more accurate analysis processing should be performed.
[0066] [D. Processing Result Database 70 and Processing Status Database 80] Next, specific examples of the processing result database 70 and the processing status database 80 will be described.
[0067] Fig. 5 is a schematic diagram showing an example of a processing result database 70 generated by information processing system 1 according to the present embodiment. Fig. 5 shows an example in which processing result database 70 is implemented using a relational database.
[0068] 5, the processing result database 70 has processing results 71 as entries, and each of the processing results 71 is associated with identification information (id) 78. The processing result 71 includes a unit area 72, latitude and longitude information 73, a phrase 74, a phrase type 75, a semantic category 76, and a photo attachment attribute 77.
[0069] By employing the processing result database 70 using such a relational database, flexible queries can be realized using SQL.
[0070] Fig. 6 is a schematic diagram showing another example of a processing result database 70 generated by the information processing system 1 according to the present embodiment. Fig. 6 shows an example in which the processing result database 70 is implemented using a key-value store (KVS) database. In a key-value store database, an index corresponding to a key used for a search is prepared in advance.
[0071] Referring to FIG. 6, the processing result database 70 includes, for example, a main database 70A, an area index database 70B, and a category index database 70C.
[0072] The main database 70A has processing results 71 as entries. Each processing result 71 has identification information 78 set as a key, and stores a unit area 72, latitude and longitude information 73, phrase 74, phrase type 75, semantic category 76, and photo attachment attribute 77 as corresponding values.
[0073] In the area index database 70B, the unit area 72 is set as a key, and the identification information 78 is stored as a value.
[0074] In the category index database 70C, semantic categories 76 are set as keys, and identification information 78 is stored as values.
[0075] For example, by specifying identification information 78 in the main database 70A, it is possible to search for the corresponding unit area 72, latitude and longitude information 73, phrase 74, phrase type 75, semantic category 76, and photo attachment attribute 77. By specifying the unit area 72 in the area index database 70B, it is possible to search for the corresponding identification information 78. By specifying the semantic category 76 in the category index database 70C, it is possible to search for the corresponding identification information 78.
[0076] By performing set operations on the search results of multiple index databases, searches can be performed that include logical operations such as logical sum and logical product. In the example shown in Figure 6, an index search is performed using the area index database 70B and / or the category index database 70C to obtain a set of identification information 78 (id), and the obtained set of identification information 78 (id) is used to obtain the desired processing result 71 from the main database 70A.
[0077] Using such a key-value store database enables high-speed searches.
[0078] Fig. 7 is a schematic diagram showing an example of a processing status database 80 generated by information processing system 1 according to the present embodiment. Fig. 7 shows an example in which processing status database 80 is implemented using a key-value store database.
[0079] 7, the processing state database 80 has entries each containing a processing state 81. Each processing state 81 is set with a combination of a unit area 72, a phrase type 75, and a semantic category 76 as a key, and an execution state value 82 is stored as a corresponding value.
[0080] The databases are not limited to the implementation forms shown in FIGS. 5 to 7, and any implementation form can be adopted.
[0081] [E. Differences in accuracy of analysis processing programs] Next, the difference in accuracy between a plurality of analysis processing programs will be described.
[0082] Fig. 8 is a diagram showing an example of differences in processing results due to differences in the analysis processing programs in the information processing system 1 according to this embodiment. Fig. 8 shows an example of processing results obtained by executing analysis processes using the second analysis processing program 128 (SVM) and the third analysis processing program 130 (deep learning) on the same text 54.
[0083] As an example of the processing result, the phrase 74, phrase type 75, and semantic category 76 are output, but the present invention is not limited to this and more information may be output.
[0084] 8, for the text 54 "It looks like there was a fire," the second analysis processing program 128 extracts the phrase 74 "There is a fire," and outputs the phrase type 75 and semantic category 76 corresponding to the extracted phrase 74. In contrast, the third analysis processing program 130 does not extract the phrase 74. This is because the third analysis processing program 130 has determined that the uncertain information "It looks like there was a fire" is not information that should be collected, and the information processing system 1 considers the processing result of the third analysis processing program 130 to be correct processing.
[0085] Furthermore, for the text 54, "There is no way to say that a fire is breaking out," the second analysis processing program 128 extracts the phrase 74, "A fire is breaking out," and outputs the phrase type 75, "request / problem," and the semantic category 76, "disaster: fire," corresponding to the extracted phrase 74. In contrast, the third analysis processing program 130 extracts the same phrase 74, "A fire is breaking out," but outputs the phrase type 75, "contradiction," and the semantic category 76, "disaster: fire," corresponding to the extracted phrase 74. Since the text 54, "There is no way to say that a fire is breaking out," does not imply that a fire is actually breaking out, the value of the phrase type 75, "contradiction," output by the third analysis processing program 130, is the correct processing result.
[0086] In this way, the third analysis processing program 130 can achieve higher accuracy than the second analysis processing program 128 (and the first analysis processing program 126), but requires more processing time and resources.
[0087] [F. Analysis process details] Next, a more detailed procedure of the analysis process provided by the information processing system 1 according to the present embodiment will be described.
[0088] 9 is a flowchart showing an example of a processing procedure of analysis processing provided by information processing system 1 according to the present embodiment. Each step shown in FIG. 9 is typically realized by a processor (CPU 102 and / or GPU 104) of analysis device 100 executing a program including processing management program 122.
[0089] 9, analysis device 100 executes a process of receiving report content 50, which is a processing target and includes location information and text. More specifically, analysis device 100 determines whether or not report content 50 has been received from SNS server 200 (step S100). If report content 50 has not been received from SNS server 200 (NO in step S100), the process of step S100 is repeated.
[0090] If report content 50 has been received from SNS server 200 (YES in step S100), analysis device 100 decodes report content 50 so that it can be handled as an internal variable of the program (step S102). For example, if report content 50 in JSON format is received, values are stored in the following variables (character strings beginning with $):
[0091] $latlong=“(34.74529,135.76016)” $text="There's a fire" $pic="none" Next, the analysis device 100 executes a process to determine a unit area corresponding to the location information included in the report content 50 to be processed. More specifically, the analysis device 100 executes the reverse geocode program 124 to acquire an address notation corresponding to the latitude and longitude information 53 (value of $latlong) included in the report content 50 (step S104). For example, if the latitude and longitude information 53 indicates (34.74529, 135.76016), the address notation "Hikaridai 3-chome, Seika-cho, Soraku-gun, Kyoto Prefecture" is acquired. The acquired address notation becomes the unit area 72.
[0092] Next, analysis device 100 executes a process of analyzing the text using the analysis process with the lowest accuracy (fastest processing speed) and acquiring the processing results. More specifically, analysis device 100 executes first analysis process program 126 to analyze text 54 (the value of $text) included in report content 50 (step S106).
[0093] For example, for the text 54 "There is a fire," the following processing results are obtained:
[0094] Phrase 74: "Fire breaks out" Phrase Type 75: "Request / Problem" Semantic Category 76: “Disaster: Fire” Next, the analysis device 100 determines whether or not a processing result has been acquired by executing the first analysis processing program 126 (step S108). If a processing result has not been acquired (NO in step S108), the processing from step S116 onwards is executed. That is, even when the analysis device 100 is unable to acquire a processing result by the analysis processing using the first analysis processing program 126, it analyzes the text 54 by the analysis processing using the third analysis processing program 130 (or the second analysis processing program 128), as will be described later, and acquires the processing result.
[0095] If a processing result has been acquired (YES in step S108), the analysis device 100 refers to the processing state database 80 and acquires an execution state value 82 corresponding to the combination of the unit area 72 acquired in step S104, the phrase type 75 included in the acquired processing result, and the semantic category 76 included in the acquired processing result (step S110). That is, the analysis device 100 searches the processing state database 80 for a corresponding processing state 81 using the unit area 72 and the semantic flag (phrase type 75 and semantic category 76) as keys.
[0096] In the above example, the unit area 72 would be "Kyoto Prefecture: Soraku District: Seika Town: Hikaridai," the phrase type 75 would be "request / problem," and the semantic category 76 would be "disaster: fire." Using the combination of these three values as a key, the corresponding value, the execution status value 82, is searched for.
[0097] If there is no entry that exactly matches the unit area 72 and semantic flag, the search target may be expanded to a range that can be considered equivalent, such as other unit areas that exist within a predetermined range from the target unit area 72 and / or other semantic flags that are at a predetermined semantic distance. In this way, if the processing state 81 for the determined unit area 72 does not exist in the processing state database 80, the analysis device 100 may search for the processing state 81 for other unit areas 72 that exist within a predetermined range from the determined unit area 72.
[0098] Next, analysis device 100 determines whether or not the processing results of another processing target (report content 50) for the determined unit area have already been stored in processing result database 70. More specifically, analysis device 100 determines whether or not the acquired execution status value 82 indicates that the analysis processing by third analysis processing program 130 has been executed (step S112). In the above example, analysis device 100 determines whether or not the corresponding execution status value 82 is "processed with results (p3)."
[0099] If the acquired execution status value 82 indicates that the analysis process by the third analysis process program 130 has been executed (YES in step S112), analysis device 100 updates the processing result database 70 and the processing status database 80 based on the information acquired in steps S104 and S106 (step S114), and the process then ends.
[0100] That is, after the analysis device 100 analyzes the text 54 through analysis processing by the first analysis processing program 126 and obtains the processing results, if the processing results of another processing target (report content 50) for the determined unit area have already been stored, the analysis device 100 adds the processing results by the first analysis processing program 126 to the processing result database 70.
[0101] In this case, since it is shown that another report content 50 has already been analyzed by the third analysis processing program 130, which has the highest accuracy, for the combination of the target unit area 72, phrase type 75, and semantic category 76, it can be determined that there is little need to analyze the report content 50 received this time using another analysis processing program, and therefore the execution of the more accurate analysis processing program is skipped.
[0102] On the other hand, if the acquired execution status value 82 does not indicate that the analysis processing by the third analysis processing program 130 has been completed (NO in step S112), the analysis device 100 determines whether the load state is such that the third analysis processing program 130 can be executed (step S116).
[0103] If the load state allows execution of third analysis processing program 130 (YES in step S116), analysis device 100 executes third analysis processing program 130 and analyzes text 54 (value of $text) included in report content 50 (step S118). Then, analysis device 100 determines whether processing results have been acquired by execution of third analysis processing program 130 (step S120).
[0104] If the processing results have been acquired (YES in step S120), analysis device 100 updates processing result database 70 and processing status database 80 based on the information acquired in steps S104 and S118 (step S122), and the processing then ends.
[0105] That is, after the analysis device 100 analyzes the text 54 through analysis processing by the first analysis processing program 126 and obtains the processing results, if the processing results of another processing target (report content 50) for the determined unit area are not stored, the analysis device 100 analyzes the text 54 through analysis processing by the third analysis processing program 130 and obtains the processing results, and stores the processing results in the processing result database 70.
[0106] On the other hand, if the processing result has not been acquired (NO in step S120), the analysis device 100 updates the execution state value 82 corresponding to the combination of the unit area 72 acquired in step S104, the phrase type 75 included in the processing result acquired in step S106, and the semantic category 76 included in the processing result acquired in step S106 to a value indicating that the analysis process by the third analysis processing program 130 has been executed but the processing result could not be acquired (step S124). That is, "Processed without result (p3: target id)" is stored as the execution state value 82. Then, the processing ends.
[0107] If the processing result has not been acquired in the previous step S106, the corresponding execution state value 82 cannot be identified, and therefore the update process of the processing state database 80 in step S122 is skipped.
[0108] On the other hand, if the load state is not such that the third analysis processing program 130 can be executed (NO in step S116), the analysis device 100 determines whether the load state is such that the second analysis processing program 128 can be executed (step S126).
[0109] If the load state allows the second analysis processing program 128 to be executed (YES in step S126), the analysis device 100 executes the second analysis processing program 128 to analyze the text 54 (the value of $text) included in the report content 50 (step S128). In this way, if the load state does not allow the analysis processing by the third analysis processing program 130 to be executed, the analysis device 100 analyzes the text 54 by the analysis processing by the second analysis processing program 128 instead of the third analysis processing program 130.
[0110] Next, the analysis device 100 determines whether or not a processing result has been acquired by executing the second analysis processing program 128 (step S130).
[0111] If the processing results have been acquired (YES in step S130), analysis device 100 updates processing result database 70 and processing status database 80 based on the information acquired in steps S104 and S128 (step S132), and the processing then ends.
[0112] That is, after the analysis device 100 analyzes the text 54 through analysis processing by the first analysis processing program 126 and obtains the processing results, if the processing results of another processing target (report content 50) for the determined unit area are not stored, the analysis device 100 analyzes the text 54 through analysis processing by the second analysis processing program 128 and obtains the processing results, and stores the processing results in the processing result database 70.
[0113] On the other hand, if the processing result has not been acquired (NO in step S130), the analysis device 100 updates the execution state value 82 corresponding to the combination of the unit area 72 acquired in step S104, the phrase type 75 included in the processing result acquired in step S106, and the semantic category 76 included in the processing result acquired in step S106 to a value indicating that the analysis process by the second analysis processing program 128 has been executed but the processing result could not be acquired (step S134). That is, "Processed without result (p2: target id)" is stored as the execution state value 82. Then, the processing ends.
[0114] If the processing result has not been acquired in the previous step S106, the corresponding execution state value 82 cannot be identified, and therefore the update process of the processing state database 80 in step S134 is skipped.
[0115] On the other hand, if the load state is not such that second analysis processing program 128 can be executed (NO in step S126), analysis device 100 updates processing result database 70 and processing status database 80 based on the information acquired in steps S104 and S106 (step S136), and the processing then ends.
[0116] The above-described processing procedure is repeatedly executed every time the report content 50 is received. It is assumed that the above-described processing procedure is executed in parallel.
[0117] [G. Example of analysis processing] In actual operation, many of the unit areas 72 registered in the processing status database 80 are updated to "processed" over time. As a result, the frequency with which highly accurate analysis processing such as the third analysis processing program 130 is executed may decrease. Therefore, the execution status value 82 corresponding to a combination of a predetermined range of unit areas 72, phrase types 75, and semantic categories 76 registered in the processing status database 80 may be initialized to "unprocessed" every predetermined time (for example, every 24 hours) or every time a predetermined condition is met (for example, 80% of the registered unit areas 72 are set to "processed").
[0118] Furthermore, there may be cases where high accuracy is not required for a specific semantic category 76 and / or a specific phrase type 75, but high accuracy analysis processing is desired for analysis processing targeting other categories. To meet such a demand, the execution status value 82 corresponding to the combination with the specific semantic category 76 and / or the specific phrase type 75 for all unit areas registered in the processing status database 80 may be initialized to "processed with results." By setting the execution status value 82 to "processed with results," the report content 50 corresponding to the corresponding combination will always be processed by the first analysis processing program 126, thereby achieving processing that meets the demand.
[0119] Conversely, there may be cases where it is desired to always perform highly accurate analysis processing for a specific semantic category 76 and / or a specific phrase type 75. To meet such a demand, the execution status value 82 corresponding to the combination of the target semantic category 76 and phrase type 75 for all unit areas registered in the processing status database 80 may be maintained as "unprocessed" even if it should be updated to "processed." By maintaining the execution status value 82 as "unprocessed," the report content 50 corresponding to the corresponding combination will be analyzed by the second analysis processing program 128 or the third analysis processing program 130.
[0120] 9 is repeatedly executed each time report content 50 is received, but additional analysis processing may be executed afterward. For example, when the number of report contents 50 received per unit time decreases and the load on analysis device 100 is alleviated, the entry (processing status 81) in processing status database 80 may be referenced, and the processing currently being performed by analysis processing program pk may be additionally performed by a more accurate analysis processing program pl (l>k). In other words, analysis device 100 may additionally execute analysis processing with higher accuracy than the analysis processing used to obtain processing results for any unit area 72. By additionally executing such analysis processing, more accurate and high-quality processing results can be collected.
[0121] [H. Application Examples] In the above explanation, information collection during a disaster has been mainly used as an example, but the present invention is not limited to this and can be applied to any situation where processing objects including location information and text must be processed efficiently.
[0122] For example, the present invention can be applied to situations where information is to be efficiently collected from many participants in an event held over a wide area, such as the Olympics.
[0123] [I. Summary] The information processing system 1 according to this embodiment efficiently processes report content 50 including location information and text. That is, there is a need to analyze and process all processing targets including location information and text with high accuracy, but high-accuracy analysis requires cost and processing time. Therefore, when the number of processing targets becomes enormous, it becomes difficult to process all processing targets with high accuracy.
[0124] Therefore, the information processing system 1 according to the present embodiment optimizes the overall processing by selecting the processing target and the processing content based on the characteristics of the location information. More specifically, the information processing system 1 manages whether the report content 50 is "processed" and the processing content for each unit of location information (unit area 72) and semantic flag (phrase type 75 and semantic category 76). Then, for report content 50 that is likely to acquire information similar to information already extracted for a combination of location information and semantic flag (i.e., report content 50 that is estimated to have low novelty), the information processing system 1 completes the processing only with low-cost (i.e., low-accuracy) analysis processing. By selecting the processing target in this way, at each point in time, report content 50 related to unit areas 72 for which no information has been acquired up to that point is processed preferentially. In particular, for unit areas 72 for which no information has been acquired, high-accuracy analysis processing is executed preferentially at a higher cost. On the other hand, for unit areas 72 for which information has already been acquired, the cost required for analysis processing can be reduced.
[0125] As a result, costs can be reduced overall without significantly reducing the usefulness of the information. In other words, instead of processing all of the vast amount of report content 50 in real time, by determining processing priorities based on location information, etc., analysis can be performed while maintaining a balance between accuracy and processing speed.
[0126] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the description of the above embodiments, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]
[0127] 1 Information processing system, 50 Report content, 51, 78 Identification information, 52 User information, 53, 73 Latitude and longitude information, 54 Text, 55, 77 Photo attachment attributes, 70 Processing result database, 70A Main database, 70B Area index database, 70C Category index database, 71 Processing result, 72 Unit area, 74 Phrase, 75 Phrase type, 76 Semantic category, 80 Processing status database, 81 Processing status, 82 Execution status value, 100 Analysis device, 102 CPU, 104 GPU, 106 Main memory, 108 Display, 110 Network interface, 112 Input device, 114 Optical drive, 116 Optical disk, 118 Internal bus, 120 Secondary storage device, 122 Processing management program, 124 Reverse geocoding program, 126 First analysis processing program, 128 Second analysis processing program, 130 Third analysis processing program, 200 SNS servers, 250 chatbots, 300 mobile devices.
Claims
1. a processing result database for storing processing results of processing targets including location information and text; an area determination means for determining a unit area corresponding to the location information included in the processing target; an analysis means for analyzing the text by a specified analysis process among a plurality of analysis processes including a first analysis process and a second analysis process having higher accuracy than the first analysis process, and outputting a processing result; and a control means for, after analyzing the text by the first analysis process to obtain a first processing result, if a processing result for another processing target has already been stored for the determined unit area, adding the first processing result to the processing result database, and, if a processing result for another processing target has not been stored for the determined unit area, analyzing the text by the second analysis process to obtain a second processing result and storing it in the processing result database.
2. the plurality of analysis processes further includes a third analysis process having higher accuracy than the first analysis process and lower accuracy than the second analysis process; 2. The information processing system according to claim 1, wherein the control means analyzes the text by the third analysis process instead of the second analysis process when the load state is such that the second analysis process cannot be executed.
3. 3. The information processing system according to claim 1, wherein the processing result includes a phrase that is a character string indicating the information to be extracted, and semantic information indicating the meaning of the phrase.
4. An information processing method comprising the following steps executed by a computer, wherein the following steps: receiving an object to be processed, the object including location information and text; determining a unit area corresponding to the location information included in the processing target; analyzing the text through a first analysis process to obtain a first processing result; a step of determining whether or not a processing result of another processing target for the determined unit area has already been stored in a processing result database; adding the first processing result to the processing result database if a processing result of another processing target for the determined unit area has already been stored in the processing result database; If the processing result of another processing target for the determined unit area is not stored in the processing result database, the text is analyzed using a second analysis process that is more accurate than the first analysis process to obtain a second processing result, and the second processing result is stored in the processing result database.
5. An information processing program for causing a computer to execute the information processing method according to claim 4.
Citation Information
Patent Citations
Method for dependency analysis
JP2008269391A
Data analysis device, precision estimation device, data analysis method, and storage medium
WO2019208411A1