Summarization device, summarization method, program, and summarization system

Abstract

Provided is a summarization device comprising: a modal sensor input means for receiving an input of a sensing result from a modal sensor that is installed in an observation environment; a modal risk information storage means for storing sensor incidental information indicating a range that is affected by a predetermined event when the modal sensor senses the predetermined event, and sensor-related text information necessary for generating text; a map-drawing means for generating a map image in which risk information that is related to a risk identified on the basis of the sensing result and the sensor incidental information is displayed; an instruction generation means for generating, on the basis of the text information, an instruction that is related to generation of a summary text for summarizing a situation of an area that is indicated by the map image; and a text generation means for generating the summary text on the basis of the map image and the instruction.

Description

Abstract device, abstract method, program, and abstract system

[0001] The present disclosure relates to an abstract device, an abstract method, a program, and an abstract system.

[0002] Patent Document 1 discloses a judgment support device that can support a user's judgment on a disaster situation according to a user who receives information at the time of a disaster.

[0003] International Publication No. 2021 / 192014

[0004] However, when predicting disaster information using a large language model (LLM) or a visual language model (VLM), it is necessary to input detailed geographical information where each sensor is placed. However, it is often difficult to describe in words, and the explanatory text also becomes long, and there are cases where the LLM or VLM cannot accept it. Therefore, an object of the present disclosure is to provide an abstract device that can grasp the early on-site situation and the possible future situations by aggregating and reporting the disaster occurrence situation and the map situation within the number of characters that can be accepted by the LLM or VLM.

[0005] The abstract device of the present disclosure includes: modal sensor input means for receiving an input of a sensing result from a modal sensor installed in an observation environment; modal risk information storage means for storing sensor - attached information indicating a range affected by a predetermined event when the modal sensor senses the predetermined event, and text information related to the sensor necessary for text generation; map drawing means for generating a map image on which risk information related to a risk identified based on the sensing result and the sensor - attached information is displayed; instruction generation means for generating an instruction related to generation of an abstract text summarizing the situation of the area indicated by the map image based on the text information; and text generation means for generating the abstract text based on the map image and the instruction.

[0006] The summarization method of this disclosure is a summarization method which includes: receiving input of sensing results from a modal sensor installed in an observation environment; storing sensor-related information indicating the range affected by a predetermined event when the modal sensor senses a predetermined event, and text information relating to the sensor necessary for generating text; generating a map image displaying risk information relating to the risk identified based on the sensing results and the sensor-related information; generating instructions for generating a summary sentence that summarizes the situation of the area shown in the map image based on the text information; and generating the summary sentence based on the map image and the instructions.

[0007] The program of this disclosure is a program that causes an information processing device to perform the following actions: receive input of sensing results from a modal sensor installed in an observation environment; store sensor-related information indicating the range affected by a predetermined event when the modal sensor senses a predetermined event, and text information about the sensor necessary for generating text; generate a map image displaying risk information related to the risk identified based on the sensing results and the sensor-related information; generate instructions for generating a summary sentence that summarizes the situation of the area shown in the map image based on the text information; and generate the summary sentence based on the map image and the instructions.

[0008] The summarization system of this disclosure comprises: a modal sensor installed in an observation environment; a modal sensor input means for receiving input of sensing results from the modal sensor; a modal risk information storage means for storing sensor-related information indicating the range affected by a predetermined event when the modal sensor senses a predetermined event, and text information relating to the sensor necessary for generating text; a map drawing means for generating a map image displaying risk information relating to the risk identified based on the sensing results and the sensor-related information; an instruction generation means for generating instructions for generating a summary sentence that summarizes the situation of the area shown in the map image based on the text information; and a text generation means for generating the summary sentence based on the map image and the instructions.

[0009] This disclosure provides a summarization device that can summarize and report disaster occurrences and map conditions within the character limit accepted by LLM or VLM, enabling early on-site situation assessment and understanding of potential future situations.

[0010] This is a map illustrating related technologies. This is a map summarized by the summarization device according to this disclosure. This is a block diagram showing the first configuration of the summarization device according to this disclosure. This is a flowchart of the summarization method according to this disclosure. This is a block diagram showing the second configuration of the summarization device according to this disclosure. This is a block diagram showing the third configuration of the summarization device according to this disclosure. This is a block diagram showing the fourth configuration of the summarization device according to this disclosure. This is a block diagram showing the configuration of the information processing device according to this disclosure.

[0011] (Explanation of related technologies) Figure 1 is a map illustrating the related technologies. The related technologies will be explained while referring to Figure 1.

[0012] Large-scale language models (LLMs) that can accept images, such as ChatGPT® and Gemini®, and vision language models (VLMs) that represent the situation in an image as text, can be interpreted based on map information. For example, if the following prompt is given to the map in Figure 1, it will return the result of interpreting the map.

[0013] (Prompt) This is a map image. Hatching indicates areas with water, such as lakes and rivers. Higher elevations are represented by white, and higher elevations by dark green (not shown). If you place a ball at the location of the black circle, which way will the ball go?

[0014] (Response) The location of the black circle is on the side of Mt. Fuji, in a high-altitude area. Therefore, if a ball were placed there, it would roll downwards due to the slope of the terrain. Specifically, it is highly likely that it would roll along the slope of Mt. Fuji, following gravity, towards the lower ground. The direction will change depending on the detailed slope and shape of the map, but from the location of the black circle, the ball is likely to roll southeast or southwest.

[0015] However, summarizing multimodal sensor information requires prompting with text information about each sensor, including its installation location. Modal sensor information includes data from water level sensors, flood sensors, vibration sensors, etc. A multimodal sensor is defined as having multiple types of modal sensors. Creating these prompts was difficult in LLM or VLM, which have limitations on the number of input characters.

[0016] (Description of the summarization device according to the embodiment) Figure 2 is a map summarized by the summarization device according to the present disclosure. Figure 3 is a block diagram showing the first configuration of the summarization device according to the present disclosure. The summarization device according to the embodiment will be described with reference to Figures 2 and 3.

[0017] The summarization device according to this embodiment includes the sensor installation location, risk situation, and risk impact area on a map image, and creates prompts using Visual Prompt Engineering. Risk situation refers to the status of risks such as flooding, fire, and earthquake. Risk impact area refers to the area affected by the disaster. Visual Prompt Engineering is a method of providing an image to an LLM or VLM that can accept images by marking the image and instructing the LLM or VLM to express in detail the information of the image around the mark. Applying this concept, the information necessary for summarizing the sensors is included on the map image, and the image is provided to the LLM or VLM. The summarization device then inputs the created map image and prompts into the LLM or VLM, causing the LLM or VLM to summarize the situation.

[0018] For example, the prompt is created as follows, referring to Figure 2. (Prompt) Figure 2 is a map image. On this map, hatched dots represent mountainous areas, and white represents lowlands. Black circles indicate the location of vibration sensors, and black arrows indicate the direction in which landslides are likely to occur. Triangles indicate flood sensors, and white arrows around flood sensors indicate the flooded area. Squares are water level sensors that have detected an abnormally high river level. Where should you evacuate to in this diagram?

[0019] (Example response) Judging from this diagram, the following evacuation locations may be appropriate: ・Highlands and mountainous areas - The map shows flood risk from the center to the lower right, so mountainous areas are relatively safe. However, there is a risk of landslides indicated by the black arrows, so choose a place that is not affected by them. ・Upper left area - The upper left part of the map is mountainous and appears to have a low risk of flooding. It is also far from the risk of vibrations and landslides. Evacuating to this area may be appropriate. ・Areas not affected by flood sensors - It is important to stay away from the area of ​​flood sensors. Therefore, areas outside the area affected by the sensors, i.e., the left and upper parts of the map, are recommended as evacuation locations. Specifically, the upper left area can be considered a safe evacuation location, but it is very important to check the latest evacuation information and instructions from the government. Also, when evacuating, it is necessary to always check the surrounding situation and make appropriate judgments.

[0020] To create such a prompt, the configuration of the summarization device 300 according to the embodiment is required. The summarization device 300 comprises a modal 1 sensor input unit 301, a modal N sensor input unit 302, a modal 1 risk information unit 303, a modal N risk information unit 304, a map drawing unit 305, an instruction generation unit 306, and a text generation unit 307.

[0021] The Modal 1 sensor input unit 301 and Modal N sensor input unit 302 indicate that there are multiple modal sensor input units. However, there may be only one modal sensor input unit. The modal sensor input unit receives sensing results from modal sensors that sense the observation environment. The modal sensor input unit may accept input of sensing results sensed by multiple types of modal sensors. The observation environment is a disaster environment in a certain area, such as flooding, fire, or earthquake. The observation environment may also be an area set by the user as the target of observation. Modal sensors can include water level sensors, flood sensors, vibration sensors, etc. The modal sensor may also be a multimodal sensor with multiple types. In addition to the above, the modal sensor may also be synthetic aperture radar, multispectral sensors from satellites, ground-based optical fiber sensing, surveillance images, vehicle-mounted images, etc. The modal sensor input unit acquires information from the modal sensor regarding the determination result of whether the continuous value or threshold of each sensor has been exceeded. The result of this determination is represented, for example, as "ON" if the threshold is exceeded, and "OFF" if it is not exceeded.

[0022] Modal 1 Risk Information Unit 303 and Modal N Risk Information Unit 304 indicate that there are multiple Modal Risk Information Units. There are the same number of Modal Risk Information Units as there are Modal Sensor Input Units. Therefore, there may be only one Modal Risk Information Unit. The Modal Risk Information Unit stores information about possible events based on the characteristics and installation location of each sensor. The characteristics of the Modal Sensor are the objects it senses, such as water, temperature, and acceleration. The possible events are predetermined events such as floods, inundation, fires, and earthquakes. The Modal Risk Information Unit stores sensor-related information indicating the range affected by the predetermined event when the Modal Sensor senses the observed environment, and text information about the sensor necessary for generating text. Specifically, this includes the map locations of the inundation sensor and water level sensor in the map of Figure 2, information about the flood range to be drawn on the map, and information to escape inundation to be written in the prompt. Sensor-related information, or information drawn on the map, includes the form of information to be drawn (such as points, arrows, shapes, and designated areas), the drawing position, the drawing direction, the drawing range, and the drawing color. Text information, or information written in prompts, includes things like a red triangle indicating the location of the flood sensor, and a light red area indicating the area where flooding is expected when the flood sensor is activated.

[0023] The map drawing unit 305 generates a map image displaying risk information related to the risks identified based on the sensing results and sensor-related information. Based on the sensor information and the corresponding risk information, the map drawing unit 305 creates a visual prompt on the map that displays the sensor information. If the input from the modal sensor is indicated as ON / OFF, the sensor information will be displayed when it is ON; if it is an analog value, the intensity of the color will be changed, allowing for representations that correspond to the magnitude of the risk.

[0024] The instruction generation unit 306 instructs a situation summary policy that requests an explanation of the map image content based on the text information. The instruction generation unit 306 acquires sensor information to be written on the map corresponding to the modal sensor for which sensor information was received at the modal sensor input unit. The instruction generation unit 306 generates a descriptive text for the map image and instructions for extracting the information requested by the user from the map image. The instructions are prompts to be entered into the LLM or VLM.

[0025] The text generation unit 307 generates a summary text, which is a summary of the environmental conditions shown in the map image, based on the map image containing risk information and instructions. The text generation unit 307 has an LLM or VLM that accepts images. Based on the input of the drawn map image and a text prompt containing instructions generated by the instruction generation unit 306, the text generation unit 307 outputs a summary text that explains the environmental conditions shown in the map image.

[0026] With the above configuration, a summarization device is provided that allows for early onset of on-site conditions and identification of potential future situations by aggregating and reporting disaster occurrence status and map conditions within the character limit accepted by LLM or VLM. The information stored in the modal risk information unit may be manually designed information or information obtained through learning based on past data. Past data refers to past disaster occurrence situations, such as data from modal sensors detecting flood conditions.

[0027] The modal sensor input unit, modal risk information unit, map drawing unit, instruction generation unit, and text generation unit are also referred to as modal sensor input means, modal risk information storage means, map drawing means, instruction generation means, and text generation means, respectively.

[0028] (Description of the summarization method and program according to the embodiment) Figure 4 is a flowchart of the summarization method according to the present disclosure. Figure 8 is a block diagram showing the configuration of the information processing device according to the present disclosure. The summarization method and program according to the embodiment will be described with reference to Figures 4 and 8.

[0029] As shown in Figure 4, first the summarization device 300 waits for a certain period of time for sensor information to be collected in the modal 1 sensor input unit 301 and the modal N sensor input unit 302 (step S401). Next, the summarization device 300 determines whether or not there is sensor information in the modal 1 sensor input unit 301 and the modal N sensor input unit 302 (step S402). If there is no sensor information in the modal 1 sensor input unit 301 and the modal N sensor input unit 302 (No. in step S402), the process returns to step S401. In other words, the summarization device 300 waits until sensor information is entered into the modal 1 sensor input unit 301 and the modal N sensor input unit 302.

[0030] If sensor information is available in the Modal 1 Sensor Input Unit 301 and the Modal N Sensor Input Unit 302 (Yes in step S402), the Modal Sensor Input Unit inputs the sensing results from the modal sensor that senses the observation environment to the map drawing unit 305 and the instruction generation unit 306. The Modal Risk Information Unit transmits sensor-related information indicating the affected area when the modal sensor senses something, and text information about the sensor necessary for text generation, to the map drawing unit 305. The map drawing unit 305 draws the sensor information and risk information on the map (step S403). In this case, the map drawing unit 305 writes the risk information on the map image based on the sensing results and sensor-related information. The instruction generation unit 306 also generates a text prompt.

[0031] Next, the summarization device 300 generates text based on the map image and text prompts (step S404). The text generation unit 307 generates summary text that describes the environmental conditions shown on the map based on the map image containing risk information and instructions.

[0032] The above configuration provides a summarization method that allows for early assessment of the situation on the ground and understanding of potential future situations by aggregating and reporting disaster occurrences and map conditions within the character limit accepted by LLM or VLM.

[0033] The above method is carried out by an information processing device executing a program and performing processing. As shown in Figure 8, the information processing device 800 includes a processor 801 that executes a program and performs processing, and a memory 802 that stores the program. The information processing device 800 may consist of one device or multiple devices. The information processing device 800 may also be a cloud server that processes some or all of its functions in a distributed manner.

[0034] (Description of the summarization device according to Embodiment 1) Figure 5 is a block diagram showing the second configuration of the summarization device according to the present disclosure. The summarization device according to Embodiment 1 will be described with reference to Figure 5.

[0035] As shown in Figure 5, the summarization device 500 according to Embodiment 1 differs in that it comprises the summarization device 300 according to Embodiment 300, a modal 1 sensor determination unit 501, and a modal N sensor determination unit 502.

[0036] The modal 1 sensor determination unit 501 and modal N sensor determination unit 502 indicate that there are multiple modal sensor determination units. There are as many modal sensor determination units as there are modal sensor input units. Therefore, there may be just one modal sensor determination unit. A modal sensor determination unit is also called a modal sensor determination means.

[0037] The modal sensor determination unit identifies the sensor information corresponding to the modal sensor input obtained from the modal sensor input unit and performs a risk determination. The risk determination determines the degree of risk. In the risk determination, the modal sensor determination unit performs threshold processing on the input value in a rule-based manner to assign risk levels such as 1, 2, 3, etc. Risk levels are divided into stages such as very dangerous, slightly dangerous, and not dangerous at all. The modal sensor determination unit may also output the results of analysis processing using a recognition engine in the risk determination. For example, the recognition engine may determine from satellite imagery or in-vehicle camera data that a house is damaged, a bridge has been washed away, etc.

[0038] First, the summarization device 500 waits for a certain period of time for sensor information to be collected at the modal 1 sensor input unit 301 and the modal N sensor input unit 302. Next, the summarization device 500 determines whether or not there is sensor information at the modal 1 sensor input unit 301 and the modal N sensor input unit 302. If there is no sensor information at the modal 1 sensor input unit 301 and the modal N sensor input unit 302 (No. in step S402), the summarization device 300 remains in standby mode.

[0039] If sensor information is available in the Modal 1 Sensor Input Unit 301 and the Modal N Sensor Input Unit 302, the Modal Sensor 1 Judgment Unit 501 and the Modal N Sensor Judgment Unit, which correspond to the Modal 1 Sensor Input Unit 301 and the Modal N Sensor Input Unit 302, perform a risk judgment. The Modal Sensor Judgment Unit inputs the sensing result and risk judgment result from the modal sensor that senses the observation environment to the map drawing unit 305 and the instruction generation unit 306. The Modal Risk Information Unit transmits sensor-related information indicating the affected area when the modal sensor senses something, and text information about the sensor necessary for text generation, to the map drawing unit 305. The map drawing unit 305 draws the sensor information, risk information, and risk judgment result on the map. In this case, the map drawing unit 305 writes the risk information on the map image based on the sensing result and the sensor-related information. The instruction generation unit 306 also generates a text prompt.

[0040] Next, the text generation unit 307 generates text based on the map image and text prompts. The text generation unit 307 generates a summary text that describes the environmental conditions shown on the map, based on the map image containing risk information and the instructions.

[0041] With the above configuration, information on risk assessment can be sent to the map drawing unit 305 and the instruction generation unit 306.

[0042] (Description of the summarization device according to Embodiment 2) Figure 6 is a block diagram showing the third configuration of the summarization device according to the present disclosure. The summarization device according to Embodiment 2 will be described with reference to Figure 6.

[0043] As shown in FIG. 6, the summary device 600 according to Embodiment 2 is different from the summary device 300 according to the embodiment in that the modal 1 sensor input unit 301 is changed to the modal 1 sensor information unit 601 and the modal N sensor input unit 302 is changed to the modal N sensor information unit 602.

[0044] The modal 1 sensor information unit 601 and the modal N sensor information unit 602 indicate that there are a plurality of modal sensor information units. There may be one modal sensor information unit. The modal sensor information unit is also referred to as sensor information storage means.

[0045] The sensing result obtained by the modal sensor input unit corresponding to the modal sensor information unit may be temporarily stored in the modal sensor information unit and retrieved at a predetermined timing. By doing so, it is also possible to use batch processing to draw sensor information in the map drawing unit 305 afterwards and have the text generation unit 307 process the map image.

[0046] The summary device 300 waits for sensor information to be collected in the modal 1 sensor information unit 601 and the modal N sensor information unit 602 for a certain period of time. The sensor information collected by the modal 1 sensor information unit 601 and the modal N sensor information unit 602 is input to the map drawing unit 305 and the instruction generation unit 306. The modal risk information unit transmits sensor supplementary information indicating the range affected when the modal sensor senses and text information regarding the sensor necessary for text generation to the map drawing unit 305. The map drawing unit 305 draws the sensor information and the risk information on the map. In that case, the map drawing unit 305 describes the risk information in the map image based on the sensing result and the sensor supplementary information. Also, the instruction generation unit 306 generates a text prompt.

[0047] Next, the summary device 300 performs text generation based on the map image and the text prompt. The text generation unit 307 generates a summary text that describes the environmental situation indicated by the map based on the map image with the risk information described and the instruction.

[0048] (Description of the Summarization Device According to Embodiment 3) FIG. 7 is a block diagram showing a fourth configuration of the summarization device according to the present disclosure. The summarization device according to Embodiment 3 will be described while referring to FIG. 7.

[0049] As shown in FIG. 7, the summarization device 700 according to Embodiment 3 is different in that it includes the summarization device 300 according to the embodiment and an image generation unit 701. The image generation unit is also referred to as image generation means.

[0050] The image generation unit 701 generates an image in which information indicating a response policy to risks is drawn on a map based on the text generated by the text generation unit 307. The response policy includes, for example, an evacuation policy, but is not limited thereto. The image generation unit 701 outputs the generated image.

[0051] With the above configuration, the user can obtain an image with additional information such as an evacuation pointer and can grasp the situation more accurately.

[0052] As described above, the present disclosure has been described with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure. And each embodiment can be combined with other embodiments as appropriate.

[0053] Each drawing is merely an example for explaining one or more embodiments. Each drawing is not associated with only one specific embodiment, but may be associated with one or more other embodiments. As can be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with the features or steps shown in one or more other drawings to create, for example, an embodiment that is not explicitly illustrated or described. Not all of the features or steps shown in any one drawing for explaining exemplary embodiments are necessarily essential, and some features or steps may be omitted. The order of the steps described in any drawing may be changed as appropriate.

[0054] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A summarization device comprising: a modal sensor input unit that inputs sensing results from a modal sensor that senses the observation environment; a modal risk information unit having sensor-related information indicating the range affected when the modal sensor senses and text information about the sensor necessary for generating text; a map drawing unit that writes risk information on a map image based on the sensing results and the sensor-related information; an instruction generation unit that instructs a situation summarization policy that requests an explanation of the contents of the map image based on the text information; and a text generation unit that generates text summarizing the situation of the environment shown on the map based on the map image in which the risk information is written and the instruction. (Note 2) The summarization device according to Note 1, wherein the text generation unit uses a large-scale language model. (Note 3) The summarization device according to Note 1, wherein the modal risk information unit has information obtained by learning based on past data. (Note 4) The summarization device according to Note 1, further comprising a modal sensor determination unit that identifies sensor information obtained from a modal sensor input unit and performs risk determination. (Note 5) The summarization device according to Note 1, further comprising a sensor information unit that stores the sensing results obtained from the modal sensor input unit, the sensor information unit providing the sensing results to the map drawing unit at the necessary timing. (Note 6) The summarization device according to Note 1, further comprising an image generation unit that generates an image of evacuation policies drawn on a map based on text generated by a text generation unit. (Note 7) The summarization device according to Note 1, wherein the modal sensor input means accepts input of the sensing results sensed by each of the multiple types of modal sensors.(Note 8) A summarization method comprising: inputting a sensing result from a modal sensor that senses the observation environment; having sensor-related information indicating the range affected when the modal sensor senses; and text information relating to the sensor necessary for generating text; writing risk information on a map image based on the sensing result and the sensor-related information; instructing a situation summary policy that requests an explanation of the contents of the map image based on the text information; and generating the text summarizing the environmental conditions shown on the map based on the map image with the risk information written on it and the instruction. (Note 9) A program that causes an information processing device to perform the following: inputting a sensing result from a modal sensor that senses the observation environment; having sensor-related information indicating the range affected when the modal sensor senses; and text information relating to the sensor necessary for generating text; writing risk information on a map image based on the sensing result and the sensor-related information; instructing a situation summary policy that requests an explanation of the contents of the map image based on the text information; and generating the text summarizing the environmental conditions shown on the map based on the map image with the risk information written on it and the instruction. (Note 10) A summarization system comprising: a modal sensor for sensing the observation environment; a modal sensor input unit for inputting sensing results from the modal sensor; a modal risk information unit having sensor-related information indicating the range affected when the modal sensor senses something, and text information about the sensor necessary for generating text; a map drawing unit for writing risk information on a map image based on the sensing results and the sensor-related information; an instruction generation unit for instructing a situation summarization policy that requests an explanation of the contents of the map image based on the text information; and a text generation unit for generating text that summarizes the environmental conditions shown on the map based on the map image with the risk information written on it and the instruction.

[0055] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 7 that are dependent on Appendice 1 {e.g., device} may also be dependent on Appendice 8 {e.g., method}, Appendice 9 {e.g., program}, and Appendice 10 (e.g., system) in the same way as in Appendices 2 to 7. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0056] Although the present invention has been described above with reference to embodiments, the present invention is not limited thereto. Various modifications to the structure and details of the present invention can be made that are understandable to those skilled in the art within the scope of the invention.

[0057] This application claims priority based on Japanese Patent Application No. 2025-025239, filed on 19 February 2025, and incorporates all of its disclosures herein.

[0058] 300 Summarization device, 301 Modal 1 sensor input unit, 302 Modal N sensor input unit, 303 Modal 1 risk information unit, 304 Modal N risk information unit, 305 Map drawing unit, 306 Instruction generation unit, 307 Text generation unit, 500 Summarization device, 501 Modal 1 sensor determination unit, 502 Modal N sensor determination unit, 600 Summarization device, 601 Modal 1 sensor information unit, 602 Modal N sensor information unit, 700 Summarization device, 701 Image generation unit, 800 Information processing device, 801 Processor, 802 Memory

Claims

1. A summarization device comprising: modal sensor input means for receiving input of sensing results from a modal sensor installed in an observation environment; modal risk information storage means for storing sensor-related information indicating the range affected by a predetermined event when the modal sensor senses a predetermined event, and text information relating to the sensor necessary for generating text; map drawing means for generating a map image displaying risk information relating to the risk identified based on the sensing results and the sensor-related information; instruction generation means for generating instructions for generating a summary sentence that summarizes the situation of the area shown in the map image based on the text information; and text generation means for generating the summary sentence based on the map image and the instructions.

2. The summarization device according to claim 1, wherein the text generation means uses a language model that outputs the summary text based on the input of the map image and the instructions.

3. The summarizing device according to claim 1, wherein the modal risk information storage means has information obtained through learning based on past data.

4. The summarization device according to claim 1, further comprising a modal sensor determination means for identifying sensor information obtained by a modal sensor input means and performing a risk determination to determine the degree of the risk.

5. The summarization device according to claim 1, further comprising a sensor information storage means for storing the sensing result obtained by the modal sensor input means, wherein the sensor information storage means outputs the sensing result to the map drawing means at a predetermined timing.

6. The summarization device according to claim 1, further comprising an image generation means that generates an image on a map showing information indicating a policy for responding to the risks, based on the text generated by the text generation means.

7. The summarization device according to claim 1, wherein the modal sensor input means receives input of the sensing result sensed by each of the multiple types of modal sensors.

8. A summarization method comprising: receiving input of sensing results from a modal sensor installed in an observation environment; storing sensor-related information indicating the range affected by a predetermined event when the modal sensor senses a predetermined event, and text information about the sensor necessary for generating text; generating a map image displaying risk information related to the risk identified based on the sensing results and the sensor-related information; generating instructions for generating a summary sentence that summarizes the situation of the area shown in the map image based on the text information; and generating the summary sentence based on the map image and the instructions.

9. The summarization method according to claim 8, wherein the generation of the text uses a language model that outputs the summary text based on the input of the map image and the instructions.

10. The summarization method according to claim 8, wherein the storage of the sensor-related information and the text information includes information obtained through learning based on past data.

11. The summary method according to claim 8, further comprising identifying sensor information obtained by receiving input of sensing results from the modal sensor and performing a risk determination to determine the degree of the risk.

12. The summarization method according to claim 8, wherein the sensing result obtained by receiving input of the sensing result from the modal sensor is stored, and the sensor-related information and the text information are stored, and the sensing result is output to the map drawing means at a predetermined timing.

13. The summarization method according to claim 8, further comprising generating an image on a map that displays information indicating a policy for responding to the risks, based on the text generated by the text generation.

14. The summary method according to claim 8, wherein receiving the input of the sensing result from the modal sensor means receiving the input of the sensing result sensed by each of the multiple types of modal sensors.

15. A program that causes an information processing device to perform the following actions: receive input of sensing results from a modal sensor installed in the observation environment; store sensor-related information indicating the range affected by a predetermined event when the modal sensor senses a predetermined event, and text information about the sensor necessary for generating text; generate a map image displaying risk information related to the risk identified based on the sensing results and the sensor-related information; generate instructions for generating a summary sentence that summarizes the situation of the area shown in the map image based on the text information; and generate the summary sentence based on the map image and the instructions.

16. The program according to claim 15, wherein the text generation uses a language model that outputs the summary text based on the input of the map image and the instructions.

17. The program according to claim 15, wherein the sensor information and the text information are stored, and the information is obtained through learning based on past data.

18. The program according to claim 15, further comprising: identifying sensor information obtained by receiving input of sensing results from the modal sensor and performing a risk determination to determine the degree of the risk.

19. The program according to claim 15, wherein the program stores the sensing result obtained by receiving input of the sensing result from the modal sensor, stores the sensor-related information and the text information, and outputs the sensing result to the map drawing means at a predetermined timing.

20. A summarization system comprising: a modal sensor installed in an observation environment; a modal sensor input means for receiving input of sensing results from the modal sensor; a modal risk information storage means for storing sensor-related information indicating the range affected by a predetermined event when the modal sensor senses a predetermined event, and text information about the sensor necessary for generating text; a map drawing means for generating a map image displaying risk information related to the risk identified based on the sensing results and the sensor-related information; an instruction generation means for generating instructions for generating a summary sentence that summarizes the situation of the area shown in the map image based on the text information; and a text generation means for generating the summary sentence based on the map image and the instructions.

Images