Information processing apparatus, information processing system, information processing method, and program

Abstract

To support smooth guidance from the inside of a building to an exit.SOLUTION: The information processing apparatus 10 includes a reception unit 11 that receives captured images captured by a plurality of image capturing apparatuses 20 disposed to capture images of states of a plurality of exits inside a building, a distribution information generation unit 12 that generates distribution information of positions of persons in the building on the basis of the plurality of captured images, and a display control unit 13 that displays, on the basis of input data including map information indicating an internal structure of the building, exit information indicating positions of a plurality of exits of the building, position information of the image capturing apparatuses, position information of a plurality of display apparatuses 30 dispersedly disposed inside the building, the captured images, and the distribution information, the system includes a guide information generation unit 13 that causes a machine learning model to generate guide information for guiding a person to an exit, a display information generation unit 14 that generates display information for each display device 30 based on the guide information corresponding to the display device, and a transmission unit 15 that transmits the display information corresponding to the display device to each display device 30.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] Conventionally, technologies for supporting evacuation in the event of a disaster have been studied.

[0003] For example, a technology has been disclosed that enables residents to evacuate quickly and smoothly to the most suitable evacuation shelter along the optimal evacuation route in the event of a disaster, without the need for prior evacuation plans or on-the-spot information gathering (Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technology does not take into consideration the guidance of people inside buildings when a disaster occurs. For example, when trying to evacuate from a large hospital, commercial facility, hotel, etc., depending on the disaster situation, evacuees may become confused and not know which direction to go. Furthermore, it is possible that many people will try to exit from the same exit, or the flow of people may become uneven, which could result in a long evacuation time.

[0005] The present invention has been made in consideration of the above points, and aims to support smooth guidance from inside a building to an exit. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device has: a receiving unit that receives captured images captured by a plurality of imaging devices arranged so as to capture the status of at least a plurality of exits inside a building; a distribution information generation unit that generates distribution information of people's positions inside the building based on the plurality of captured images; a guidance information generation unit that causes a machine learning model to generate guidance information for guiding people to the exits based on input data including map information that indicates the internal structure of the building, exit information that indicates the positions of a plurality of exits of the building, position information of the imaging devices, position information of a plurality of display devices that are arranged dispersedly inside the building, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; a display information generation unit that generates display information for each of the display devices based on the guidance information that corresponds to the display device; and a transmission unit that transmits the display information corresponding to the display device to each of the display devices, and the machine learning model has learned the correspondence between the input data and the guidance information for each of the display devices. [Effects of the Invention]

[0007] It can help guide people smoothly from inside the building to the exit. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an example of a configuration of an information processing system according to a first embodiment. [Figure 2] 1 is a diagram illustrating an example of a hardware configuration of an information processing device 10 according to a first embodiment. [Figure 3] 1 is a diagram illustrating an example of a functional configuration of an information processing device 10 according to a first embodiment. [Figure 4] 3 is a diagram for explaining an example of input and output of a guide information generation model 16. FIG. [Figure 5] 3 is a diagram for explaining an example of learning data of the guide information generation model 16. FIG. [Figure 6] FIG. 10 is a diagram illustrating an example of constraints in a building. [Figure 7]FIG. 10 is a diagram illustrating an example of a thought process when a learning data creator considers an evacuation route. [Figure 8] FIG. 10 is a diagram illustrating an example of constraints based on attribute information of evacuees. [Figure 9] FIG. 10 is a diagram showing an example of distribution of evacuees taking into consideration attribute information of the evacuees. [Figure 10] 10 is a flowchart illustrating an example of a processing procedure executed by information processing device 10 according to the first embodiment. [Figure 11] 10A and 10B are diagrams for explaining an example of a method for generating distribution information and attribute information sequences. [Figure 12] FIG. 10 is a diagram illustrating an example of the configuration of a display device table. [Figure 13] FIG. 3 is a diagram showing a display example of display information in the first embodiment. [Figure 14] FIG. 10 is a diagram showing a display example of display information in the second embodiment. [Figure 15] FIG. 13 is a diagram showing a display example of display information in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Fig. 1 is a diagram showing an example of the configuration of an information processing system in a first embodiment. In Fig. 1, a plurality of image capturing devices 20 and a plurality of display devices 30 arranged in a building B1 are connected to an information processing device 10 via a network such as the Internet.

[0010] Building B1 is a structure with many people entering and exiting, such as a hospital, commercial facility, office building, complex, or educational facility.

[0011] The multiple image capture devices 20 are distributed and arranged at multiple locations (i.e., located at different locations) inside building B1 (hereinafter referred to as "inside the building"). Each image capture device 20 is an apparatus that captures a different location and is arranged so that at least the exits of building B1 are captured. The multiple image capture devices 20 may be capable of capturing all locations within the building, or only some locations. Furthermore, the range that one image capture device 20 can capture may not overlap with the range that another image capture device 20 can capture, or may overlap. In this embodiment, images captured by the image capture devices 20 (captured images) are used to determine the distribution of people within the building and to identify damaged areas within the building in the event of a disaster such as an earthquake or fire. Therefore, the image capture devices 20 may be arranged in locations suitable for such purposes. Note that the image capture device 20 may be a camera that captures in one direction, or a camera that can capture in multiple directions, such as a 360-degree celestial camera.

[0012] The display devices 30 are distributed across multiple locations within the building. The display devices 30 are used to display guidance information during disasters such as earthquakes and fires. The guidance information is information for guiding people (evacuees) in the building to exits. An exit indicates a location from which to leave the building. Exits are not limited to the main entrance of the building, but may also include emergency exits, employee entrances, and windows. Therefore, the display devices 30 may be placed in locations suitable for such purposes. For example, the display devices 30 may be placed at regular intervals along passages that can be used as evacuation routes in the event of a disaster in building B1. Since the positions of the display devices 30 are different, the routes (evacuation routes) from the positions of the display devices 30 to the exits may also be different. Therefore, the guidance information displayed on each display device 30 may also be different. The display devices 30 may be equipped with a touch panel that allows the display content to be changed in response to user operation.

[0013] The information processing device 10 is one or more computers that, in the event of a disaster, generate guidance information for each display device 30 based on images captured by the multiple image capturing devices 20, and generate display information for displaying the guidance information on the display devices 30. As described above, evacuation routes from each display device 30 may differ. Therefore, the information processing device 10 generates guidance information, etc. for each display device 30.

[0014] Fig. 2 is a diagram showing an example of the hardware configuration of information processing device 10 in the first embodiment. As shown in Fig. 2, information processing device 10 is constructed by a computer, and includes CPU 101, ROM 102, RAM 103, HD 104, HDD (Hard Disk Drive) controller 105, display 106, external device connection I / F (Interface) 108, network I / F 109, data bus 110, keyboard 111, pointing device 112, DVD-RW (Digital Versatile Disk Rewritable) drive 114, and media I / F 116.

[0015] Of these, the CPU 101 controls the overall operation of the information processing device 10. The ROM 102 stores programs used to drive the CPU 101, such as an IPL. The RAM 103 is used as a work area for the CPU 101. The HD 104 stores various data, such as programs. The HDD controller 105 controls the reading and writing of various data from and to the HD 104 under the control of the CPU 101. The display 106 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 108 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories, printers, etc. The network I / F 109 is an interface for data communication using a communication network. The data bus 110 is an address bus, data bus, etc. for electrically connecting the components, such as the CPU 101, shown in FIG. 2.

[0016] The keyboard 111 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 112 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 114 controls reading and writing of various data from a DVD-RW 113, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 116 controls reading and writing (storing) of data from a recording medium 115, such as a flash memory.

[0017] FIG. 3 is a diagram illustrating an example of a functional configuration of the information processing device 10 according to the first embodiment. In FIG. 3, the information processing device 10 includes a receiving unit 11, a distribution information generating unit 12, a guide information generating unit 13, a display information generating unit 14, a transmitting unit 15, and a guide information generation model 16. These units are realized by processing executed by a CPU 101 in accordance with one or more programs installed in the information processing device 10. The information processing device 10 also uses storage units such as a map information storage unit 121, an exit information storage unit 122, a shooting position storage unit 123, and a display position storage unit 124. These storage units can be realized using, for example, the HD 104 or a storage device connectable to the information processing device 10 via a network.

[0018] The map information storage unit 121 stores map information for the building. The map information is a map (interior map) of the building that shows the internal structure of the building B1 and the layout of stores and other facilities within the building. The map information may or may not be expressed in three dimensions (for example, it may be expressed in two dimensions). When the map information is not expressed in three dimensions, the term "three dimensions" below may be replaced with the dimension corresponding to the map information. For example, the map information is point cloud data obtained by scanning the interior of the building B1 with a 3D scanner. Point cloud data refers to a collection of points, each of which has three-dimensional coordinate values. The map information may also be data that allows an image of the interior of the building B1 to be synthesized with the point cloud data. By using the map information, the interior of the damaged building B1 can be reproduced in the virtual space of the information processing device 10 using digital twin technology. In this embodiment, the three-dimensional coordinate system of the interior of the building B1 used to express the map information is referred to as a "map coordinate system." That is, each point of the point cloud data as the map information has a three-dimensional coordinate value in the map coordinate system.

[0019] The exit information storage unit 122 stores position information of each of a plurality of exits of the building B1 (hereinafter referred to as "exit information").

[0020] The photographing position storage unit 123 stores the position information of each photographing device 20.

[0021] The display position storage unit 124 stores position information of each display device 30.

[0022] The position information of each exit, each image capturing device 20, and each display device 30 is information expressed by coordinate values ​​(three-dimensional coordinate values) in a map coordinate system.

[0023] The receiving unit 11 receives images captured by a plurality of image capturing devices 20 distributed within the building B1. The captured images refer to images captured by the image capturing devices 20. The receiving unit 11 may receive the captured images from each of the plurality of image capturing devices 20, or may receive the captured images from a server that collects the images captured by the plurality of image capturing devices 20 (for example, a server to which each image capturing device 20 uploads the captured images).

[0024] The distribution information generation unit 12 generates distribution information of the positions of people (evacuees) inside building B1 based on multiple captured images (respective captured images from multiple image capture devices 20). The distribution information is information indicating the positions of people inside building B1, that is, whether people are concentrated or dispersed inside building B1. The distribution information can be generated by detecting people from each captured image using object recognition and identifying position information (coordinate values) in a map coordinate system for each detected person. The position information in the coordinate system of the captured image can be converted to position information in the map coordinate system using a function prepared in advance.

[0025] The distribution information generation unit 12 also extracts (detects) attribute information of each person from the captured image. In this embodiment, the attribute information of a person refers to, for example, information indicating attributes that affect movement to an exit within building B1. For example, the attribute information may include some or all of the following items: whether the person is able-bodied, whether they are an adult or a child, whether they are injured or disabled, whether they use a wheelchair, whether they have a pet, whether they have a lot of luggage, etc., or it may include additional items. Such attribute information can be extracted using known skeletal structure detection. That is, the skeletal structure may differ depending on whether the person is able-bodied, whether they are injured, whether they use a wheelchair, whether they have a pet, whether they have a lot of luggage, etc. Therefore, the distribution information generation unit 12 can extract the attribute information of each person based on the skeletal structure obtained for each person by skeletal structure detection.

[0026] In the event of a disaster in building B1, the guidance information generation unit 13 generates guidance information for each display device 30 to guide people (evacuees) in building B1 to the exits of building B1. In the first embodiment, information (e.g., an arrow) indicating the direction in which evacuees should proceed (travel direction) will be described as an example of the guidance information. In this case, the actual substance of the guidance information may be an angle indicating the travel direction (the direction of the arrow). The guidance information generation unit 13 may also generate location information of disaster-affected areas (hereinafter referred to as "information about the disaster-affected areas"). A disaster-affected area refers to an area where damage has occurred due to a disaster. For example, a location where a fire has occurred or a location that has been deformed (e.g., damaged) due to a fire or earthquake is an example of a disaster-affected area. The information about the disaster-affected area is expressed, for example, by coordinate values ​​in a map coordinate system. The information about the disaster-affected area may also be expressed using icons or text to indicate the extent of the damage (e.g., impassable areas, damaged and dangerous areas, fire areas, etc.).

[0027] When generating the guide information, the guide information generating unit 13 uses a guide information generation model 16, which is a machine learning model (for example, a neural network) that has been trained in advance.

[0028] Here, machine learning is a technology that allows computers to acquire human-like learning abilities, and refers to a technology in which a computer autonomously generates algorithms necessary for making judgments such as data identification from training data that is input in advance, and applies these to new data to make predictions.

[0029] Fig. 4 is a diagram for explaining an example of input and output of the guide information generation model 16. As shown in Fig. 4, the guide information generation model 16 receives input data including map information, an exit information sequence, a shooting position sequence, a display position sequence, a shot image sequence, and distribution information (attribute information sequence), and outputs a guide information sequence and a disaster location sequence.

[0030] The map information is stored in the map information storage unit 121.

[0031] The exit information sequence is an array of exit information stored in the exit information storage unit 122. The n-th element of the array corresponds to the n-th exit in a sequence previously assigned to one or more exits.

[0032] The sequence of captured images refers to an array of captured images for each camera device 20, and the nth element of the array corresponds to the nth camera device 20 in the order assigned in advance to the multiple camera devices 20.

[0033] The image capture position sequence refers to an array of position information of a plurality of image capture devices 20, and the nth element of the array corresponds to the nth image capture device 20 in the order assigned to the plurality of image capture devices 20 in advance.

[0034] The display position sequence refers to an array of position information of a plurality of display devices 30, and the n-th element of the array corresponds to the n-th display device 30 in the order assigned to the plurality of display devices 30 in advance.

[0035] The distribution information (attribute information sequence) may be distribution information only, or may include distribution information and an attribute information sequence. The attribute information sequence refers to an array of attribute information of each evacuee that constitutes the distribution information.

[0036] The guidance information sequence refers to an arrangement of guidance information for each display device 30, and the nth element of the sequence corresponds to the nth display device 30 in the order previously assigned to the multiple display devices 30. However, if the guidance information generation model 16 also inputs attribute information, the nth element of the guidance information sequence includes multiple pieces of guidance information corresponding to the attribute information of the evacuee for the nth display device 30. In other words, if the guidance information generation model 16 also inputs attribute information, guidance information corresponding to the attribute information is estimated for each display device 30. "According to attribute information" refers to each type of attribute information or each classification of attribute information.

[0037] The damaged location sequence is an array of information related to the damaged locations. Since multiple locations within building B1 may be damaged, an array of information related to the damaged locations is output.

[0038] Fig. 5 is a diagram for explaining an example of learning data of the guide information generation model 16. As shown in Fig. 5, one piece of learning data for learning the guide information generation model 16 having input and output as described in Fig. 4 includes a set of map information, an exit information sequence, a shooting position sequence, a display position sequence, a photographed image sequence, distribution information (attribute information sequence), a guide information sequence, and a disaster location sequence. Among these, the map information, the exit information sequence, the shooting position sequence, the display position sequence, the photographed image sequence, and the distribution information (attribute information sequence) are input data used as input to the guide information generation model 16, and the guide information sequence and the disaster location sequence are data (correct answer data) as a correct answer to an output from the guide information generation model 16 to which the input data is input.

[0039] The guide information generation model 16 is trained based on such a set of training data so that the output when input data is input approaches correct data (guidance information, disaster location sequence). Training of the guide information generation model 16 means updating the model parameters of the guide information generation model 16. The guide information generator 13 may execute the training process of the guide information generation model 16.

[0040] The guide information generation model 16 thus trained is a machine learning model that has learned the correspondence between the input data including map information, exit information sequence, shooting position sequence, display position sequence, photographed image sequence, and distribution information (attribute information sequence), and the guidance information sequence and disaster area sequence. As a result, the guide information generation model 16 becomes able to estimate the guidance information and disaster area for each display device 30 according to the internal structure of the building indicated by the map information, the position of each exit indicated by the exit information sequence, the multiple photographed images related to the photographed image sequence, the position of each photographing device 20 indicated by the shooting position sequence, the position of each display device 30 indicated by the display position sequence, and the distribution of the positions of evacuees inside the building (attribute information sequence) of each evacuee (attributes of each evacuee).

[0041] Note that a guide information generation model 16 that does not output a sequence of disaster-stricken areas may be used. In this case, the sequence of disaster-stricken areas is not necessary in the learning data.

[0042] When creating a sequence of damaged locations as a correct answer corresponding to input data, a person who creates training data (hereinafter referred to as the "training data creator") may identify one or more damaged locations based on images in the sequence of photographed images that correspond to the damaged locations. Since it is difficult to collect photographed images of damaged locations under normal circumstances, a pseudo-photographed image corresponding to any damaged location may be generated by combining images taken under normal circumstances with images showing a damaged situation (such as an image of a fire or an image of damage). Alternatively, a sequence of damaged locations may be determined in advance, and a sequence of photographed images may be generated to correspond to the sequence of damaged locations.

[0043] Furthermore, when creating the correct answer for the guidance information of each display device 30 corresponding to each input data, the learning data creator considers the internal structure indicated by the map information to determine an evacuation route that can be safely traveled from the position corresponding to each position information included in the display position sequence included in the input data to any of the exits (any of the exits related to the exit information sequence). Being able to safely travel to an exit means being able to travel while avoiding disaster-stricken areas. In this case, the learning data creator creates the evacuation route taking into consideration constraints such as passages and exits that can be used as evacuation routes within building B1.

[0044] Fig. 6 is a diagram showing an example of constraint information for a building. Constraint information is information related to the structure that restricts people's passage within the building. For example, Fig. 6 shows an example of constraints related to the exits of a building. In the example of Fig. 6, building B1 has four exits, exits e1 to e4, and for each exit, the presence or absence of stairs, the width of the passage, the presence or absence of handrails, etc. are indicated.

[0045] Although Figure 6 only shows constraints related to exits, the creator of the learning data creates evacuation routes by taking into account constraints such as the width of passageways that could serve as evacuation routes.

[0046] For a certain evacuee (or group), the shortest route is not necessarily the safest evacuation route for that evacuee. If multiple groups pass through the same evacuation route at the same time, the evacuees may end up being guided to a route that is actually impassable. Therefore, the creator of the learning data considers, depending on the distribution of evacuees (or groups), what route and which exit each evacuee (or group) should be guided to in order to complete the evacuation smoothly and in the shortest time.

[0047] For example, FIG. 7 is a diagram illustrating an example of the thought process when a learning data creator considers an evacuation route. In FIG. 7, the distribution of the positions of evacuees within building B1 (distribution based on the distribution information of the learning data) is shown by a distribution of circles, along with the internal structure of building B1. The positions of all or some of the image capture devices 20 and display devices 30 are also shown. Furthermore, disaster-stricken areas are represented by crosses. In this case, the learning data creator considers an evacuation route, for example, as follows:

[0048] (1) Exits E1 to E4 can all be used as emergency exits.

[0049] (2) The closest exit to group P is exit e1, but there are many people nearby and it takes a long time to pass through exit e1, so group P needs to be guided to another exit.

[0050] (3) There are people near exit e2, but they can pass by before group P arrives, so it is better to guide group P to exit e2.

[0051] The arrows in FIG. 7 indicate the direction of travel of each group.

[0052] When the learning data creator has completed the study of evacuation routes, he / she assigns a direction of travel along the evacuation route to each display device 30 associated with the evacuation route. The result is a guidance information sequence that serves as the correct answer.

[0053] Furthermore, if the guidance information generation model 16 also inputs attribute information of each evacuee and outputs guidance information according to the attribute information, the learning data creator will consider evacuation routes using the attribute information of the evacuee as a constraint.

[0054] Fig. 8 shows an example of restrictions based on the attribute information of evacuees. In Fig. 8, the types of attribute information of evacuees are classified into able-bodied, injured / disabled, wheelchair users, those with pets, and those with a lot of baggage, and for each type, whether stairs can be used and whether handrails are required for passage are shown.

[0055] When attribute information is included in the input data for the learning data, the creator of the learning data must also take into consideration the attribute information of the evacuees in addition to the distribution of evacuees shown in FIG.

[0056] 9 is a diagram showing an example of the distribution of evacuees taking into consideration the attribute information of the evacuees. In FIG. 9, the same parts as in FIG. 7 are given the same reference numerals.

[0057] The distribution of evacuees in Figure 9 is the same as in Figure 7, but the attribute information for each evacuee is differentiated. Differences in attribute information are expressed by different backgrounds of the circles. In this case, the person creating the learning data must consider the constraints shown in Figure 8 and determine the correct guidance information corresponding to this distribution information and attribute information. For example, for wheelchair users, an evacuation route that does not use stairs must be considered.

[0058] The training data creator performs the above-mentioned examination for each pattern (validation) of input data in the training data. The guidance information generation model 16 trained based on the training data thus created can estimate guidance information corresponding to the above-mentioned thought logic.

[0059] Returning to Fig. 3, the display information generation unit 14 generates display information for each display device 30 based on the guide information corresponding to that display device 30. The display information is information to be displayed on each display device, which is generated based on the guide information corresponding to each display device.

[0060] The transmitting unit 15 transmits, to each display device 30, display information corresponding to that display device 30.

[0061] The following describes the processing procedure executed by the information processing device 10. Fig. 10 is a flowchart for explaining an example of the processing procedure executed by the information processing device 10 in the first embodiment. The processing procedure in Fig. 10 may be executed during normal times, or may be started in response to the occurrence of a disaster.

[0062] In step S101, the receiving unit 11 receives current captured images from each of the photographing devices 20. The photographing devices 20 may transmit captured images to the receiving unit 11 in response to a request from the receiving unit 11, or may actively transmit captured images to the receiving unit 11. As described above, the receiving unit 11 may receive captured images from a server that collects captured images captured by each of the photographing devices 20.

[0063] Next, the distribution information generating unit 12 generates distribution information and attribute information strings of evacuees based on the plurality of captured images (S102).

[0064] FIG. 11 is a diagram illustrating an example of a method for generating distribution information and attribute information sequences. As shown in FIG. 11, the distribution information generation unit 12 generates (extracts) data d1 for each camera device 20 by performing skeletal structure detection on the captured image from the camera device 20. The data d1 includes location information for each evacuee included in the captured image and attribute information that can be estimated from the evacuee's skeleton. The location information is location information in a map coordinate system. The distribution information generation unit 12 combines the data d1 extracted for each camera device 20 into one piece to generate one piece of data d2. The data d2 is a distribution information and attribute information sequence. When combining the data d1, the distribution information generation unit 12 may combine location information and attribute information that are likely to belong to the same person into one piece. Whether or not the data belong to the same person may be determined based on the similarity between the location information and the attribute information.

[0065] Returning to Fig. 10, following step S102, the guide information generation unit 13 generates a guide information sequence and a disaster location sequence by inputting input data into the guide information generation model 16 (S103). Here, the input data includes map information stored in the map information storage unit 121, an exit information sequence stored in the exit information storage unit 122, a shooting position sequence stored in the shooting position storage unit 123, a display position sequence stored in the display position storage unit 124, a captured image sequence in which the captured images received by the receiving unit 11 in step S101 are arranged in a predetermined order, and distribution information (attribute information sequence) generated by the distribution information generation unit 12 in step S102.

[0066] Next, the display information generation unit 14 generates, for each display device 30, display information based on the guide information in the guide information sequence that corresponds to that display device 30 (S104). In the first embodiment, an image indicating the traveling direction is generated as the display information.

[0067] Next, the transmitting unit 15 transmits, for each piece of display information generated for each display device 30, the display information generated for that display information to that display device 30 (S105). For example, the information processing device 10 stores a display device table that is information related to the display devices 30.

[0068] FIG. 12 is a diagram showing an example of the configuration of the display device table. In FIG. 12, one row (record) corresponds to one display device 30. As shown in FIG. 12, the display device table stores information such as the location information, IP address, and WebAPI of each display device 30. The transmission unit 15 determines which display information to send to each display device 30 based on the display device table. For example, the transmission unit 15 identifies the IP address of each display device 30 based on the display device table, and also identifies the WebAPI for sending the display information. In addition, the information processing device 10 stores a format for displaying on the display device 30. This format may use symbols such as characters or arrows, or may be an image such as a map.

[0069] Next, the information processing device 10 waits for a certain period of time (No in S106), and after the certain period of time has elapsed (Yes in S106), repeats step S101 and subsequent steps. By repeating step S101 and subsequent steps, it is possible to change the guidance information from moment to moment in response to, for example, changes in the disaster area or changes in the distribution of evacuees' positions.

[0070] Each display device 30 that has received the display information displays the display information.

[0071] Fig. 13 is a diagram showing an example of display information in the first embodiment. Fig. 13 shows an example of display information when an earthquake occurs.

[0072] (1) shows an example of display information when the attribute information of the evacuees is not distinguished (the attribute information is not used as an input for the guidance information generation model 16) or when the guidance information is the same for all types of attribute information. (1) shows that the evacuees should go to the right (the direction of travel is right).

[0073] (2) shows an example of information displayed according to the attribute information of evacuees. In addition to (1), (2) indicates that wheelchair users should move to the left.

[0074] Each evacuee can determine the evacuation direction by referring to such guidance information. During the evacuation process, evacuees who encounter multiple display devices 30 move according to the guidance information for each display device 30. The guidance information is generated by the guidance information generation model 16, which is trained to estimate evacuation information based on a safe evacuation route. Therefore, a smooth evacuation can be expected if each evacuee evacuates according to the evacuation information.

[0075] In addition to evacuation guidance, it can also be used to smoothly guide people to the exit at places where large numbers of people gather at once, such as event venues.

[0076] As described above, according to the first embodiment, it is possible to support smooth evacuation from a damaged building.

[0077] Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be described. Therefore, unless otherwise specified, the second embodiment may be the same as the first embodiment.

[0078] In the second embodiment, an example will be described in which the guidance information corresponding to each display device 30 is information indicating a route (evacuation route) from the position of the display device 30 to one of the exits (hereinafter referred to as "route information").

[0079] In this case, the guidance information sequence among the learning data of the guide information generation model 16 described in Fig. 5 may be an array of route information for each display device 30. When the guide information generation model 16 also receives attribute information as input, the guidance information sequence is an array of elements including route information according to the attribute information for each display device 30. How the learning data creator should consider evacuation routes is as described in the first embodiment.

[0080] Here, the route information may be expressed, for example, as in known techniques, by graph structure data, which is a set of nodes and a set of edges connecting the nodes. Each node can be assigned a coordinate value in a map coordinate system. In this way, the route can be expressed in the map coordinate system.

[0081] In this case, in step S103 of FIG. 10, the guide information generating unit 13 generates route information for each display device 30, or route information according to attribute information for each display device 30, and a disaster location sequence as the guide information sequence.

[0082] In step S104, the display information generating unit 14 generates, for each display device 30, display information indicating an evacuation route from the display device 30 to one of the exits, based on the guidance information (route information) corresponding to the display information. At this time, the display information generating unit 14 generates display information in which information indicating the evacuation route is superimposed on the map information stored in the map information storage unit 121. If coordinate values ​​in the map coordinate system are assigned to the route information as described above, the evacuation route can be easily superimposed on the map information, which is point cloud data in the map coordinate system.

[0083] The display information generating unit 14 may also superimpose, on the display information, information (graphics or images) indicating each of the disaster-affected locations indicated by the disaster-affected location sequence generated in step S103.

[0084] In step S105, the transmission unit 15 transmits the display information to each display device 30. As a result, in the second embodiment, for example, display information such as that shown in FIG.

[0085] Fig. 14 is a diagram showing an example of display information in the second embodiment. In Fig. 14, arrows r1 to r4 representing an evacuation route from the position p1 of the display device 30 to the exit e1 are superimposed on map information (which indicates the internal structure of the building). In addition, information (image) indicating a disaster area p2 is superimposed on the map information. Fig. 14 corresponds to an example of a fire, and an image of flames is superimposed on the disaster area p2.

[0086] The display information generating unit 14 may also superimpose information about the interior of building B1 on the map information. The information about the interior of building B1 is information about facilities within the building and information about equipment within the building. The information about facilities within the building may be names of facilities within the building, etc., or images of the interior of the building. The names of facilities within the building, etc., and images of the interior of the building may be stored in the map information storage unit 121 in association with their respective location information (coordinate values ​​in the map coordinate system). Furthermore, the information about equipment within the building is information indicating precautions for equipment within building B1. For example, information such as locations where doors cannot be opened or locations that are aging may be stored in advance. Such information may be cited, for example, from an equipment management ledger or the like.

[0087] As described above, according to the second embodiment, each evacuee can understand the route to the exit by referring to the guidance information displayed on the display device 30. The route is generated by the trained guidance information generation model 16. Therefore, similar to the first embodiment, it is possible to support smooth evacuation from a damaged building.

[0088] Next, a third embodiment will be described. In the third embodiment, differences from the above-described embodiments will be described. Therefore, unless otherwise specified, the third embodiment may be the same as the first or second embodiment.

[0089] When a disaster occurs, evacuees tend to become more anxious if they are unable to accurately grasp the extent of the damage. In the third embodiment, an example will be described in which the damage situation can be communicated to evacuees. Specifically, in the third embodiment, an example will be described in which live video (images currently captured) of the affected area captured by the image capturing device 20 during the occurrence of a disaster is displayed on each display device 30.

[0090] In the third embodiment, the process executed by the display information generating unit 14 in step S104 in FIG. 10 is different from that in the above-described embodiments.

[0091] In step S104, the display information generator 14 generates, for each display device 30, display information based on the guidance information corresponding to that display device 30 in the guidance information sequence and based on the captured image from the imaging device 20 corresponding to any of the disaster-stricken locations included in the disaster-stricken location sequence. The display information may include the captured image together with the guidance information, or may be display information in which the display target can be switched between the guidance information and the captured image by a predetermined operation by the user on the display device 30.

[0092] Fig. 15 is a diagram showing a display example of display information in the third embodiment. Fig. 15 shows an example in which a photographed image g1 corresponding to a disaster-stricken area is added to the display example of Fig. 13 (1).

[0093] The camera device 20 corresponding to the affected area may be identified based on, for example, the position information of the camera device 20 stored in the camera position storage unit 123. Alternatively, information indicating, in a map coordinate system, range information corresponding to images captured by each camera device 20 may be stored in advance in the camera position storage unit 123, and the camera device 20 corresponding to the range information including the affected area may be identified to identify the camera device 20 corresponding to the affected area.

[0094] In step S105, the transmission unit 15 transmits the display information to each display device 30. As a result, in the third embodiment, a captured image including an evacuation location is displayed on each piece of display information.

[0095] 10 shows a processing procedure for receiving captured images at regular intervals, but in the third embodiment, for example, the receiving unit 11 may receive live images (video) from each of the image capturing devices 20 by streaming, etc. This allows evacuees to check live video of the disaster-stricken areas.

[0096] As described above, according to the third embodiment, live images of the disaster area are displayed on the display device 30, which can reduce the anxiety of evacuees.

[0097] In the above-described embodiments, the output from the guide information generation model 16 is guide information, and the display information generating unit 14 generates display information based on the guide information. However, the output from the guide information generation model 16 may be display information based on the guide information for each piece of display information. In this case, the guide information sequence as the correct answer in the learning data shown in FIG. 5 may be replaced with a display information sequence. The display information sequence refers to an arrangement of display information for each display device 30.

[0098] The information processing device 10 is not limited to a general-purpose server computer as long as it is a device having a communication function and a calculation function. The information processing device 10 may be, for example, a PJ (Projector), an IWB (Interactive White Board: a white board with an electronic blackboard function that allows mutual communication), an output device such as digital signage, a HUD (Head Up Display) device, industrial machinery, an imaging device, a sound collection device, medical equipment, a network home appliance, a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, a desktop PC, or the like.

[0099] Each function of each embodiment can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to execute each function described above.

[0100] Additionally, the devices described in the above embodiments are merely illustrative of one of several computing environments for implementing the embodiments disclosed herein.

[0101] In one embodiment, information processing apparatus 10 includes multiple computing devices, such as a server cluster, configured to communicate with each other over any type of communications link, including a network, shared memory, etc., and to perform the processes disclosed herein.

[0102] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

[0103] For example, aspects of the present invention are as follows.

[0104] <1> a receiving unit that receives images captured by a plurality of image capturing devices arranged inside the building so as to capture images of at least a plurality of exits; a distribution information generating unit that generates distribution information of people's positions inside the building based on the plurality of captured images; a guidance information generation unit that causes a machine learning model to generate guidance information for guiding people to the exits based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of a plurality of display devices dispersedly disposed inside the building, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; and a display information generating unit that generates display information for each of the display devices based on the guidance information corresponding to the display device; a transmitting unit that transmits the display information corresponding to each of the display devices to the display devices; and the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; 1. An information processing device comprising:

[0105] <2> the distribution information generation unit extracts attribute information of each of the people from the captured image; the guidance information generation unit causes the machine learning model to generate the guidance information according to the attribute information for each of the display devices, based on the input data further including the attribute information and constraint information that is information indicating constraints on the attribute information; the machine learning model has learned a correspondence relationship between the input data including the attribute information and constraint information that is information indicating constraints on the attribute information, and guidance information corresponding to the attribute information for each display device. Characterized by <1> The information processing device described in

[0106] <3> The guidance information is information indicating a traveling direction relative to an exit. Characterized by <1> or <2> The information processing device described in

[0107] <4> The machine learning model has further learned a correspondence relationship between the input data and location information of damaged areas in the building, The guidance information generation unit further causes a machine learning model to generate position information of a damaged area in the building based on the input data, the display information generation unit generates display information including information about the affected area. Characterized by <1> ~ <3> 2. The information processing device according to claim 1 ,

[0108] <5> the display information generation unit generates display information in which information indicating the disaster-stricken area is superimposed on the map information. Characterized by <4> The information processing device described in

[0109] <6> the display information generation unit generates the display information including the photographed image corresponding to the disaster-stricken area. Characterized by <4> or <5> The information processing device described in

[0110] <7> the display information generation unit further generates display information in which information about the interior of the building is superimposed on the map information. Characterized by <4> ~ <6> 2. The information processing device according to claim 1 ,

[0111] <8> The guidance information is information indicating a route from the position of the display device to an exit of the building. Characterized by <1> The information processing device described in

[0112] <9> the display information generation unit generates display information in which information indicating the route is superimposed on the map information. Characterized by <8> The information processing device described in

[0113] <10> The map information is point cloud data. Characterized by <1> ~ <9> 2. The information processing device according to claim 1 ,

[0114] <11> the receiving unit receives the captured images from the plurality of image capturing devices. Characterized by <1> The information processing device described in

[0115] <12> An information processing system including a plurality of image capturing devices arranged inside a building so as to capture images of situations of at least a plurality of exits, a plurality of display devices arranged in a dispersed manner inside the building, and an information processing device, The information processing device includes: a receiving unit that receives images captured by the plurality of imaging devices; a distribution information generating unit that generates distribution information of people's positions inside the building based on the plurality of captured images; a guidance information generation unit that causes a machine learning model to generate guidance information for guiding people to the exits based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of the plurality of display devices, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; and a display information generating unit that generates display information for each of the display devices based on the guidance information corresponding to the display device; a transmitting unit that transmits the display information corresponding to each of the display devices to the display devices; and the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; An information processing system comprising:

[0116] <13> a receiving step of receiving images captured by a plurality of image capturing devices arranged inside the building so as to capture images of at least a plurality of exits; a distribution information generation step of generating distribution information of positions of people inside the building based on the plurality of captured images; a guidance information generation step of causing a machine learning model to generate guidance information for guiding people to the exits, based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of a plurality of display devices dispersedly disposed inside the building, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; a display information generating step of generating, for each of the display devices, display information based on the guidance information corresponding to the display device; a transmission step of transmitting the display information corresponding to each of the display devices to the display devices; The computer executes the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; An information processing method comprising:

[0117] <14> a receiving step of receiving images captured by a plurality of image capturing devices arranged inside the building so as to capture images of at least a plurality of exits; a distribution information generation step of generating distribution information of positions of people inside the building based on the plurality of captured images; a guidance information generation step of causing a machine learning model to generate guidance information for guiding people to the exits, based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of a plurality of display devices dispersedly disposed inside the building, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; a display information generating step of generating, for each of the display devices, display information based on the guidance information corresponding to the display device; a transmission step of transmitting the display information corresponding to each of the display devices to the display devices; on the computer, the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; A program characterized by: [Explanation of symbols]

[0118] 10. Information processing equipment 11 Receiving unit 12 Distribution information generation section 13 Guidance information generation section 14 Display information generation section 15 Transmitter 16 Guidance Information Generation Model 20 Imaging equipment 30 Display device 121 Map information storage unit 122 Exit information storage unit 123 Shooting position memory unit 124 Display position storage section [Prior art documents] [Patent documents]

[0119] [Patent Document 1] Japanese Patent Publication No. 2023-102161

Claims

1. a receiving unit that receives images captured by a plurality of image capturing devices arranged inside the building so as to capture images of at least a plurality of exits; a distribution information generating unit that generates distribution information of people's positions inside the building based on the plurality of captured images; a guidance information generation unit that causes a machine learning model to generate guidance information for guiding people to the exits based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of a plurality of display devices dispersedly disposed inside the building, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; and a display information generating unit that generates display information for each of the display devices based on the guidance information corresponding to the display device; a transmitting unit that transmits the display information corresponding to each of the display devices to the display devices; and the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; 1. An information processing device comprising:

2. the distribution information generation unit extracts attribute information of each of the people from the captured image; the guidance information generation unit causes the machine learning model to generate the guidance information according to the attribute information for each of the display devices, based on the input data further including the attribute information and constraint information that is information indicating constraints on the attribute information; the machine learning model has learned a correspondence relationship between the input data including the attribute information and constraint information that is information indicating constraints on the attribute information, and guidance information corresponding to the attribute information for each display device.

2. The information processing apparatus according to claim 1, wherein:

3. The guidance information is information indicating a traveling direction relative to an exit.

2. The information processing apparatus according to claim 1, wherein:

4. The machine learning model has further learned a correspondence relationship between the input data and location information of damaged areas in the building, The guidance information generation unit further causes a machine learning model to generate position information of a damaged area in the building based on the input data, the display information generation unit generates display information including information about the affected area.

2. The information processing apparatus according to claim 1, wherein:

5. the display information generation unit generates display information in which information indicating the disaster-stricken area is superimposed on the map information.

5. The information processing apparatus according to claim 4,

6. the display information generation unit generates the display information including the photographed image corresponding to the disaster-stricken area.

5. The information processing apparatus according to claim 4,

7. the display information generation unit further generates display information in which information about the interior of the building is superimposed on the map information.

5. The information processing apparatus according to claim 4,

8. The guidance information is information indicating a route from the position of the display device to an exit of the building.

2. The information processing apparatus according to claim 1, wherein:

9. the display information generation unit generates display information in which information indicating the route is superimposed on the map information.

9. The information processing apparatus according to claim 8,

10. The map information is point cloud data.

2. The information processing apparatus according to claim 1, wherein:

11. the receiving unit receives the captured images from the plurality of image capturing devices.

2. The information processing apparatus according to claim 1, wherein:

12. An information processing system including a plurality of image capturing devices arranged inside a building so as to capture images of situations of at least a plurality of exits, a plurality of display devices arranged in a dispersed manner inside the building, and an information processing device, The information processing device includes: a receiving unit that receives images captured by the plurality of imaging devices; a distribution information generating unit that generates distribution information of people's positions inside the building based on the plurality of captured images; a guidance information generation unit that causes a machine learning model to generate guidance information for guiding people to the exits based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of the plurality of display devices, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; and a display information generating unit that generates display information for each of the display devices based on the guidance information corresponding to the display device; a transmitting unit that transmits the display information corresponding to each of the display devices to the display devices; and the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; An information processing system comprising:

13. a receiving step of receiving images captured by a plurality of image capturing devices arranged inside the building so as to capture images of at least a plurality of exits; a distribution information generation step of generating distribution information of positions of people inside the building based on the plurality of captured images; a guidance information generation step of causing a machine learning model to generate guidance information for guiding people to the exits, based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of a plurality of display devices dispersedly disposed inside the building, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; a display information generating step of generating, for each of the display devices, display information based on the guidance information corresponding to the display device; a transmission step of transmitting the display information corresponding to each of the display devices to the display devices; The computer executes the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; An information processing method comprising:

14. a receiving step of receiving images captured by a plurality of image capturing devices arranged inside the building so as to capture images of at least a plurality of exits; a distribution information generation step of generating distribution information of positions of people inside the building based on the plurality of captured images; a guidance information generation step of causing a machine learning model to generate guidance information for guiding people to the exits, based on input data including map information indicating the internal structure of the building, exit information indicating the positions of a plurality of exits of the building, position information of the image capturing device, position information of a plurality of display devices dispersedly disposed inside the building, the captured images, and the distribution information, the guidance information corresponding to each of the display devices; a display information generating step of generating, for each of the display devices, display information based on the guidance information corresponding to the display device; a transmission step of transmitting the display information corresponding to each of the display devices to the display devices; on the computer, the machine learning model has learned a correspondence relationship between the input data and the guidance information for each of the display devices; A program characterized by:

Images