Information processing device, information processing method, and program

Abstract

This information processing device (10) has an acquisition unit (11) that acquires an image captured at a site at which trouble has occurred, an information generation unit (12) that detects people in the image and generates information about the detected people on the basis of the image, and an output unit (13) that outputs the information about the detected people.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Techniques related to the present invention are disclosed in Patent Documents 1 and 2 and Non-Patent Document 1.

[0003] Patent Document 1 discloses a technology that uses a wearable eyeglasses-type terminal to support firefighting activities by firefighters, etc. Specifically, it discloses that the layout of a structure, vital signs, etc. are displayed via the wearable eyeglasses-type terminal.

[0004] Patent Document 2 discloses a technology that calculates the feature values ​​of each of multiple key points of a human body contained in an image, searches for images containing human bodies with similar postures or movements based on the calculated feature values, and classifies images with similar postures or movements together.

[0005] Patent Document 3 discloses a technology for supporting rescue operations based on location information of mobile devices carried by the person in need of rescue and the rescue team members.

[0006] Patent Document 4 discloses a technology for detecting emergency situations in public facilities, buildings, vehicles and transportation networks.

[0007] Non-Patent Document 1 discloses a technique related to human skeleton estimation. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Special table number 2015-504616 [Patent Document 2] International Publication No. 2021 / 084677 [Patent Document 3] Patent Publication No. 2018-142338 [Patent Document 4] Special table number 2019-534488 [Non-patent literature]

[0009] [Non-Patent Document 1] Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, P. 7291-7299 Summary of the Invention [Problem to be solved by the invention]

[0010] An object of the present invention is to provide a new technology that supports workers who carry out rescue operations at the scene of a trouble. [Means for solving the problem]

[0011] According to the present invention, an acquisition means for acquiring an image taken at the site where the trouble occurred; an information generating means for detecting a person from the image and generating information about the detected person based on the image; an output means for outputting information about the detected person; An information processing device having the above configuration is provided.

[0012] Further, according to the present invention, The computer an acquisition step of acquiring images taken at the site where the trouble occurred; an information generating step of detecting a person from the image and generating information about the detected person based on the image; an output step of outputting information about the detected person; An information processing method is provided that performs the above.

[0013] Further, according to the present invention, Computer, an acquisition means for acquiring images taken at the site where the trouble occurred; an information generating means for detecting a person from the image and generating information about the detected person based on the image; an output means for outputting information about the detected person; A program is provided to function as a [Effects of the Invention]

[0014] According to the present invention, a new technique is provided to support workers who carry out rescue operations at the scene of a trouble. [Brief explanation of the drawings]

[0015] The above and other objects, features and advantages are described below. Suitable This will become more apparent from the following embodiments and the accompanying drawings.

[0016] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional block diagram of a support system. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the apparatus. [Figure 3] FIG. 1 is a diagram illustrating an example of a functional block diagram of an information processing apparatus. [Figure 4] FIG. 10 is a diagram for explaining processing by an information generating unit. [Figure 5] 10 is a flowchart illustrating an example of a processing flow of an information processing device. [Figure 6] FIG. 10 is a diagram illustrating another example of a functional block diagram of the support system. [Figure 7] FIG. 2 is a diagram schematically illustrating an example of information output by an information processing device. [Figure 8]FIG. 10 is a diagram schematically illustrating another example of information output by the information processing device. [Figure 9] FIG. 10 is a diagram schematically illustrating another example of information output by the information processing device. [Figure 10] FIG. 1 is a diagram illustrating an example of a functional block diagram of an information processing apparatus. [Figure 11] FIG. 2 is a diagram schematically illustrating an example of information processed by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0018] First Embodiment "overview" The information processing device of this embodiment uses image analysis technology to support workers who are conducting rescue operations at the scene of a problem (hereinafter, sometimes simply referred to as the "scene"). Specifically, when the information processing device acquires an image taken at the scene of the problem, it detects a person from the image and generates information about the detected person based on the image. Then, the information processing device outputs the generated information.

[0019] Image analysis techniques are used to detect people and generate information about the detected people. Examples of image analysis techniques used by information processing devices include face recognition, human shape recognition, posture recognition, movement recognition, appearance attribute recognition, image gradient feature detection, image color feature detection, and object recognition.

[0020] "Overall view of support systems with information processing devices" An example of a functional block diagram of the support system is shown in Fig. 1. As shown in the figure, the support system includes an information processing device 10 and a worker terminal 20.

[0021] The outline of the information processing device 10 is as described above. The detailed configuration of the information processing device 10 will be described later.

[0022] The worker terminal 20 is a terminal carried by a worker who performs rescue operations at the scene of a trouble. The worker terminal 20 has at least a calculation function, a camera function, a communication function, an output function, and an input function.

[0023] The calculation function provided in the worker terminal 20 is a function for processing input data and program instructions.

[0024] The camera function of the worker terminal 20 may be a function to detect visible light and create an image, a function to detect infrared light and create an image, a camera to detect ultraviolet light and create an image, a function to detect other electromagnetic waves and create an image, or a combination of these functions. The image is a concept that includes at least one of a moving image and a still image.

[0025] The communication function provided in the worker terminal 20 is a function for communicating with an external device. The worker terminal 20 uses this communication function to communicate with the information processing device 10. The communication function provided in the worker terminal 20 may be a function for communicating using a public line such as the Internet, or a function for communicating using a dedicated communication line.

[0026] The output function of the worker terminal 20 is a function for outputting information to the worker. Examples of the output function of the worker terminal 20 include, but are not limited to, a display, a projector, a speaker, etc.

[0027] The input function of the worker terminal 20 is a function for receiving input from the worker. Examples of the input function of the worker terminal 20 include, but are not limited to, a touch panel, physical buttons, a microphone, a camera that receives gesture input, and the like.

[0028] Examples of worker terminals 20 equipped with such functions include, but are not limited to, wearable terminals (glasses, helmets, watches, etc.), smartphones, mobile phones, tablet terminals, etc. Considering that the terminals will be used during rescue operations, a wearable terminal is preferable.

[0029] The worker terminal 20 transmits an image generated by the camera function to the information processing device 10 using the communication function. The information processing device 10 detects a person from the received image and generates information about the detected person based on the image. 10 The information processing device 10 transmits information about the detected person to the worker terminal 20. The worker terminal 20 outputs the information received from the information processing device 10 via an output function.

[0030] "Hardware Configuration" Next, an example of the hardware configuration of the information processing device 10 and the worker terminal 20 will be described. Each functional unit of the information processing device 10 and the worker terminal 20 is realized by any combination of hardware and software, centered on a central processing unit (CPU) of any computer, memory, programs loaded into the memory, a storage unit such as a hard disk that stores the programs (this can store programs that are pre-loaded when the device is shipped, as well as programs downloaded from storage media such as CDs (compact discs) or servers on the Internet), and a network connection interface. Those skilled in the art will understand that there are many variations in the realization methods and devices.

[0031] FIG. 2 is a block diagram illustrating an example of the hardware configuration of the information processing device 10 and the worker terminal 20. As shown in FIG. 2, the information processing device 10 and the worker terminal 20 have a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The information processing device 10 and the worker terminal 20 do not necessarily have the peripheral circuit 4A. Note that the information processing device 10 and the worker terminal 20 may be configured as multiple devices that are physically and / or logically separated. In this case, each of the multiple devices can have the above hardware configuration.

[0032] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to transmit and receive data among them. The processor 1A is an arithmetic processing device such as a CPU or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input / output interface 3A includes an interface for acquiring information from an input device, an external device, an external server, an external sensor, a camera, etc., and an interface for outputting information to an output device, an external device, an external server, etc. Examples of input devices include a keyboard, a mouse, a microphone, physical buttons, a touch panel, etc. Examples of output devices include a display, a speaker, a printer, a mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0033] "Function Configuration" Next, the functional configuration of the information processing device 10 of this embodiment will be described in detail. An example of a functional block diagram of the information processing device 10 of this embodiment is shown in Fig. 3. As shown in the figure, the information processing device 10 has an acquisition unit 11, an information generation unit 12, and an output unit 13.

[0034] The acquisition unit 11 acquires images taken at the site where a problem has occurred. Specifically, the acquisition unit 11 acquires images generated by the worker terminal 20. The acquisition unit 11 acquires the images generated by the worker terminal 20 in real time.

[0035] The information generating unit 12 detects people from the image acquired by the acquiring unit 11, and generates information about the detected people based on the image.

[0036] In this embodiment, as shown in Fig. 4, an image analysis system 40 is provided that analyzes an image and outputs the analysis results. The image analysis system 40 may be a part of the information processing device 10, or may be an external device that is physically and / or logically independent from the information processing device 10. The image analysis system 40 may also be a part of the assistance system of this embodiment, or may be an external device that is physically and / or logically independent from the assistance system. The information generation unit 12 uses the image analysis system 40 to detect people as described above and generate information about the detected people.

[0037] Here, we will explain the image analysis system 40. The image analysis system 40 has at least one of a face recognition function, a human shape recognition function, a posture recognition function, a movement recognition function, an appearance attribute recognition function, an image gradient feature detection function, an image color feature detection function, and an object recognition function.

[0038] The face recognition function extracts a person's facial features and locates the position of the person's face within the image. It may also compare and calculate the similarity between facial features (to determine whether they are the same person, etc.). The human form recognition function extracts a person's physical features (e.g., overall characteristics such as whether they are fat or thin, height, clothing, etc.) and locates the position of the person within the image. It may also compare and calculate the similarity between physical features (to determine whether they are the same person, etc.).

[0039] The posture recognition function and movement recognition function detect the joint points of a person and connect the joint points to form a stick figure model. Then, information about the stick figure model is used to estimate the person's height, extract posture features, identify movement based on changes in posture, and identify the person's position within the image. Furthermore, similarities between posture features and movement features may be compared and calculated (e.g., to determine whether the posture or movement is the same). The posture recognition function and movement recognition function may be realized by the technology disclosed in Patent Document 2 and Non-Patent Document 1 above.

[0040] The appearance attribute recognition function recognizes the appearance attributes associated with a person (for example, clothing color, shoe color, hairstyle, wearing a hat or tie, etc., there are a total of more than 100 appearance attributes). Furthermore, it may be possible to compare and calculate the similarity of the recognized appearance attributes (it is possible to determine whether they are the same attributes).

[0041] Image gradient feature detection functions include SIFT, SURF, RIFF, ORB, BRISK, CARD, and HOG. Image color feature detection functions generate data that indicates the color characteristics of an image, such as a color histogram. Object recognition functions are realized using engines such as YOLO (which can extract general objects (e.g., cars, bicycles, chairs, etc.) and people). Using object recognition functions, you can detect objects from images and identify their positions within the image.

[0042] 4, the information generating unit 12 inputs an image to the image analysis system 40. Then, the information generating unit 12 acquires the analysis result of the image output from the image analysis system 40.

[0043] The analysis results output from the image analysis system 40 described above include results of detecting people from within the image (for example, analysis results using the face recognition function, analysis results using the human shape recognition function, analysis results using the posture recognition function, analysis results using the movement recognition function, and analysis results using the object recognition function).

[0044] Furthermore, the information generation unit 12 can use the analysis results output from the image analysis system 40 as information about the detected person, such as the analysis results from the face recognition function, the human form recognition function, the posture recognition function, the movement recognition function, the appearance attribute recognition function, the image gradient feature detection function, the image color feature detection function, and the object recognition function. Furthermore, the information generation unit 12 can use information obtained by performing statistical processing, editing, correction, etc. on the analysis results output from the image analysis system 40 as information about the detected person. Specific examples of information generated by the information generation unit 12 will be described in the following embodiments.

[0045] The output unit 13 outputs the information generated by the information generation unit 12. For example, the output unit 13 outputs information about a person detected from an image. The information output by the output unit 13 is transmitted to the worker terminal 20. Then, the worker terminal 20 outputs the information via an output device such as a display.

[0046] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of FIG.

[0047] First, the information processing device 10 acquires an image captured at the site where the trouble occurred (S10). In this embodiment, the information processing device 10 acquires an image generated by the worker terminal 20.

[0048] Next, the information processing device 10 detects a person from the acquired image and generates information about the detected person based on the image (S11). The processing of S11 is realized by using the image analysis system 40.

[0049] Then, the information processing device 10 outputs the information generated in S11, for example, information about the detected person (S12). In this embodiment, the information processing device 10 transmits the information generated in S11 to the worker terminal 20.

[0050] "Action and effect" The information processing device 10 of this embodiment can acquire an image taken at the scene of a trouble, detect a person from the image, and generate and output information about the detected person based on the image. Such information processing device 10 can use image analysis technology to support workers performing rescue operations at the scene of a trouble.

[0051] <Second embodiment> "Overall view of support systems with information processing devices" An example of a functional block diagram of the support system of this embodiment is shown in FIG. 6. As shown in the figure, the support system of this embodiment process The system includes a device 10, a worker terminal 20, and a center terminal 30.

[0052] The configurations of the information processing device 10 and the worker terminal 20 are the same as those in the first embodiment.

[0053] The center terminal 30 is a terminal used at a center that manages rescue operations being carried out at the site where a problem has occurred. For example, a leader or supervisor of the rescue operations uses the center terminal 30. The center terminal 30 has at least a calculation function, a communication function, an output function, and an input function.

[0054] The calculation function provided in the center terminal 30 is a function for processing input data and program instructions.

[0055] The communication function of the center terminal 30 is a function for communicating with external devices. The center terminal 30 uses this communication function to communicate with the information processing device 10. The communication function of the center terminal 30 may be a function for communicating using a public line such as the Internet, or a function for communicating using a dedicated communication line.

[0056] The output function of the center terminal 30 is a function for outputting information to a user. Examples of the output function of the center terminal 30 include, but are not limited to, a display, a projector, a speaker, and the like.

[0057] The input function of the center terminal 30 is a function for accepting input from a user. Examples of the input function of the center terminal 30 include, but are not limited to, a touch panel, physical buttons, a microphone, and a camera that accepts gesture input.

[0058] Examples of the center terminal 30 having such a function include, but are not limited to, a tablet terminal, a personal computer, a computer dedicated to the purpose, a smartphone, a mobile phone, a wearable terminal, and the like.

[0059] The worker terminal 20 transmits images generated by the camera function to the information processing device 10 using the communication function. The center terminal 30 may also obtain images generated by cameras installed at the site where the trouble occurred and sensor data generated by other sensors (temperature sensors, smoke sensors, etc.) by any means, and transmit them to the information processing device 10. The information processing device 10 detects people from the received images and generates information about the detected people based on the images. The information processing device 10 also generates various information based on the received sensor data. Then, the information processing device 10 transmits information about the detected person and information generated based on the sensor data to at least one of the worker terminal 20 and the center terminal 30. At least one of the worker terminal 20 and the center terminal 30 outputs the information received from the information processing device 10 via an output function.

[0060] "Hardware Configuration" An example of the hardware configuration of the information processing device 10 and the worker terminal 20 is the same as that of the first embodiment. Each functional unit of the center terminal 30 is realized by any combination of hardware and software, centered on the CPU of any computer, memory, programs loaded into the memory, a storage unit such as a hard disk that stores the programs (this can store programs that are pre-loaded when the device is shipped, as well as programs downloaded from storage media such as CDs or servers on the Internet), and a network connection interface. Those skilled in the art will understand that there are many variations in the realization methods and devices.

[0061] FIG. 2 is a block diagram illustrating an example of the hardware configuration of the center terminal 30. As shown in FIG. 2, the center terminal 30 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The center terminal 30 does not necessarily have to have the peripheral circuit 4A. Note that the center terminal 30 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices can have the above hardware configuration.

[0062] "Function Configuration" Next, a description will be given of the functional configuration of the information processing device 10. An example of a functional block diagram of the information processing device 10 according to this embodiment is shown in FIG.

[0063] The acquisition unit 11 acquires images captured at the site where the trouble occurred. Specifically, the acquisition unit 11 acquires images generated by the worker terminal 20. The acquisition unit 11 acquires the images generated by the worker terminal 20 in real time. The acquisition unit 11 may also acquire images generated by cameras installed at the site where the trouble occurred and sensor data generated by other sensors (temperature sensors, smoke sensors, etc.). For example, if a server that collects and stores images generated by cameras installed at the site where the trouble occurred and sensor data generated by other sensors (temperature sensors, smoke sensors, etc.) in real time is provided in advance, the acquisition unit 11 may acquire the data from the server by any means. As an example, a worker working at a center that manages rescue operations being performed at the site where the trouble occurred may operate the center terminal 30 to access the server and acquire the data. The acquired data may then be transmitted from the center terminal 30 to the information processing device 10. Note that this means is merely one example, and means for acquiring images generated by cameras installed at the site where the trouble occurred and sensor data generated by other sensors (temperature sensors, smoke sensors, etc.) are not limited to this.

[0064] The information generation unit 12 inputs at least one of an image generated by the worker terminal 20 and an image generated by a camera installed at the site where the trouble occurred into the image analysis system 40, and acquires analysis results output from the image analysis system 40 (e.g., analysis results by a face recognition function, analysis results by a human form recognition function, analysis results by a posture recognition function, analysis results by a movement recognition function, analysis results by an appearance attribute recognition function, analysis results by an image gradient feature detection function, analysis results by an image color feature detection function, and analysis results by an object recognition function). The information generation unit 12 can also generate information by subjecting the analysis results output from the image analysis system 40 to statistical processing, editing, correction, etc. The information generation unit 12 can also generate information by subjecting sensor data generated by sensors (temperature sensors, smoke sensors, etc.) installed at the site where the trouble occurred to statistical processing, editing, correction, etc.

[0065] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of FIG.

[0066] First, the information processing device 10 acquires an image taken at the site where the trouble occurred (S10). In this embodiment, the information processing device 10 acquires an image generated by the worker terminal 20. Furthermore, the information processing device 10 may acquire an image generated by a camera installed at the site where the trouble occurred, or sensor data generated by other sensors (temperature sensor, smoke sensor, etc.).

[0067] Next, the information processing device 10 detects a person from the acquired image and generates information about the detected person based on the image (S11). The processing of S11 is realized using the image analysis system 40. The information processing device 10 can also generate information by performing statistical processing, editing processing, correction processing, etc. on sensor data generated by sensors (temperature sensors, smoke sensors, etc.) installed at the site where the trouble occurred.

[0068] Then, the information processing device 10 outputs the information generated in S11, for example, information about the detected person (S12). In this embodiment, the information processing device 10 transmits the information generated in S11 to at least one of the worker terminal 20 and the center terminal 30.

[0069] Other configurations of the information processing device 10 of this embodiment are the same as those of the first embodiment.

[0070] "Action and effect" The information processing device 10 of this embodiment achieves the same effects as those of the first embodiment. Furthermore, the information processing device 10 of this embodiment can further use images generated by cameras installed at the site where the trouble has occurred and sensor data generated by other sensors (temperature sensors, smoke sensors, etc.) to support workers performing rescue operations at the site where the trouble has occurred.

[0071] <Third embodiment> In this embodiment, information about the detected person generated by the information generation unit 12 is embodied. In this embodiment, the information about the detected person includes location information of the detected person. That is, in this embodiment, a person present at the site where a problem has occurred is detected by image analysis, and location information indicating the location of the detected person is output.

[0072] The information generation unit 12 generates, for example, position information indicating the position within an image of a person detected in the image. Then, the output unit 13 outputs the position information. For example, the output unit 13 may use an AR (Augmented Reality) technique to output information indicating the position within the image of a person detected in the image.

[0073] Alternatively, the information generation unit 12 may generate location information indicating the real-world location of a person detected in an image. For example, the information generation unit 12 identifies an object in the vicinity of the detected person as the location information. This identification is achieved, for example, by the object recognition function of the image analysis system 40. The output unit 13 then outputs information indicating the identified object. For example, the output unit 13 may output information such as "There is a person around the blackboard." In this case, the "blackboard" is the "identified object."

[0074] If a map (such as a floor map) of the site where the problem occurred has been generated in advance, the information processing device 10 may acquire data on the map by any means, such as user input. The map indicates the positions of objects within the site, such as a blackboard, a door, and a window. The information generation unit 12 may then use the map to determine the relative relationship (such as direction and distance) between the worker's current position and the position of the identified object. The output unit 13 may then output information indicating the determined relationship. The worker's current position may be determined, for example, by a global positioning system (GPS) provided in the worker terminal 20, or by detecting the worker from an image generated by a camera installed at the site. An example of the information indicating the relative relationship is, for example, "There is a person around the blackboard, approximately 3 meters north." Alternatively, another example of the information indicating the relative relationship may be, for example, "There is a person around the blackboard, approximately 10 meters ahead, at 10 o'clock to the left." In this example, the information generator 12 needs to identify the direction of the worker at that time (direction of gaze, direction of face, or direction of body). There are various means for achieving this, but examples include "attaching a magnetic sensor to the wearable terminal (worker terminal 20) and using data from the magnetic sensor," "detecting the worker from an image captured by a camera installed at the site, and then calculating based on the installation position of the camera, the orientation of the camera, and the state of the worker in the image (which eye or ear is visible in the image, direction of gaze, direction of face, direction of body, etc.)," ​​and "comparing the image from the wearable camera with the image from the camera installed at the site."

[0075] Other configurations of the information processing device 10 of this embodiment are the same as those of the first and second embodiments.

[0076] The information processing device 10 of this embodiment achieves the same effects as those of the first and second embodiments. Furthermore, the information processing device 10 of this embodiment detects people present at the scene of a problem through image analysis and outputs location information indicating the location of the detected people. As a result, it is possible to assist in the task of searching for people to be rescued at the scene. It is also possible to prevent inconveniences such as overlooking people to be rescued.

[0077] <Fourth embodiment> In this embodiment, information about detected people is embodied, which is generated by the information generating unit 12. In this embodiment, the information about detected people indicates at least one of the number of people taking each of a plurality of postures, the number of people performing each of a plurality of movements, the number of people in each age group, and the number of people of each gender.

[0078] "The number of people taking each of several postures" The multiple postures include, but are not limited to, "lying down," "sitting," "standing," "crouching," etc. The posture recognition function of the image analysis system 40 can recognize the posture of each detected person. The information generation unit 12 can then calculate the number of people in each posture by aggregating the recognition results. The output unit 13 outputs the number of people in each of the multiple postures. For example, the output unit 13 may output information such as "people lying down: 3 people, people sitting: 5 people, person crouching: 1 person." Based on this information, workers can quickly grasp the situation on site.

[0079] "The number of people performing each of several movements" The multiple movements include, but are not limited to, "moving while standing," "moving while sitting," "crawling," and "not moving." The movement recognition function of the image analysis system 40 can recognize the movement of each detected person. The information generation unit 12 can then calculate the number of people performing each movement by aggregating the recognition results. The output unit 13 outputs the number of people performing each of the multiple movements. For example, the output unit 13 may output information such as "people moving while standing: 2 people, people moving while sitting: 5 people, people crawling: 4 people, people not moving: 2 people." Based on this information, workers can quickly grasp the situation on site.

[0080] The number of people in each age group The image analysis system 40 may further have a function to estimate the age group of the detected people. The image analysis system 40 can implement this function by employing any known technology. The information generation unit 12 can then calculate the number of people in each age group by aggregating the estimation results. The output unit 13 outputs the number of people in each age group. For example, the output unit 13 may output information such as "People under 10: 0 people, People in their teens: 2 people, People in their 20s to 40s: 7 people, People in their 50s to 60s: 5 people, People in their 70s or older: 3 people." Based on this information, workers can quickly grasp the situation at the site.

[0081] The number of people of each gender The image analysis system 40 may further have a function to estimate the gender of detected people. The image analysis system 40 can realize this function by employing any known technology. The information generation unit 12 can then calculate the number of people of each gender by aggregating the estimation results. The output unit 13 outputs the number of people of each gender. For example, the output unit 13 may output information such as "Male: 5, Female: 2." Based on this information, workers can quickly grasp the situation on site.

[0082] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to third embodiments.

[0083] The information processing device 10 of this embodiment achieves the same effects as those of the first to third embodiments. Furthermore, the information processing device 10 of this embodiment detects people present at the scene where a problem has occurred through image analysis, and outputs information indicating at least one of the number of people taking each of a plurality of postures, the number of people performing each of a plurality of movements, the number of people in each age group, and the number of people of each gender. As a result, it is possible to assist in the task of grasping the situation at the scene.

[0084] <Fifth embodiment> This embodiment embodies information about detected people generated by the information generating unit 12. In this embodiment, the information about detected people indicates at least one of the number of people and their location information that meets a condition defined using at least one of posture, movement, age group, and gender.

[0085] The conditions are defined so that they are met by, for example, people who need rescue urgently or who have a high priority for rescue, etc. Examples of the conditions include, but are not limited to, "(posture) lying down or crouching" and "(age group) under 10 years old or over 70 years old, and (movement) not moving."

[0086] The information generation unit 12 can acquire location information of people who meet the conditions based on the results output from the image analysis system 40. Furthermore, the information generation unit 12 can identify the number of people who meet the conditions by aggregating the results output from the image analysis system 40. The output unit 13 outputs information indicating at least one of the number of people who meet the conditions and the location information. For example, the output unit 13 may output information such as "There are three people who meet the conditions." The method for outputting the location information is as described above. Based on the information, the worker can quickly grasp the situation at the scene, for example, the number and locations of people who need rescue urgently or who have a high priority for rescue.

[0087] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to fourth embodiments.

[0088] The information processing device 10 of this embodiment achieves the same effects as those of the first to fourth embodiments. Furthermore, the information processing device 10 of this embodiment detects people present at the site where a problem has occurred through image analysis, and outputs at least one of the number of people and their location information that meets conditions defined using at least one of posture, movement, age group, and gender. As a result, it is possible to assist in the task of understanding the situation at the site.

[0089] Sixth Embodiment In this embodiment, the information about the detected people generated by the information generating unit 12 is embodied. In this embodiment, the information about the detected people indicates a rescue priority for each detected person determined based on at least one of posture, movement, age group, and gender.

[0090] When multiple people are detected from an image, the information generation unit 12 determines rescue priorities for the multiple detected people based on at least one of posture, movement, age group, and gender. The analysis results of the image output from the image analysis system 40 indicate the posture, movement, age group, gender, etc. of each detected person. The information generation unit 12 determines rescue priorities for each of the multiple people detected from the image based on priority determination rules registered in advance in the information processing device 10.

[0091] The content of the priority determination rules is not particularly limited, and for example, conditions for assigning points may be set such as "(Gender) Female: 3 points, Male: 1 point," "(Age group) Under 10 years old: 5 points, Teenagers: 4.5 points, ...," and "(Posture) Lying down: 8 points, Crouching: 7 points, ...." In this case, the information generation unit 12 detects the conditions that apply to each detected person and calculates the total points by adding up the points for the detected conditions. Then, the information generation unit 12 determines the priority in descending order of the total points. Note that the examples given here are merely examples and are not limiting.

[0092] The output unit 13 outputs the priority of each of the detected people. For example, the output unit 13 may use AR technology to output information indicating the priority of each of the detected people in the image. Based on this information, the worker can quickly understand the situation at the scene and the priority of rescuing the multiple people.

[0093] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to fifth embodiments.

[0094] The information processing device 10 of this embodiment achieves the same effects as those of the first to fifth embodiments. Furthermore, the information processing device 10 of this embodiment detects people present at the scene of a problem through image analysis, and outputs a rescue priority determined using at least one of posture, movement, age group, and gender. As a result, it is possible to assist in the task of grasping the situation at the scene.

[0095] Seventh Embodiment In this embodiment, the information generating unit 12 generates information about the environment of the site, and the output unit 13 outputs the generated information about the environment of the site.

[0096] The information about the environment of the site indicates at least one of the current temperature of the site, the trend of time-varying temperature of the site, the current smoke condition of the site, the trend of time-varying smoke condition of the site, the current status of multiple pieces of equipment installed at the site, and the trend of time-varying status of multiple pieces of equipment installed at the site.

[0097] Current temperature at the site The information generating unit 12 can grasp the temperature of the site based on sensor data generated by a temperature sensor installed at the site. Alternatively, the information generating unit 12 may grasp the temperature of the site based on thermography generated by the worker terminal 20. The output unit 13 outputs information indicating the current temperature of the site.

[0098] "Temperature change trends at the site" The information generator 12 can grasp the trend of temperature change over time at the site based on sensor data generated by temperature sensors installed at the site. The trend is classified, for example, as "temperature rising," "temperature falling," or "no temperature change." There are various ways to classify the temperature, but for example, by comparing the temperature T0 from a predetermined time ago with the current temperature T1, if the difference between T0 and T1 is less than a reference value, it is classified as "no temperature change," if the difference between T0 and T1 is equal to or greater than the reference value and T0 > T1, it is classified as "temperature falling," and if the difference between T0 and T1 is equal to or greater than the reference value and T1 > T0, it is classified as "temperature rising."

[0099] Additionally, "temperature falling" may be further subdivided into "slowly falling" and "rapidly falling" based on the magnitude of the difference between T0 and T1. The same applies to "temperature rising."

[0100] "Current smoke situation at the scene" The information generating unit 12 can grasp the smoke condition at the site based on sensor data generated by a smoke sensor installed at the site. The output unit 13 outputs information indicating the current smoke condition at the site. The smoke condition may be expressed as smoke density.

[0101] "Temporal changes in smoke conditions at the site" The information generator 12 can grasp the trend of time-dependent changes in the smoke condition at the site based on sensor data generated by a smoke sensor installed at the site. The trend is classified, for example, into categories such as "smoke density increasing," "smoke density decreasing," and "no change in smoke density." There are various ways of classification, but for example, the classification may be performed by comparing a smoke density D0 from a predetermined time ago with a current smoke density D1, and if the difference between D0 and D1 is less than a reference value, the classification may be "no change in smoke density." If the difference between D0 and D1 is equal to or greater than the reference value and D0 > D1, the classification may be "smoke density decreasing." If the difference between D0 and D1 is equal to or greater than the reference value and D1 > D0, the classification may be "smoke density increasing."

[0102] In addition, "smoke density falling" may be further subdivided into "slowly falling" and "rapidly falling" based on the magnitude of the difference between D0 and D1. The same applies to "smoke density rising."

[0103] "The current status of multiple pieces of equipment installed on-site" The information generating unit 12 is a camera installed at the site where the trouble occurred. La andThe current status of other sensors (temperature sensors, smoke sensors, etc.), specifically, whether or not they have failed after a problem occurs at the site, is grasped. There are various means for realizing this grasping. For example, if a camera or sensor is configured to transmit failure information indicating the failure to a predetermined server when a function other than the communication function of the camera or sensor fails, the information generation unit 12 may grasp whether or not the camera or sensor has failed based on the reception status of the failure information. The reception status of the failure information stored in the predetermined server is input to the information processing device 10 by any means. Furthermore, if the server is unable to communicate with the camera or sensor, the information generation unit 12 may determine that the camera or sensor has failed. The information that the server is unable to communicate with the camera or sensor is input to the information processing device 10 by any means.

[0104] "Temporal changes in the status of multiple pieces of equipment installed on-site" The trend of time-varying changes in the status of multiple devices installed at a site is classified into, for example, "no device failure has occurred," "no change in the number of failed devices," "the number of failed devices is slowly increasing," "the number of failed devices is rapidly increasing," etc. There are various ways of classification, but for example, by comparing the number of failed devices B0 from a predetermined time ago with the current number of failed devices B1, if the difference between B0 and B1 is "0" and both B0 and B1 are "0," it is "no change in the number of failed devices" if the difference between B0 and B1 is "0" and both B0 and B1 are 1 or greater, it is "no change in the number of failed devices," if the difference between B0 and B1 is greater than 0 and less than a first reference value, it is "the number of failed devices is slowly increasing," and if the difference between B0 and B1 is greater than the first reference value, it is "the number of failed devices is rapidly increasing."

[0105] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to sixth embodiments.

[0106] The information processing device 10 of this embodiment achieves the same effects as those of the first to sixth embodiments. Furthermore, the information processing device 10 of this embodiment outputs at least one of the following: the current temperature at the site where the trouble occurred, the trend of temperature change over time at the site, the current state of smoke at the site, the trend of smoke change over time at the site, the current states of multiple devices installed at the site, and the trend of change over time in the states of multiple devices installed at the site. As a result, workers and others can quickly grasp the environment at the site.

[0107] Eighth Embodiment In this embodiment, the information generating unit 12 generates information about the local environment of the site. The environment of the site may vary locally. Therefore, the information generating unit 12 generates information about the local environment of the site.

[0108] The information generating unit 12 treats the site environment identified based on each of the multiple cameras and sensors as information indicating the environment of only a predetermined area (e.g., an area within a predetermined distance from the reference) identified based on the installation position of each camera or sensor. The site environment is identified by the method described in the seventh embodiment. Then, the output unit 13 outputs information about the local site environment thus generated.

[0109] Fig. 7 shows a schematic example of information output by the output unit 13. Fig. 7 is an image in which information about the local environment of the site where the trouble occurred is mapped onto a map of the site.

[0110] In FIG. 7, the current smoke concentration identified by the sensing data of the smoke sensor 1 is below the reference value, so information that "little smoke" is displayed in association with the area surrounding the smoke sensor 1.

[0111] In addition, in FIG. 7, the current smoke concentration identified by the sensing data of the smoke sensor 2 is greater than the reference value, so information that "heavy smoke" is associated with the surrounding area of ​​the smoke sensor 2 is displayed.

[0112] In addition, in FIG. 7, camera 1 broke down after a problem occurred at the site, so information "Possible danger" is displayed in association with the area surrounding camera 1.

[0113] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to seventh embodiments.

[0114] According to the information processing device 10 of this embodiment, the same effects as those of the first to seventh embodiments are realized. Furthermore, according to the information processing device 10 of this embodiment, information indicating the local environment of the site where the trouble occurred is output. As a result, workers and the like can quickly grasp the local environment of the site.

[0115] <Ninth embodiment> In this embodiment, when multiple people are detected from an image, the information generation unit 12 determines the rescue priorities for the detected people based on the information about the local environment of the scene created in the eighth embodiment. The information generation unit 12 determines the priorities so that people in more dangerous positions are given higher priority.

[0116] For example, the information generating unit 12 determines the rescue priority of each of the multiple people detected from the image based on a priority determination rule registered in advance in the information processing device 10.

[0117] The priority determination rule may include, for example, the following conditions for assigning points: "(Smoke density) D2 or higher: 8 points, less than D2 and D3 or higher: 6 points, ...", "(Temperature) T2 or higher: 6 points, less than T2 and T3 or higher: 4 points, ...", or "(Device status) Broken device present within first distance: 3 points, no broken device present within first distance: 1 point, ...". In this case, the information generation unit 12 detects, for each detected person, a condition corresponding to the location of the person (the person's location indicated by the location information described above) and calculates a total point by adding up the points for the detected conditions. The information generation unit 12 then determines the priority in descending order of total points. Note that the example given here is merely an example and is not limited thereto. For example, the priority determination rule may further include the content described in the sixth embodiment.

[0118] The output unit 13 outputs the priority of each of the detected people. For example, the output unit 13 may use AR technology to output information indicating the priority of each of the detected people in the image. Based on this information, the worker can quickly understand the situation at the scene and the priority of rescuing the multiple people.

[0119] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to eighth embodiments.

[0120] The information processing device 10 of this embodiment achieves the same effects as those of the first to eighth embodiments. Furthermore, the information processing device 10 of this embodiment detects people present at the scene of trouble by image analysis, and outputs a rescue priority determined based on the environment of the location of the detected people. As a result, it is possible to assist in the task of grasping the situation at the scene.

[0121] <Tenth embodiment> In this embodiment, the information generating unit 12 calculates an evacuation route based on the information about the local environment of the site created in the eighth embodiment.

[0122] An example is shown in Figure 8. In the example shown, the route from the worker's current location to the exit is calculated. The calculated route is indicated by an arrow. There are various methods for calculating the route. An example will be explained below.

[0123] First, the information generation unit 12 searches for multiple routes from the current location to a destination such as an exit based on a floor map of the site and the worker's current location. Then, the information generation unit 12 calculates a risk score for each of the multiple routes that have been searched.

[0124] For example, the information generating unit 12 calculates a risk score for each route based on a risk score determination rule registered in advance in the information processing device 10.

[0125] The content of the risk score determination rule is not particularly limited, but the conditions for giving points may be set as follows: "(Smoke density) D2 or more: 8 points, less than D2 and D3 or more: 6 points, ...", "(Temperature) T2 or more: 6 points, less than T2 and T3 or more: 4 points, ...", "(Device status) There is a broken device within the first distance: 3 points, there is no broken device within the first distance: 1 point, ...". In this case, the information generating unit 12 search For each route, the information generator 12 detects conditions that apply to each area that the route passes through, adds up the points for the detected conditions, and calculates a risk score.The information generator 12 then determines the route with the lowest risk score as the evacuation route.

[0126] Then, the output unit 13 outputs the calculated evacuation route. For example, the output unit 13 may output information showing the evacuation route on a floor map of the site, as shown in FIG.

[0127] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to ninth embodiments.

[0128] According to the information processing device 10 of this embodiment, the same effects as those of the first to ninth embodiments are realized. Furthermore, according to the information processing device 10 of this embodiment, a safer evacuation route is calculated and output based on information about the local environment of the site. As a result, it is possible to assist in the task of understanding the situation at the site.

[0129] <Eleventh embodiment> In this embodiment, the information generating unit 12 generates risk level information indicating the risk level for each area within a site, based on the information about the local environment of the site created in the eighth embodiment.

[0130] For example, the information generating unit 12 calculates the risk level for each area of ​​the site based on risk level calculation rules registered in advance in the information processing device 10.

[0131] The content of the risk calculation rule is not particularly limited, and the conditions for awarding points may be set as follows: "(Smoke density) D2 or more: 8 points, less than D2 and D3 or more: 6 points, ...", "(Temperature) T2 or more: 6 points, less than T2 and T3 or more: 4 points, ...", "(Device status) There is a broken device within the first distance: 3 points, there is no broken device within the first distance: 1 point, ...". In this case, the information generation unit 12 detects the conditions that apply to each area and calculates the risk by adding up the points for the detected conditions. Note that the information generation unit 12 may use the total points as the risk level, or may convert the total points into another index based on criteria such as "total points P1 or more: high risk", "total points less than P1 and P2 or more: medium risk", or "total points less than P2: low risk", and use the converted index as the risk level.

[0132] Then, the output unit 13 outputs the calculated risk level for each area. For example, the output unit 13 may output information indicating the risk level for each area on a floor map of the site, as shown in FIG.

[0133] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to tenth embodiments.

[0134] The information processing device 10 of this embodiment achieves the same effects as those of the first to tenth embodiments. Furthermore, the information processing device 10 of this embodiment calculates and outputs the risk level for each area of ​​the site based on information about the local environment of the site. As a result, it is possible to assist in the task of understanding the situation at the site.

[0135] <Twelfth embodiment> In this embodiment, the information generating unit 12 determines the worker who will rescue the detected person based on the position information of the detected person described in the above embodiment and the position information of each worker. Then, the output unit 13 outputs information indicating the worker who will rescue the detected person.

[0136] For example, the information generation unit 12 determines the worker closest to the detected person as the worker who will rescue that person. Furthermore, if the information generation unit 12 determines the worker closest to the detected person as the worker who will rescue another person, the information generation unit 12 may also determine the next closest worker as the worker who will rescue that person.

[0137] The output unit 13 may output information about the detected person, such as location information, posture, movement, age group, gender, surrounding environment, etc., to the worker terminal 20 of the worker who has decided to rescue the detected person. The method for generating this information is as described in the above embodiment.

[0138] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to eleventh embodiments.

[0139] The information processing device 10 of this embodiment achieves the same effects as those of the first to eleventh embodiments. Furthermore, the information processing device 10 of this embodiment can determine and output the most appropriate worker to rescue each detected person based on the position information of the person detected by image analysis and the position information of the worker. As a result, it can assist in the task of understanding the situation at the scene.

[0140] <Thirteenth embodiment> 10 shows an example of a functional block diagram of the information processing device 10 according to this embodiment. As shown in the figure, the information processing device 10 includes an acquisition unit 11, an information generation unit 12, an output unit 13, a safety management unit 14, and a database 15.

[0141] A list of people present at the site is registered in database 15. The list includes the appearance characteristics of each person (facial characteristics, characteristics of belongings, etc.). The list was created before any trouble occurred at the site. For example, an entrance / exit management system that manages people entering and exiting the site at the entrance / exit creates the list by taking pictures of the appearances of entrants.

[0142] The safety management unit 14 compares the appearance feature of a person detected from an image taken at the scene with the appearance feature of a person registered in the database 15. Then, based on the comparison result, the safety information registered in the database 15 is updated. FIG. 11 schematically shows an example of safety information registered in the database 15. The safety information shown in the figure links the serial numbers assigned to multiple people, the facial information of each person, and the results of safety confirmation. The result of safety confirmation for people detected from images taken at the scene is "completed," and the result of safety confirmation for people not detected from images taken at the scene is "unconfirmed."

[0143] Although not shown, the posture and movements of the person at the time of detection, as well as the surrounding environment, may be registered in association with the person whose safety confirmation result is "completed."

[0144] Other configurations of the information processing device 10 of this embodiment are the same as those of the first to twelfth embodiments.

[0145] According to the information processing device 10 of this embodiment, the same effects as those of the first to twelfth embodiments can be achieved. Furthermore, according to the information processing device 10 of this embodiment, it is possible to confirm the safety of people at the scene by image analysis.

[0146] <Modification> When the worker terminal 20 is a wearable terminal, the direction in which the worker terminal 20 captures images may be the same as or different from the worker's line of sight. When the worker terminal 20 captures images in a direction different from the worker's line of sight (for example, behind), it is preferable because information about the worker's blind spots can be conveyed to the worker.

[0147] Furthermore, if the worker terminal 20 is a wearable terminal and is a glasses-type wearable terminal that displays information superimposed on the user's field of vision (a terminal that displays information in the lens portion of the glasses), it may be possible to switch between a state in which an image captured by the camera is displayed in the lens portion and a state in which the image captured by the camera is not displayed in the lens portion in response to instructions from the worker.

[0148] Furthermore, the frame rate of the video transmitted from the worker terminal 20 to the information processing device 10 may be switchable. For example, the frame rate may be relatively high while a person is detected in the image, and relatively low while a person is not detected in the image. The worker terminal 20 may determine whether a person is detected in the image. Alternatively, the information processing device 10 may perform the above determination and transmit the result to the worker terminal 20.

[0149] As described above, the image analysis system 40 has a plurality of image analysis functions, and a trigger for executing each function may be registered in advance. Then, when each trigger is satisfied, the information generator 12 may input an instruction to the image analysis system 40 to perform image analysis using the function corresponding to the trigger. The details of the trigger for executing each function are not particularly limited.

[0150] The information processing device 10 may also check the safety of each worker based on images acquired from the worker terminal 20. For example, it may determine whether or not the worker is moving based on changes in the image. The information processing device 10 may determine that the worker is safe if there is a change in the image, i.e., the worker is moving, and may determine that the worker is not safe if there is no change in the image, i.e., the worker is not moving.

[0151] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations may be adopted. The configurations of the above-described embodiments may be combined with each other, or some of the configurations may be replaced with other configurations. Furthermore, various modifications may be made to the configurations of the above-described embodiments without departing from the spirit of the invention. Furthermore, the configurations and processes disclosed in the above-described embodiments and modified examples may be combined with each other.

[0152] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0153] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. 1. A means for acquiring images taken at the scene where a problem has occurred; an information generating means for detecting a person from the image and generating information about the detected person based on the image; an output means for outputting information about the detected person; An information processing device having the above. 2. The information processing device according to 1, wherein the information relating to the detected person includes location information of the detected person. 3. An information processing device according to 1 or 2, wherein the information about the detected people indicates at least one of the number of people taking each of a plurality of postures, the number of people performing each of a plurality of movements, the number of people in each age group, and the number of people of each gender. 4. An information processing device described in any one of 1 to 3, wherein the information about the detected people indicates at least one of the number of people and their location information that meets conditions defined using at least one of posture, movement, age group, and gender. 5. When the number of detected people is more than one, the information generating means determines rescue priorities for the detected people based on at least one of posture, movement, age group, and gender; 5. The information processing device according to any one of 1 to 4, wherein the output means outputs the priority order. 6. The information generating means generates information about the environment of the site, the output means outputs information about the environment of the site; An information processing device described in any one of 1 to 5, wherein the information regarding the environment of the site indicates at least one of the current temperature of the site, the trend of time-dependent changes in temperature at the site, the current smoke state at the site, the trend of time-dependent changes in smoke at the site, the current state of multiple devices installed at the site, and the trend of time-dependent changes in the state of multiple devices installed at the site. 7. The information processing device according to 6, wherein the information about the site environment includes information about the local environment of the site. 8. When the number of detected people is more than one, the information generating means determines rescue priorities for the detected people based on information about the local environment of the scene and location information of the detected people; 8. The information processing device according to claim 7, wherein the output means outputs the priority order. 9. The information generating means calculates an evacuation route based on information about the local environment of the site and a map of the site; 9. The information processing device according to 7 or 8, wherein the output means outputs the evacuation route. 10. The information generating means generates risk level information indicating the risk level for each area within the site based on information about the local environment of the site, 10. The information processing device according to any one of 7 to 9, wherein the output means outputs the risk level information. 11. The information generating means determines a worker who will rescue the detected person based on the position information of each of the multiple workers performing rescue activities at the site and the position information of the detected person; 11. The information processing device according to any one of 1 to 10, wherein the output means outputs information indicating a worker who will rescue the detected person. 12. An information processing device described in any one of 1 to 11, further comprising a safety management means for comparing the detected facial image of the person with facial images of people pre-registered in a database, and updating the safety information of the people registered in the database based on the comparison results. 13. An information processing device according to any one of 1 to 12, wherein the acquisition means acquires the images taken by a worker terminal carried by a worker performing rescue operations at the site. 14. The computer an acquisition step of acquiring images taken at the site where the trouble occurred; an information generating step of detecting a person from the image and generating information about the detected person based on the image; an output step of outputting information about the detected person; An information processing method that performs the above. 15. Computer, an acquisition means for acquiring images taken at the site where the trouble occurred; an information generating means for detecting a person from the image and generating information about the detected person based on the image; an output means for outputting information about the detected person; A program that functions as a [Explanation of symbols]

[0154] 10. Information processing equipment 11 Acquisition Department 12 Information generation section 13 Output section 14 Safety Management Department 15 Databases 20 Worker terminal 30 Center terminal 40 Image Analysis System 1A processor 2A Memory 3A input / output I / F 4A peripheral circuit 5A Bus

Claims

1. an acquisition means for acquiring images taken at the scene where a problem has occurred, the images including worker surrounding images taken of the surroundings of the worker by a camera mounted on a worker terminal carried by the worker rescuing the person to be rescued, or scene images taken by a camera installed at the scene; a detection means for detecting the person to be rescued and an object around the person to be rescued from the image; a position specifying means for specifying the current position of the worker by a positioning process using a GPS (global positioning system) function installed in the worker terminal, or by a process for detecting the worker from the site image by matching the site image with an image of the worker's surroundings; an information generating means for generating information including a relative positional relationship between the worker and the object based on the image and a floor map of the site, the information indicating the position of the object in terms of a direction and distance based on the current position of the worker and indicating that there is a person in the vicinity of the object; an output means for outputting the information; An information processing device having the above.

2. The information is the location of the person to be rescued; At least one of the number of people to be rescued in each of a plurality of postures, the number of people to be rescued performing each of a plurality of movements, the number of people to be rescued in each age group, and the number of people to be rescued in each gender, or At least one of the number and location of the people to be rescued who satisfy a condition defined using at least one of posture, movement, age group, and gender; indicates, The information generating means If there are a plurality of people to be rescued, determining rescue priorities for the plurality of people to be rescued based on at least one of posture, movement, age group, and gender; The output means The information processing apparatus according to claim 1 , wherein the priority order is output.

3. The information generating means generating device information indicating trends in time-varying states of a plurality of devices installed at the site; The output means outputting the device information; The device information includes: The information processing apparatus according to claim 1 , wherein the information processing apparatus includes any one of a state in which no equipment failure has occurred, a state in which the number of failed equipment has not changed, and a state in which the number of failed equipment has increased.

4. the information generating means generates information about the environment of the site; the output means outputs information about the environment of the site; 4. An information processing device as claimed in any one of claims 1 to 3, wherein the information regarding the environment of the site indicates at least one of the current temperature of the site, the trend of time-varying temperature change of the site, the current smoke state of the site, the trend of time-varying smoke state change of the site, the current state of a plurality of devices installed at the site, and the trend of time-varying state change of a plurality of devices installed at the site.

5. the information about the site environment includes information about the site local environment; The information generating means If there are multiple people to be rescued, a process of determining a priority of rescue for the multiple people to be rescued based on information about the local environment of the scene and location information of the people to be rescued; calculating an evacuation route based on information about the local environment of the site and a map of the site; or A process of generating risk level information indicating the risk level for each area within the site based on information about the local environment of the site; Run The output means The information processing device according to claim 4 , wherein the information on the priority order, the evacuation route, or the degree of danger is output.

6. the information generating means determines a worker who will rescue the person to be rescued based on position information of each of a plurality of workers performing rescue activities at the scene and position information of the person to be rescued; The information processing apparatus according to claim 1 , wherein the output unit outputs information indicating a worker who will rescue the person to be rescued.

7. An information processing device described in any one of claims 1 to 6, further having a safety management means for comparing the facial image of the person to be rescued with facial images of people pre-registered in a database, and updating the safety information of people registered in the database based on the comparison results.

8. The information processing apparatus according to claim 1 , wherein the acquisition unit acquires the image taken by a worker terminal carried by a worker performing rescue operations at the site.

9. The computer an acquisition step of acquiring images taken at the scene where the trouble occurred, the images including worker surrounding images taken around the worker by a camera mounted on a worker terminal carried by the worker rescuing the person to be rescued, or scene images taken by a camera installed at the scene; a detection step of detecting the person to be rescued and an object around the person to be rescued from the image; a position specifying step of specifying the current position of the worker by a positioning process using a GPS function installed in the worker terminal or a process of detecting the worker from the work site image by matching the work site image with an image of the worker's surroundings; an information generating step of generating information including a relative positional relationship between the worker and the object based on the image and a floor map of the site, the information indicating the position of the object in terms of a direction and distance based on the current position of the worker and indicating that there is a person in the vicinity of the object; an output step of outputting the information; An information processing method that performs the above.

10. Computer, an acquisition means for acquiring images taken at the scene where the trouble has occurred, the images including worker surrounding images taken of the surroundings of the worker by a camera mounted on a worker terminal carried by the worker rescuing the person to be rescued, or scene images taken by a camera installed at the scene; a detection means for detecting the person to be rescued and an object around the person to be rescued from the image; a position specifying means for specifying the current position of the worker by a positioning process using a GPS function installed in the worker terminal, or a process for detecting the worker from the work site image by matching the work site image with an image of the worker's surroundings; an information generating means for generating information including a relative positional relationship between the worker and the object based on the image and a floor map of the site, the information indicating the position of the object in terms of a direction and distance based on the current position of the worker and indicating that there is a person in the vicinity of the object; an output means for outputting the information; A program that functions as a

Images