Information processing device, monitoring system, data generation method, and program

Abstract

To provide an information processing device capable of displaying information useful to perform a monitoring job.SOLUTION: An information processing device comprises: a generation section for generating three-dimensional data including a monitoring object facility; a data acquisition section for acquiring sensor data detected in at least one sensor for monitoring the monitoring object facility; and a display control section for generating display data to display, on the three-dimensional data, abnormality data which is obtained by combining the plurality of elements of sensor data or combining at least the one sensor data with information related to the monitoring object facility and also which indicates an abnormality state that has occurred in the monitoring object facility.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a monitoring system, a data generating method, and a program. [Background technology]

[0002] When inspecting equipment installed in a substation, the equipment may be inspected using images captured by sensors such as LiDAR (Light Detection And Ranging). The person inspecting the equipment checks the three-dimensional shape of each piece of equipment to determine its normality.

[0003] Patent Document 1 discloses an inspection support system that enables an inspector to inspect equipment at an inspection site from a monitoring location remote from the inspection site. The inspection support system includes an unmanned mobile body equipped with a sensor arranged at the inspection site and a microphone that collects surrounding sounds. The inspector remotely monitors the inspection site by visually checking an image that displays information collected using the sensor and the unmanned mobile body. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-149349 A Summary of the Invention [Problem to be solved by the invention]

[0005] The inspection support system disclosed in Patent Document 1 displays on a device a 3D image of the equipment to be inspected, as well as the source of sounds collected by the unmanned vehicle, the type of odor, etc. Here, when remotely monitoring facilities such as substations, there is a demand for displaying more useful information on a display device or the like so that the monitor can efficiently perform the monitoring task.

[0006] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing device, a monitoring system, a data generation method, and a program that are capable of displaying information useful for performing monitoring tasks. [Means for solving the problem]

[0007] An information processing device according to a first aspect of the present disclosure includes a generation unit that generates three-dimensional data including a monitored equipment, a data acquisition unit that acquires sensor data detected by at least one sensor monitoring the monitored equipment, and a display control unit that generates display data that displays abnormality data indicating an abnormal condition that has occurred in the monitored equipment on the three-dimensional data, the abnormality data being a combination of multiple types of the sensor data or a combination of at least one of the sensor data and information related to the monitored equipment.

[0008] A monitoring system according to a second aspect of the present disclosure comprises an information processing device having at least one sensor monitoring a monitored facility, a generating device generating three-dimensional data including the monitored facility, a data acquisition unit acquiring sensor data detected by at least one of the sensors, and a display control unit generating display data displaying abnormality data indicating an abnormal condition that has occurred in the monitored facility on the three-dimensional data, the abnormality data being a combination of multiple types of the sensor data or a combination of at least one of the sensor data and information related to the monitored facility, and a display device displaying the display data.

[0009] A data generation method according to a third aspect of the present disclosure generates three-dimensional data including a monitored equipment, acquires sensor data detected by at least one sensor monitoring the monitored equipment, and generates display data on the three-dimensional data that displays abnormality data indicating an abnormal condition that has occurred in the monitored equipment, by combining multiple types of the sensor data or by combining at least one of the sensor data and information related to the monitored equipment.

[0010] A program according to a fourth aspect of the present disclosure causes a computer to generate three-dimensional data including a monitored equipment, acquire sensor data detected by at least one sensor monitoring the monitored equipment, and generate display data on the three-dimensional data that displays abnormality data indicating an abnormal condition that has occurred in the monitored equipment, the abnormality data being a combination of multiple types of the sensor data or a combination of at least one of the sensor data and information related to the monitored equipment. Effect of the Invention

[0011] The present disclosure makes it possible to provide an information processing device, a monitoring system, a data generating method, and a program that are capable of displaying information useful for performing monitoring tasks. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a configuration diagram of an information processing device according to the present disclosure. [Diagram 2] FIG. 13 is a diagram showing the flow of data generation processing according to the present disclosure. [Diagram 3] FIG. 1 is a configuration diagram of an information processing device according to the present disclosure. [Figure 4] FIG. 2 is a diagram showing three-dimensional data according to the present disclosure. [Diagram 5] FIG. 1 is a diagram showing three-dimensional data according to the present disclosure. [Figure 6] FIG. 1 is a diagram showing three-dimensional data according to the present disclosure. [Figure 7] FIG. 11 is a diagram showing a flow of data display processing according to the present disclosure. [Figure 8] FIG. 1 is a configuration diagram of an information processing device according to the present disclosure. [Figure 9] FIG. 1 is a configuration diagram of a monitoring system according to the present disclosure. [Figure 10] FIG. 11 is a diagram showing a flow of a process for generating additional data according to the present disclosure. [Figure 11] FIG. 1 is a configuration diagram of a monitoring system according to the present disclosure. [Figure 12]FIG. 1 is a configuration diagram of an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Embodiment 1) An example of the configuration of the information processing device 10 will be described below with reference to Fig. 1. The information processing device 10 may be a computer device that operates when a processor executes a program stored in a memory.

[0014] The information processing device 10 has a generation unit 11, a data acquisition unit 12, and a display control unit 13. The generation unit 11, the data acquisition unit 12, and the display control unit 13 may be software or modules that are executed by a processor executing a program stored in a memory. Alternatively, the generation unit 11, the data acquisition unit 12, and the display control unit 13 may be hardware such as a circuit or a chip.

[0015] The generating unit 11 generates three-dimensional data including the monitored facility. The monitored facility may be a facility that needs to be monitored periodically or at any timing. The monitored facility may be, for example, a building, a road, a device, a part, etc. Specifically, the monitored facility may be a facility disposed in a substation, a power plant, etc.

[0016] The three-dimensional data may be, for example, data indicating a three-dimensional position of a monitored facility in a predetermined three-dimensional space. The three-dimensional data may be generated based on data measured using a sensor or the like. Alternatively, the three-dimensional data may be generated using software that generates three-dimensional data from two-dimensional data such as a drawing. The three-dimensional data may be, for example, point cloud data. Alternatively, the three-dimensional data may be mesh data, polygon data, or the like generated using point cloud data. Alternatively, the three-dimensional data may be data represented using a depth map.

[0017] The data acquisition unit 12 acquires sensor data detected by at least one sensor that monitors the monitored facility. The at least one sensor may be built into the information processing device 10. Alternatively, the at least one sensor may be externally attached to the information processing device 10. Alternatively, the at least one sensor may be installed at a location different from the information processing device 10 and communicate with the information processing device 10 via a network.

[0018] The sensor for monitoring the monitored facility may be a camera, a LiDAR (Light Detection And Ranging) device, a temperature sensor, a humidity sensor, a vibration sensor, an acceleration sensor, a microphone, an odor sensor, a GPS (Global Positioning System) sensor, etc. The temperature sensor may be, for example, a thermograph. Furthermore, the sensor for monitoring the monitored facility is not limited to these, and a device for detecting various data may be used.

[0019] For example, a camera or LiDAR device generates sensor data that can be used to visually determine whether or not an abnormality has occurred. A temperature sensor, a humidity sensor, a vibration sensor, and an acceleration sensor generate sensor data that can be used to tactilely determine whether or not an abnormality has occurred. A microphone generates sensor data that can be used to hear whether or not an abnormality has occurred. An odor sensor generates sensor data that can be used to olfactoryly determine whether or not an abnormality has occurred. A GPS sensor may be used as a device that generates auxiliary data for determining whether or not an abnormality has occurred.

[0020] The display control unit 13 generates display data that displays abnormal data obtained by combining multiple types of sensor data on three-dimensional data. The abnormal data is data that indicates an abnormal state that has occurred in the monitored equipment. Alternatively, the display control unit 13 generates display data that displays abnormal data obtained by combining at least one sensor data and information related to the monitored equipment on three-dimensional data.

[0021] Displaying the abnormal data on the three-dimensional data may mean combining the abnormal data with the three-dimensional data, or displaying the abnormal data on the three-dimensional data may mean simultaneously displaying the abnormal data and the three-dimensional data.

[0022] The multiple types of sensor data may be sensor data generated by two or more sensors that generate different types of data. The types of data may be, for example, sensor data that can determine whether or not an abnormality has occurred by each of the senses of vision, touch, hearing, and smell.

[0023] The information regarding the monitored equipment may be, for example, the location of the monitored equipment, the size of the monitored equipment, the weight of the monitored equipment, the materials that make up the monitored equipment, the functions of the monitored equipment, substances flowing within the monitored equipment, services provided or used by the monitored equipment, etc.

[0024] A supervisor of the monitored equipment can determine whether or not an abnormality has occurred in the monitored equipment by checking the display data displayed on a display device or the like.

[0025] Next, the process flow of the data generating method executed in the information processing device will be described with reference to FIG. 2. First, the generating unit 11 generates three-dimensional data including the monitored equipment (S11). Next, the data acquiring unit 12 acquires sensor data detected by at least one sensor monitoring the monitored equipment (S12). Next, the display control unit 13 generates display data that displays abnormality data indicating an abnormal state occurring in the monitored equipment, which is a combination of multiple types of sensor data, on the three-dimensional data (S13). Alternatively, the display control unit 13 generates display data that displays abnormality data indicating an abnormal state occurring in the monitored equipment, which is a combination of at least one sensor data and information related to the monitored equipment, on the three-dimensional data (S13).

[0026] As described above, the information processing device 10 generates abnormality data indicating an abnormal state that has occurred in the monitored equipment by combining multiple types of sensor data or by combining at least one sensor data and information on the monitored equipment. Furthermore, the information processing device 10 generates display data that displays the abnormality data on three-dimensional data.

[0027] Since the abnormality data is a combination of multiple types of sensor data, it indicates useful information that cannot be obtained from information indicated by a single piece of sensor data. Furthermore, since the abnormality data is a combination of at least one piece of sensor data and information related to the monitored equipment, it indicates useful information that cannot be obtained from information indicated by a single piece of sensor data. The useful information may be, for example, the cause of the abnormality, the severity of the abnormality, etc. By monitoring the display data that combines the three-dimensional data showing the monitored equipment and the abnormality data, a monitor can efficiently identify the cause of the abnormality, identify measures to be taken when an abnormality occurs, etc.

[0028] (Embodiment 2) Next, a configuration example of the information processing device 20 will be described with reference to Fig. 3. The information processing device 20 has a configuration in which an abnormality determination unit 21, an abnormality location identification unit 22, and a display unit 23 are added to the information processing device 10 of Fig. 1. In the information processing device 20, detailed description of the same components as those of the information processing device 10 will be omitted. The components constituting the information processing device 20 may be software or modules in which processing is performed by a processor executing a program stored in a memory. Alternatively, the components constituting the information processing device 20 may be hardware such as a circuit or a chip.

[0029] The generating unit 11 generates point cloud data as three-dimensional data including the monitored equipment. FIG. 4 shows an example of the point cloud data. FIG. 4 shows equipment in a substation using point cloud data. FIG. 4 shows a simplified view of the equipment in order to easily show the shape of the equipment, but in reality, the shape of the equipment is shown as a set of points. Specifically, points are distributed on the boundary line of the equipment and on the surface of the equipment. The boundary line of the equipment may be a line indicating the boundary between the equipment and the background, or the boundary between the equipment and other equipment. The point cloud data is generated using, for example, a LiDAR device. The LiDAR device may be called a laser scanner. The generating unit 11 may use a LiDAR device. Alternatively, the generating unit 11 may acquire the point cloud data generated in the LiDAR device from the LiDAR device via a network or the like.

[0030] The LiDAR device measures the distance to an object using, for example, a ToF (Time of Flight) method, and generates points that indicate the shape of the object. A collection of points that indicate the shape of the object becomes point cloud data. The points that indicate the shape of the object may be specified using three-dimensional coordinates in a predetermined space.

[0031] The data acquiring unit 12 acquires sensor data from each of the multiple types of sensors. The data acquiring unit 12 may also acquire point cloud data including the monitored facility from the generating unit 11. The sensors may be placed in the vicinity of the monitored facility or may be attached to the monitored facility.

[0032] For example, the LiDAR device may be installed at a position where point cloud data of the monitored equipment can be acquired. In other words, the LiDAR device may be installed at a position where the monitored equipment can be photographed. The thermography used as a temperature sensor may be installed at a position where the temperature of the surface of the monitored equipment can be measured. The vibration sensor may be attached to the surface of the monitored equipment and detect the occurrence of vibration. The microphone may be installed at a position where the sound emitted by the monitored equipment can be collected. The odor sensor may be installed at a position where the odor emitted by the monitored equipment can be detected. Only one of each sensor may be installed, or multiple sensors may be installed.

[0033] The abnormality determination unit 21 determines whether or not an abnormality has occurred in the monitored equipment using the sensor data acquired by the data acquisition unit 12. For example, the abnormality determination unit 21 may input point cloud data generated using a LiDAR device to a learning model and receive a determination result of whether or not an abnormality has occurred. The learning model is assumed to have learned point cloud data of the monitored equipment in a normal state or point cloud data of the monitored equipment in a state in which an abnormality has occurred by machine learning in advance. The learning model may input the point cloud data and output a determination result indicating whether or not an abnormality has occurred in the shape indicated by the point cloud data. Furthermore, the learning model may output data indicating the location where the abnormality has occurred when an abnormality has occurred.

[0034] The abnormality determination unit 21 may determine that an abnormality has occurred in a location indicating a temperature higher than a predetermined temperature. Alternatively, the abnormality determination unit 21 may determine that an abnormality has occurred in a location indicating a temperature lower than a predetermined temperature. The abnormality determination unit 21 identifies the distance and direction from the location where the temperature sensor is installed to the location where the abnormality has occurred.

[0035] When a vibration sensor attached to the surface of the monitored equipment detects the occurrence of vibration, the abnormality determination unit 21 may determine that an abnormality has occurred at the location where the vibration sensor is attached. Alternatively, the abnormality determination unit 21 may detect the occurrence of vibration using an optical vibration analysis technique that captures minute movements of the surface from an image of a structure captured by a camera. The abnormality determination unit 21 may determine that an abnormality has occurred at the location where the vibration has occurred. The abnormality determination unit 21, for example, identifies the distance and direction from the location where the camera is installed to the location where the abnormality has occurred.

[0036] The abnormality determination unit 21 may identify the sound source using the arrival time difference between sound waves collected by microphones installed at two or more locations around the monitored equipment. Alternatively, the abnormality determination unit 21 may form an acoustic beam by taking the sum of the sound waves detected by each microphone. The abnormality determination unit 21 may identify the sound source using a preference pattern of the acoustic beam. When the abnormality determination unit 21 detects a volume equal to or higher than a predetermined level or a predetermined dB (decibels), the abnormality determination unit 21 may determine that an abnormality has occurred at the location of the identified sound source. The abnormality determination unit 21 identifies the distance and direction from a position determined based on the microphone positions, such as a position where any microphone is installed or the center position of a figure with multiple microphones as vertices, to a location where an abnormality has occurred.

[0037] The abnormality determination unit 21 may specify in advance the position from which an odor will be generated in the monitored equipment when an abnormality occurs. For example, a monitor may input information regarding the position from which an odor will be generated in the event of an abnormality to the abnormality determination unit 21. The information regarding the position from which an odor will be generated in the event of an abnormality may be, for example, a position within the point cloud data of FIG. 4. When the odor value indicated by the sensor data exceeds a predetermined value, the abnormality determination unit 21 may determine that an abnormality has occurred in the position specified in advance as the position from which an odor will be generated in the event of an abnormality.

[0038] The abnormality location identifying unit 22 generates abnormality location data indicating an abnormality location that has occurred in the monitored equipment. Specifically, the abnormality location identifying unit 22 identifies the location where the abnormality has occurred in the point cloud data shown in Fig. 4. More specifically, the abnormality location identifying unit 22 identifies the location where the abnormality has been determined to have occurred by the abnormality determining unit 21, and the location of the point cloud data shown in Fig. 4.

[0039] The abnormality location identifying unit 22 identifies in advance the distance between the LiDAR device and each sensor used to generate the point cloud data shown in Fig. 4. Furthermore, the abnormality location identifying unit 22 identifies in advance the direction or orientation in which each sensor is installed with respect to the LiDAR device. That is, the abnormality location identifying unit 22 identifies in advance the position in which each sensor is installed with respect to the LiDAR device. The position with respect to the LiDAR device may be a position on a coordinate system determined with the position of the LiDAR device as the origin.

[0040] For example, the abnormality location identifying unit 22 identifies which position in the point cloud data based on the position of the LiDAR device corresponds to the distance and direction to the location where the abnormality has occurred identified by the abnormality determination unit 21. In other words, the abnormality location identifying unit 22 uses the position of the LiDAR device as a starting point to apply the distance and direction to the location where the abnormality has occurred identified by the abnormality determination unit 21. The abnormality location identifying unit 22 identifies which position in the point cloud data corresponds to the position indicated by the distance and direction to the location where the abnormality has occurred, based on the position of the LiDAR device.

[0041] Furthermore, the abnormality location identifying unit 22 corrects the position of the identified point cloud data according to the positional relationship between the LiDAR device and each sensor. For example, when the position of the sensor is a distance A (A is a positive number, and the unit is meters, centimeters, millimeters, etc.) away from the LiDAR device to the right, the abnormality location identifying unit 22 identifies a position that is the distance A away to the right from the identified position of the point cloud data as the location where the abnormality has occurred.

[0042] The abnormality location identifying unit 22 does not need to make corrections to the abnormality locations detected based on the shape of the point cloud data generated using the LiDAR device. Also, the abnormality location identifying unit 22 does not need to make corrections to the locations identified as the locations where the abnormality occurred using the odor sensor.

[0043] 5 shows the positions identified as abnormal positions by the abnormal position identifying unit 22 using curved dotted lines. FIG. 5 shows the positions identified as abnormal positions by the abnormal position identifying unit 22 superimposed on the point cloud data generated by the generating unit 11. The positions identified as abnormal positions may be shown using curved dotted lines, solid lines, or shapes other than curved lines, as shown in FIG. 5. Furthermore, the positions identified as abnormal positions may be shown using different shapes, different colors, different patterns, different types of lines, or the like, for each piece of sensor data used to determine the occurrence of an abnormality.

[0044] Returning to Fig. 3, the display control unit 13 generates display data to be displayed on the display unit 23. The display unit 23 may be, for example, a display device such as a display. The display data is data in which point cloud data is associated with abnormality data indicating an abnormal state that has occurred in the monitored equipment. The abnormality data includes abnormality location data indicating the location where the abnormality has occurred and additional data related to the abnormality that has occurred in the monitored equipment.

[0045] The additional data may be, for example, data indicating the cause of an abnormality that has occurred in the monitored equipment. Alternatively, the additional data may be data that highlights and displays the abnormality. Alternatively, the additional data may include both data indicating the cause of an abnormality that has occurred in the monitored equipment and data that highlights and displays the abnormality.

[0046] The display control unit 13 may generate display data to be displayed on the display unit 23 based on the data shown in FIG. 5 identified by the abnormality location identification unit 22. For example, when two or more abnormality location data exist within a predetermined range, or when the ranges indicated by two or more abnormality location data overlap, the display control unit 13 may highlight the abnormality by, for example, blinking the two or more abnormality location data. When two or more abnormality location data exist within a predetermined range, the severity of the abnormality may be considered to be high. Examples of highlighting the abnormality include, in addition to blinking, using a color different from other abnormality location data, using a specific pattern, surrounding the range indicated by two or more abnormality location data with a dotted line or a solid line, and the like. The display control unit 13 may also display the cause of the abnormality of the abnormality as shown in FIG. 6.

[0047] Furthermore, even if the abnormality determination unit 21 determines that an abnormality exists from the image data captured by the camera, the abnormality determination unit 21 may determine that an abnormality does not exist in the point cloud data related to the same location as the location captured by the camera from the sensor data. In such a case, the display control unit 13 may prioritize the determination using the point cloud data and determine that an abnormality does not exist. Furthermore, the display control unit 13 may determine in advance the priority order between the determination result using the point cloud data and the determination result using each sensor data, and perform the abnormality determination based on the priority order. Performing the abnormality determination based on the priority order may mean preferentially adopting the determination result based on the sensor data with a higher priority order.

[0048] The display control unit 13 may generate additional data based on information about the monitored equipment and the determination result by the abnormality determination unit 21. For example, the display control unit 13 holds information indicating that the monitored equipment contains gas or oil, or that gas or oil is flowing through the monitored equipment, as information about the monitored equipment. In this state, when the abnormality determination unit 21 detects an abnormality based on the sensor data generated in the LiDAR device and the sensor data generated in the odor sensor, the display control unit 13 determines that the cause of the abnormality is a gas leak or an oil leak. The display control unit 13 generates additional data indicating that the observed abnormality is a gas leak or an oil leak.

[0049] Moreover, the display control unit 13 holds information indicating that the monitored facility is near the sea as information regarding the monitored facility. Near the sea may mean that the distance from the monitored facility to the sea is closer than a predetermined distance. In this state, when the abnormality determination unit 21 detects an abnormality based on the sensor data generated by the LiDAR device, the display control unit 13 determines that the cause of the abnormality is salt damage, and generates additional data indicating that the cause of the abnormality is salt damage.

[0050] The display control unit 13 is not limited to these examples, and may predetermine the cause of an abnormality when combining the information on the monitored equipment and the judgment result in the abnormality judgment unit 21. Furthermore, the display control unit 13 may predetermine the assumed cause of an abnormality when two or more abnormal location data exist within a predetermined range.

[0051] Next, a data display method in the information processing device 20 will be described with reference to Fig. 7. In Fig. 7, it is assumed that the generating unit 11 generates point cloud data including the monitored facility in advance. Alternatively, it is assumed that the generating unit 11 acquires or holds point cloud data including the monitored facility in advance.

[0052] First, the data acquisition unit 12 acquires sensor data detected by at least one sensor monitoring the monitored equipment (S21). The data acquisition unit 12 may acquire sensor data at different timings for each sensor, or may acquire sensor data from all sensors at the same timing. Alternatively, the data acquisition unit 12 may acquire sensor data from some of all sensors at the same timing. Alternatively, each sensor may output or transmit sensor data to the data acquisition unit 12 at the timing when it detects a change in state. Detecting a change in state may be detecting a sound, an odor, a vibration, or the like. Alternatively, detecting a change in state may be detecting sensor data indicating a value exceeding a predetermined value. Alternatively, detecting a change in state may be detecting that the difference between a value indicated by the latest sensor data and a value indicated by sensor data generated so far is greater than a threshold value.

[0053] Next, the abnormality determination unit 21 determines whether or not there is sensor data indicating that an abnormality has occurred among the acquired sensor data (S22). If the abnormality determination unit 21 determines that there is no sensor data indicating that an abnormality has occurred, the process of step S21 is repeated.

[0054] When it is determined that there is sensor data indicating that an abnormality has occurred, the abnormality determination unit 21 identifies an abnormal location (S23). The sensor data indicating that an abnormality has occurred may be, for example, sensor data that allows for determining whether or not an abnormality has occurred by each of the senses of vision, touch, hearing, and smell. When sensor data indicating that an abnormality has occurred by at least one of the senses of vision, touch, hearing, and smell is included, the abnormality determination unit 21 may determine that there is sensor data indicating that an abnormality has occurred. Furthermore, the abnormality determination unit 21 identifies an abnormal location in the point cloud data as shown in FIG. 5.

[0055] Next, the display control unit 13 generates display data including the abnormality location data and the additional data (S24). The display control unit 13 may display the abnormality location data and the additional data by superimposing them on the point cloud data including the monitored equipment. The position of the additional data superimposed on the point cloud data may be, for example, a position that does not overlap with the abnormality location data, or may be a predetermined position.

[0056] Next, the display control unit 13 displays the display data on the display unit (S25). The user operating the information processing device 20 can visually recognize the display data displayed on the display unit .

[0057] As described above, the information processing device 20 generates abnormal data including abnormal location data and additional data by combining multiple types of sensor data, or by combining at least one sensor data and information on the monitored equipment. Furthermore, the information processing device 20 displays the abnormal data. This allows the user of the information processing device 20 to confirm the additional data that cannot be obtained from only a single sensor data. As a result, the user of the information processing device 20 can efficiently perform the monitoring work of the monitored equipment.

[0058] Furthermore, FIG. 3 shows a configuration in which the information processing device 20 has a generation unit 11, a data acquisition unit 12, an abnormality determination unit 21, an abnormality location identification unit 22, a display control unit 13, and a display unit 23, but the information processing device is not limited to such a configuration.

[0059] For example, the information processing device may have a data acquisition unit 12, an abnormality determination unit 21, an abnormality location identification unit 22, and a display control unit 13, as in the information processing device 30 shown in FIG. 8. The generation unit 11 may be used as a generation device 31, which is a device different from the information processing device 30. Furthermore, the display unit 23 may be used as a display device 32, which is a device different from the information processing device 30. The generation device 31 and the display device 32 may be computer devices that operate by a processor executing a program stored in a memory. The generation device 31 and the display device 32 may be connected to the information processing device 30 via a network, or may be connected to the information processing device 30 via a cable or the like. Alternatively, the generation device 31 and the display device 32 may be connected to the information processing device 30 via a wireless communication line.

[0060] 9, the information processing device 30 may configure a monitoring system together with a generating device 31, a display device 32, and a sensor 33 that monitors the monitored facility. The monitoring system may include the information processing device 20 and the sensor 33. The devices and sensors that configure the monitoring system may be connected via a network.

[0061] (Embodiment 3) Next, the flow of the additional data generation process in the information processing device 20 will be described with reference to Fig. 10. Steps S31 and S32 in Fig. 10 are similar to steps S21 and S22 in Fig. 7, and therefore detailed description thereof will be omitted.

[0062] Next, the display control unit 13 generates additional data (S33) without identifying an abnormality in the abnormality location identifying unit 22 shown in Fig. 7. For example, when sensor data indicating that an abnormality has occurred is present, the abnormality location identifying unit 22 may determine that an abnormality has occurred at any location in the monitored equipment. In other words, when sensor data indicating that an abnormality has occurred is present, the abnormality location identifying unit 22 may determine that an abnormality has occurred in the monitored equipment without identifying the detailed abnormality location.

[0063] The display control unit 13 may generate data indicating the cause of an abnormality that has occurred in the monitored equipment by using multiple types of sensor data or at least one type of sensor data and information on the monitored equipment as the additional data. Next, the display control unit 13 displays the additional data on the display unit 23 (S34).

[0064] As described above, in the additional data generation process in Fig. 10, the display control unit 13 generates additional data without identifying detailed abnormal points in the monitored equipment. As a result, for example, for equipment with a simple structure that does not require identifying detailed abnormal points, additional data is generated by processing with a lower load compared to the case of identifying detailed abnormal points.

[0065] (Embodiment 4) Next, an abnormality data generation process using the intrusion detection system 40 will be described with reference to Fig. 11. Fig. 11 shows a monitoring system including the intrusion detection system 40. In the monitoring system, an information processing device 30 may be used instead of the information processing device 20. Furthermore, the monitoring system may have a sensor 33.

[0066] The intrusion detection system 40 may be, for example, a system that provides a service used by a manager of a monitored facility to monitor the monitored facility. The intrusion detection system 40 provides, for example, a service that detects a person intruding into the monitored facility. A person intruding into the monitored facility may be a person intruding into a predetermined area that includes the monitored facility. The intrusion detection system may detect a person using, for example, a temperature sensor, an infrared sensor, a camera, or the like.

[0067] The information processing device 20 may be connected to the intrusion detection system 40 via a network. When the information processing device 20 receives a message notifying an intrusion of a person from the intrusion detection system 40, the information processing device 20 may display a warning message or the like indicating the intrusion of a person on the display unit 23.

[0068] Here, it is assumed that the intrusion detection system 40 can detect humans, but cannot detect animals or living things other than humans. For example, if the intrusion detection system 40 detects human intrusion by detecting human characteristics such as human body temperature, it cannot detect animals or living things other than humans.

[0069] The display control unit 13 may determine that an animal or living thing other than a human has invaded the monitored facility when it is determined that an abnormality has occurred due to visual detection using a camera or the like, even though it has not been notified of information indicating that a human has been detected by the intrusion detection system 40. Alternatively, the display control unit 13 may determine that an animal or living thing other than a human has invaded the monitored facility when it is determined that an abnormality has occurred due to auditory detection using a microphone or the like, even though it has not been notified of information indicating that a human has been detected by the intrusion detection system 40. The display control unit 13 may also determine that an animal or living thing other than a human has invaded the monitored facility when it is determined that an abnormality has occurred due to olfactory detection using an odor sensor or the like, even though it has not been notified of information indicating that a human has been detected by the intrusion detection system 40.

[0070] In this way, the information processing device 20 can detect the intrusion of animals or living things other than humans, using information notified from an intrusion detection system that can detect human intrusion. Also, in Fig. 11, the operation of the information processing device 20 linked with the intrusion detection system has been described, but the system with which the information processing device 20 links is not limited to the intrusion detection system. The information processing device 20 may generate useful information to be displayed on the display unit 23, using information notified from a system that provides other services that monitor the monitored equipment, or other services used by companies or the like that own the monitored equipment.

[0071] Fig. 12 is a block diagram showing a configuration example of information processing devices 10, 20, and 30 (hereinafter referred to as information processing devices 10, etc.). Referring to Fig. 12, the information processing device 10, etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) that complies with the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0072] The processor 1202 reads out and executes software (computer programs) from the memory 1203 to perform the processing of the information processing device 10 and the like described using the flowcharts in the above-mentioned embodiments. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0073] The memory 1203 is configured by a combination of volatile memory and non-volatile memory. The memory 1203 may include storage located away from the processor 1202. In this case, the processor 1202 may access the memory 1203 via an I / O (Input / Output) interface (not shown).

[0074] 12, the memory 1203 is used to store a group of software modules. The processor 1202 reads out and executes the group of software modules from the memory 1203, thereby performing the processing of the information processing device 10 and the like described in the above-mentioned embodiment.

[0075] As described with reference to FIG. 12, each of the processors included in the information processing device 10 or the like executes one or more programs including a group of instructions for causing a computer to execute the algorithm described with reference to the drawings.

[0076] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0077] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be appropriately combined with other embodiments.

[0078] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but to one or more other embodiments. As can be understood by those skilled in the art, various features or steps described with reference to any one drawing may be combined with features or steps shown in one or more other drawings to create an embodiment not explicitly shown or described, for example. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0079] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) A generation unit that generates three-dimensional data including a monitored facility; a data acquisition unit that acquires sensor data detected by at least one sensor that monitors the monitored equipment; and a display control unit that generates display data that displays abnormality data indicating an abnormal condition that has occurred in the monitored equipment, on the three-dimensional data, the abnormality data being a combination of multiple types of the sensor data, or a combination of at least one of the sensor data and information related to the monitored equipment. (Appendix 2) The information processing device according to claim 1, wherein the abnormality data includes abnormality location data indicating an abnormality location that has occurred in the monitored equipment and additional data related to the abnormality that has occurred in the monitored equipment. (Appendix 3) 3. The information processing device according to claim 2, wherein the additional data indicates a cause of an abnormality that has occurred in the monitored equipment. (Appendix 4) The information processing device according to claim 2, wherein the additional data highlights the abnormal area. (Appendix 5) 5. The information processing device according to claim 1, wherein the information about the monitored equipment indicates an installation location of the monitored equipment. (Appendix 6) 5. The information processing device according to claim 1, wherein the information relating to the monitored equipment is information relating to an intrusion detection service used by the monitored equipment. (Appendix 7) 7. The information processing device according to any one of claims 1 to 6, further comprising an identification unit that identifies an abnormal location of the monitored equipment included in the three-dimensional data based on a distance between a measuring device that generates the three-dimensional data and at least one of the sensors. (Appendix 8) 8. The information processing device according to claim 1, further comprising a display unit that displays the display data. (Appendix 9) At least one sensor monitoring the monitored equipment; A generating device for generating 3D data including the monitored facility; an information processing device having: a data acquisition unit that acquires sensor data detected by at least one of the sensors; and a display control unit that generates display data that displays, on the three-dimensional data, abnormality data that indicates an abnormal state that has occurred in the monitored equipment, the abnormality data being a combination of a plurality of types of the sensor data, or a combination of at least one of the sensor data and information related to the monitored equipment; A monitoring system comprising: a display device that displays the display data. (Appendix 10) Generate 3D data including the monitored equipment, Acquire sensor data detected by at least one sensor monitoring the monitored equipment; A data generation method for generating display data that displays abnormality data indicating an abnormal condition that has occurred in the monitored equipment on the three-dimensional data, the abnormality data being obtained by combining multiple types of sensor data, or by combining at least one of the sensor data and information related to the monitored equipment. (Appendix 11) Generate 3D data including the monitored equipment, Acquire sensor data detected by at least one sensor monitoring the monitored equipment; A program that causes a computer to execute the following: generating display data that displays, on the three-dimensional data, abnormality data that indicates an abnormal condition that has occurred in the monitored equipment, the abnormality data being a combination of multiple types of the sensor data, or a combination of at least one of the sensor data and information related to the monitored equipment.

[0080] Some or all of the elements (e.g., configurations and functions) described in Supplementary Note 2 to Supplementary Note 8 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Note 9, Supplementary Note 10, and Supplementary Note 11 in the same dependent relationship as Supplementary Note 2 to Supplementary Note 8. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0081] 10. Information processing device 11 Generation part 12 Data Acquisition Section 13 Display control section 20 Information processing device 21 Abnormality determination section 22 Abnormal location identification part 23 Display section 30 Information processing device 31 Generator 32 Display device 33 Sensors 40 Intrusion Detection System

Claims

1. A generation unit that generates three-dimensional data including a monitored facility; a data acquisition unit that acquires sensor data detected by at least one sensor that monitors the monitored equipment; and a display control unit that generates display data that displays abnormality data indicating an abnormal condition that has occurred in the monitored equipment, the abnormality data being a combination of multiple types of the sensor data, or a combination of at least one of the sensor data and information related to the monitored equipment, on the three-dimensional data.

2. The information processing apparatus according to claim 1 , wherein the abnormality data includes abnormality location data indicating an abnormality location that has occurred in the monitored equipment and additional data related to the abnormality that has occurred in the monitored equipment.

3. The information processing device according to claim 2 , wherein the additional data indicates a cause of an abnormality that has occurred in the monitored facility.

4. The information processing apparatus according to claim 2 , wherein the additional data displays the abnormal portion in an emphasized manner.

5. The information processing apparatus according to claim 1 , wherein the information relating to the monitored facility indicates an installation location of the monitored facility.

6. The information processing apparatus according to claim 1 , wherein the information relating to the monitored facility is information relating to an intrusion detection service used by the monitored facility.

7. 5. The information processing device according to claim 1, further comprising an identification unit that identifies an abnormality location of the monitored equipment included in the three-dimensional data based on a distance between a measuring device that generates the three-dimensional data and at least one of the sensors.

8. At least one sensor for monitoring the monitored equipment; A generating device for generating three-dimensional data including a monitored facility; an information processing device having: a data acquisition unit that acquires sensor data detected by at least one of the sensors; and a display control unit that generates display data that displays, on the three-dimensional data, abnormality data that indicates an abnormal state that has occurred in the monitored equipment, the abnormality data being a combination of a plurality of types of the sensor data or a combination of at least one of the sensor data and information related to the monitored equipment; A monitoring system comprising: a display device that displays the display data.

9. Generate 3D data including the monitored equipment, Acquire sensor data detected by at least one sensor monitoring the monitored equipment; A data generation method for generating display data that displays abnormality data indicating an abnormal condition that has occurred in the monitored equipment on the three-dimensional data, the abnormality data being a combination of multiple types of sensor data, or a combination of at least one of the sensor data and information related to the monitored equipment.

10. Generate 3D data including the monitored equipment, Acquire sensor data detected by at least one sensor monitoring the monitored equipment; A program that causes a computer to execute the following: Generate display data that displays, on the three-dimensional data, abnormality data that indicates an abnormal condition that has occurred in the monitored equipment, by combining multiple types of the sensor data, or by combining at least one of the sensor data and information related to the monitored equipment.

Images