VIDEO MONITORING DEVICE, VIDEO MONITORING SYSTEM, AND VIDEO MONITORING METHOD

Abstract

To support an action of a user responding to detected abnormality by monitoring abnormality of a target facility while suppressing introduction cost.SOLUTION: In a video monitoring system 100, a video monitoring device 1 comprises: a camera video analysis unit 15 for detecting abnormality of a facility 23 from a video shot by a monitoring camera 22 and identifying a content of the abnormality; a responding action selection unit 16 for acquiring an action responding to the identified abnormality content on the basis of, responding action information in which an action responding to abnormality is associated for each abnormality content; and a visualization processing unit 14 which generates display data for displaying a monitoring picture displaying the video of the facility 23 on a client operation terminal 3 and displays the monitoring screen on the client operation terminal using the display data. In a case where abnormality of the facility is detected, the visualization processing unit 14 generates display data for abnormal time including an indication of a video, in which abnormality is detected, and a presentation of the action responding to the abnormality and displays the monitoring screen in such a mode that a determination operation of the responding action can be performed.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video monitoring device, a video monitoring system, and a video monitoring method, and is suitable for use in a video monitoring device, a video monitoring system, and a video monitoring method that monitors abnormalities in target equipment using video captured by a surveillance camera. [Background technology]

[0002] Conventionally, there is known a technology in which, when an abnormality is detected by an abnormality detection sensor such as a fire detector and a camera associated with the sensor, the image captured by the camera is displayed on a terminal operated by a user and a report is sent to a pre-set destination (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-089092 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the anomaly reporting system described in Patent Document 1 has a problem in that the user can only check the video when an anomaly occurs, and it is not convenient enough for a building manager who monitors various facilities in a building collectively. Also, the anomaly reporting system described in Patent Document 1 requires the installation of an anomaly detection sensor and a camera in each of the various facilities in the building to be monitored, which increases the introduction cost.

[0005] The present invention has been made in consideration of the above points, and aims to propose a video monitoring device, a video monitoring system, and a video monitoring method that are capable of assisting a user in taking action in response to a detected abnormality. [Means for solving the problem]

[0006] In order to solve such problems, the present invention provides a video monitoring device that monitors abnormalities in a monitored facility using video from a surveillance camera that captures the facility, the video monitoring device comprising a camera video acquisition unit that acquires video data of the facility captured by the surveillance camera, a camera video analysis unit that detects abnormalities in the facility from the video of the video data and identifies the contents of the detected abnormality, customer information that associates information about a customer who has concluded a maintenance contract for monitoring the facility, the facility that is the subject of monitoring under the maintenance contract, and the surveillance camera that captures the facility, and a Selectable by said customer The next action to be taken is determined for each abnormality. 1 or more a storage device that stores the associated response action information; and a storage device that stores the associated response action information, which is associated with the abnormality content identified by the camera image analysis unit based on the response action information stored in the storage device. 1 or more a response action selection unit that acquires the response action; and a visualization processing unit that generates display data for displaying a monitoring screen that displays the image of the image data acquired by the camera image acquisition unit on a customer operation terminal operated by the customer, and causes the customer operation terminal to display the monitoring screen using the display data. When the camera image analysis unit detects an abnormality in the equipment, the visualization processing unit displays the image in which the abnormality was detected and the 1 or more Response actions List and generating display data for an abnormality including the display of the abnormality. One or more of the above Response actions The customer selects which action to take from the list. The monitoring screen is displayed in a state in which a decision operation can be executed. On the customer operation terminal A video monitoring device is provided that displays the video.

[0007] In order to solve the above problem, the present invention provides a video monitoring system for monitoring abnormalities in a monitored facility, the system comprising: a surveillance camera for capturing images of the facility; a camera video recorder for recording image data of the facility captured by the surveillance camera; a customer operation terminal operated by a customer who has signed a maintenance contract for monitoring the facility; and a system for detecting an abnormality in the facility using the image data of the facility, Selectable by said customer Next steps List of and a video monitoring device that presents on the customer operation terminal a camera video acquisition unit that acquires video data of the equipment from the camera video recording device, a camera video analysis unit that detects an abnormality in the equipment from the video of the video data and identifies the content of the detected abnormality, and a customer information that associates information on the customer who has concluded the maintenance contract, the equipment that is a monitoring target of the maintenance contract, and the monitoring camera that captures the equipment, and the response action to the abnormality is provided for each of the abnormality contents of the abnormality. 1 or more a storage device that stores the associated response action information; and a storage device that stores the associated response action information, which is associated with the abnormality content identified by the camera image analysis unit based on the response action information stored in the storage device. 1 or more A video monitoring system is provided that has a response action selection unit that acquires the response action, and a visualization processing unit that generates display data for displaying a monitoring screen that displays the video of the video data acquired by the camera video acquisition unit on a customer operation terminal operated by the customer, and uses the display data to display the monitoring screen on the customer operation terminal. In this video monitoring system, when the camera video analysis unit detects an abnormality in the equipment, the visualization processing unit displays the video in which the abnormality was detected and the 1 or more Response actions List and generating display data for an abnormality including the display of the abnormality. One or more of the above Response actions The customer selects which action to take from the list. The monitoring screen is displayed in a state in which a decision operation can be executed. On the customer operation terminal The customer operation terminal displays In response to an operation by the customer,The response action presented on the monitoring screen Of the list Then, decide what response action to take.

[0008] In addition, in order to solve the above problems, the present invention provides a video monitoring method using a video monitoring device that monitors abnormalities in a monitored facility using video from a surveillance camera that captures the facility, the video monitoring device collecting customer information that is associated with information on a customer who has concluded a maintenance contract for monitoring the facility, the facility that is the subject of the maintenance contract, and the surveillance camera that captures the facility, and when an abnormality in the facility is detected, Selectable by said customer The next action to be taken is determined for each abnormality. 1 or more a camera image acquisition step in which the video monitoring device acquires video data of the equipment captured by the surveillance camera, a camera image analysis step in which the video monitoring device detects an abnormality in the equipment from the video of the video data and identifies the abnormality content of the detected abnormality, and 1 or more a response action selection step of acquiring a response action; and a visualization processing step of the video monitoring device generating display data for displaying a monitoring screen displaying the image of the video data acquired in the camera video acquisition step on a customer operation terminal operated by the customer, and displaying the monitoring screen on the customer operation terminal using the display data, wherein when an abnormality of the equipment is detected in the camera video analysis step, the video monitoring device in the visualization processing step displays the image in which the abnormality was detected and the 1 or more Response actions List and generating display data for an abnormality including the display of the abnormality. One or more of the above Response actions The customer selects which action to take from the list. The monitoring screen is displayed in a state in which a decision operation can be executed. On the customer operation terminal A video monitoring method is provided. Effect of the Invention

[0009] According to the present invention, it is possible to support a user in taking action to deal with a detected abnormality. Problems, configurations, and effects other than those described above will become apparent from the description of the embodiments described below. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a video monitoring system 100 according to an embodiment of the present invention. [Diagram 2] FIG. 4 is a diagram showing an example of customer information. [Diagram 3] FIG. 11 is a diagram illustrating an example of response action information. [Figure 4] FIG. 2 is a diagram showing an example of a processing flow of the entire processing in the video monitoring system 100. [Diagram 5] FIG. 13 is a diagram showing an image of data reference etc. in video data analysis processing. [Figure 6] FIG. 13 is a diagram showing an image of data reference etc. in the response action acquisition process. [Figure 7] 11 is a diagram for explaining an example of a screen configuration of a monitoring screen. FIG. [Figure 8] FIG. 13 is a diagram showing an example of a monitoring screen display when an abnormality occurs. [Figure 9] FIG. 13 is a diagram showing another example of the monitoring screen when an abnormality occurs; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In the following description, an "interface apparatus" may refer to one or more interface devices. The one or more interface devices may be at least one of the following: One or more I / O (Input / Output) interface devices. The I / O interface device is an interface device for at least one of the I / O device and a remote display computer. The I / O interface device for the display computer may be a communications interface device. The at least one I / O device may be a user interface device, for example, either an input device such as a keyboard or a pointing device, or an output device such as a display device. One or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)), or two or more heterogeneous communication interface devices (e.g., a NIC and an HBA (Host Bus Adapter)).

[0012] In the following description, a "memory" refers to one or more memory devices, typically a primary storage device. At least one of the memory devices in the memory may be a volatile memory device or a non-volatile memory device.

[0013] In the following description, a "persistent storage device" refers to one or more persistent storage devices. A persistent storage device is typically a non-volatile storage device (e.g., an auxiliary storage device), and specifically, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0014] In the following description, a "storage device" refers to at least one of the above-mentioned memory and persistent storage devices.

[0015] In the following description, a "processor" refers to one or more processor devices. The at least one processor device is typically a microprocessor device such as a CPU (Central Processing Unit), but may be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may be a processor core. Alternatively, the at least one processor device may be a broader processor device such as a hardware circuit (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0016] In the following description, functions may be described using expressions such as "kkk unit", but the functions may be realized by one or more computer programs being executed by a processor, or by one or more hardware devices (e.g., FPGA or ASIC), or by a combination thereof. The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

[0017] In the following description, information that provides an output for an input may be described using an expression such as "xxxDB," but the information may be data of any structure (for example, it may be structured data or unstructured data) or a learning model such as a neural network that generates an output for an input. Therefore, "xxxDB" may be referred to as "xxx information." In the following description, the configuration of each DB is an example, and one DB may be divided into two or more DBs, or all or part of two or more DBs may be realized by one DB. Note that "DB" is an abbreviation for database.

[0018] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this specification, when describing elements of the same kind (elements having the same function) without distinction, a common part (part excluding the suffix and the branch number) of the reference number including the suffix and the branch number is used, and when describing elements of the same kind with distinction, a reference number including the suffix and the branch number may be used. For example, when describing a surveillance camera without any particular distinction, it may be described as "surveillance camera 22", whereas when describing each surveillance camera 22 with distinction, it may be described as "surveillance camera 22a", "surveillance camera 22b", "surveillance camera 22c", etc. In addition, the repetition of the description of elements having the same function as described above will be omitted. In addition, the description of general components constituting an elevator, specifically, the main rope, the counterweight, the car, the hoist, the security camera, the video recording device, etc. will be omitted.

[0019] (1) Composition Fig. 1 is a block diagram showing an example of the configuration of a video monitoring system 100 according to an embodiment of the present invention. As shown in Fig. 1, the video monitoring system 100 includes a video monitoring device 1, a camera video recording device 21 and a surveillance camera 22 in a building (structure) 2, and a customer operation terminal 3. The video monitoring device 1 is communicatively connected to the camera video recording device 21 via a communication network 4, and communicatively connected to the customer operation terminal 3 via a communication network 5.

[0020] In this embodiment, the equipment 23 in the building 2 (specifically, the rooftop 23a, the elevator 23b, the fire equipment 23c, the automatic door 23d, and the parking lot 23e) are monitored, and the surveillance cameras 22a to 22e are installed to capture the respective equipment 23. The images captured by each surveillance camera 22 are stored in the camera image recording device 21. The camera image recording device 21 may be, for example, a general computer or storage device, and may be a device having at least a storage function for recording image data and a communication function for transmitting the recorded image data to the image monitoring device 1. Note that the equipment 23 shown in FIG. 1 is merely an example, and is not limited thereto. For example, the equipment 23 to be monitored may be an outside passageway, a garbage collection area, or the like.

[0021] The building 2 in which the surveillance camera 22 that monitors the facility 23 to be monitored is installed is not limited to one, and may be multiple. In this embodiment, the surveillance camera 22 is treated as one management unit, but the management unit is not limited to this. For example, a group of buildings (buildings) 2 may be the management unit. The facility 23 to be monitored is not limited to facilities within a building, and facilities in structures other than buildings, such as factories and stations, may be monitored, and these structures may be the management units.

[0022] The customer operation terminal 3 is an information processing terminal owned or used by a customer, and includes a display device, an input device, and an interface device. The input device is, for example, a keyboard and a mouse, but may also be, for example, a touch panel integrated with the display device. Specific examples of the customer operation terminal 3 are not particularly limited, but include, for example, a personal computer, a tablet, or a smartphone. The number of customer operation terminals 3 is not limited to one, and may be multiple. Note that the "customer" in this embodiment refers to a person or organization (contractor) that has concluded a contract (hereinafter, "maintenance contract") regarding the maintenance and management of the building 2.

[0023] The communication network 4 connecting the video monitoring device 1 and the camera video recording device 21, and the communication network 5 connecting the video monitoring device 1 and the customer operation terminal 3 may be the same type of communication network or different types of communication networks. Each of the communication networks 4 and 5 may be, for example, the Internet, a mobile communication network, a wired LAN (Local Area Network), a wireless LAN, or a dedicated line, or a combination of these. Also, the communication networks 4 and 5 may be a single communication network (for example, the Internet).

[0024] The video monitoring device 1 may be an information processing device including an interface device (not shown), a storage device 12, and a processor connected to them, or may be a device realized on the device (e.g., a cloud platform). The interface device inputs and outputs data to and from the video monitoring device 1. Specifically, the video monitoring device 1 may be provided with a keyboard and a display, or an operation terminal (not shown) used by an operator or salesperson may be used as the interface device for the video monitoring device 1.

[0025] In the video monitoring device 1, the processor is, for example, a CPU (Central Processing Unit). The storage device 12 includes a memory, for example, a ROM (Read Only Memory) and a RAM (Random Access Memory). In the video monitoring device 1, the CPU reads and executes program codes stored in the ROM, and the software and hardware work together to realize the functions of each processing unit (customer information input unit 11, camera image acquisition unit 13, visualization processing unit 14, camera image analysis unit 15, and response action selection unit 16) described below. The storage device 12 may be realized by including a persistent storage device such as an HDD (Hard Disk Drive) in addition to the above-mentioned ROM and RAM. The communication unit 17 is realized by an interface device having a communication function.

[0026] As shown in FIG. 1, the video monitoring device 1 includes a customer information input unit 11, a storage device 12, a camera image acquisition unit 13, a visualization processing unit 14, a camera image analysis unit 15, a response action selection unit 16, and a communication unit 17.

[0027] The customer information input unit 11 has a function of inputting information (customer information) about a customer who has concluded a maintenance contract for video surveillance of the facility 23 and the target of surveillance into the customer information DB 12a of the storage device 12. The customer information may include information indicating a person receiving the maintenance service, a contractor, a building ID for identifying the building 2, and a building address. A specific example of customer information is shown in FIG. 2, which will be described later. In this embodiment, information about the facility 23 that is the target of surveillance in the building 2 and information about the surveillance camera 22 that monitors the facility 23 are registered in the customer information in association with each other.

[0028] The storage device 12 has a function of storing predetermined information. In this embodiment, the storage device 12 is configured with different DBs (databases) depending on the type of information to be stored, and specifically, the storage device 12 has a customer information DB 12a, a camera image management DB 12b, an abnormality record DB 12c, and a response action DB 12d.

[0029] The customer information DB 12a stores information (customer information) about customers who have concluded a maintenance contract for video monitoring of the facility 23 and the target of monitoring. As described above, the customer information is input from the customer information input unit 11, and a specific example is shown in FIG. 2, which will be described later. The camera video management DB 12b stores information (camera video information) about video data captured by each monitoring camera 22. The camera video information is acquired from the camera video recording device 21 and stored in the camera video management DB 12b. The abnormality record DB 12c stores information (abnormality information) about abnormalities detected by analyzing the camera video information of each monitoring camera 22. The camera video information is analyzed by the camera video analysis unit 15. The response action DB 12d stores information (response action information) about response actions that are options for responding to a detected abnormality. A specific example of the response action information is shown in FIG. 3, which will be described later.

[0030] The camera image acquisition unit 13 connects to the camera image recording device 21, which records the image data of each monitoring camera 22, via the communication network 4 based on each camera (monitoring camera 22) indicated in the customer information stored in the customer information DB 12a at a predetermined interval (for example, once every 5 minutes). The camera image acquisition unit 13 then periodically acquires monitoring images (image data) of the facility 23 from each monitoring camera 22 from the camera image recording device 21, and registers the camera image information based on the acquired image data in the camera image management DB 12b. The camera image acquisition unit 13 may be configured to be constantly connected to the camera image recording device 21. The camera image acquisition unit 13 may also register the image data acquired from the camera image recording device 21 as camera image information as it is in the camera image management DB 12b, but preferably registers the image data as camera image information in the camera image management DB 12b after adding predetermined information related to the image data (for example, information related to the image data itself, various information registered in the customer information related to the shooting environment of the image data, etc.).

[0031] The visualization processing unit 14 has a function of processing data to be output to the customer operation terminal 3, and a function of generating screen data to be output to the customer operation terminal 3. Specifically, for example, the visualization processing unit 14 can generate a screen for requesting input of a user name and password, a monitoring screen for the equipment 23 to be monitored in the building 2, a response selection screen when an abnormality is detected, and the like, and provide them to the customer operation terminal 3.

[0032] The camera video analysis unit 15 has a function of detecting anomalies from the camera video information of the surveillance camera 22 acquired from the camera video recording device 21, and identifying the type of the detected anomaly (anomaly type). In analyzing the camera video information (i.e., detecting the anomaly and identifying the anomaly type described above), the camera video analysis unit 15 obtains an analysis result by referring to the customer information DB 12a and the camera video management DB 12b and inputting necessary data into a predetermined algorithm for detecting anomalies. The camera video analysis unit 15 then registers the obtained analysis result in the anomaly record DB 12c as anomaly information. The camera video analysis unit 15 also has a function of outputting the obtained analysis result to the visualization processing unit 14.

[0033] The response action selection unit 16 has a function of receiving the analysis result (detection of anomaly, identification of anomaly type) by the camera image analysis unit 15 and acquiring a recommended response action from among the response actions associated with the identified anomaly type. Specifically, the response action selection unit 16 refers to the anomaly record DB 12c and the response action DB 12d, and selects and acquires information on an action (response action) corresponding to the anomaly recorded in the anomaly information from the response action information. The response action selection unit 16 also has a function of outputting information on the acquired response action to the visualization processing unit 14.

[0034] The communication unit 17 functions as an input / output unit that enables each part of the video monitoring device 1 to input and output data between the camera video recording device 21, the customer operation terminal 3, or an operation terminal (not shown) of a maintenance staff or sales staff, etc.

[0035] (2)Various information In the following, various types of information stored in the storage device 12 will be described in detail, focusing on the customer information and the response behavior information that are set before the video surveillance is performed.

[0036] First, the customer information will be described. The customer information is information about a customer who has concluded a maintenance contract, and is registered in the customer information DB 12a by the customer information input unit 11 in response to input operations by a sales person who performs sales activities or a maintenance person who performs maintenance work.

[0037] Fig. 2 is a diagram showing an example of customer information. Customer information 121 shown in Fig. 2 is a specific example of customer information registered in customer information DB 12a. When a video surveillance maintenance contract is concluded with a customer on a building 2 basis, customer information 121 is formed from a customer record that compiles information on the customer and the monitored object for each building 2 with which the video surveillance maintenance contract is concluded.

[0038] 2, the customer record is composed of the following items: customer 1211, customer ID 1212, building ID 1213, building address 1214, building equipment 1215, building equipment ID 1216, camera 1217, and camera ID 1218. Note that the items shown in FIG. 2 are merely examples, and the customer information according to this embodiment may be such that a customer record is formed including items other than those exemplified in FIG. 2.

[0039] The name of a customer who has concluded a maintenance contract is registered in the customer 1211, and an identifier (customer ID) assigned to uniquely identify the customer is registered in the customer ID 1212. The building ID 1213 is an identifier (building ID) assigned to uniquely identify a building 2 in which a facility subject to video monitoring under the maintenance contract is installed, and the building address 1214 is an address of the building 2. The building facility 1215 is a name of a facility subject to video monitoring (facility 23), and the building facility ID 1216 is an identifier (building facility ID) assigned to identify the facility 23. The camera 1217 is a name of a surveillance camera 22 that monitors the facility subject to video monitoring (facility 23), and the camera ID 1218 is an identifier (camera ID) assigned to identify the surveillance camera 22.

[0040] Incidentally, although not shown in FIG. 1, in this embodiment, a user DB may be stored in the storage device 12. In this embodiment, a "user" refers to a person who belongs to a customer (for example, an employee of the customer's company). The user DB may have a user record for each user, and it is desirable for each user record to include an individual user ID, password, and customer ID. The user ID and password are authentication information for identifying a user who can access the video monitoring device 1.

[0041] Here, the usage of the user ID and password will be described. The customer operation terminal 3 is connected to the video monitoring device 1 through the communication network 5 by the user's operation. In response to this connection, the visualization processing unit 14 requests the user to input the user ID and password. That is, the visualization processing unit 14 outputs information prompting the user to input the user ID and password to the customer operation terminal 3. Then, only when a combination of the user ID and password registered in the user DB is input from the customer operation terminal 3, the visualization processing unit 14 can output (for example, display) various information acquired from the customer information DB 12a, the camera video management DB 12b, the abnormality record DB 12c, and the response action DB 12d to the customer operation terminal 3. Note that the output data created by the visualization processing unit 14 is not limited to a specific data format, and specifically, for example, an image format, a video format, an HTML format, an XML format, a CSV format, or the like can be adopted.

[0042] Furthermore, in this embodiment, reference is mainly made to video data from surveillance camera 22, abnormalities (type and content of abnormality) detected by analyzing the video data, and actions (response actions) taken against the abnormalities, but instead of or in addition to this information, video monitoring device 1 according to the present invention may display responses to inquiries about building facilities, scheduled inspection dates, inspection results, etc. By configuring in this way, video monitoring device 1 according to the present invention can handle various types of contract information about building facilities.

[0043] Next, the response action information will be described. The response action information is information on recommended actions (response actions) for an abnormality detected in the target equipment (equipment 23) of video monitoring under a maintenance contract, and one or more response actions are set in advance according to the abnormality content. The response action information may be registered in the response action DB 12d before video monitoring is performed by the video monitoring device 1, and various methods can be adopted for the registration method. Specifically, for example, the response action information may be registered in the response action DB 12d in advance by a system administrator, or the response action information for representative abnormality content may be registered in the response action DB 12d by a manufacturer when the video monitoring device 1 is manufactured or sold. In addition, the response action information may be appropriately customized by the customer side (for example, the owner or management company of the building 2).

[0044] Fig. 3 is a diagram showing an example of the response action information. The response action information 124 shown in Fig. 3 is a specific example of the response action information registered in the response action DB 12d, and the response action information 124 is formed from a response action record in which one or more response actions to be taken next when an abnormality is detected are linked for each combination of the equipment 23 of the building 2 for which a maintenance contract for video monitoring by the surveillance camera 22 has been concluded and the abnormality content.

[0045] Specifically, in the response action information 124 in FIG. 3, the response action record is composed of the following items: building facility 1241, building facility ID 1242, anomaly type 1243, anomaly content 1244, anomaly content priority 1245, action number 1246, action content 1247, and contact 1248. Note that the items shown in FIG. 3 are merely examples, and the response action information according to this embodiment may be one in which a response action record is formed including items other than those shown in FIG. 3. Also, in FIG. 3, a response action record is shown for "fire equipment" with building facility ID "S001" and "elevator" with building facility ID "E001", but in the actual response action information 124, a response action record may be registered for all the facilities shown in the building facility 1215 in the customer information 121 in FIG. 2.

[0046] The building equipment 1241 and the building equipment ID 1242 correspond to the building equipment 1215 and the building equipment ID 1216 in the customer information 121 shown in Fig. 2. The abnormality type 1243 indicates the type of abnormality that can be detected by the video data analysis by the camera video analysis unit 15. In the case of Fig. 3, "smoke / ignition" belonging to fire abnormalities and "abnormal behavior" belonging to human behavior abnormalities are exemplified.

[0047] In the abnormality content 1244, an abnormality content indicating a specific abnormality event is registered. The abnormality content can be regarded as a detailed classification of the abnormality type 1243. In the abnormality content priority 1245, a priority corresponding to each abnormality content registered in the abnormality content 1244 is registered. The priority corresponding to each abnormality content is set in advance, but may be changed by an administrator or the like during operation. In the abnormality content priority 1245, a priority may be set for each abnormality type 1243, but considering the case where abnormalities belonging to multiple abnormality types occur simultaneously, it is preferable to set a priority for all abnormality contents 1244 belonging to all abnormality types 1243. In the example shown in FIG. 3, three types of abnormality contents belonging to "smoke / fire" are registered: "smoke detection" meaning an abnormality in which smoke is detected from video data, "fire detection" meaning an abnormality in which a flame is detected from video data, and "operation without smoke / fire" meaning an abnormality in which neither smoke nor flame is detected from video data but smoke or fire is estimated from other situations in the video (e.g., blinking of a lamp, etc.). Furthermore, as abnormality contents belonging to "abnormal behavior", three types are registered: "abnormal behavior (unruly) detection", which means an abnormality where a person is detected to be violent from video data; "abnormal behavior (staying) detection", which means an abnormality where a person is detected to be staying from video data; and "abnormal behavior (other) detection", which means an abnormality in human behavior other than violence and staying staying is detected from video data. For the above six types of abnormality contents 1244, priorities of "1" to "6" are set in the abnormality content priority 1245, with the smaller the priority value, the higher the priority. For example, "ignition detection" is the most dangerous abnormality among the above six types of abnormality contents 1244, and therefore the highest priority "1" is set.

[0048] In the action content 1247, one or more response actions recommended when the abnormality content 1244 is detected are registered, and in the contact 1248, a contact to be contacted when the corresponding response action is executed is registered. In addition, in the action number 1246, for one or more response actions (action content 1247) recommended for each abnormality content 1244, an order in which each response action is recommended is registered. In the case of FIG. 3, for example, when the detected abnormality is "smoke detection", "call a fire engine" is registered in the action content 1247 as a response action linked to the "first" recommended action number 1246, and in the contact 1248, "119", which is the telephone number of the fire department, is registered as a contact for the corresponding response action. In addition, for "smoke detection", "announce by PA system" is registered in the action content 1247 as a response action linked to the "second" recommended action number 1246, but since a contact is not required for the corresponding response action, no specific contact is registered in the contact 1248. In this embodiment, as described above, multiple response actions can be registered for one abnormality, and a recommended order for these response actions can be set, making it possible to clearly present to the customer what action they should take when an abnormality is detected.

[0049] 3, since the abnormality type 1243 differs for each building facility 1241, the abnormality content 1244 belonging to the abnormality type 1243 and the action content 1247 corresponding to the abnormality content 1244 also differ for each building facility 1241, but the response action information 124 according to this embodiment may register common or overlapping abnormality types 1243 even for different building facilities 1241. Furthermore, in the response action information 124 (response action record) according to this embodiment, it is sufficient that at least the action content 1247 is individually defined for each abnormality content 1244.

[0050] Furthermore, the response action information 124 (response action DB 12d) is registered in advance by, for example, a system administrator, but if the user is an employee of a management company, it may be necessary to register contact information that the maintenance company cannot anticipate, such as the need to contact the building owner when an abnormality is detected. Therefore, as a modified example of this embodiment, the response action information 124 may be configured so that the user can change or newly register the action number 1246, action content 1247, or contact information 1248.

[0051] As described above, the anomaly information DB 12c stores anomaly information regarding anomalies detected from video data. In this anomaly information, the data items regarding the detected anomalies use a data format corresponding to the response action information 124 (specifically, anomaly type 1243, anomaly content 1244, etc.).

[0052] (3) Processing The following describes the process in which the video monitoring system 100 of this embodiment provides a monitoring screen to the customer operation terminal 3 using video data from the surveillance camera 22, monitors the occurrence of abnormalities from the video data from the surveillance camera 22, and when an abnormality is detected, notifies the customer operation terminal 3 to assist in appropriate response actions to the abnormality (e.g., reporting the abnormality).

[0053] Fig. 4 is a diagram showing an example of a processing flow of the entire processing in the video monitoring system 100. Fig. 4 mainly shows the processing related to the display of information on the monitoring screen and the operations on the monitoring screen, and omits an explanation of basic operations such as logging in and logging out of the system (video monitoring device 1).

[0054] 4, first, the camera video recording device 21 constantly acquires and records video data of the equipment 23 captured by the surveillance camera 22 installed in the building 2 (step S11). Then, the camera video acquisition unit 13 of the video monitoring device 1 constantly or periodically accesses the camera video recording device 21, acquires the video data of the surveillance camera 22, and registers the camera video information based on the video data in the camera video management DB 12b (step S21). Note that in step S21, the camera video acquisition unit 13 may also acquire various information required to generate the camera video information based on the video data from the camera video recording device 21 and the customer information DB 12a.

[0055] Next, when a predetermined condition is satisfied in the video monitoring device 1, the camera video analysis unit 15 starts a video data analysis process for each video data. The above-mentioned predetermined condition may be, for example, a timing of a predetermined cycle after the monitoring screen is started in the customer operation terminal 3, or a timing of a predetermined cycle regardless of the function of the monitoring screen. In the former case, an abnormality is detected only when the monitoring screen is started in the customer operation terminal 3, so that the processing load in the video monitoring device 1 can be reduced. On the other hand, in the latter case, regardless of the operation status in the customer operation terminal 3, the video data analysis is periodically performed on all the video data acquired from the camera video recording device 21, so that the processing load is high but the occurrence of an abnormality can be detected quickly and reliably. In addition, a predetermined condition other than the above may be set.

[0056] In the video data analysis process, the camera video analysis unit 15 selects one unanalyzed video data from among the video data to be analyzed registered in the camera video management DB 12b, and acquires the camera video information and customer information related to the selected video data from the camera video management DB 12b and the customer information DB 12a (step S22). For example, when the monitoring screen is activated on the customer operation terminal 3, the customer ID and building ID entered by the customer when the screen is activated are transmitted to the video monitoring device 1, so that the video data of the building facility 1215 linked to these IDs in the customer information 121 (i.e., the video data captured by the camera 1217) becomes the video data to be analyzed.

[0057] Next, the camera image analysis unit 15 uses the information acquired in step S22 to perform image analysis to determine whether there is an abnormality in the image of the image data to be analyzed (step S23). In the image analysis in step S23, an abnormality corresponding to the abnormality type 1243 and the abnormality content 1244 in the response action information 124 (see FIG. 3) is detected.

[0058] Next, the camera video analysis unit 15 determines whether or not an abnormality is detected in the image analysis of step S23 (step S24). If no abnormality is detected in step S24 (NO in step S24), the image (still image or moving image) of the video data in which no abnormality is detected is transmitted to the visualization processing unit 14, and the process proceeds to step S28 described below. If an abnormality is detected in step S24 (YES in step S24), the camera video analysis unit 15 identifies the type and content of the detected abnormality, generates abnormality information from the analysis results including the identified information, and registers the generated abnormality information in the abnormality record DB 12c (step S25).

[0059] FIG. 5, which will be described later, shows an image of data reference and the like in the video data analysis process in steps S22 to S25.

[0060] After the process of step S25, the response action selection unit 16 performs a response action acquisition process to acquire a response action associated with the detected abnormality.

[0061] In the response action acquisition process, the response action selection unit 16 refers to the anomaly information registered in the anomaly record DB 12c in step S25, refers to the response action DB 12d using the anomaly type and the anomaly content as keys, and acquires all the response action records that match the anomaly type 1243 and the anomaly content 1244 (step S26). If the video monitoring device 1 has a pop-up notification function for notifying the detection of an anomaly, the response action selection unit 16 may notify the visualization processing unit 14 of the detection of the anomaly at the timing of step S26, and the visualization processing unit 14 may transmit the pop-up notification to the customer operation terminal 3 via the communication unit 17. The timing for the pop-up notification may be any of steps S27 to S29 described later, but by adopting an early timing such as step S26, the detection of the anomaly can be promptly notified to the customer.

[0062] Next, the response action selection unit 16 refers to the abnormality content priority 1245 of the response action record acquired in step S26, and if an abnormality has been detected in the video data analysis performed so far on other video data, rearranges information on the detection of multiple abnormalities based on the abnormality content priority 1245 (step S27). The process of step S27 is a process for determining which abnormality should be notified preferentially (for example, displayed at the top of the monitoring screen) when abnormalities are detected in multiple pieces of equipment 23 at the same time. Therefore, if no abnormality has been detected in the video data analysis performed so far on other video data, the process of step S27 may be skipped. After the process of step S27 is completed, the response action selection unit 16 transmits information on one or more response action records after rearrangement to the visualization processing unit 14, and proceeds to step S28.

[0063] FIG. 6, which will be described later, shows an image of data reference and the like in the response action acquisition process in steps S26 to S27.

[0064] In step S28, a predetermined processing unit of the video monitoring device 1 (for example, it may be the camera video analysis unit 15 or an overall control unit (not shown)) determines whether any unanalyzed video data remains. If unanalyzed video data remains (YES in step S28), the process returns to step S22 and the above-mentioned process is repeated. If video data analysis has been completed for all video data to be analyzed (NO in step S28), the process proceeds to step S29. Here, "all video data to be analyzed" may be replaced with video data from the surveillance cameras 22 for all equipment 23 in the building 2.

[0065] In step S29, the visualization processing unit 14 generates display data for the monitoring screen using various information received from the camera image analysis unit 15 and the response action selection unit 16 in the processes of steps S22 to S28. The display data generated here includes not only display data related to the equipment 23 in which an abnormality has been detected, but also display data related to the equipment 23 in which no abnormality has been detected. Then, the visualization processing unit 14 transmits the display data to the customer operation terminal 3, whereby the customer operation terminal 3 can display the received display data on the activated monitoring screen (corresponding to step S33 described later).

[0066] In the processing flow of FIG. 4, the visualization processing unit 14 generates display data after the video data analysis processing (and corresponding action acquisition processing) is completed for all video data to be analyzed. However, as a variant, display data may be generated each time the video data analysis processing (and corresponding action acquisition processing) is completed for each video data, and the display data may be transmitted to the customer operation terminal 3 at any stage.

[0067] When the process of step S29 is completed, in principle, the video monitoring device 1 ends the series of processes. However, when the execution of a specific response action is decided from the customer operation terminal 3, the video monitoring device 1 may execute that response action (step S30). For example, when a response action that can be executed by a response action processing unit (not shown) of the video monitoring device 1 (specifically, "announce via the public address system") is set, the response action processing unit executes that response action in response to an operation on the monitoring screen of the customer operation terminal 3 (step S35 described later).

[0068] Next, the processing in the customer operation terminal 3 will be described with reference to Fig. 4. Note that the processing flow shown in Fig. 4 is merely one example of processing in the customer operation terminal 3, and is not limited thereto. For example, before receiving a pop-up notification of abnormality detection, the customer may operate the customer operation terminal 3 to launch a monitoring screen.

[0069] 4, when the customer operation terminal 3 receives a pop-up notification from the visualization processing unit 14 of the video monitoring device 1 in step S26, the customer operation terminal 3 issues a pop-up notification by sound, display, vibration, or the like to notify the customer of the detection of an abnormality (step S31). The function of the pop-up notification is a generally known technology, so a detailed description will be omitted.

[0070] Next, in response to the pop-up notification in step S31, or by the customer's will, the customer performs a predetermined operation, causing the customer operation terminal 3 to launch a monitoring screen (step S32). When the monitoring screen is launched, the customer is requested to input a customer ID (user ID) assigned to the customer and a building ID of the building 2 in which the equipment 23 to be monitored is provided, and these customer ID and building ID are transmitted to the video monitoring device 1 via the communication network 5. When the monitoring screen is launched, authentication may be performed using the user ID and password described above.

[0071] After the monitoring screen is started, the customer operation terminal 3 displays the monitoring screen using the display data received from the visualization processing unit 14 of the video monitoring device 1 (step S33).

[0072] The customer then checks whether or not an abnormality has been detected by looking at the monitoring screen displayed in step S33 (step S34). If no abnormality has been detected, the monitoring screen displays "Normal" (NO in step S34), and the customer does not need to perform any special operation; the process returns to step S33, and the display of the monitoring screen (including updating of the display content) continues. At this time, the customer may, for example, close the monitoring screen. On the other hand, if an abnormality has been detected, the monitoring screen displays "Abnormal" (YES in step S34), and in this case the customer is requested to perform the operation of step S35.

[0073] FIG. 7, which will be described later, shows examples of the "normal" and "abnormal" monitoring screen configurations.

[0074] When the message "Abnormal" is displayed on the monitoring screen, in step S35, the customer checks the video in which the abnormality was detected and presses the response action display button (response action display button 3022 in FIG. 7 described later) to determine the response action to be taken from the response actions displayed in a list (response action list 3203 in FIG. 7 described later).

[0075] Next, the customer performs a predetermined operation to execute the response action determined in step S35, and the customer operation terminal 3 or the video monitoring device 1 executes the response action (step S30 or step S36). The predetermined operation to execute the response action corresponds to, for example, pressing the "contact" button for the response action displayed on the monitoring screen. It may be considered that which of the customer operation terminal 3 or the video monitoring device 1 executes the response action is set in advance in the response action information (response action DB 12d). Specifically, for example, in the case of a response action in which a specific telephone number is registered in the contact, it is more efficient for the customer operation terminal 3 to make a call directly in step S36. Also, in the case of a response action in which a specific telephone number is not registered in the contact and an announcement is made in the building 2, it may be more efficient for the video monitoring device 1 to take the initiative in executing the response action in step S30.

[0076] After the response action is taken, the customer can end the operation on the monitoring screen, but it is preferable that the information continue to be displayed on the monitoring screen while being updated until the abnormality is actually resolved.

[0077] It is possible to adopt various configurations for the execution of the response action, as follows. For example, when contacting a contact person according to the response action, the video in which the abnormality is detected may be repeatedly played on the monitoring screen in order to make it easier to convey the abnormal situation. Also, for example, as a response action in the event of an abnormality, the customer operation terminal 3 may be directly or indirectly connected to the in-house broadcasting equipment of the building 2 or an intercom in the elevator car, so that questions can be made to users of the building 2. Also, for example, as a response action in the event of an abnormality, the customer operation terminal 3 may be directly or indirectly connected to a signage in the building 2 or a liquid crystal display in the elevator car, so that the video of the abnormality is distributed, so that it is easier to convey information to security guards and emergency personnel who rush to the scene. Also, for example, as a response action in the event of an abnormality, a response action such as remote control of equipment (such as disaster prevention equipment) in the building 2 may be provided.

[0078] Furthermore, if no abnormality is detected, the monitoring screen may be updated periodically, and each time an update is performed, the video data may be analyzed and the video may be displayed on the monitoring screen.

[0079] Furthermore, regarding the presence or absence of abnormality detection and the notification at the time of abnormality detection, history information (logs) may be saved and accumulated in the storage device 12 of the video monitoring device 1. With such a configuration, the accumulated logs can be used for post-mortem verification of the detected abnormality, etc.

[0080] 4 shows a procedure in which a monitoring screen is launched in the customer operation terminal 3 after a pop-up notification is issued when an abnormality is detected, but the process flow when an abnormality is detected in this embodiment is not limited to the above procedure. For example, when an abnormality is detected while the monitoring screen is being displayed, the customer operation terminal 3 may be notified of the detection of the abnormality separately by a predetermined means such as a pop-up notification. Also, for example, when an abnormality is detected while the monitoring screen is being displayed, the display of the monitoring screen may be switched to a monitoring screen for abnormality, which will be described later in FIG. 8 and FIG. 9, without a pop-up notification.

[0081] In the above description, the camera video (video data) is analyzed in step S22 based on the state of the customer operation terminal 3, such as when the monitoring screen is started, but the trigger for analyzing the video data in this embodiment is not limited to this. For example, the camera video analysis unit 15 may analyze the video data every time the camera video acquisition unit 13 acquires video data from the camera video recording device 21.

[0082] As described above, the overall processing in the video monitoring system 100 has been explained with reference to Figure 4. Below, as a detailed explanation, the video data analysis processing and corresponding action acquisition processing in the video monitoring device 1, and the display of the monitoring screen in the customer operation terminal 3 will be explained with reference to the drawings.

[0083] (3-1) Video data analysis and processing Fig. 5 is a diagram showing an image of data reference in video data analysis processing. As shown inside camera video analysis unit 15 in Fig. 5, data reference (reference, acquisition, transmission, recording) shown in Fig. 5 corresponds to the processing of steps S22 to S25 in Fig. 4.

[0084] The data flow in the video data analysis process will be described with reference to FIG.

[0085] First, when a request to display a monitoring screen is made in response to operation of a customer's customer operation terminal 3, the visualization processing unit 14 receives the request via the communication unit 17 and transmits to the camera image analysis unit 15 the customer ID (user ID) of the customer operating the customer operation terminal 3 and the building ID of the building 2 to be monitored, which are included in the request.

[0086] Next, the camera video analysis unit 15 refers to the customer information DB 12a using the received customer ID and building ID as keys to acquire the camera ID 1218 of the corresponding customer record. Furthermore, the camera video analysis unit 15 refers to the camera video management DB 12b using the acquired camera ID 1218 and the above-mentioned customer ID and building ID as keys to acquire the "video recording date and time (start)" and "video recording date and time (end)" indicating the recording date and time of the target video data, and the "video data" which is the main body of the target video data, as information necessary for analyzing the video data.

[0087] Next, the camera image analysis unit 15 performs image analysis (image data analysis) on the acquired image data to check whether any abnormality has occurred. The camera image analysis unit 15 is equipped with a program including an anomaly detection algorithm for camera images (still images or moving images) using machine learning or the like, and the anomaly detection algorithm is capable of detecting, for example, equipment failure such as an automatic door not closing, smoke, fire, abnormal behavior of building users (being violent or staying still), suspicious behavior (mischief such as graffiti), and the like.

[0088] When the camera image analysis unit 15 detects equipment failure such as an automatic door not closing, smoke, fire, abnormal behavior of building users (unruly or staying), suspicious behavior (mischief such as graffiti), etc., by analyzing the image data, it identifies the type and content of the abnormality, generates abnormality information from the analysis result including these, and registers it in the abnormality record DB 12c. A specific data configuration example of the abnormality information is as shown in FIG. 5, and includes the abnormality type and content, various information acquired from the customer information DB 12a and the camera image management DB 12b in step S22, the date and time of abnormality detection, and image data including an image of the abnormal state. Note that the "image data" of the abnormality information registered in the abnormality record DB 12c does not need to be the entire image data that has been subjected to image analysis. For example, it is preferable to generate image data showing an image for 5 seconds before and after the date and time of abnormality detection (total of 10 seconds) and use this as the "image data" of the abnormality information. In this way, by using a part of the original image data that has been subjected to image analysis as the "image data" of the abnormality information, it is possible to suppress an increase in the data volume of the abnormality information while securing the image showing the detected abnormality.

[0089] (3-2) Response action acquisition process 6 is a diagram showing an image of data reference etc. in the response action acquisition process. The data reference etc. (reference, acquisition, transmission) shown in FIG. 6 corresponds to the processes in steps S26 to S27 in FIG.

[0090] The data flow in the response action acquisition process will be described with reference to FIG.

[0091] After the camera image analysis unit 15 analyzes the video data and stores the analysis result as anomaly information in the anomaly record DB 12c, the response action selection unit 16 refers to the anomaly record DB 12c to acquire the anomaly type and anomaly content of the detected anomaly. Next, the response action selection unit 16 refers to the response action DB 12d using the acquired anomaly type and anomaly content as keys to acquire the action number 1246, action content 1247, and contact information 1248 of the corresponding response action record.

[0092] Thereafter, the response action selection unit 16 combines the abnormality information (including a part of the customer information as shown in FIG. 5) acquired from the abnormality record DB 12c with the response action information acquired from the response action DB 12d, and transmits the combined information to the visualization processing unit 14. The visualization processing unit 14 then generates display data for a monitoring screen using the received information, and outputs the display data to the customer operation terminal 3 via the communication unit 17 and the communication network 5, thereby displaying information on the detected abnormality on the monitoring screen of the customer operation terminal 3. On the other hand, if no abnormality is detected, the visualization processing unit 14 generates display data for a monitoring screen based on information acquired from the camera image management DB 12b by the camera image analysis unit 15 described in FIG. 5, and information on normal times is displayed on the monitoring screen of the customer operation terminal 3. In this way, the customer (user) can check the camera image of the monitoring camera 22 capturing the equipment 23 through the monitoring screen in either normal times or abnormal times, and in abnormal times, it is possible to check the response action to the abnormality, as described below.

[0093] (3-3) Monitoring screen 7 is a diagram for explaining an example of the screen configuration of the monitoring screen. The monitoring screen 300 displayed on the customer operation terminal 3 is, for example, a GUI (Graphical User Interface), and displays the video of the facility 23 captured by the surveillance camera 22, anomalies detected from the video of the facility 23, and corresponding actions, etc., based on the display data generated by the visualization processing unit 14.

[0094] As shown in FIG. 7, the monitoring screen 300 has different screen configurations: a monitoring screen 301 for normal times (when no abnormalities are detected) and a monitoring screen 302 for abnormal times (when one or more abnormalities are detected).

[0095] The monitoring screen 301 under normal conditions is composed of a building information display section 3011 that displays information about the building 2 in which the monitored equipment 23 is installed, a message display section 3012 that displays a message to the effect that no abnormalities have been detected, and a camera image display section 3013 that displays the name of each monitored equipment 23 and camera images (still images or video images) taken of the equipment 23.

[0096] The abnormality monitoring screen 302, in addition to a building information display field 3011 and a camera image display field 3013 similar to those of the normal monitoring screen 301, is configured to include an abnormality detection information display field 3021 that displays information regarding the detected abnormality, a response action display button 3022 for displaying a list of recommended response actions (actions to be taken next) for the detected abnormality, and a response action list 3023 that displays a list of response actions when the response action display button 3022 is pressed and can accept an operation to decide to carry out a response action.

[0097] Here, the abnormality detection information display field 3021 includes at least the name of the equipment where the abnormality was detected, the video at the time of the abnormality occurrence, the date and time of the abnormality occurrence, and the abnormality content. The abnormality detection information display field 3021 is preferably displayed in a more emphasized manner than in normal times so that the abnormality detection information display field 3021 can be intuitively recognized as not being in a normal state. In this case, the abnormality detection information display field 3021 is displayed in a different manner from the message display field 3012 displayed in normal times, and specifically, the display frame of the video where the abnormality was detected is displayed in red (e.g., green or black in normal times) or flashes (no flashing in normal times). In addition, the color of the characters displayed in the abnormality detection information display field 3021 may be a color different from that in normal times. In addition, in the monitoring screen 302 in abnormal times, as shown in FIG. 7, it is also preferable that the display mode of the abnormality detection information display field 3021 displaying the camera video of the equipment where the abnormality was detected is displayed higher (in other words, in a more emphasized manner) than the camera video display field 3013 displaying the camera video of the equipment where no abnormality was detected.

[0098] The response actions that can be displayed in the response action list 3023 are the action contents 1247 of the response action information linked to the detected abnormality contents, and the number of the displayed actions may be one or more. Also, in the response action list 3023, similar to the display mode of the abnormality detection information display field 3021 described above, the text color may be changed to red to indicate that an abnormality has occurred. Furthermore, the monitoring screen 302 during an abnormality may be provided with a button for acquiring and switching between an image at the time of the abnormality occurrence and a current image in addition to the configuration exemplified in Fig. 7.

[0099] The display areas 3011 to 3013 and 3021 to 3023 on the above-mentioned monitoring screens 301 and 302 are realized by, for example, GUI components.

[0100] By configuring the monitoring screen 300 as described above, it is possible to display that no abnormality has been detected under normal circumstances, and when an abnormality occurs, the detected abnormality state and the corresponding action associated with the content of the abnormality can be displayed in an intuitive and easy-to-understand manner, thereby supporting the management work of customers such as building managers.

[0101] Fig. 8 is a diagram showing an example of the monitoring screen display when an abnormality occurs. Fig. 8 shows a monitoring screen 303 when an abnormality of "smoke detection" is detected in the firefighting equipment, and a monitoring screen 304 when an abnormality of "abnormal behavior (violence)" is detected in the elevator, and the screen configurations of these monitoring screens 303 and 304 are based on the screen configuration of the monitoring screen 302 described in Fig. 7. Note that Fig. 8 and Fig. 9 described later will explain the monitoring screen 300 that displays the camera video of the monitored target, targeting the customer record in which the customer 1211 in the customer information 121 in Fig. 2 is "XX Management Association", the customer ID 1212 is "001", and the building ID 1213 is "Building A".

[0102] The monitoring screen 303 is an example of a display when an abnormality in "smoke detection" is detected in the firefighting equipment.

[0103] On the monitoring screen 303, the building information display field 3031 outputs the building ID “Building A” and the building address “○○ 1-2-3, □□ City, Ibaraki Prefecture” obtained from the customer information DB 12a (the specific values ​​are listed in building ID 1213 and building address 1214 in Figure 2).

[0104] In the abnormality detection information display field 3032, the building equipment "firefighting equipment" (specific values ​​are described in the customer information 121 in FIG. 2), the video data, the abnormality detection date and time, and the abnormality content "smoke detection" acquired from the abnormality record DB 12c are output. As described above with respect to the monitoring screen 302 in an abnormal state shown in FIG. 7, it is preferable that the abnormality detection information display field 3032 has a display form different from that in a normal state so that an abnormality can be intuitively recognized. Specifically, for example, when the display frame of the camera image of other equipment (such as "automatic door" in the figure) in which no abnormality has been detected is green, the display frame of the camera image displayed in the abnormality detection information display field 3032 may be colored red.

[0105] In the response action list 3033, values ​​of the action number 1246, the action content 1247, and the contact 1248 in the response action record corresponding to the combination of the abnormality type "smoke / fire" of the detected abnormality and the abnormality content "smoke detected" are output. Specifically, in the response action list 3033, "call a fire engine" is displayed as the action content corresponding to the action number "1", and the telephone number "119" is displayed as the contact. In addition, "announce by public address system" is displayed as the action content corresponding to the action number "2", and a button with the notation "Execute public address" is displayed instead of the contact. In this case, if a smartphone is used as the customer operation terminal 3, the customer can call the displayed contact or execute the public address system by tapping "119" or "Execute public address system" displayed in the response action list 3033.

[0106] Monitoring screen 304 is a display example when an abnormality of "abnormal behavior" is detected in an elevator. The screen configuration of monitoring screen 304 (e.g., building information display field 3041, abnormality detection information display field 3042, response action list 3043) is the same as monitoring screen 303, and the acquisition of various information output to monitoring screen 304 can be considered to be similar to the explanation for monitoring screen 303, so a detailed explanation will be omitted.

[0107] Fig. 9 is a diagram showing another example of the monitoring screen when an abnormality occurs. The monitoring screen 305 shown in Fig. 9 is a display example of the monitoring screen 300 when an abnormality is detected in a plurality of pieces of equipment 23 that are the monitoring targets.

[0108] 9, an abnormality of "fire detection" in the firefighting equipment and an abnormality of "smoke detection" in the automatic door are detected. In this example, in the response action information 124 registered in the response action DB 12d, the abnormality content priority 1245 of the fire detection in the firefighting equipment is higher than the abnormality content priority 1245 of the smoke detection in the automatic door.

[0109] At this time, on the monitoring screen 305 of Fig. 9, a first abnormality detection information display field 3052 is arranged below a building information display field 3051, and a second abnormality detection information display field 3054 is arranged below the first abnormality detection information display field 3052. The first abnormality detection information display field 3052 displays information on an abnormality having the highest abnormality content priority among the abnormalities detected at the present time, and the second abnormality detection information display field 3054 displays information on an abnormality having the next highest abnormality content priority. That is, the first abnormality detection information display field 3052 displays the detection of a fire in a firefighting facility, and the second abnormality detection information display field 3054 displays the detection of a smoke in an automatic door. In addition, a normality information display field 3055 showing an image of the facility 23 (elevator, parking lot, rooftop) in which no abnormality is detected is arranged below the second abnormality detection information display field 3054.

[0110] In the monitoring screen 305 displayed as above, for example, when a response action display button corresponding to the first abnormality detection information display field 3052 is pressed, a list of response actions linked to the detection of fire in the firefighting equipment is displayed in a response action list 3053, and the customer can select and execute a response action from among the response actions of "call a fire engine" or "announce over the public address system." Similarly, when a response action display button corresponding to the second abnormality detection information display field 3054 is pressed, a list of response actions linked to the detection of smoke in the automatic door is displayed in the response action list, and the customer can select a response action to execute.

[0111] As shown in the monitoring screen 305, when multiple abnormalities are detected, the abnormality with the highest importance is displayed at the top (in a prominent location) based on the abnormality priority set in the response action information 124, allowing the customer to intuitively recognize the abnormality that should be dealt with as a priority and efficiently take action to respond to the abnormality.

[0112] As described above, the video monitoring system 100 (video monitoring device 1) according to this embodiment can determine an abnormal state by analyzing the video from the surveillance camera 22 associated with the monitored facility 23 without the need for an abnormality detection sensor, and the next action to be taken according to the determination result, that is, information on the response action, is displayed on the customer operation terminal 3 of the user (customer) together with the video in which the abnormality was detected, thereby supporting the user's response action to the detected abnormality while keeping implementation costs down. In addition, by providing the customer operation terminal 3 with not only the video from the surveillance camera 22 but also an intuitive and easy-to-understand display screen, the burden of management work on the building manager and the like can be reduced.

[0113] The present invention is not limited to the above-described embodiment and modified examples, and further includes various modified examples. For example, the above-described embodiment has been described in detail to easily explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. In addition, it is possible to add, delete, or replace part of the configuration of the embodiment with other configurations.

[0114] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0115] In addition, the control lines and information lines in the drawings are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0116] 1. Video monitoring equipment 2. Building 3 Customer operation terminal 4.5 Communication Network 11 Customer information input section 12 Storage device 12a Customer information DB 12b Camera image management DB 12c Abnormality Record DB 12d Response Action DB 13 Camera image acquisition section 14 Visualization processing section 15 Camera image analysis department 16 Response Action Selection Section 17 Communications Department 21 Camera video recording device 22 Surveillance Cameras 23 Equipment 100 Video Monitoring System 121 Customer information 124 Response Action Information 300 Monitoring screen

Claims

1. A video monitoring device that monitors abnormalities in a monitored facility using video from a surveillance camera that captures the facility, a camera image acquisition unit that acquires image data of the facility captured by the surveillance camera; A camera image analysis unit that detects an abnormality in the equipment from the image of the video data and identifies the content of the detected abnormality; a storage device that stores customer information that associates a customer who has concluded a maintenance contract for monitoring the facility, the facility that is the subject of monitoring under the maintenance contract, and information about the monitoring camera that captures the facility, and response action information that associates one or more response actions that can be selected by the customer when the abnormality is detected with each of the abnormality contents; a response action selection unit that acquires one or more of the response actions associated with the abnormality content identified by the camera image analysis unit based on the response action information stored in the storage device; a visualization processing unit that generates display data for displaying a monitoring screen displaying the image of the image data acquired by the camera image acquisition unit on a customer operation terminal operated by the customer, and displays the monitoring screen on the customer operation terminal using the display data; Equipped with When the camera image analysis unit detects an abnormality in the facility, the visualization processing unit generates display data for abnormality situations including a display of an image in which the abnormality is detected and a list of the one or more response actions acquired by the response action selection unit in response to the detection of the abnormality, and causes the customer operation terminal to display the monitoring screen in a manner that allows the customer to perform an operation of determining which response action to execute from the list of the one or more response actions using the display data for abnormality. A video monitoring device comprising:

2. In the response action information, a plurality of response actions can be registered by assigning different action priorities as the response action corresponding to one of the abnormality contents, When the plurality of response actions are associated in the response action information with the abnormality content identified from the abnormality detected by the camera image analysis unit, The response action selection unit acquires all of the plurality of corresponding response actions, The visualization processing unit generates display data for the abnormality so as to present the list of the plurality of response actions acquired by the response action selection unit on the monitoring screen in a display mode according to the action priority.

2. The video monitoring device according to claim 1.

3. When the camera image analysis unit detects the abnormality, the visualization processing unit executes a predetermined output process for notifying the customer operation terminal of the occurrence of the abnormality.

2. The video monitoring device according to claim 1.

4. When the customer operation terminal requests the start of the monitoring screen, the visualization processing unit permits the display of the monitoring screen on the customer operation terminal only when authentication using the customer information is satisfied.

2. The video monitoring device according to claim 1.

5. When there are a plurality of pieces of equipment that are to be monitored under the maintenance contract, the visualization processing unit generates the display data so as to display images of each of the plurality of pieces of equipment on the monitoring screen; When the camera image analysis unit detects an abnormality in any one of the plurality of pieces of equipment, The visualization processing unit generates the display data for the abnormality so that an image of a facility in which an abnormality is detected is displayed in a more emphasized display mode than an image of a facility in which an abnormality is not detected on the monitoring screen.

2. The video monitoring device according to claim 1.

6. When the camera image analysis unit detects an abnormality in two or more pieces of equipment among the plurality of pieces of equipment, the camera image analysis unit identifies the abnormality content for each of the detected abnormalities, the response action selection unit acquires, for each of the detected abnormalities, one or more response actions associated with respective abnormality contents based on the response action information; The visualization processing unit generates display data for the abnormality so that, for each of the detected abnormalities, a video in which the abnormality is detected and a list of the one or more response actions to the abnormality are displayed on the monitoring screen.

6. The video monitoring device according to claim 5.

7. An abnormality priority is assigned to each of the plurality of abnormality contents registered in the response action information, When the camera image analysis unit detects an abnormality in two or more pieces of equipment among the plurality of pieces of equipment and identifies different abnormality contents for each abnormality, the response action selection unit acquires, for each of the detected abnormalities, one or more response actions associated with respective abnormality contents based on the response action information; The visualization processing unit generates display data for the abnormality so that, for each of the detected abnormalities, a video in which the abnormality is detected and a list of the one or more response actions for the abnormality are displayed on the monitoring screen in a display order according to the abnormality priority level assigned to the abnormality content of each of the detected abnormalities.

6. The video monitoring device according to claim 5.

8. a response action processing unit that executes a specific response action when an operation for deciding to execute the specific response action is performed on the monitoring screen displayed on the customer operation terminal using the display data for abnormality.

2. The video monitoring device according to claim 1.

9. A video monitoring system for monitoring abnormalities in a monitored facility, comprising: A surveillance camera that photographs the facility; a camera image recording device that records image data of the facility captured by the surveillance camera; a customer operation terminal operated by a customer who has concluded a maintenance contract for monitoring the facility; a video monitoring device that detects an abnormality in the equipment using video data of the equipment, and presents, when the abnormality is detected, a list of next response actions that can be selected by the customer on the customer operation terminal; Equipped with The video monitoring device comprises: a camera image acquisition unit that acquires image data of the facility from the camera image recording device; A camera image analysis unit that detects an abnormality in the equipment from the image of the video data and identifies the content of the detected abnormality; a storage device that stores customer information in which a customer who has concluded the maintenance contract, the equipment that is a monitoring target of the maintenance contract, and information on the monitoring camera that captures the equipment are associated with each other, and response action information in which the response action to the abnormality is associated with one or more of the abnormality contents of the abnormality; a response action selection unit that acquires one or more of the response actions associated with the abnormality content identified by the camera image analysis unit based on the response action information stored in the storage device; a visualization processing unit that generates display data for displaying a monitoring screen displaying the image of the image data acquired by the camera image acquisition unit on a customer operation terminal operated by the customer, and displays the monitoring screen on the customer operation terminal using the display data; having When the camera image analysis unit detects an abnormality in the equipment, the visualization processing unit generates display data for an abnormality including a display of the video in which the abnormality is detected and a list of the one or more response actions acquired by the response action selection unit in response to the detection of the abnormality, and displays the monitoring screen on the customer operation terminal using the display data for an abnormality in a manner that allows the customer to perform an operation of deciding which response action to execute from the list of the one or more response actions; The customer operation terminal determines a response action to be performed from the list of response actions presented on the monitoring screen in response to an operation by the customer. A video monitoring system comprising:

10. A video monitoring method using a video monitoring device that monitors an abnormality in a monitored facility using video from a surveillance camera that captures the facility, comprising: The video monitoring device stores in a storage device customer information that associates information regarding a customer who has concluded a maintenance contract for monitoring the facility, the facility that is the subject of monitoring under the maintenance contract, and the monitoring camera that captures the facility, and response action information that associates one or more response actions that can be selected by the customer when an abnormality is detected in the facility with each type of the abnormality, a camera image acquisition step in which the image monitoring device acquires image data of the facility captured by the surveillance camera; A camera image analysis step in which the video monitoring device detects an abnormality in the equipment from the video of the video data and identifies the content of the detected abnormality; a response action selection step in which the video monitoring device acquires the one or more response actions associated with the abnormality content identified in the camera video analysis step based on the response action information stored in the storage device; a visualization processing step in which the video monitoring device generates display data for displaying a monitoring screen displaying the video of the video data acquired in the camera video acquisition step on a customer operation terminal operated by the customer, and displays the monitoring screen on the customer operation terminal using the display data; Equipped with When an abnormality of the facility is detected in the camera image analysis step, in the visualization processing step, the video monitoring device generates display data for abnormality situations including a display of the image in which the abnormality is detected and a list of the one or more response actions acquired in the response action selection step in response to the detection of the abnormality, and displays the monitoring screen on the customer operation terminal using the display data for abnormality in a manner that allows the customer to perform an operation of deciding which response action to execute from the list of the one or more response actions. A video monitoring method comprising:

Images