Information provision device, information provision method, and information provision program

Abstract

To effectively understand the status of the monitored object. [Solution] The monitoring server 10 acquires video data of the monitored object captured by the camera 30, and calculates an evaluation value that evaluates the state of the monitored object according to the ratio of the white value, which represents the value of white, and the black value, which represents the value of black, based on the acquired video data and the respective setting values ​​for white and black.

Description

[Technical Field]

[0001] This disclosure relates to an information provision device, an information provision method, and an information provision program. [Background technology]

[0002] Surveillance cameras are used for security purposes in building interiors and other areas, as well as for monitoring the progress and status of manufacturing processes in plants and checking for any abnormalities. Operators can monitor the footage captured by the surveillance cameras in real time, and can also review the footage later that has been stored on a storage device. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2004-336171 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, effectively understanding the status of the monitored object is difficult. For example, in real-time monitoring, operators need to monitor the video and check the status, which places a heavy burden on operators as they must avoid overlooking anything. Also, while reviewing video stored on a storage device eliminates the risk of operators overlooking something, it requires them to review the video later, which also increases the burden on operators.

[0005] This disclosure is made in view of the above and aims to effectively grasp the status of the monitored object. [Means for solving the problem]

[0006] An information providing device according to one embodiment of the present disclosure includes a processor, which acquires video data of a monitored object captured by a camera, and calculates an evaluation value for evaluating the state of the monitored object according to the ratio of the white value, which represents the numerical value of white, and the black value, which represents the numerical value of black, based on the acquired video data and setting values ​​corresponding to white and black, respectively.

[0007] An information provision method according to one embodiment of the present disclosure involves an information provision device that acquires video data of a monitored object captured by a camera, and performs a process to calculate an evaluation value that evaluates the state of the monitored object according to the ratio of the white value, which represents the numerical value of white, and the black value, which represents the numerical value of black, based on the acquired video data and setting values ​​corresponding to white and black, respectively.

[0008] An information provision program according to one embodiment of the present disclosure causes an information provision device to perform a process that acquires video data of a monitored object captured by a camera, and calculates an evaluation value for evaluating the state of the monitored object according to the ratio of the white value, which represents the numerical value of white, and the black value, which represents the numerical value of black, based on the acquired video data and the respective setting values ​​corresponding to white and black. [Effects of the Invention]

[0009] According to this disclosure, there is an effect of being able to effectively understand the status of the monitored object. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example configuration and processing example of a monitoring system according to the embodiment. [Figure 2] This figure shows an example of the processing for calculating evaluation values ​​in the monitoring system according to the embodiment. [Figure 3] This block diagram shows an example of the configuration of each device in the monitoring system according to the embodiment. [Figure 4] This figure shows an example of a configuration information storage unit for a monitoring server according to the embodiment. [Figure 5]This figure shows an example of a video data storage unit of a monitoring server according to the embodiment. [Figure 6] This figure shows an example of the evaluation value storage unit of the monitoring server according to the embodiment. [Figure 7] This is a diagram illustrating the hue values ​​in the HWB model used in the monitoring system according to the embodiment. [Figure 8] This figure shows a specific example 1 of the evaluation value calculation process of the monitoring system according to the embodiment. [Figure 9] This figure shows a specific example of a conversion function for the monitoring system according to the embodiment. [Figure 10] This figure shows a specific example 2 of the evaluation value calculation process of the monitoring system according to the embodiment. [Figure 11] This figure shows a specific example of the settings screen of the monitoring system according to the embodiment. [Figure 12] This figure shows a specific example 1 of the evaluation value notification screen of the monitoring system according to the embodiment. [Figure 13] This figure shows a specific example 2 of the evaluation value notification screen of the monitoring system according to the embodiment. [Figure 14] This figure shows a specific example 3 of the evaluation value notification screen of the monitoring system according to the embodiment. [Figure 15] This flowchart shows an example of the overall processing flow of the monitoring system according to the embodiment. [Figure 16] This flowchart shows an example of the flow of the configuration information management process for the monitoring system according to the embodiment. [Figure 17] This flowchart shows an example of the video data management process flow of the surveillance system according to the embodiment. [Figure 18] This flowchart shows an example of the flow of evaluation value management processing for the monitoring system according to the embodiment. [Figure 19] This flowchart shows an example of the alarm management process flow of the monitoring system according to the embodiment. [Figure 20] This figure shows an example of a hardware configuration according to the embodiment. [Modes for carrying out the invention]

[0011] An information provision device, information provision method, and information provision program according to embodiments of this disclosure will be described in detail with reference to the drawings. However, this disclosure is not limited to the embodiments described below.

[0012] The following describes the configuration and processing of the monitoring system 100 according to the embodiment, the configuration and processing of each device of the monitoring system 100, specific examples of each process of the monitoring system 100, the flow of each process of the monitoring system 100, the effects of the embodiment, and examples of applications of the embodiment.

[0013] [1. Configuration and processing of the monitoring system 100] Referring to Figure 1, the configuration and processing of the monitoring system 100 according to the embodiment will be described in detail. Figure 1 is a diagram showing an example of the configuration and processing of the monitoring system 100 according to the embodiment. Below, an example of the overall configuration of the monitoring system 100, an example of the overall processing of the monitoring system 100, the evaluation value calculation process of the monitoring system 100, and the effects of the monitoring system 100 will be described. In this embodiment, monitoring in the context of building security measures will be used as an example, but the monitoring target and application field are not limited.

[0014] (1-1. Example of the overall configuration of monitoring system 100) An example of the overall configuration of the monitoring system 100 will be described. The monitoring system 100 consists of a monitoring server 10, an operator terminal 20, and a camera 30. Here, the monitoring server 10, the operator terminal 20, and the camera 30 are connected via a communication network N (not shown) so that they can communicate by wired or wireless means. Various communication networks such as the internet or dedicated lines can be used for the communication network N.

[0015] (1-1-1. Monitoring Server 10) The monitoring server 10 is an information providing apparatus that calculates an evaluation value E for evaluating the state of the monitored object based on video data I acquired from the camera 30. For example, the monitoring server 10 can be implemented in a cloud environment, on-premise environment, edge environment, etc. Note that the monitoring system 100 shown in Figure 1 may include multiple monitoring servers 10.

[0016] (1-1-2. Operator terminal 20) Operator terminal 20 is a user terminal used by operator O, who is the administrator of the monitored system. Note that the monitoring system 100 shown in Figure 1 may include multiple operator terminals 20.

[0017] (1-1-3. Camera 30) Camera 30 is an imaging device that photographs the object being monitored. Note that the monitoring system 100 shown in Figure 1 may include multiple cameras 30.

[0018] (1-2. Example of the entire monitoring system 100 process) This section describes an example of the overall processing of the monitoring system 100. The following sections describe the configuration information reception process, video data acquisition process, evaluation value calculation process, and alarm notification process.

[0019] (1-2-1. Configuration Information Reception Processing) Firstly, the monitoring server 10 receives configuration information from the operator terminal 20 (see Figure 1(1)). For example, the monitoring server 10 receives and saves configuration information entered by operator O into the configuration screen of the operator terminal 20.

[0020] Here, the setting information refers to information related to the settings used when calculating the evaluation value E of the monitored object, and includes, for example, a specified range, region divisions, and conversion information. The specified range is the range set by operator O that includes each pixel in the video data I from which the evaluation value E is calculated. The region division is the division of the video data I into colored or achromatic regions that classifies the pigment (color information, as appropriate) of the pixels, set by operator O. The conversion information is a conversion table, conversion function, fixed value, etc., that shows each conversion value C corresponding to each pigment, set by operator O.

[0021] (1-2-2. Video data acquisition process) Secondly, the monitoring server 10 acquires video data I from the camera 30 (see Figure 1(2)). For example, the monitoring server 10 acquires video data I of still images taken every second from the camera 30 installed inside the building. Alternatively, the monitoring server 10 may acquire video data I of moving images from the camera 30 installed inside the building.

[0022] (1-2-3. Evaluation Value Calculation Process) Thirdly, the monitoring server 10 calculates an evaluation value E from the video data I (see Figure 1(3)). For example, the monitoring server 10 uses the configuration information set by the operator O (e.g., specified range, area division, conversion information) and the video data I acquired from the camera 30 to calculate an evaluation value E for evaluating the condition of the interior of the building.

[0023] In the example shown in Figure 1, the monitoring server 10 calculates evaluation values ​​E for each time interval within a specified range (see dashed rectangle in Figure 1) corresponding to the vicinity of a window from the video data I of the building's interior, specifically {Time: "1:00", Evaluation Value: "40"}, {Time: "1:30", Evaluation Value: "40"}, and {Time: "2:00", Evaluation Value: "70"}, and monitors for the presence of suspicious persons inside the building. Details of the evaluation value calculation process will be described later in (1-3. Evaluation Value Calculation Process of Monitoring System 100).

[0024] (1-2-4. Alarm notification processing) Fourth, the monitoring server 10 notifies the operator O of an alarm (see Figure 1(4)). For example, if the calculated evaluation value E is greater than or equal to the threshold X set by the operator O, the monitoring server 10 sends alarm data indicating an abnormal state to the operator terminal 20, and displays the alarm on the operator terminal 20's display. The monitoring server 10 can also send table data or graph data showing the evaluation value E to the operator terminal 20, and display a table or graph (e.g., line graph, histogram) of the evaluation value E on the operator terminal 20's display.

[0025] (1-3. Processing for calculating evaluation values ​​of monitoring system 100) Referring to Figure 2, the evaluation value calculation process of the monitoring system 100 according to the embodiment will be described. Figure 2 is a diagram showing an example of the evaluation value calculation process of the monitoring system 100 according to the embodiment. The following describes the specified range identification process, HWB value conversion process, area classification process, converted value output process, and converted value aggregation process.

[0026] (1-3-1. Processing to identify the specified range) Firstly, the monitoring server 10 identifies a specified range of video data I (see Figure 2(1)). In the example in Figure 2(1), the monitoring server 10 identifies a specified range of video data I of the building's interior that corresponds to the area near the window, which has been set by operator O via the settings screen of operator terminal 20.

[0027] (1-3-2. HWB value conversion process) Secondly, the monitoring server 10 converts the color of each pixel included in the specified range of the video data I into an HWB value (see Figure 2(2)). In the example in Figure 2(2), the monitoring server 10 converts the color of each pixel (e.g., light blue, red, white, black) included in the specified range of the video data I of the building's interior, which corresponds to the area near the window, into an HWB value, which is a combination of a hue value (H) which is a numerical value (0 to 360°) indicating the difference in color, a white value (W) which is a numerical value (0 to 1) indicating white, and a black value (B) which is a numerical value (0 to 1) indicating black.

[0028] (1-3-3. Region Classification and Classification Processing) Thirdly, the monitoring server 10 classifies the pigments of each pixel included in the specified range of the video data I into regions (see Fig. 2(3)). In the example of Fig. 2(3), the monitoring server 10 classifies cyan (wide diamond mesh) and red (narrow diamond mesh) pigments into colored regions with hues, and classifies white (plain) and black (black background) pigments into achromatic regions without hues. At this time, the monitoring server 10 calculates the total value T of the white value W and the black value B indicated by each pixel B , A , W and classifies it into a colored region if it is less than the boundary value TH. On the other hand, the monitoring server 10 calculates the total value T of the white value W and the black value B indicated by each pixel A and classifies it into an achromatic region if it is greater than or equal to the boundary value TH.

[0029] (1-3-4. Conversion Value Output Processing) [[ID=�]](1-3-4. Conversion Value Output Processing) Fourthly, the monitoring server 10 outputs the conversion value C of each pixel using the conversion information set according to the region classification (see Fig. 2(4a) and (4b)). In the example of Fig. 2(4a), for cyan and red classified into the colored region, the monitoring server 10 shows the set value V corresponding to the hue value H of 0 to 360° as the conversion information H and uses the conversion function, which is an equation between the hue value H and the set value V H to output the conversion value C in the colored region C . Also, in the example of Fig. 2(4b), for white and black classified into the achromatic region, the monitoring server 10 uses the set value V when the white value W is 1 W and the set value V when the black value B is 1 B to output the conversion value C in the achromatic region weighted according to the ratio of white and black A . At this time, the monitoring server 10 can output an appropriate conversion value C according to the ratio of white and black even for the gray pigment, which is an intermediate color between white and black, by weighting according to the ratio of white and black A .

[0030] As described above, the monitoring server 10 repeatedly performs HWB value conversion processing, region classification processing, and converted value output processing for all pixels included in the specified range of the video data I.

[0031] (1-3-5. Processing of converted values) Fifth, the monitoring server 10 aggregates the conversion values ​​C of all pixels and calculates the evaluation value E (see Figure 2(5)). In the example in Figure 2(5), the monitoring server 10 aggregates the conversion values ​​"10" for light blue (wide diamond mesh), "20" for red (narrow diamond mesh), "80" for white (plain), and "30" for black (black background), and calculates the evaluation value E as the sum of all pixels T V The value "140" is calculated. Also, in the example in Figure 2(5), the monitoring server 10 aggregates the conversion values ​​of light blue (wide diamond mesh) "10", red (narrow diamond mesh) "20", white (plain) "80", and black (black background) "30", and uses the average value A of all pixels as the evaluation value E. V The result is "35".

[0032] (1-3-6. Others) The monitoring server 10 can also further convert the calculated evaluation value into a value desired by operator O. For example, the monitoring server 10 can convert the average value A of all pixels calculated as the evaluation value E into a value corresponding to the temperature near a window inside a building. V By multiplying this by a predetermined coefficient, it can also be converted to the temperature near the window.

[0033] (1-4. Effects of the monitoring system 100) The effects of monitoring system 100 will be explained. Below, the problems of monitoring system 100-P related to the reference technology will be explained, followed by an overview and effects of monitoring system 100.

[0034] (1-4-1. Problems with the 100-P monitoring system) In the monitoring system 100-P related to the reference technology, cameras 30, such as surveillance cameras, are used for purposes such as building security measures inside buildings, monitoring the progress and status of manufacturing processes in plants, and confirming whether or not abnormalities occur. In this case, the operator O can monitor the video captured by the cameras 30 in real time, and can also review the video later that has been stored in a storage device. Furthermore, the monitoring system 100-P can also utilize AI (Artificial Intelligence) to determine whether the state is normal or abnormal.

[0035] However, the monitoring system 100-P has the following problems. Firstly, in the case of real-time monitoring with the monitoring system 100-P, operator O needs to monitor the video and check the status, so if it is required that no oversights be made, the burden on operator O will be great. Secondly, when checking video stored in a memory device, the risk of operator O overlooking something is eliminated, but it becomes necessary to review the video later, which increases the burden on operator O. Thirdly, when using AI, it is necessary to train the AI ​​on the normal and abnormal states of the relevant monitored object, which increases the burden on operator O.

[0036] (1-4-2. Overview of Monitoring System 100) The monitoring system 100 performs the following processes. First, the monitoring server 10 receives setting information from operator O, including a specified range, area division, and conversion information. Second, the monitoring server 10 acquires still images and video data I from the camera 30 installed on the monitored object. Third, the monitoring server 10 uses the setting information set by operator O and the video data I acquired from the camera 30 to calculate an evaluation value E for evaluating the state of the monitored object. At this time, the monitoring server 10 classifies the pigment of each pixel into a colored area and an achromatic area, and for the colored area, it assigns a conversion value C corresponding to the hue value H. C The following is calculated: For achromatic colors, a conversion value C is calculated according to the ratio of the white value W and the black value B. AThe value is calculated. Fourth, if the calculated evaluation value E is greater than or equal to the threshold X set by operator O, the monitoring server 10 notifies operator O of an alarm indicating an abnormal condition.

[0037] By performing the above process, the monitoring system 100 can provide a mechanism for converting the color in the monitored video into a numerical value desired by operator O. Furthermore, the monitoring system 100 can label the state of the monitored object not for the entire video, but for a specified range set by operator O (dividing the area to be monitored into smaller sections, assigning identifiable names to them, and managing them).

[0038] (1-4-3. Effects of the monitoring system 100) The monitoring system 100 has the following advantages: Firstly, with the monitoring system 100, operator O can grasp the changes in color and size (spread) within a specified range set by operator O in a time series by displaying the trend of numerical data. Secondly, with the monitoring system 100, operator O can grasp the state of the monitored object by the magnitude of the numerical values ​​without having to check the image from camera 30. Thirdly, with the monitoring system 100, operator O can appropriately grasp the state of the monitored object even if the image from camera 30 is a black and white image.

[0039] As described above, the monitoring system 100 can effectively grasp the status of the monitored object.

[0040] [2. Configuration and Processing of Each Device in the Monitoring System 100] Referring to Figure 3, the configuration and processing of each device in the monitoring system 100 shown in Figure 1 will be described. Figure 3 is a block diagram showing an example of the configuration of each device in the monitoring system 100 according to the embodiment. Below, an example of the overall configuration of the monitoring system 100 according to the embodiment will be described, followed by a detailed description of the configuration and processing examples of the monitoring server 10, the operator terminal 20, and the camera 30.

[0041] (2-1. Example of the overall configuration of monitoring system 100) Referring to Figure 3, an example of the overall configuration of the monitoring system 100 shown in Figure 1 will be described. As shown in Figure 3, the monitoring system 100 consists of a monitoring server 10, an operator terminal 20, and a camera 30. The monitoring server 10, the operator terminal 20, and the camera 30 are connected to each other via a communication network N, which is implemented via the internet or a dedicated line.

[0042] The monitoring server 10 is installed in a cloud environment, on-premises environment, edge environment, etc. The operator terminal 20 is installed in a monitoring room of a facility or equipment managed by operator O. The camera 30 is installed at the monitoring target site, which is the monitoring site of the facility or equipment.

[0043] (2-2. Example configuration and processing of monitoring server 10) Referring to Figure 3, an example configuration and processing example of the monitoring server 10 will be described. The monitoring server 10 is an information providing device and includes an input unit 11, an output unit 12, a communication unit 13, a storage unit 14, and a control unit 15.

[0044] (2-2-1. Input section 11) The input unit 11 is responsible for inputting various types of information to the monitoring server 10. For example, the input unit 11 can be implemented using a mouse or keyboard, and it accepts various types of information input to the monitoring server 10.

[0045] (2-2-2. Output section 12) The output unit 12 is responsible for outputting various types of information from the monitoring server 10. For example, the output unit 12 is implemented as a display or the like and displays various types of information stored on the monitoring server 10.

[0046] (2-2-3. Communications Section 13) The communication unit 13 is responsible for data communication with other devices. For example, the communication unit 13 performs data communication with each communication device via a router or the like. The communication unit 13 can also perform data communication with terminals (not shown).

[0047] (2-2-4. Storage section 14) The storage unit 14 stores various information that the control unit 15 references when it operates, and various information acquired when the control unit 15 operates. The storage unit 14 is composed of a setting information storage unit 14a, a video data storage unit 14b, and an evaluation value storage unit 14c. Here, the storage unit 14 can be implemented as, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disc. In the example in Figure 3, the storage unit 14 is installed inside the monitoring server 10, but it may be installed outside the monitoring server 10, or multiple storage areas may be installed.

[0048] (2-2-4-1. Setting information storage unit 14a) The configuration information storage unit 14a stores configuration information. For example, the configuration information storage unit 14a stores configuration information received by the reception unit 15a of the control unit 15, which will be described later. Here, an example of the data stored by the configuration information storage unit 14a will be explained with reference to Figure 4. Figure 4 is a diagram showing an example of the configuration information storage unit 14a of the monitoring server 10 according to the embodiment. In the example in Figure 4, the configuration information storage unit 14a has items such as "monitoring target", "specified range", "area division", and "conversion information".

[0049] "Monitored object" refers to identification information used to identify facilities, equipment, or areas whose status is being evaluated. For example, "monitored object" could be the identification number or code for a room in a building. Alternatively, "monitored object" could be the identification number or code for a manufacturing process in a plant.

[0050] The "specified range" refers to the area of ​​the video data I that includes each pixel from which the evaluation value E is calculated, as set by the user. For example, the "specified range" is an area enclosed by a rectangle, circle, sector, polygon, or any other shape specified by the operator O on the settings screen.

[0051] "Region classification" refers to the classification of pixel pigments within the video data I, as set by the user. For example, "Region classification" is a division between a colored region corresponding to colored pigments and an achromatic region corresponding to colorless pigments, defined by the boundary value TH entered by the operator O on the settings screen. Furthermore, "Region classification" is the boundary value TH that indicates the boundary between the colored region and the achromatic region.

[0052] "Conversion information" indicates the setting value V corresponding to the pigment, as set by the user. For example, "Conversion information" is the setting value V corresponding to each hue value H in the color range, as entered by operator O on the settings screen. H This shows the transformed value C in the colored region. C This is a conversion table or conversion function for calculating [the value]. Furthermore, "conversion information" is the setting value V corresponding to the maximum value of the white value W in the achromatic region, which is entered by operator O on the settings screen. W This is a fixed value that indicates [something]. Furthermore, "Conversion Information" is a setting value V corresponding to the maximum value of the black value B in the achromatic region, which is entered by operator O on the settings screen. B This is a fixed value that indicates [something]. Furthermore, "conversion information" is the conversion value C in the achromatic region from the white value W and black value B. A These are conversion tables and conversion functions for calculating [the value].

[0053] In other words, Figure 4 shows an example in which data for "Monitoring Target #1" is stored in the setting information storage unit 14a, with the following characteristics: {Specified range: "Specified range #1", Area classification: "Area classification #1", Conversion information: "Conversion information #1"}, {Specified range: "Specified range #2", Area classification: "Area classification #2", Conversion information: "Conversion information #2"}, {Specified range: "Specified range #3", Area classification: "Area classification #3", Conversion information: "Conversion information #3"}, ...

[0054] (2-2-4-2. Video data storage unit 14b) The video data storage unit 14b stores video data I. For example, the video data storage unit 14b stores video data I acquired by the acquisition unit 15b of the control unit 15, which will be described later. Now, with reference to Figure 5, an example of the data stored by the video data storage unit 14b will be explained. Figure 5 is a diagram showing an example of the video data storage unit 14b of the monitoring server 10 according to the embodiment. In the example in Figure 5, the video data storage unit 14b has items such as "monitoring target", "monitoring equipment", "time", and "video data". Note that the item "monitoring target" is the same as that of the setting information storage unit 14a, so its explanation will be omitted.

[0055] "Monitoring equipment" refers to identification information used to identify a camera. For example, "monitoring equipment" could be the identification number or code of camera 30, which is a surveillance camera inside a building. Alternatively, "monitoring equipment" could be the identification number or code of camera 30, which is a surveillance camera for the manufacturing process in a plant.

[0056] "Time" indicates the time of capture by the camera. For example, "Time" is the time of capture by camera 30, which is a surveillance camera inside a building, and is expressed in year, month, day, hour, minute, and second. Also, "Time" is the time of capture by camera 30, which is a surveillance camera for the manufacturing process in a plant, and is expressed in year, month, day, hour, minute, and second.

[0057] "Video data" refers to video data I at the time of shooting. For example, "video data" may include still image video data, video video data, or video video data including audio data, acquired every second.

[0058] In other words, Figure 5 shows an example in which data such as {time: "time #1", video data: "video data #1"}, {time: "time #2", video data: "video data #2"}, {time: "time #3", video data: "video data #3"}, ... is stored in the video data storage unit 14b for camera 30, which is a surveillance device identified by "surveillance target #1" and "camera #1".

[0059] (2-2-4-3. Evaluation value storage unit 14c) The evaluation value storage unit 14c stores the evaluation value E. For example, the evaluation value storage unit 14c stores the evaluation value E calculated by the calculation unit 15c of the control unit 15, which will be described later. Here, an example of the data stored by the evaluation value storage unit 14c will be explained with reference to Figure 6. Figure 6 is a diagram showing an example of the evaluation value storage unit 14c of the monitoring server 10 according to the embodiment. In the example in Figure 6, the evaluation value storage unit 14c has items such as "monitoring target", "specified range", "time", and "evaluation value". Note that the items "monitoring target" and "specified range" are the same as those of the setting information storage unit 14a, so their explanation will be omitted. Also, the item "time" is the same as that of the video data storage unit 14b, so its explanation will be omitted.

[0060] The "evaluation value" is a numerical value that assesses the state of the monitored object. For example, the "evaluation value" could be a numerical value corresponding to the temperature near a window inside a building, or a numerical value indicating hazard or safety. Alternatively, the "evaluation value" could be a numerical value corresponding to the temperature of the burner flame in a plant's manufacturing process, or a numerical value indicating hazard or safety.

[0061] In other words, Figure 6 shows an example in which data such as {Time: "Time #1", Evaluation Value: "Evaluation Value #1"}, {Time: "Time #2", Evaluation Value: "Evaluation Value #2"}, {Time: "Time #3", Evaluation Value: "Evaluation Value #3"}, ... is stored in the evaluation value storage unit 14c for evaluation value E identified by "Monitoring Target #1" and "Specified Range #1".

[0062] (2-2-5. Control Unit 15) The control unit 15 is responsible for controlling the entire monitoring server 10. The control unit 15 includes a reception unit 15a, an acquisition unit 15b, a calculation unit 15c, and a notification unit 15d. Here, the control unit 15 can be implemented by electronic circuits such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or integrated circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0063] (2-2-5-1. Reception area 15a) The reception unit 15a receives various types of information. The reception unit 15a then stores the received information in the storage unit 14. The setting information reception process (specified range reception process, area division reception process, and conversion information reception process) will be described below.

[0064] (Configuration information reception processing) The reception unit 15a performs configuration information reception processing. For example, the reception unit 15a receives the specified range, area division, and conversion information set on the configuration screen of operator O's operator terminal 20 as configuration information.

[0065] A specific example of the configuration information reception process will be explained. The reception unit 15a receives configuration information set by operator O, who manages "Monitoring Target #1", via operator terminal 20, including {Specified range: "Specified range #1", Area classification: "Area classification #1", Conversion information: "Conversion information #1"}, {Specified range: "Specified range #2", Area classification: "Area classification #2", Conversion information: "Conversion information #2"}, {Specified range: "Specified range #3", Area classification: "Area classification #3", Conversion information: "Conversion information #3"}, ..., and stores it in the configuration information storage unit 14a.

[0066] (Processing of requests within the specified range) The reception unit 15a performs a specified range reception process as a setting information reception process. For example, the reception unit 15a receives a specified range of video data I. The reception unit 15a also receives a specified range of video data I specified by operator O on the video displayed on the setting screen of operator O's operator terminal 20.

[0067] (Area classification acceptance processing) The reception unit 15a performs area division reception processing as a setting information reception process. For example, the reception unit 15a receives area divisions indicating colored areas and achromatic areas. At this time, the reception unit 15a receives boundary values ​​TH indicating the boundaries of the area divisions entered by operator O via the setting screen displayed on operator terminal 20 of operator O.

[0068] (Conversion information reception processing) The reception unit 15a performs a conversion information reception process as a setting information reception process. For example, the reception unit 15a receives conversion information (conversion table, conversion function, fixed value) indicating the setting value V corresponding to the pigment. It also receives the setting values ​​corresponding to white, black, and each hue value H entered by operator O via the setting screen displayed on operator terminal 20 of operator O. At this time, the reception unit 15a receives the setting value V corresponding to each hue value H in the HWB model. H It accepts conversion information including the following. In addition, the receiving unit 15a accepts a set value V corresponding to the maximum value 1 of the white value W in the HWB model. W It accepts conversion information including the following. In addition, the receiving unit 15a accepts a set value V corresponding to the maximum value of black value B, which is 1, in the HWB model. B It accepts conversion information including [specific elements]. Below, we will describe conversion tables, conversion functions, and fixed values ​​as examples of conversion information.

[0069] (Conversion Table) The reception unit 15a receives the hue value H and the setting value V as conversion information. H It accepts a conversion table that shows the relationship between the two values. For example, the receiving unit 15a accepts each setting value V corresponding to each hue value H from 0 to 360°. H We accept conversion tables that show the following.

[0070] (Conversion function) The reception unit 15a receives the hue value H and the setting value V as conversion information. H It accepts a conversion function that shows the relationship between the two values. For example, the receiving unit 15a accepts a conversion function that shows a linear equation between predetermined hue values ​​H. The receiving unit 15a also accepts a conversion function that shows a curved equation between predetermined hue values ​​H.

[0071] (Fixed value) The reception unit 15a receives fixed values ​​for white and black, which are classified as achromatic regions, as conversion information. For example, the reception unit 15a receives a set value V corresponding to the maximum value of the white value W in the achromatic region. WIt accepts a fixed value that indicates the value. In addition, the reception unit 15a accepts a set value V corresponding to the maximum value of the black value B in the achromatic region. B It accepts a fixed value that indicates [something].

[0072] (2-2-5-2. Acquisition part 15b) The acquisition unit 15b acquires various types of information. The acquisition unit 15b then stores the acquired information in the storage unit 14. The video data acquisition process will be described below.

[0073] (Video data acquisition process) The acquisition unit 15b performs video data acquisition processing. For example, the acquisition unit 15b acquires video data I of the monitored object captured by the camera 30, which is a shooting device. At this time, the acquisition unit 15b acquires still image data or video data output by the camera 30. The acquisition unit 15b also acquires still image data or video data output by a camera 30 installed inside a building. Furthermore, the acquisition unit 15b acquires still image data or video data output by a camera 30 installed in the manufacturing process of a plant.

[0074] The acquisition unit 15b acquires video data I containing only white and black, output by the camera 30. The acquisition unit 15b also acquires monochrome image data output by the camera 30 installed inside a building. The acquisition unit 15b also acquires monochrome image data output by the camera 30 installed in the manufacturing process of a plant.

[0075] A specific example of the video data acquisition process will be explained. The acquisition unit 15b acquires video data such as {time: "time #1", video data: "video data #1"}, {time: "time #2", video data: "video data #2"}, {time: "time #3", video data: "video data #3"}, ... as video data generated by camera 30, which is a monitoring device identified by "camera #1" installed on "monitoring target #1", and transmitted to the monitoring server 10, and stores it in the video data storage unit 14b.

[0076] (2-2-5-3. Calculation part 15c) The calculation unit 15c calculates various types of information. The calculation unit 15c refers to the various types of information stored in the storage unit 14. The calculation unit 15c also stores the calculation results in the storage unit 14. The following describes the domain classification process and the evaluation value calculation process.

[0077] (Area classification processing) The calculation unit 15c performs region classification processing. For example, the calculation unit 15c classifies the pigment of each pixel in the video data I into either a colored region or a gray region.

[0078] Firstly, the calculation unit 15c converts the color information of each pixel into an HWB model and calculates the hue value H, white value W, and black value B. Secondly, the calculation unit 15c calculates the sum of the white value W and black value B in the HWB model of each pixel T. A The calculation unit 15c calculates the following: Thirdly, the calculation unit 15c refers to the boundary value TH that indicates the boundary of the area division stored in the setting information storage unit 14a. Fourthly, the calculation unit 15c calculates the total value T A If the value is less than the boundary value TH, the pigment of each pixel is classified into the colored region. Fifth, the calculation unit 15c calculates the total value T A If the value is greater than or equal to the boundary value TH, the pigment of each pixel is classified as achromatic.

[0079] The calculation unit 15c changes the boundary value TH, which indicates the boundary of the region division, according to the change in brightness of the video data I. The calculation unit 15c also uses the changed boundary value TH to classify the pigment of each pixel in the video data I into either a colored region or a gray region.

[0080] Firstly, the calculation unit 15c calculates the sum of the white value W and black value B of each pixel in the entire video data I, T AFirst, the calculation unit 15c calculates the time-dependent average values ​​μ{μ(1), μ(2), ..., μ(t-1), μ(t)} as the respective brightness values. Second, the calculation unit 15c calculates the difference Δμ{Δμ(2), ..., Δμ(t-1), Δμ(t)} of the average values ​​μ as the change in each brightness value. Third, the calculation unit 15c updates the boundary value TH at each time interval by adding a term obtained by multiplying the previous boundary value TH(t-1) by the coefficient α and a term obtained by multiplying the difference Δμ(t) representing brightness by the coefficient β.

[0081] Details of the area classification process performed by the calculation unit 15c will be described later in [3. Specific Examples of Each Process of the Monitoring System 100] (3-2. Specific Examples of Area Classification Process).

[0082] (Evaluation value calculation process) The calculation unit 15c performs the evaluation value calculation process. For example, the calculation unit 15c calculates an evaluation value E to evaluate the state of the monitored object based on the acquired video data I and the received conversion information (conversion table, conversion function, fixed value).

[0083] The calculation unit 15c calculates the acquired video data I and the set values ​​V corresponding to each hue value H of each pixel. H Based on this, an evaluation value E corresponding to the hue value H is calculated. For example, if the pigment of each pixel of the video data I included in the specified range is classified as a colored area, the calculation unit 15c calculates each set value V corresponding to each hue value H in the HWB model of each pixel. H The evaluation value E is calculated based on the hue value H.

[0084] The calculation unit 15c calculates each set value V corresponding to each hue value H of each pixel in the video data I converted by the conversion function. H The evaluation value E is calculated using the following: For example, the calculation unit 15c calculates each set value V corresponding to each hue value H of each pixel in the video data I converted by a conversion function that shows linear or curved equations. H Use this to calculate the evaluation value E.

[0085] The calculation unit 15c calculates an evaluation value E based on the acquired video data I and the respective setting values ​​V corresponding to white and black, according to the ratio of the white value W, which indicates the value of white, and the black value B, which indicates the value of black. For example, if the pigment of each pixel in the video data I is classified as achromatic, the calculation unit 15c calculates the setting value V corresponding to white. W The value obtained by multiplying the white value W by the setting value V corresponding to black. B The evaluation value E is calculated by adding the value obtained by multiplying this value by the black value B.

[0086] The calculation unit 15c calculates the evaluation value E as the sum of the setting values ​​V corresponding to the dye of each pixel in the video data I, T. V The calculation unit 15c calculates the set value V corresponding to each hue value H of each pixel in the video data I. H The transformed value C in the coloring region C The conversion value C in the achromatic region is calculated based on the ratio of the white value W and black value B of each pixel in the video data I. A Calculated as, and the converted value C C and converted value C A By adding them together, the total value T V The evaluation value E is calculated by the calculation unit 15c. Furthermore, the calculated total value T is used as the evaluation value E. V By multiplying this value by a predetermined coefficient set by operator O, it can also be converted into a value desired by operator O (e.g., temperature near a window inside a building, temperature of a burner flame).

[0087] The calculation unit 15c calculates the evaluation value E as the average value A of each set value V corresponding to the dye of each pixel in the video data I. V The calculation unit 15c calculates the set value V corresponding to each hue value H of each pixel in the video data I. H The transformed value C in the coloring region C The conversion value C in the achromatic region is calculated based on the ratio of the white value W and black value B of each pixel in the video data I. A Calculated as, and the converted value C C and converted value C A The mean value A is obtained by taking the arithmetic mean mean VThe evaluation value E is calculated by the calculation unit 15c. Furthermore, the calculated average value A is used as the evaluation value E. V By multiplying this value by a predetermined coefficient set by operator O, it can be converted into a value desired by operator O (e.g., the temperature near a window inside a building, the temperature of a burner flame).

[0088] A specific example of the evaluation value calculation process will be explained below. Firstly, the calculation unit 15c refers to {specified range: "specified range #1", area division: "area division #1", conversion information: "conversion information #1"} stored by the setting information storage unit 14a. Secondly, the calculation unit 15c refers to {time: "time #1", video data: "video data #1"}, {time: "time #2", video data: "video data #2"}, and {time: "time #3", video data: "video data #3"} stored by the video data storage unit 14b. Thirdly, the calculation unit 15c calculates {time: "time #1", evaluation value: "evaluation value #1"}, {time: "time #2", evaluation value: "evaluation value #2"}, and {time: "time #3", evaluation value: "evaluation value #3"} as evaluation values ​​and stores them in the evaluation value storage unit 14c. Details of the evaluation value calculation process performed by the calculation unit 15c will be described later in [3. Specific Examples of Each Process of the Monitoring System 100] (3-3. Specific Example 1 of the Evaluation Value Calculation Process) and (3-5. Specific Example 2 of the Evaluation Value Calculation Process).

[0089] (2-2-5-4. Notification section 15d) The notification unit 15d notifies various types of information. The notification unit 15d may also obtain various types of information from the storage unit 14. The alarm notification process and the evaluation value notification process will be described below.

[0090] (Alarm notification processing) The notification unit 15d performs alarm notification processing. For example, if the calculated evaluation value is greater than or equal to threshold X, the notification unit 15d notifies operator O of an alarm indicating an abnormality in the monitored system.

[0091] A specific example of alarm notification processing will be explained. If the evaluation value "evaluation value #3" calculated by the calculation unit 15c is greater than or equal to threshold X, the notification unit 15d sends "abnormal alarm #1" as alarm data indicating an abnormal state of "monitored target #1" to the operator terminal 20, and notifies the operator O by displaying the alarm indicated by "abnormal alarm #1" on the display of the operator terminal 20.

[0092] (Evaluation value notification process) The notification unit 15d performs evaluation value notification processing. For example, the notification unit 15d notifies operator O of a table or graph showing the calculated evaluation value E. In this case, the notification unit 15d notifies operator O of a table showing the calculated evaluation value E, which is a table showing the evaluation value E for each shooting time. The notification unit 15d also notifies operator O of a graph showing the calculated evaluation value E, such as a line graph showing the evaluation value E for each shooting time, or a histogram showing the distribution of the converted value C and the evaluation value E.

[0093] A specific example of the evaluation value notification process will be described. Firstly, the notification unit 15d refers to {time: "time #1", evaluation value: "evaluation value #1"}, {time: "time #2", evaluation value: "evaluation value #2"}, and {time: "time #3", evaluation value: "evaluation value #3"} stored by the evaluation value storage unit 14c. Secondly, the notification unit 15d generates table data "table #1~3" showing the evaluation value E for each shooting time, line graph data "line graph #1~3" showing the evaluation value E for each shooting time, and histogram data "histogram #1~3" showing the distribution of the converted value C and evaluation value E. Thirdly, the notification unit 15d transmits "Tables #1-3", "Line Graphs #1-3", and "Histograms #1-3" to the operator terminal 20, and notifies the operator O by displaying the tables shown in "Tables #1-3", the line graphs shown in "Line Graphs #1-3", and the histograms shown in "Histograms #1-3" on the display of the operator terminal 20.

[0094] (2-3. Example configuration and processing of operator terminal 20) Referring to Figure 3, an example of the configuration and processing of the operator terminal 20 will be described. The operator terminal 20 consists of an input / output unit 21, a transmitting / receiving unit 22, and a communication unit 23.

[0095] (2-3-1. Input / output section 21) The input / output unit 21 is responsible for inputting various types of information to the operator terminal 20. For example, the input / output unit 21 can be implemented as a mouse, keyboard, touch panel, etc., and accepts various types of information input to the operator terminal 20. The input / output unit 21 is also responsible for displaying various types of information from the operator terminal 20. For example, the input / output unit 21 can be implemented as a display, etc., and displays various types of information stored in the operator terminal 20.

[0096] Furthermore, the input / output unit 21 displays a settings screen that allows the operator O to input setting information. The input / output unit 21 also displays alarms indicated by alarm data transmitted from the monitoring server 10, which is an information providing device. The input / output unit 21 also displays tables indicated by table data or graphs indicated by graph data transmitted from the monitoring server 10.

[0097] (2-3-2. Transceiver Unit 22) The transmitting / receiving unit 22 transmits various types of information. For example, the transmitting / receiving unit 22 transmits configuration information entered by operator O via the configuration screen to the monitoring server 10.

[0098] The transmitting / receiving unit 22 receives various types of information. For example, the transmitting / receiving unit 22 receives alarm data transmitted from the monitoring server 10. The transmitting / receiving unit 22 also receives table data or graph data transmitted from the monitoring server 10.

[0099] (2-3-3. Communications Section 23) The communication unit 23 is responsible for data communication with other devices. For example, the communication unit 23 performs data communication with each communication device via a router or the like. The communication unit 23 can also perform data communication with terminals (not shown).

[0100] (2-4. Example configuration and processing of camera 30) Referring to Figure 3, an example of the configuration and processing of the camera 30 will be described. For example, the camera 30 is realized by a camera 30, which is a photographic device such as a surveillance camera or security camera installed in a facility managed by operator O, and is composed of a generation unit 31 and a communication unit 32.

[0101] (2-4-1. Generation unit 31) The generation unit 31 generates video data I. For example, the generation unit 31 generates video data I, such as still images or videos, by taking pictures of the interior of a facility, etc., every second.

[0102] The generation unit 31 transmits the generated video data I to the monitoring server 10. For example, the generation unit 31 transmits the generated still image or video data I to the monitoring server 10.

[0103] (2-4-2. Communications Section 32) The communication unit 32 is responsible for data communication with other devices. For example, the communication unit 32 performs data communication with each communication device via a router or the like. The communication unit 32 can also perform data communication with terminals (not shown).

[0104] [3. Specific examples of each process of the monitoring system 100] Referring to Figures 7 to 14, specific examples of each process of the monitoring system 100 according to the embodiment will be described. Below, the basic principle of the HWB model, one of the color models used in the monitoring system 100, will be explained, and then specific examples of each process of the monitoring system 100 will be described, including a specific example of area classification processing, specific example 1 of evaluation value calculation processing, a specific example of a conversion function, specific example 2 of evaluation value calculation processing, a specific example of a settings screen, and specific examples 1 to 3 of evaluation value notification screens.

[0105] (3-1. Basic Principles of the HWB Model) This section explains the basic principles of the HWB model used in the monitoring system 100. The HWB model is a color model that represents color by combining a hue value H, which is a numerical value (0 to 360°) indicating the difference in color, a white value W, which is a numerical value (0 to 1) indicating white, and a black value B, which is a numerical value (0 to 1) indicating black. Below, we will explain the characteristics of the HWB model, and then describe the hue value H, white value W, and black value B as elements of the HWB model used in the monitoring system 100.

[0106] (3-1-1. Characteristics of the HWB model) This section explains the characteristics of the HWB model. The HWB model is a common color model used in web pages and other similar applications. Furthermore, the HWB model is adopted as the color model for CSS (Cascading Style Sheets) by the W3C (World Wide Web Consortium), a web standardization organization, and is widely used in web design. Additionally, the HWB model can be calculated using a simple formula, resulting in low computational load and enabling high-speed calculations.

[0107] (3-1-2. Hue Value H) Referring to Figure 7, the hue value H will be explained as an element of the HWB model used in the monitoring system 100. Figure 7 is a diagram illustrating the hue value H in the HWB model used in the monitoring system 100 according to this embodiment.

[0108] As shown in Figure 7(1), the hue value H is represented on a color wheel where "yellow" corresponds to "H60", "green" to "H120", "cyan" to "H180", "blue" to "H240", "magenta" to "H300", and "red" to "H360" ("H0"). Also, as shown in Figure 7(2), the hue value H is represented on a hue scale showing a scale from 0 to 360° and the corresponding pigments.

[0109] (3-1-3. White value W) The white value W is described as an element of the HWB model used in the monitoring system 100. Below, the calculation process of the white value W performed by the monitoring server 10 is described.

[0110] Firstly, the monitoring server 10 converts the pigment of each pixel into a combination of red (R), green (G), and blue (B) in the RGB model (RGB values). At this time, the monitoring server 10 normalizes each (R, G, B) value by dividing it by the maximum value of 255.

[0111] Secondly, the monitoring server 10 selects the minimum value from the normalized (R, G, B) values ​​and sets it as MIN(R, G, B).

[0112] Thirdly, the monitoring server 10 calculates the white value W for each pixel using the selected MIN(R,G,B) values. That is, the monitoring server 10 calculates W = MIN(R,G,B).

[0113] (3-1-4. Black value B) The black value B is described as an element of the HWB model used in the monitoring system 100. Below, the calculation process for black value B performed by the monitoring server 10 is described.

[0114] Firstly, the monitoring server 10 converts the pigment of each pixel into a combination of red (R), green (G), and blue (B) in the RGB model (RGB values). At this time, the monitoring server 10 normalizes each (R, G, B) value by dividing it by the maximum value of 255.

[0115] Secondly, the monitoring server 10 selects the maximum value from the normalized (R, G, B) values ​​and sets it to MAX(R, G, B).

[0116] Thirdly, the monitoring server 10 calculates the black value B for each pixel by subtracting the selected MAX(R,G,B) value from 1. That is, the monitoring server 10 calculates B = 1 - MAX(R,G,B).

[0117] (3-2. Specific Examples of Domain Classification Processing) This section describes specific examples of the area classification process of the monitoring system 100. Below, we will describe Specific Example 1, which uses a set boundary value TH, and Specific Example 2, which dynamically changes the boundary value TH.

[0118] (3-2-1. Specific Example of Domain Classification Processing 1) This section describes a specific example (1) of using the set boundary value TH as part of the region division classification process. The monitoring server 10 classifies the region into two areas, a colored region and a grayscale region, based on the calculated white value W and black value B.

[0119] Firstly, the monitoring server 10 determines the sum of the white values ​​W and black values ​​B T A The monitoring server 10 calculates the set boundary value TH and the total value T. A By comparing these, the pigment of each pixel is classified as either a colored or achromatic region.

[0120] Secondly, the monitoring server 10 will determine the total value T A If the value is less than the boundary value TH, it is classified as a colored region. That is, the monitoring server 10 classifies a pixel as a colored region if the white value W and black value B are low as pigments for each pixel, and uses the hue value H to convert the value C in the colored region. C Calculate the converted value C. C Details of the calculation process will be described later in (3-3. Specific Example 1 of Evaluation Value Calculation Process) and (3-4. Specific Example of Conversion Function).

[0121] Thirdly, the monitoring server 10 has a total value T A If the value is greater than or equal to the boundary value TH, it is classified as achromatic. That is, the monitoring server 10 classifies a pixel as achromatic when there are many white values ​​W and black values ​​B as the pigments of each pixel, and the converted value C in the achromatic region is determined by weighting the white values ​​W and black values ​​B. A Calculate the converted value C. A Details of the calculation process will be described later in (3-5. Specific Example of Evaluation Value Calculation Process 2).

[0122] The white value W is a number greater than or equal to 0 and less than or equal to 1. The black value B is a number greater than or equal to 0 and less than or equal to 1. The total value T A The value is greater than or equal to 0 and less than or equal to 1. The boundary value TH is also greater than or equal to 0 and less than or equal to 1.

[0123] (3-2-2. Specific Example of Domain Classification Processing 2) This section describes a specific example (2) of dynamically changing the boundary value TH as part of the region classification process. The monitoring server 10 calculates the brightness of the video data I based on the HWB model, etc., and automatically sets the boundary value TH according to the brightness difference and brightness distribution per frame.

[0124] Firstly, the monitoring server 10 determines the sum of the white value W and black value B of each pixel T A The average value μ is calculated. For example, if "pixel 1" is (W,B)=(0.1,0.8) and "pixel 2" is (W,B)=(0.1,0.4), the monitoring server 10 calculates μ={(0.1+0.8)+(0.1+0.4)} / 2=0.7. Here, the average value μ(t) over time changes with time t, so it can be expressed as μ(t)=mean(W+B). In this case, when the monitoring server 10 obtains the brightness around the monitored object, it calculates the average value μ not only for the specified range but also for the entire image, i.e., per frame.

[0125] Secondly, the monitoring server 10 detects the overall change in brightness by calculating the difference Δμ(t) of the average value μ(t) over time. That is, the monitoring server 10 calculates Δμ(t) = μ(t) - μ(t-1) as the change in brightness at time t-1 and time t.

[0126] Thirdly, the monitoring server 10 dynamically changes the latest boundary value TH(t) according to the difference from the previous boundary value TH(t-1). At this time, the monitoring server 10 sequentially updates the value by adding a term obtained by multiplying the previous boundary value TH(t-1) by the coefficient α and a term obtained by multiplying the difference Δμ(t), which represents brightness, by the coefficient β, so that it changes smoothly. In other words, the monitoring server 10 changes the boundary value TH by calculating TH(t) = α × TH(t-1) + β × Δμ(t). Specifically, if the surroundings become brighter, that is, if the overall white component increases, the monitoring server 10 raises the boundary value TH from TH(t-1) = 0.9 to TH(t) = 0.95. Also, if the surroundings become darker, that is, if the overall black component increases, the monitoring server 10 raises the boundary value TH from TH(t-1) = 0.9 to TH(t) = 0.95.

[0127] As described above, the monitoring server 10 can acquire the brightness around the monitored target and dynamically change the boundary value TH in accordance with the changes in the acquired brightness, thereby suppressing fluctuations in the evaluation value E due to the presence or absence of sunlight or electric lights.

[0128] (3-3. Specific Example 1 of Evaluation Value Calculation Process) Referring to Figure 8, a specific example 1 of the evaluation value calculation process of the monitoring system 100 will be described. Figure 8 is a diagram showing a specific example 1 of the evaluation value calculation process of the monitoring system 100 according to the embodiment. Below, a specific example of the conversion value output process and evaluation value calculation process for each pixel classified into a colored area, executed by the monitoring server 10 of the monitoring system 100, will be described. Here, the monitoring server 10 receives the set value V input by operator O. H Using this, the transformed value C in the color region corresponds to the hue value H. C It is possible to calculate this.

[0129] (3-3-1. Specific example of outputting converted values) This section describes a specific example of the conversion value output process for each pixel classified into the colored area of ​​the monitoring system 100. In the example shown in Figure 8(1), the monitoring server 10 outputs a conversion value of "10" for the light blue (horizontal lines), which is the pigment of the colored area, using the conversion information. The monitoring server 10 also outputs a conversion value of "30" for the yellow (diagonal lines), which is the pigment of the colored area, using the conversion information. Furthermore, the monitoring server 10 outputs a conversion value of "50" for the orange (vertical lines), which is the pigment of the colored area, using the conversion information.

[0130] (3-3-2. Specific Examples of Evaluation Value Calculation Process) A specific example of the process for calculating the evaluation value of each pixel classified into the colored area of ​​the monitoring system 100 will be explained. In the example in Figure 8(2), the monitoring server 10 calculates an evaluation value E of "200" for the total value and "12.5" for the average value, because out of the 16 pixels (number of pigments) included in the specified range, the total of the light blue conversion values ​​C is 10 × 14 = 140, the total of the yellow conversion values ​​C is 30 × 2 = 60, and the total of the orange conversion values ​​C is 50 × 0 = 0.

[0131] In the example shown in Figure 8(3), the monitoring server 10 calculates an evaluation value E of "240" for the total value and "15" for the average value, based on the fact that out of the 16 pixels (number of pigments) included in the specified range, the sum of the light blue conversion values ​​C is 10 × 12 = 120, the sum of the yellow conversion values ​​C is 30 × 4 = 120, and the sum of the orange conversion values ​​C is 50 × 0 = 0.

[0132] In the example shown in Figure 8(4), the monitoring server 10 calculates an evaluation value E of "440" for the total and "27.5" for the average, based on the fact that out of the 16 pixels (number of pigments) included in the specified range, the sum of the light blue conversion values ​​C is 10 × 6 = 60, the sum of the yellow conversion values ​​C is 30 × 6 = 180, and the sum of the orange conversion values ​​C is 50 × 4 = 200.

[0133] (3-4. Specific examples of transformation functions) Referring to Figure 9, a specific example of the conversion function of the monitoring system 100 will be described. Figure 9 is a diagram showing a specific example of the conversion function of the monitoring system 100 according to the embodiment. Below, the assumptions of the conversion function used in the conversion value output processing performed by the monitoring server 10 of the monitoring system 100 will be explained, and then, as specific examples of the conversion function, linear equations and curve equations will be described.

[0134] (3-4-1. Prerequisites for the transformation function) The assumptions for the conversion function used in the conversion value output processing of the monitoring system 100 are described below. In the monitoring system 100, operator O receives each set value V corresponding to each hue value H as conversion information. H It is necessary to set this. However, for example, for each integer value of hue value H from 0 to 360°, each setting value V H Setting these values ​​places a heavy burden on operator O. Therefore, the monitoring system 100 controls the hue value H and the set value V. H The converted value C can be calculated using a conversion function created by plotting several points. That is, in the monitoring system 100, by using the conversion function, the set value V corresponding to all hue values ​​H can be calculated. H This eliminates the need for configuration, further reducing the burden on operator O.

[0135] (3-4-2. Specific Example of a Transformation Function 1: Linear Equation) As a concrete example of a conversion function used in the conversion value output processing of the monitoring system 100, we will describe a linear equation. As shown in the example in Figure 9(1), the monitoring server 10 has the hue value H on the horizontal axis and the set value V on the vertical axis. H This shows a linear conversion function that can convert a hue value H between 0 and 360° into a converted value C. In the example in Figure 9(1), the monitoring server 10 outputs a converted value "25" corresponding to a hue value "150" using a linear equation.

[0136] (3-4-3. Specific Example of a Transformation Function 2: Curve Equations) As a second specific example of a conversion function used in the conversion value output processing of the monitoring system 100, we will describe a curve equation. As shown in the example in Figure 9(2), the monitoring server 10 uses the hue value H on the horizontal axis and the set value V on the vertical axis. H This shows a curved conversion function that can convert a hue value H between 0 and 360° into a converted value C. In the example in Figure 9(2), the monitoring server 10 uses a curved equation to output a converted value "10" corresponding to a hue value "150".

[0137] (3-5. Specific Example of Evaluation Value Calculation Process 2) Referring to Figure 10, a specific example 2 of the evaluation value calculation process of the monitoring system 100 will be described. Figure 10 is a diagram showing a specific example 2 of the evaluation value calculation process of the monitoring system 100 according to the embodiment. Below, a specific example of the conversion value output process and evaluation value calculation process for each pixel classified as achromatic area, executed by the monitoring server 10 of the monitoring system 100, will be described. Here, the monitoring server 10 receives the set value V input by operator O. W and the set value V B Using this, the conversion value C in the achromatic region is determined according to the ratio of white value W and black value B. A It is possible to calculate this.

[0138] (3-5-1. Specific example of outputting converted values) A specific example of the conversion value output processing for each pixel classified as achromatic in the monitoring system 100 is described. In the example in Figure 10, the white value W corresponds to the setting value V, which is the maximum value in the HWB model, W=1. W This is set to "100". Also, the setting value V corresponds to B=1, which is the maximum value in the HWB model where the black value B is. B This value is set to "0". Additionally, the boundary value TH is set to "0.9".

[0139] In the example shown in Figure 10, the monitoring server 10 uses the conversion information to convert the color (B,W)=(0.2,0.8) in the achromatic region to the converted value C. A It outputs the following. At this time, the monitoring server 10 outputs the conversion value C in the achromatic region. A C A=100 × 0.8 + 0 × 0.2 = 80 is calculated. In other words, the monitoring server 10 calculates the conversion value C in the achromatic region when the pigments in the achromatic region contain a large amount of white. A It outputs a large number.

[0140] In the example shown in Figure 10, the monitoring server 10 uses the conversion information to convert the color (B,W)=(0.6,0.3) in the achromatic region to obtain the converted value C. A It outputs the following. At this time, the monitoring server 10 outputs the conversion value C in the achromatic region. A C A =100 × 0.3 + 0 × 0.6 = 30 is calculated. In other words, the monitoring server 10 calculates the conversion value C in the achromatic region when the pigments in the achromatic region contain a large amount of black. A It outputs a small number.

[0141] (3-5-2. Specific Examples of Evaluation Value Calculation Process) This section describes a specific example of the process for calculating the evaluation value of each pixel classified as achromatic in the monitoring system 100. The monitoring server 10 uses all converted values ​​C as the evaluation value E for the number of pixels (number of pigments) included in the specified range. A The sum of the added values ​​T V The monitoring server 10 calculates the value E of the number of pixels (colors) included in the specified range, and all the converted values ​​C. A The mean A is the arithmetic mean of the values. V The monitoring server 10 can calculate the conversion value C not only for video data I containing only pigments in the achromatic region, but also for video data I containing pigments in the colored region, and similarly calculate the evaluation value E.

[0142] (3-6. Specific examples of the settings screen) Referring to Figure 11, a specific example of the settings screen of the monitoring system 100 will be described. Figure 11 is a diagram showing a specific example of the settings screen of the monitoring system 100 according to the embodiment. Below, a specific example of the settings screen displayed by the operator terminal 20 of the monitoring system 100 will be described.

[0143] (3-6-1. Setting the specified range screen) As shown in FIG. 11(1), the operator terminal 20 displays a designated range setting screen for displaying the video captured by the camera 30. In the example of FIG. 11(1), the operator terminal 20 is displaying the video near the window in the interior of the building as the designated range setting screen. At this time, the operator O can specify a predetermined range on the displayed video by surrounding it with a pointer (see FIG. 11(2)), and can set the designated range of the video data by clicking an image cutout button (see FIG. 11(6)).

[0144] At this time, as the shape of the designated range, the operator O can specify a range of an arbitrary shape in addition to shapes such as a rectangle, a circle, a sector, and a polygon. Further, the operator O can specify the range by inputting the coordinate values corresponding to the designated range of the video data.

[0145] (3-6-2. Setting Information Input Screen) As shown in FIG. 11(3), the operator terminal 20 displays a setting information input screen for inputting a setting value V corresponding to a pigment and a boundary value TH indicating the boundary of the region division. In the example of FIG. 11(3), the operator terminal 20 is displaying text boxes capable of inputting the setting value V and the boundary value TH as the setting information input screen. At this time, the operator O can set the conversion information by inputting the setting value V and the boundary value TH into each text box. In the example of FIG. 11(3), the operator O sets the setting value V corresponding to the hue value H “0” H to “36”, the setting value V corresponding to the hue value H “60” H to “40”, the setting value V corresponding to the hue value H “120” H to “0”, the setting value V corresponding to the hue value H “180” H to “18”, the setting value V corresponding to the hue value H “240” H to “24”, and the setting value V corresponding to the hue value H “300” H is input as “30”. Further, the operator O sets the setting value V corresponding to the maximum value of the white value W W to “80”, and the setting value V corresponding to the maximum value of the black value B Bis inputting "0". Also, operator O is inputting the boundary value TH as "0.9". Further, operator O can execute the update of the setting information by clicking the "Setting Information Change" button. Also, operator O can save the table data of the conversion table and the formula data of the conversion function on the setting information input screen.

[0146] Further, the operator terminal 20 can display a line graph (see Fig. 11(4)) showing the set value V corresponding to the hue value H. Also, when operator O inputs a conversion value into each text box, operator O can also select the hue value H for which the set value V H is to be input on the hue scale (see Fig. 11(5)). H

[0147] (3-7. Specific Example of Evaluation Value Notification Screen) Referring to Figs. 12 to 14, a specific example of the evaluation value notification screen of the monitoring system 100 will be described. Hereinafter, specific examples 1 to 3 of the evaluation value notification screen displayed on the operator terminal 20 of the monitoring system 100 will be described.

[0148] (3-7-1. Specific Example 1 of Evaluation Value Notification Screen) Referring to Fig. 12, a specific example 1 of the evaluation value notification screen displayed on the operator terminal 20 will be described. Fig. 12 is a diagram showing a specific example 1 of the evaluation value notification screen of the monitoring system 100 according to the embodiment.

[0149] As shown in Fig. 12(1), operator O sets a specified range with a lot of black in the specified range setting screen for displaying the video captured by camera 30. At this time, the operator terminal 20 displays an evaluation value notification screen including the evaluation value E calculated by the monitoring server 10. In the example of Fig. 12(2), the operator terminal 20 displays a histogram and a B-W scatter diagram.

[0150] As shown in Fig. 12(2), the operator terminal 20 shows the number of pixels n classified into the colored area and the achromatic area, and the average value A of the conversion value C in each of the colored area and the achromatic area V ​It displays the average value A for the colored area. In the example in Figure 12(2), the operator terminal 20 displays the average value A for the colored area. V It displays "Average = 0.0" and "n = 0", which is the number of pixels n. Furthermore, the operator terminal 20 displays the average value A for the achromatic area. V The average value A is displayed as "Average = 33.1" and the number of pixels n is displayed as "n = 3965". The operator terminal 20 also displays the average value A for the whole. V The display shows "Average = 33.1" and the number of pixels n, "n = 3965". The operator terminal 20 also displays "Average: 33.1" as the evaluation value E.

[0151] Furthermore, as shown in Figure 12(2), the operator terminal 20 displays a histogram showing the evaluation value E. In the example in Figure 12(2), the operator terminal 20 displays a histogram with the converted value C on the horizontal axis and the number of pixels n on the vertical axis, where the converted value C is maximized around the converted value "34".

[0152] Furthermore, as shown in Figure 12(2), the operator terminal 20 displays a scatter plot showing the evaluation value E in the achromatic region. In the example in Figure 12(2), the operator terminal 20 displays a scatter plot with the black value B on the horizontal axis and the white value W on the vertical axis, where there are many converted values ​​C around (B,W)=(0.6,0.4).

[0153] As described above, the operator terminal 20 displays a small evaluation value E when the video data I contains a lot of black, and also displays a histogram and scatter plot, thereby displaying an evaluation value notification screen that is easy for the operator O to understand intuitively and visually.

[0154] (3-7-2. Specific Example of Evaluation Value Notification Screen 2) Referring to Figure 13, a specific example 2 of the evaluation value notification screen displayed by the operator terminal 20 will be described. Figure 13 is a diagram showing a specific example 2 of the evaluation value notification screen of the monitoring system 100 according to the embodiment.

[0155] As shown in Figure 13(1), operator O sets a specified range with a large amount of white in the specified range setting screen that displays the video captured by camera 30. At this time, operator terminal 20 displays an evaluation value notification screen that includes the evaluation value E calculated by monitoring server 10. In the example in Figure 13(2), operator terminal 20 displays a histogram and a BW scatter plot.

[0156] As shown in Figure 13(2), the operator terminal 20 has n pixels classified into colored and achromatic regions, and the average value A of the conversion value C in the colored and achromatic regions, respectively. V It displays the average value A for the colored area. In the example in Figure 13(2), the operator terminal 20 displays the average value A for the colored area. V It displays "Average = 0.0" and "n = 0", which is the number of pixels n. Furthermore, the operator terminal 20 displays the average value A for the achromatic area. V The average value A is displayed as "Average = 54.2" and the number of pixels n is displayed as "n = 8798". The operator terminal 20 also displays the average value A for the whole. V The display shows "Average = 54.2" and the number of pixels n, "n = 8798". The operator terminal 20 also displays "Average: 54.2" as the evaluation value E.

[0157] Furthermore, as shown in Figure 13(2), the operator terminal 20 displays a histogram showing the evaluation value E. In the example in Figure 13(2), the operator terminal 20 displays a histogram with the converted value C on the horizontal axis and the number of pixels n on the vertical axis, where the number of converted values ​​C is maximized around the converted value "60".

[0158] Furthermore, as shown in Figure 13(2), the operator terminal 20 displays a scatter plot showing the evaluation value E in the achromatic region. In the example in Figure 13(2), the operator terminal 20 displays a scatter plot with the black value B on the horizontal axis and the white value W on the vertical axis, showing that there are many converted values ​​C around (B,W)=(0.0,1.0) to around (B,W)=(0.6,0.4).

[0159] As described above, the operator terminal 20 displays a large evaluation value E when the video data I contains a lot of white, and also displays a histogram and scatter plot, thereby displaying an evaluation value notification screen that is easy for the operator O to understand intuitively and visually.

[0160] (3-7-3. Specific Example of Evaluation Value Notification Screen 3) Referring to Figure 14, a specific example 3 of the evaluation value notification screen displayed by the operator terminal 20 will be described. Figure 14 is a diagram showing a specific example 3 of the evaluation value notification screen of the monitoring system 100 according to the embodiment.

[0161] As shown in Figure 14(1), operator O sets a specified range in the specified range setting screen for displaying the video captured by camera 30, where white and black are included in roughly equal proportions. At this time, operator terminal 20 displays an evaluation value notification screen that includes the evaluation value E calculated by monitoring server 10. In the example in Figure 14(2), operator terminal 20 displays a histogram and a BW scatter plot.

[0162] As shown in Figure 14(2), the operator terminal 20 has n pixels classified into colored and achromatic regions, and the average value A of the conversion value C in the colored and achromatic regions, respectively. V It displays the average value A for the colored area. In the example in Figure 14(2), the operator terminal 20 displays the average value A for the colored area. V It displays "Average = 0.0" and "n = 0", which is the number of pixels n. Furthermore, the operator terminal 20 displays the average value A for the achromatic area. V The average value A is displayed as "Average = 47.0" and the number of pixels n is displayed as "n = 3654". The operator terminal 20 also displays the average value A for the whole. V The display shows "Average = 47.0" and the number of pixels n, "n = 3654". The operator terminal 20 also displays "Average: 47.0" as the evaluation value E.

[0163] Furthermore, as shown in Figure 14(2), the operator terminal 20 displays a histogram showing the evaluation value E. In the example in Figure 14(2), the operator terminal 20 displays a histogram with the converted value C on the horizontal axis and the number of pixels n on the vertical axis, where the number of converted values ​​C is maximized around the converted value "50".

[0164] Furthermore, as shown in Figure 14(2), the operator terminal 20 displays a scatter plot showing the evaluation value E in the achromatic region. In the example in Figure 14(2), the operator terminal 20 displays a scatter plot with the black value B on the horizontal axis and the white value W on the vertical axis, showing that there are many converted values ​​C around (B,W)=(0.0,1.0) to around (B,W)=(0.8,0.2).

[0165] As described above, the operator terminal 20 displays a moderate evaluation value E when the video data I contains an equal amount of white and black, and also displays a histogram and scatter plot, thereby displaying an evaluation value notification screen that is easy for the operator O to understand intuitively and visually.

[0166] [4. Flow of each process in the monitoring system 100] Referring to Figures 15 to 19, the flow of each process in the monitoring system 100 according to the embodiment will be described below. After describing the overall flow of the monitoring system 100, the individual processes will be described as setting information management, video data management, evaluation value management, and alarm management.

[0167] (4-1. Overall processing of monitoring system 100) Referring to Figure 15, the overall processing flow of the monitoring system 100 according to the embodiment will be described. Figure 15 is a flowchart showing an example of the overall processing flow of the monitoring system 100 according to the embodiment. Note that the processes in steps S101 to S104 below can be executed in a different order. Also, some of the processes in steps S101 to S104 below may be omitted.

[0168] (4-1-1. Configuration Information Management Process) Firstly, the monitoring system 100 performs configuration information management processing (step S101). For example, the monitoring system 100 manages the configuration information set by operator O by performing the processes described in steps S201 to S204.

[0169] (4-1-2. Video Data Management Processing) Secondly, the surveillance system 100 performs video data management processing (step S102). For example, the surveillance system 100 manages the video data I of the monitored object captured by the camera 30 by performing the processes described in steps S301 to S303.

[0170] (4-1-3. Evaluation Value Management Process) Thirdly, the monitoring system 100 performs evaluation value management processing (step S103). For example, the monitoring system 100 manages evaluation values ​​E for evaluating the state of the monitored object by performing the processes described in steps S401 to S408.

[0171] (4-1-4. Alarm Management Process) Fourth, the monitoring system 100 performs alarm management processing (step S104). For example, the monitoring system 100 manages alarms indicating abnormal conditions of the monitored system by performing the processes described in steps S501 to S504.

[0172] (4-2. Configuration Information Management Process) Referring to Figure 16, the flow of the configuration information management process of the monitoring system 100 according to the embodiment will be described. Figure 16 is a flowchart showing an example of the flow of the configuration information management process of the monitoring system 100 according to the embodiment. Note that the processes in steps S201 to S204 below can be executed in a different order. Also, some of the processes in steps S201 to S204 below may be omitted.

[0173] (4-2-1. Processing of applications within a specified range) Firstly, the monitoring server 10 performs a specified range acceptance process (step S201). For example, the monitoring server 10 accepts a specified range of video data I set by operator O via operator terminal 20.

[0174] (4-2-2. Area Classification Acceptance Processing) Secondly, the monitoring server 10 performs area division acceptance processing (step S202). For example, the monitoring server 10 receives boundary values ​​TH that indicate the boundaries between colored and achromatic areas set by operator O via operator terminal 20.

[0175] (4-2-3. Processing of conversion information) Thirdly, the monitoring server 10 performs conversion information reception processing (step S203). For example, the monitoring server 10 receives conversion information such as a conversion table, conversion function, and fixed values ​​set by operator O via the operator terminal 20.

[0176] (4-2-4. Configuration Information Storage Process) Fourth, the monitoring server 10 executes the configuration information storage process (step S204) and terminates the configuration information management process. For example, the monitoring server 10 stores configuration information, including the specified range, area division, and conversion information of the video data I, in the configuration information storage unit 14a.

[0177] (4-3. Video Data Management Processing) Referring to Figure 17, the flow of video data management processing in the surveillance system 100 according to the embodiment will be described. Figure 17 is a flowchart showing an example of the flow of video data management processing in the surveillance system 100 according to the embodiment. Note that the processes in steps S301 to S303 below can be executed in a different order. Also, some of the processes in steps S301 to S303 below may be omitted.

[0178] (4-3-1. Video data generation process) Firstly, the camera 30 performs video data generation processing (step S301). For example, the camera 30 photographs the object being monitored and generates video data I, either a still image or a video.

[0179] (4-3-2. Video data acquisition process) Secondly, the monitoring server 10 performs video data acquisition processing (step S302). For example, the monitoring server 10 acquires video data I, which may be a still image or video, generated by the camera 30.

[0180] (4-3-3. Video data storage processing) Thirdly, the monitoring server 10 executes video data storage processing (step S303) and terminates the video data management processing. For example, the monitoring server 10 stores still images or video data I acquired from the camera 30 in the video data storage unit 14b.

[0181] (4-4. Evaluation Value Management Process) Referring to Figure 18, the flow of the evaluation value management process of the monitoring system 100 according to the embodiment will be described. Figure 18 is a flowchart showing an example of the flow of the evaluation value management process of the monitoring system 100 according to the embodiment. Note that the processes in steps S401 to S408 below can be executed in a different order. Also, some of the processes in steps S401 to S408 below may be omitted.

[0182] (4-4-1. Configuration Information Reference Processing) Firstly, the monitoring server 10 performs configuration information reference processing (step S401). For example, the monitoring server 10 references configuration information stored in the configuration information storage unit 14a, which includes the specified range, area division, and conversion information of the video data I.

[0183] (4-4-2. Video Data Reference Processing) Secondly, the monitoring server 10 performs video data reference processing (step S402). For example, the monitoring server 10 references the video data I stored in the video data storage unit 14b.

[0184] (4-4-3. Processing to identify the specified range) Thirdly, the monitoring server 10 performs a specified range identification process (step S403). For example, the monitoring server 10 identifies the specified range of the video data I by referring to the specified range indicated by the configuration information.

[0185] (4-4-4. HWB value conversion process) Fourth, the monitoring server 10 performs HWB value conversion processing (step S404). For example, the monitoring server 10 converts the color of each pixel included in a specified range of the video data I into an HWB value, which is a combination of hue value H, white value W, and black value B.

[0186] (4-4-5. Domain Classification Processing) Fifth, the monitoring server 10 performs region classification processing (step S405). For example, the monitoring server 10 refers to the boundary value TH indicated by the configuration information and classifies the pigment of each pixel into a colored region or a gray region.

[0187] (4-4-6. Outputting converted values) Sixth, the monitoring server 10 performs the conversion value output process (step S406). For example, the monitoring server 10 refers to the conversion information corresponding to the classified region division and outputs the conversion value C for each pixel.

[0188] (4-4-7. Processing of converted values) Seventh, the monitoring server 10 performs a conversion value aggregation process (step S407). For example, the monitoring server 10 aggregates the conversion values ​​C of each output pixel and calculates the total value T of the conversion values ​​C of all pixels. V Or average value A V This is calculated as the evaluation value E. At this time, the monitoring server 10 can also create tables or graphs showing the converted value C or the evaluation value E.

[0189] (4-4-8. Processing to store evaluation values) Eighth, the monitoring server 10 executes the evaluation value storage process (step S408) and terminates the evaluation value management process. For example, the monitoring server 10 stores the calculated evaluation value E in the evaluation value storage unit 14c.

[0190] (4-5. Alarm Management Process) Referring to Figure 19, the flow of the alarm management process of the monitoring system 100 according to the embodiment will be described. Figure 19 is a flowchart showing an example of the flow of the alarm management process of the monitoring system 100 according to the embodiment. Note that the processes in steps S501 to S504 below can be executed in a different order. Also, some of the processes in steps S501 to S504 below may be omitted.

[0191] (4-5-1. Evaluation Value Reference Processing) First, the monitoring server 10 performs evaluation value reference processing (step S501). For example, the monitoring server 10 references the evaluation value E stored in the evaluation value storage unit 14c. At this time, if the evaluation value E is greater than or equal to the threshold X (step S502: Yes), the monitoring server 10 proceeds to the process in step S503. On the other hand, if the evaluation value E is less than the threshold X (step S502: No), the monitoring server 10 proceeds to the process in step S504.

[0192] (4-5-2. Alarm notification processing) Secondly, the monitoring server 10 performs alarm notification processing (step S503). For example, the monitoring server 10 notifies the operator O of the alarm by sending alarm data indicating an abnormal state of the monitored system to the operator terminal 20 and displaying the alarm on the operator terminal 20.

[0193] (4-5-3. Evaluation Value Notification Process) Thirdly, the monitoring server 10 executes the evaluation value notification process (step S504) and terminates the alarm management process. For example, the monitoring server 10 notifies the operator O of the evaluation value E by sending table data or graph data showing the evaluation value E to the operator terminal 20 and displaying the table or graph showing the evaluation value E on the operator terminal 20.

[0194] [5. Effects of the Embodiment] The effects of the embodiment will be described below. Effects 1 to 14 corresponding to the processing according to the embodiment will be described below.

[0195] (5-1. Effect 1) Firstly, in this embodiment, the monitoring server 10 acquires video data I of the monitored object captured by the camera 30, and calculates an evaluation value E that evaluates the state of the monitored object according to the ratio of the white value W, which indicates the value of white, and the black value B, which indicates the value of black, based on the acquired video data I and the respective setting values ​​V corresponding to white and black. Therefore, in this embodiment, the state of the monitored object can be effectively grasped.

[0196] (5-2. Effect 2) Secondly, in the embodiment, the monitoring server 10, among the region divisions indicating achromatic and colored regions, if the pigment of each pixel of the video data I is classified as an achromatic region, sets a setting value V corresponding to white. W The value obtained by multiplying the white value W by the setting value V corresponding to black. B The evaluation value E is calculated by adding a value obtained by multiplying the black value B by the black value. Therefore, in this embodiment, even in a black and white image, an evaluation value E weighted for white and black can be calculated, making it possible to effectively grasp the state of the monitored object.

[0197] (5-3. Effect 3) Thirdly, in the embodiment, the monitoring server 10 determines the sum of the white value W and black value B in the HWB model of each pixel T A Calculate the sum T A If the value is greater than or equal to the boundary value TH that indicates the boundary of the region division, the pigment of each pixel is classified as achromatic, and the total value T A If the value is less than the boundary value TH, the pigment of each pixel is classified into a colored region. Therefore, in this embodiment, it is possible to classify the achromatic region and the colored region and then calculate an evaluation value E according to the region classification, thereby effectively understanding the state of the monitored object.

[0198] (5-4. Effect 4) Fourth, in the embodiment, the monitoring server 10 receives the threshold value TH input by the operator O via the setting screen displayed on the operator terminal 20 of the operator O. Therefore, in the embodiment, since the region classification can be classified based on the set threshold value TH and then the evaluation value E corresponding to the region classification can be calculated, the state of the monitoring target can be effectively grasped.

[0199] (5-5. Effect 5) Fifth, in the embodiment, the monitoring server 10 changes the threshold value TH according to the change in the brightness of the video data I. Therefore, in the embodiment, since the region classification can be classified based on the dynamically changed threshold value TH and then the evaluation value E corresponding to the region classification can be calculated, the state of the monitoring target can be effectively grasped.

[0200] (5-6. Effect 6) Sixth, in the embodiment, when the pigment of each pixel is classified into the color region, the monitoring server 10 uses each set value V corresponding to each hue value H in the HWB model of each pixel H to calculate the evaluation value E corresponding to the hue value H. Therefore, in the embodiment, since the evaluation value E corresponding to the coloration in the color image can be calculated, the state of the monitoring target can be effectively grasped.

[0201] (5-7. Effect 7) Seventh, in the embodiment, the monitoring server 10 receives each set value V corresponding to white, black, and each hue value H input by the operator O via the setting screen displayed on the operator terminal 20 of the operator O. Therefore, in the embodiment, since the evaluation value E that can be intuitively grasped by the operator O can be calculated, the state of the monitoring target can be effectively grasped.

[0202] (5-8. Effect 8) Eighth, in the embodiment, the monitoring server 10 receives a specified range of video data I specified by the operator O on the video displayed on the setting screen of the operator terminal 20 of the operator O. Therefore, in the embodiment, the operator O who monitors in real time can effectively grasp the state of the monitoring target.

[0203] (5-9. Effect 9) Ninth, in the embodiment, the monitoring server 10 calculates, as the evaluation value E, the total value T of each setting value V corresponding to the pigments of each pixel of the video data I V Thereby, in the embodiment, the state of the monitoring target can be effectively grasped by calculating the intensity of the entire video as the evaluation value E.

[0204] (5-10. Effect 10) Tenth, in the embodiment, the monitoring server 10 calculates, as the evaluation value E, the average value A of each setting value V corresponding to each hue of each pixel of the video data I V Thereby, in the embodiment, the state of the monitoring target can be effectively grasped by calculating the averaged intensity of the video as the evaluation value E.

[0205] (5-11. Effect 11) Eleventh, in the embodiment, when the calculated evaluation value is greater than or equal to the threshold value X, the monitoring server 10 notifies the operator O of an alarm indicating an abnormality of the monitoring target. Therefore, in the embodiment, the operator O can effectively grasp the state of the monitoring target without constantly monitoring.

[0206] (5-12. Effect 12) Twelfth, in the process according to the above-described embodiment, the monitoring server 10 notifies the operator O of a table or graph indicating the calculated evaluation value E. Therefore, in this process, the operator O can effectively grasp the state of the monitoring target visually.

[0207] (5-13. Effect 13) Thirteenth, in this embodiment, the monitoring server 10 acquires video data I, which includes only white and black, output by the camera 30. Therefore, in this embodiment, even if the video data I is a black and white image, the state of the monitored object can be effectively grasped.

[0208] (5-14. Effect 14) Fourteenth, in this embodiment, the monitoring server 10 acquires still image data or video data output by the camera 30. Therefore, in this embodiment, the state of the monitored object can be effectively grasped whether the video data I is a still image or a video.

[0209] [6. Examples of applications of the embodiment] Examples of applications of the embodiment will be described below. Examples of applications 1 to 10 of the embodiment will be described below.

[0210] (6-1. Application Example 1) As an application example of the embodiment, it is possible to detect impurities in iron ore flowing by a belt conveyor during the manufacturing process of a plant.

[0211] (6-2. Application Example 2) As an application example 2 of the embodiment, it is possible to detect impurities contained in the fluid flowing through piping in the manufacturing process of a plant.

[0212] (6-3. Application Example 3) As an application example 3 of the embodiment, it is possible to detect abnormalities in the water level and flow rate of fluids flowing through piping in a plant's manufacturing process.

[0213] (6-4. Application Example 4) As an application example 4 of the embodiment, it is possible to detect strain caused by pressure in pipes and tanks during the manufacturing process of a plant.

[0214] (6-5. Application Example 5) As an application example 5 of the embodiment, it is possible to detect abnormalities in products flowing by a belt conveyor in the manufacturing process of a plant.

[0215] (6-6. Application Example 6) As Application Example 6 of the embodiment, in the manufacturing process of a plant, it is possible to detect abnormalities in the smoke discharged from a chimney.

[0216] (6-7. Application Example 7) As Application Example 7 of the embodiment, in a building, farmland, etc., it is possible to detect the intrusion of people and animals.

[0217] (6-8. Application Example 8) As Application Example 8 of the embodiment, in a park, road, river, etc., it is possible to detect abnormalities.

[0218] (6-9. Application Example 9) As Application Example 9 of the embodiment, it is possible to grasp the state using the black-and-white image of a thermal camera.

[0219] (6-10. Application Example 10) As Application Example 10 of the embodiment, it is possible to grasp the state using the black-and-white image in medical image analysis.

[0220] [7. System] Regarding the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings, they can be arbitrarily changed unless otherwise specified.

[0221] Also, each component of each illustrated device is a functional concept and does not necessarily need to be configured physically as shown in the drawing. That is, the specific forms of dispersion and integration of each device are not limited to those shown in the drawing. In other words, all or part of it can be functionally or physically dispersed and integrated in any unit according to various loads, usage situations, etc.

[0222] Furthermore, each processing function performed by each device can be implemented, in whole or in part, by a CPU and a program that is analyzed and executed by the CPU, or by hardware using wired logic.

[0223] [8. Hardware] This section describes an example of the hardware configuration of the monitoring server 10, which is an information provision device. Note that other devices can also have a similar hardware configuration. Figure 20 shows an example of the hardware configuration according to this embodiment. As shown in Figure 20, the monitoring server 10 includes a communication device 10a, a storage device 10b, memory 10c, and a processor 10d. Furthermore, the components shown in Figure 20 are interconnected by a bus or the like.

[0224] The communication device 10a is a network interface card or the like, and communicates with other servers. The storage device 10b stores programs and databases that operate the functions shown in Figure 3.

[0225] The processor 10d operates a process that performs the functions described in Figure 3 by reading a program that performs the same processing as each processing unit shown in Figure 3 from the storage device 10b, etc., and loading it into memory 10c. For example, this process performs the same functions as each processing unit of the monitoring server 10. Specifically, the processor 10d reads a program that has the same functions as the reception unit 15a, acquisition unit 15b, calculation unit 15c, notification unit 15d, etc., from the storage device 10b, etc. Then, the processor 10d executes a process that performs the same processing as the reception unit 15a, acquisition unit 15b, calculation unit 15c, notification unit 15d, etc.

[0226] Thus, the monitoring server 10 operates as a device that executes various processing methods by reading and executing a program. Furthermore, the monitoring server 10 can also achieve the same functionality as the embodiment described above by reading the program from a recording medium using a media reader and executing the read program. Note that the program according to the embodiment is not limited to being executed by the monitoring server 10. For example, this disclosure can be similarly applied when another computer or server executes the program, or when they cooperate to execute the program.

[0227] The program according to the embodiment can be distributed via a network such as the Internet. Furthermore, the program according to the embodiment can be recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), or DVD (Digital Versatile Disc), and executed by reading it from the recording medium by a computer.

[0228] [9. Other] Some examples of the combinations of technical features that will be disclosed are listed below.

[0229] (1) An information providing device comprising a processor, wherein the processor acquires video data of a monitored object captured by a camera, and calculates an evaluation value for evaluating the state of the monitored object according to the ratio of the white value, which represents the value of white, and the black value, which represents the value of black, based on the acquired video data and setting values ​​corresponding to white and black, respectively.

[0230] (2) The information providing device according to (1), wherein the processor calculates the evaluation value by adding a value obtained by multiplying the setting value corresponding to white by the white value and a value obtained by multiplying the setting value corresponding to black by the black value, when the color information of each pixel of the video data is classified as the achromatic region among the region divisions indicating achromatic regions and colored regions.

[0231] (3) The information providing device according to (2), wherein the processor calculates the sum of the white value and the black value in the HWB model of each pixel, and if the sum is greater than or equal to a boundary value indicating the boundary of the region division, the color information of each pixel is classified into the achromatic region, and if the sum is less than the boundary value, the color information of each pixel is classified into the colored region.

[0232] (4) The information providing device according to (3), wherein the processor receives the boundary value entered by the user via a settings screen displayed on the user's terminal.

[0233] (5) The information providing device according to (3) or (4), wherein the processor changes the boundary value in accordance with the change in brightness of the video data.

[0234] (6) The information providing device according to any one of (2) to (5), wherein the processor calculates the evaluation value corresponding to the hue value using the setting values ​​corresponding to each hue value in the HWB model of each pixel when the color information of each pixel is classified into the colored area.

[0235] (7) The information providing device according to any one of (1) to (6), wherein the processor receives setting values ​​corresponding to white, black, and each hue value entered by the user via a setting screen displayed on the user's user terminal.

[0236] (8) The information providing device according to any one of (1) to (7), wherein the processor accepts a specified range of the video data specified by the user on the video displayed on the user terminal settings screen of the user.

[0237] (9) The information providing device according to any one of (1) to (8), wherein the processor calculates the sum of the setting values ​​corresponding to the color information of each pixel of the video data as the evaluation value.

[0238] (10) The information providing device according to any one of (1) to (9), wherein the processor calculates the average value of each setting value corresponding to the color information of each pixel of the video data as the evaluation value.

[0239] (11) The information providing device according to any one of (1) to (10), wherein the processor notifies the user of an alarm indicating an abnormality of the monitored object when the calculated evaluation value is equal to or greater than a threshold.

[0240] (12) The information providing device according to any one of (1) to (11), wherein the processor notifies the user of a table or graph showing the calculated evaluation value.

[0241] (13) The information providing device according to any one of (1) to (12), wherein the processor acquires the video data, which includes only the white and black colors, output by the imaging device that films the subject being monitored.

[0242] (14) The information providing device according to any one of (1) to (13), wherein the processor acquires still image data or video data output by the imaging device.

[0243] (15) An information provision method comprising: an information provision device acquiring video data of a monitored object captured by a camera; and calculating an evaluation value for evaluating the state of the monitored object based on the acquired video data and setting values ​​corresponding to white and black, respectively, according to the ratio of the white value representing the value of white and the black value representing the value of black.

[0244] (16) An information providing program that causes the information providing device to acquire video data of a monitored object captured by a camera, and to calculate an evaluation value for evaluating the state of the monitored object according to the ratio of the white value, which represents the value of white, and the black value, which represents the value of black, based on the acquired video data and the respective setting values ​​for white and black. [Explanation of Symbols]

[0245] 10 Monitoring Servers 10a Communication device 10b Storage device 10c memory 10d processor 11 Input section 12 Output section 13 Communications Department 14 Storage section 14a Configuration Information Storage Unit 14b Video data storage unit 14c Evaluation value storage unit 15 Control Unit 15a Reception Desk 15b Acquisition part 15c Calculation part 15d Notification Department 20 Operator terminals 21 Input / output section 22 Transceiver Unit 23 Communications Department 30 Cameras 31 Generation part 32 Communications Department 100 monitoring systems N Communication Network O Operator

Claims

1. Equipped with a processor, The aforementioned processor, The camera acquires video data of the monitored target captured by the camera. The sum of the white value, which represents the white value in the HWB model for each pixel of the acquired video data, and the black value, which represents the black value, is calculated. If the sum is greater than or equal to the boundary value indicating the boundary between the achromatic region and the colored region, the color information of each pixel is classified into the achromatic region. If the sum is less than the boundary value, the color information of each pixel is classified into the colored region. Based on the acquired video data and the respective setting values ​​corresponding to white and black, an evaluation value is calculated to evaluate the state of the monitored object according to the ratio of the white value and the black value. Information provision device.

2. The aforementioned processor, In the region divisions representing the achromatic region and the colored region, if the color information of each pixel of the video data is classified as the achromatic region, the evaluation value is calculated by adding a value obtained by multiplying the setting value corresponding to white by the white value, and a value obtained by multiplying the setting value corresponding to black by the black value. The information providing device according to claim 1.

3. The aforementioned processor, The system accepts the boundary value entered by the user via a settings screen displayed on the user's terminal. The information providing device according to claim 1.

4. The aforementioned processor, The boundary value is changed according to the change in brightness of the aforementioned video data. The information providing device according to claim 1.

5. The aforementioned processor, If the color information of each pixel is classified into the colored area, the evaluation value corresponding to the hue value is calculated using the setting values ​​corresponding to each hue value in the HWB model of each pixel. The information providing device according to claim 1.

6. The aforementioned processor, The system accepts the setting values ​​corresponding to white, black, and each hue value entered by the user via a settings screen displayed on the user's terminal. An information providing device according to any one of claims 1 to 5.

7. The aforementioned processor, On the user's terminal settings screen, the system accepts the specified range of the video data specified by the user. An information providing device according to any one of claims 1 to 5.

8. The aforementioned processor, As the evaluation value, the sum of the setting values ​​corresponding to the color information of each pixel of the video data is calculated. An information providing device according to any one of claims 1 to 5.

9. The aforementioned processor, As the evaluation value, the average value of each setting value corresponding to the color information of each pixel of the video data is calculated. An information providing device according to any one of claims 1 to 5.

10. The aforementioned processor, If the calculated evaluation value is above a threshold, an alarm indicating an abnormality in the monitored target will be notified to the user. An information providing device according to any one of claims 1 to 5.

11. The aforementioned processor, The user is notified of a table or graph showing the calculated evaluation values. An information providing device according to any one of claims 1 to 5.

12. The aforementioned processor, The camera that films the subject being monitored outputs video data containing only white and black to be acquired. An information providing device according to any one of claims 1 to 5.

13. The aforementioned processor, The still image data or video data output by the aforementioned camera is acquired. An information providing device according to any one of claims 1 to 5.

14. The information provision device, The camera acquires video data of the monitored target captured by the camera. The sum of the white value, which represents the white value in the HWB model for each pixel of the acquired video data, and the black value, which represents the black value, is calculated. If the sum is greater than or equal to the boundary value indicating the boundary between the achromatic region and the colored region, the color information of each pixel is classified into the achromatic region. If the sum is less than the boundary value, the color information of each pixel is classified into the colored region. Based on the acquired video data and the respective setting values ​​corresponding to white and black, an evaluation value is calculated to evaluate the state of the monitored object according to the ratio of the white value and the black value. A method for providing information to execute a process.

15. The information provision device, The camera acquires video data of the monitored target captured by the camera. The sum of the white value, which represents the white value in the HWB model for each pixel of the acquired video data, and the black value, which represents the black value, is calculated. If the sum is greater than or equal to the boundary value indicating the boundary between the achromatic region and the colored region, the color information of each pixel is classified into the achromatic region. If the sum is less than the boundary value, the color information of each pixel is classified into the colored region. Based on the acquired video data and the respective setting values ​​corresponding to white and black, an evaluation value is calculated to evaluate the state of the monitored object according to the ratio of the white value and the black value. A program that provides information to initiate a process.

Images