Railway line monitoring system and railway line monitoring method

Abstract

To provide a route environment monitoring system and a route environment monitoring method, which are capable of making various settings when an abnormal event is detected based on changes in color appearing in images captured by a camera to issue an alert, thereby achieving highly accurate abnormality detection and alert issue.SOLUTION: A route environment monitoring system includes: at least one camera 1 imaging a monitoring area; an image analysis server 2 executing color analysis processing of sorting, for each color system, a plurality of measurement units (one pixel or a block including a plurality of pixels) included in images captured by the camera based on color information for each of the measurement units, abnormality determination processing of comparing a current color analysis result and a past color analysis result obtained by the color analysis processing, to determine whether there is an abnormal event, and alert issuing processing of notifying the occurrence of an abnormal event; and a monitoring terminal 4 displaying a setting screen regarding processing conditions related to the color analysis processing, the abnormality determination processing, and the alert issuing processing, and displaying a notification screen to notify the occurrence of an abnormal event according to an abnormality determination result by the abnormality determination processing.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a trackside monitoring system and a trackside monitoring method that detect and issue an alert for an abnormal event occurring in a monitoring area set along a trackside through image analysis performed on an image obtained by capturing the monitoring area.

Background Art

[0002] If an event such as a landslide or an avalanche occurs along a trackside such as a highway or a railway, it impairs the safe passage of automobiles and trains, so it is desired that recovery work be carried out promptly. In addition, if overgrown weeds on the slope adjacent to a driving road are left unattended, it causes nuisance to neighboring residents, so it is desired that mowing work be performed at an appropriate timing.

[0003] In order to quickly respond to such abnormalities along the trackside, a trackside monitoring system is conventionally known in which a camera is installed on the trackside, image analysis is performed on images captured by the camera to detect and issue an alert for an abnormal event that occurs along the trackside (see Patent Document 1).

[0004] In addition, in a system for detecting abnormal events in facilities such as buildings and factories, a technology is known that measures color components of an image captured by a camera and detects the abnormal event based on a change in the color components (see Patent Document 2).

Prior Art Literature

Patent Literature

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problem to be Solved by the Invention

[0006] Now, it is conceivable to apply the anomaly detection technology disclosed in Patent Document 2 to the railway line monitoring system disclosed in Patent Document 1 to detect and report abnormal events occurring on or around highways, railways, and other roads.

[0007] On the other hand, the conditions of the monitoring area targeted for abnormal event detection vary greatly depending on the site. Therefore, in order to perform highly accurate abnormal event detection and alarm activation, it is desirable to set detailed and appropriate processing conditions according to the site conditions, such as the processing conditions for detecting abnormal events and issuing alarms based on color changes appearing in the camera's captured images. However, conventional technologies have not taken such requirements into consideration, resulting in the problem that highly accurate abnormal event detection and alarm activation cannot be performed.

[0008] Therefore, the main objective of the present invention is to provide a railway line monitoring system and method that can perform various settings regarding processing conditions when detecting abnormal events and issuing alerts based on color changes appearing in images captured by a camera, thereby achieving highly accurate abnormality detection and alerting. [Means for solving the problem]

[0009] The railway line monitoring system of the present invention is a railway line monitoring system that detects abnormal events occurring in a monitoring area by image analysis of images taken of a monitoring area set along a railway line and issues an alert, comprising: at least one camera that photographs the monitoring area; a color analysis process that sorts multiple measurement units into color systems based on the color information of each of the multiple measurement units contained in the image taken by the camera; and the current color analysis result from the color analysis process. The color analysis results are compared with those from multiple points in the past that occurred on different days. If a change exceeding a predetermined limit is observed in both, it is determined that an abnormal event has occurred. The configuration includes a server device that performs an abnormality detection process and an alerting process that notifies the occurrence of an abnormal event, and a terminal device that displays a setting screen for processing conditions related to the color analysis process, the abnormality detection process, and the alerting process, and also displays an alerting screen that notifies the occurrence of an abnormal event according to the abnormality detection result from the abnormality detection process.

[0010] Furthermore, the railway line monitoring method of the present invention is a railway line monitoring method in which a processor performs a process to detect abnormal events occurring in a monitoring area by image analysis on images taken of a monitoring area set along the railway line and to issue an alert, wherein a server device performs a color analysis process that sorts multiple measurement units by color system based on the color information for each of the multiple measurement units contained in the image taken by at least one camera that photographs the monitoring area, and the current color analysis result of the color analysis process and The color analysis results are compared with those from multiple points in the past that occurred on different days. If a change exceeding a predetermined limit is observed in both, it is determined that an abnormal event has occurred. The system performs an abnormality detection process and an alerting process to notify the occurrence of an abnormal event. The terminal device displays a setting screen in advance for the processing conditions related to the color analysis process, the abnormality detection process, and the alerting process, and displays an alerting screen to notify the occurrence of an abnormal event according to the abnormality detection result from the abnormality detection process. [Effects of the Invention]

[0011] According to the present invention, based on the color changes that appear in the image captured by the camera Appropriate detection of abnormal events By configuring various settings regarding the processing conditions for issuing alerts, it is possible to achieve highly accurate anomaly detection and alerting. [Brief explanation of the drawing]

[0012] [Figure 1] Overall configuration diagram of the monitoring system according to this embodiment [Figure 2] An explanatory diagram showing images captured by cameras in the surveillance area. [Figure 3] An explanatory diagram showing the detection area and mask area set on the camera's captured image. [Figure 4] Block diagram showing an overview of the processing performed by the image analysis server. [Figure 5] An explanatory diagram showing the color family ratio graph generated by the image analysis server. [Figure 6] An explanatory diagram showing a heatmap superimposed image generated by the image analysis server. [Figure 7] Block diagram showing the schematic configuration of the image analysis server, image management server, and monitoring terminal. [Figure 8]Flow chart showing the procedure of color component measurement processing performed in an image analysis server [Figure 9] Flow chart showing the procedure of color analysis processing performed in an image analysis server [Figure 10] Flow chart showing the procedure of abnormality determination processing performed in an image analysis server [Figure 11] Block diagram showing an outline of processing when a correlation is set for a plurality of cameras [Figure 12] Explanatory diagram showing a list display screen in an unfinished setting state displayed on a monitoring terminal [Figure 13] Explanatory diagram showing a list display screen during setting operation displayed on a monitoring terminal [Figure 14] Explanatory diagram showing a camera setting screen displayed on a monitoring terminal [Figure 15] Explanatory diagram showing an area setting screen displayed on a monitoring terminal [Figure 16] Explanatory diagram showing a color analysis setting screen displayed on a monitoring terminal [Figure 17] Explanatory diagram showing a normal list display screen in an operation mode displayed on a monitoring terminal [Figure 18] Explanatory diagram showing a notification screen displayed on a monitoring terminal and a list display screen at the time of abnormality in an operation mode [Figure 19] Explanatory diagram showing a status confirmation screen displayed on a monitoring terminal [Figure 20] Explanatory diagram showing a time-series confirmation screen displayed on a monitoring terminal [Figure 21] Explanatory diagram showing a time-series confirmation screen displayed on a monitoring terminal [Figure 22] Explanatory diagram showing a time-series confirmation screen displayed on a monitoring terminal [Figure 23] Explanatory diagram showing a time-series confirmation screen displayed on a monitoring terminal [Figure 24] Explanatory diagram showing a map display screen displayed on a monitoring terminal DETAILED DESCRIPTION OF EMBODIMENTS FOR CARRYING OUT THE INVENTION

[0013] The first invention made to solve the aforementioned problems is a railway line monitoring system that detects abnormal events occurring in a monitoring area by image analysis of images taken of a monitoring area set along the railway line and issues an alert, comprising: at least one camera that photographs the monitoring area; a color analysis process that sorts multiple measurement units into color systems based on the color information of each of the multiple measurement units contained in the image taken by the camera; and the current color analysis result from the color analysis process. The color analysis results are compared with those from multiple points in the past that occurred on different days. If a change exceeding a predetermined limit is observed in both, it is determined that an abnormal event has occurred. The configuration includes a server device that performs an abnormality detection process and an alerting process that notifies the occurrence of an abnormal event, and a terminal device that displays a setting screen for processing conditions related to the color analysis process, the abnormality detection process, and the alerting process, and also displays an alerting screen that notifies the occurrence of an abnormal event according to the abnormality detection result from the abnormality detection process.

[0014] According to this, based on the color changes that appear in the camera's captured image Appropriate detection of abnormal events By configuring various settings regarding the processing conditions for issuing alerts, it is possible to achieve highly accurate anomaly detection and alerting.

[0017] Also, Second The invention relates to the server device, current The system is configured to issue an alert of a type corresponding to the time when the change was observed if a change exceeding a predetermined limit is detected between the color analysis result and the color analysis result at any point in the past.

[0018] According to this, appropriate types of alarms can be issued according to the circumstances of the abnormal event. In this case, alarms may be issued at multiple levels (for example, three levels: alert, warning, and caution) corresponding to different degrees of abnormal events depending on when the change is detected.

[0019] Also, The third The invention is configured such that the server device performs the color analysis process and the anomaly determination process for each of the multiple detection areas set up within the monitoring area.

[0020] According to this, abnormal events can be detected with high accuracy. For example, if the captured image includes the track (e.g., railway tracks) and its surroundings (e.g., the embankment next to the tracks), detection areas may be set for the track and its surroundings, respectively.

[0021] Also, The fourth The invention provides a configuration in which the terminal device displays a confirmation screen that includes a captured image of the monitoring area and a graph showing the progress of the color analysis results over time.

[0022] According to this, users can appropriately confirm the situation of abnormal events by comparing captured images of the monitoring area with the color analysis results.

[0023] Also, Fifth The invention provides a configuration in which the terminal device displays a monitoring screen that includes a map showing the route and the arrangement of the multiple cameras, and thumbnail images for each camera generated from images captured in the monitoring area.

[0024] According to this, users can comprehensively check the status of monitoring areas set at each point along the route.

[0025] Also, The sixth The invention relates to a railway line monitoring method in which a processor performs image analysis on images taken of a monitoring area set along the railway line to detect abnormal events occurring in the monitoring area and issue an alert, wherein a server device performs a color analysis process that sorts multiple measurement units by color system based on the color information for each of the multiple measurement units contained in the image taken by at least one camera that photographs the monitoring area, and the current color analysis result from the color analysis process The color analysis results are compared with those from multiple points in the past that occurred on different days. If a change exceeding a predetermined limit is observed in both, it is determined that an abnormal event has occurred. The system performs an abnormality detection process and an alerting process to notify the occurrence of an abnormal event. The terminal device displays a setting screen in advance for the processing conditions related to the color analysis process, the abnormality detection process, and the alerting process, and displays an alerting screen to notify the occurrence of an abnormal event according to the abnormality detection result from the abnormality detection process.

[0026] According to this, similar to the first invention, based on the color changes that appear in the image captured by the camera. Appropriate detection of abnormal events By configuring various settings regarding the processing conditions for issuing alerts, it is possible to achieve highly accurate anomaly detection and alerting.

[0027] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0028] Figure 1 is an overall diagram of the monitoring system according to this embodiment.

[0029] This system detects abnormal events occurring in a designated monitoring area based on images taken within that area and issues an alert. The system comprises a camera 1, an image analysis server 2 (server device), an image management server 3, and a monitoring terminal 4 (terminal device). Camera 1, image analysis server 2, image management server 3, and monitoring terminal 4 are connected via a network such as the Internet or a closed network.

[0030] Camera 1 captures images of the monitoring area and transmits the captured images in real time to image analysis server 2 and image management server 3.

[0031] The image analysis server 2 performs image analysis processing on real-time images captured by the camera 1 to detect abnormal events occurring within the monitoring area and notifies the user (monitoring officer) of the occurrence of the abnormal event using the monitoring terminal 4. In this embodiment, color analysis is performed on the image of the detection area within the captured image, and an abnormality determination is made based on the results of the color analysis.

[0032] Image management server 3 receives captured images periodically transmitted from camera 1, stores and manages those images.

[0033] Monitoring terminal 4 displays a monitoring screen (list view screen). The monitoring screen displays real-time images captured by camera 1. This allows the user (monitoring officer) to check the current status of the monitoring area. Monitoring terminal 4 also displays a notification screen. This allows the user to quickly recognize if an abnormal event has occurred in the monitoring area. Monitoring terminal 4 also displays a confirmation screen that displays the color analysis results in chronological order. This allows the user to check the progress of the color analysis results. Furthermore, monitoring terminal 4 allows the user (administrator) to perform operations related to setting conditions for processing performed by image analysis server 2 and image management server 3.

[0034] While the image analysis server 2 and image management server 3 can be configured to operate on-premises, i.e., near the monitoring area, the image management server 3 and image analysis server 2 may also be operated in the cloud.

[0035] Furthermore, in this embodiment, various processes are performed in the image analysis server 2 and the image management server 3, but these processes may be performed on a single server. Also, the processes performed in the image analysis server 2 and the image management server 3 may be shared among multiple servers in a different combination than that of this embodiment.

[0036] Furthermore, in this embodiment, management operations are performed on the monitoring terminal 4, but a separate management terminal may be provided, and management operations (such as setting processing conditions) may be performed on that management terminal.

[0037] Next, we will explain the surveillance area captured by camera 1. Figure 2 is an explanatory diagram showing the image of the surveillance area captured by camera 1.

[0038] Camera 1 is a PTZ (pan, tilt, zoom) camera capable of pan, tilt, and zoom operation, allowing the shooting conditions (shooting angle and magnification) to be arbitrarily changed within its movable range. This enables one camera 1 to capture different areas. Therefore, by periodically switching the shooting conditions of camera 1, it is possible to detect abnormal events in multiple monitoring areas with a single camera 1.

[0039] In this embodiment, the shooting conditions of camera 1 are set as a preset position to capture two monitoring areas, a first and a second, and the shooting conditions of camera 1 are controlled to switch at predetermined time intervals (for example, every 5 minutes).

[0040] In the examples shown in Figures 2(A-1) and (A-2), the #1 preset position captures the track and its surroundings, specifically the tracks on which the train, as a moving object, travels and the embankment beside the tracks. The #2 preset position captures the track and its surroundings on the opposite side of the #1 preset position, specifically the tracks and the embankment beside the tracks.

[0041] In the examples shown in Figures 2(B-1) and (B-2), the #1 preset position captures the road on which the moving object travels, specifically the road on which the automobile travels, and the #2 preset position captures the area around the road, specifically the embankment beside the road.

[0042] Note that the example shown in Figure 2 is a case where camera 1 is a PTZ camera whose shooting angle can be changed, but it may also be a fixed camera 1 whose shooting angle cannot be changed. In this case, only an image taken at a single shooting angle will be shown, rather than images taken at multiple preset positions.

[0043] Next, we will explain the detection area and mask area set on the image captured by camera 1. Figure 3 is an explanatory diagram showing the detection area and mask area set on the image captured by camera 1.

[0044] In this embodiment, detection areas subject to anomaly detection processing and mask areas excluded from anomaly detection processing are set on the image captured by camera 1. Note that if the entire captured image is to be subject to anomaly detection processing, the setting of detection areas and mask areas can be omitted. Alternatively, only the detection area may be set, and the setting of the mask area may be omitted.

[0045] Specifically, as shown in Figures 3(A-1) and (A-2), in images taken targeting the travel path (railway) of a moving object and its surroundings (embankment next to the railway), the area around the travel path (embankment next to the railway) is set as the first detection area, and the travel path (railway) is set as the second detection area.

[0046] Furthermore, as shown in Figure 3(B-1), in the first preset position image targeting the path (road) of a moving object, the path is set as the detection area, and the area surrounding the path (the slope beside the road) is set as the mask area. On the other hand, as shown in Figure 3(B-2), in the second preset position image targeting the area surrounding the path, the area surrounding the path is set as the detection area, and the path is set as the mask area.

[0047] Next, we will explain the processing performed by the image analysis server 2. Figure 4 is a block diagram illustrating the overview of the processing performed by the image analysis server 2.

[0048] Image analysis server 2 performs image analysis on real-time images transmitted from camera 1. In the image analysis process, first, pixel information of the detection area is extracted from the captured image, and based on that pixel information, the color components of each measurement unit in the detection area are measured to obtain color information for each measurement unit (color component measurement process). The measurement unit is either a single dot (pixel) or a grid (block) containing multiple dots.

[0049] Next, the image analysis server 2 performs a process (color analysis) to sort multiple measurement units into color systems based on the color information of each measurement unit. In this embodiment, the color system percentage is obtained as a result of the color analysis. The color system percentage represents the proportion (area ratio) occupied by each color system. The color analysis results obtained in the color analysis process are registered in the database. By sorting by color system in this way, the effects of temporary environmental changes such as sunlight and weather can be minimized.

[0050] Furthermore, the image analysis server 2 performs an abnormality detection process (anomaly detection process) by comparing the current color analysis result obtained from the color analysis process with the color analysis result from a predetermined point in the past registered in the database. If it is determined that an abnormality has occurred, the monitoring terminal 4 performs a process (notification process) to notify the user of the occurrence of the abnormality. At this time, a notification screen containing notification information corresponding to the abnormality that occurred is displayed on the monitoring terminal 4.

[0051] Furthermore, the image analysis server 2 performs a process to visualize the color analysis results (analysis result visualization process) in order to present the color analysis results obtained during the color analysis process to the user in an easy-to-understand manner. In this embodiment, a color system ratio graph (see Figure 5) and a heatmap superimposed image (see Figure 6) are generated. The color system ratio graph shows the progress of the color analysis results (color system ratios) over time. The heatmap superimposed image is a heatmap representing the distribution of color systems as a result of the color analysis, superimposed on the image captured by camera 1. The color system ratio graph and the heatmap superimposed image are displayed on the screen of the monitoring terminal 4.

[0052] Next, we will explain the color scheme percentage graph generated by the image analysis server 2. Figure 5 is an explanatory diagram showing the color scheme percentage graph.

[0053] Image analysis server 2 generates a color system percentage graph. This graph shows the trend of color analysis results (color system percentages) over time. The color system percentage represents the proportion (area ratio) that each color system occupies within the detection area. In the color system percentage graph, the horizontal axis represents time (period), and the vertical axis represents the percentage. The color system percentage graph is a so-called 100% stacked area graph, with the percentage of each color system separated by line graphs.

[0054] Here, the example shown in Figure 5(A) is a case where a greening method has been applied to a slope. The example shown in Figure 5(B) is a case where a lot of concrete has been used, or a railway line where a lot of concrete has been used. The example shown in Figure 5(C) is a case where there is little concrete and soil and rocks are exposed, or a railway line where there is little concrete and soil and gravel are exposed.

[0055] In this embodiment, attributes (vegetation, snow, concrete, soil, etc.) are assigned to color systems. The attributes specify the type of object that appears in the captured image for each color system. For example, the green system is assigned the "vegetation" attribute, the white system is assigned the "snow" attribute, the brown system is assigned the "soil" attribute, and the gray system is assigned the "concrete" attribute.

[0056] For example, in the example shown in Figure 5(A), the proportion of color schemes representing "snow cover" (white) is high in February, and from March onwards, the proportion of color schemes representing "vegetation" (green) gradually increases. This indicates a state where weeds are overgrown and mowing is highly necessary. In this case, the abnormality detection process determines that an abnormal event has occurred, an alert is issued, and the user (monitoring officer) requests the dispatch of the department responsible for taking the necessary action on-site. In addition, for example, if the state changes from one where the proportion of color schemes representing "snow cover" (white) is high to one where the proportion of color schemes representing "vegetation" (green) suddenly appears partially, it is assumed to be a sign of an impending avalanche, and an alert is issued. Also, if the state changes from one where the proportion of color schemes representing "vegetation" (green) is high to one where the proportion of color schemes representing "fallen leaves" (yellow) is high, it is assumed that fallen leaves have accumulated and cleaning is necessary, and an alert is issued.

[0057] Next, we will explain the heatmap superimposed image generated by the image analysis server 2. Figure 6 is an explanatory diagram showing the heatmap superimposed image.

[0058] The image analysis server 2 generates a heatmap superimposed image that shows the distribution of color schemes in the image captured by camera 1. The heatmap superimposed image is created by superimposing a heatmap representing the color scheme at each position on the image captured by camera 1 onto the image captured by camera 1. In this embodiment, the image captured by camera 1 is divided into multiple blocks in a grid pattern, and the color scheme (representative color of each block) is acquired for each of the multiple blocks to generate a heatmap. Furthermore, the heatmap is superimposed on the image captured by camera 1 in a semi-transparent state to generate the heatmap superimposed image.

[0059] Next, we will describe the schematic configuration of the image analysis server 2, the image management server 3, and the monitoring terminal 4. Figure 7 is a block diagram showing the schematic configuration of the image analysis server 2, the image management server 3, and the monitoring terminal 4.

[0060] The image management server 3 comprises a communication unit 31, a storage unit 32, and a processor 33.

[0061] The communication unit 31 communicates with the camera 1, the image analysis server 2, and the monitoring terminal 4.

[0062] The memory unit 32 stores programs and other data executed by the processor 33. The memory unit 22 also stores captured images (camera images) received from the camera 1.

[0063] The processor 33 performs various processes by executing programs stored in the memory unit 32. In this embodiment, the processor 33 performs image management processing, etc.

[0064] In the image management process, the processor 33 stores and manages the captured images received from the camera 1 in the storage unit 32, along with additional information such as the date and time of capture. If the camera 1 has multiple preset positions set, i.e., if it is a PTZ camera that can change the shooting angle, captured images for each of the set preset positions are stored.

[0065] The image analysis server 2 comprises a communication unit 21, a storage unit 22, and a processor 23.

[0066] The communication unit 21 communicates with the camera 1, the image management server 3, and the monitoring terminal 4.

[0067] The memory unit 22 stores programs executed by the processor 33. The memory unit 22 also stores the color analysis results obtained from the color analysis process performed by the processor 23. The color analysis results should ideally be registered and managed in a database.

[0068] The processor 23 performs various processes by executing programs stored in the memory unit 22. In this embodiment, the processor 23 performs processes such as color component measurement, color analysis, anomaly detection, alarm generation, and analysis result visualization.

[0069] In the color component measurement process, the processor 23 extracts pixel information of the detection area from the captured image, measures the color components for each measurement unit in the detection area based on that pixel information, and obtains color information for each measurement unit. The measurement unit is either a single dot (pixel) or a grid (block) containing multiple dots.

[0070] In the color analysis process, the processor 23 sorts the multiple measurement units into color systems (hues) based on the color information of each measurement unit contained within the detection area, and obtains the color system for each of the multiple measurement units. The processor 23 also aggregates the color systems for each of the multiple measurement units contained within the detection area to obtain the percentage of each color system. The percentage of each color system represents the proportion (area ratio) that each color system occupies within the detection area.

[0071] In the abnormality detection process, the processor 23 determines whether or not an abnormal event has occurred based on the color analysis results (color system ratios) obtained in the color analysis process. In this embodiment, with respect to the target monitoring area, the current color analysis results obtained in the color analysis process are compared with the color analysis results from a predetermined point in the past registered in the database, and if a change exceeding a predetermined limit is observed in both, it is determined that an abnormal event has occurred.

[0072] In the notification process, if the processor 23 determines that an abnormal event has occurred in the abnormality detection process, it performs processing on the monitoring terminal 4 to notify the user of the abnormal event. Specifically, a notification screen containing notification information corresponding to the abnormal event that occurred is displayed on the monitoring terminal 4.

[0073] In the analysis result visualization process, processor 23 visualizes the color analysis results obtained in the color analysis process. Specifically, processor 23 generates a color family percentage graph (see Figure 5) and a heatmap superimposed image (see Figure 6). The color family percentage graph shows the trend of the color analysis results (color family percentages) over time. The heatmap superimposed image is a heatmap representing the distribution of color families as a result of the color analysis, superimposed on the image captured by camera 1. The color family percentage graph and the heatmap superimposed image are displayed on the screen of the monitoring terminal 4.

[0074] The monitoring terminal 4 comprises a display 41, an input device 42, a communication unit 43, a storage unit 44, and a processor 45.

[0075] The display 41 shows the screen. The input device 42 is a keyboard or mouse, and detects user input.

[0076] The communication unit 43 communicates with the image management server 3 and the image analysis server 2.

[0077] The memory unit 44 stores programs and other data executed by the processor 45.

[0078] The processor 45 performs various processes by executing programs stored in the memory unit 44. In this embodiment, the processor 45 performs display input control processing, etc.

[0079] In the display input control process, the processor 45 displays various screens, such as the list display screen 101 (see Figure 12), on the display 41, and acquires input operation information in response to the user's operation of the input device 42. The user's input operation information is sent to the image management server 3 and the image analysis server 2.

[0080] Next, we will explain the procedure for color component measurement processing performed on the image analysis server 2. Figure 8 is a flowchart showing the procedure for color component measurement processing.

[0081] In the image analysis server 2, the processor 23 first acquires the image (pixel information) of the detection area from the image captured by camera 1 (ST101).

[0082] Next, the processor 23 determines the color components for each measurement unit in the image (pixel information) of the detection area and obtains color information for each measurement unit (ST102). Here, RGB color information is obtained. The measurement unit is either a single dot (pixel) or a grid (block) containing multiple dots. The grid is, for example, a 5x5 dot area. The color information of the grid is the average of the color information of the multiple dots (pixels) contained in that grid.

[0083] Next, the processor 23 stores color information for each measurement unit (dot or grid) in the storage unit 22 (ST103). In the example shown in Figure 8, the color information (RGB values) for each dot, which is the measurement unit, is stored in the storage unit 22.

[0084] Next, we will explain the procedure for color analysis processing performed on the image analysis server 2. Figure 9 is a flowchart showing the procedure for color analysis processing.

[0085] In the image analysis server 2, first, the processor 23 acquires color information (RGB color information) for each measurement unit (dot or grid) stored in the storage unit 22 (ST201). Next, the processor 23 acquires the color system (hue) for each measurement unit based on the color information for each measurement unit (ST202). Then, the processor 23 calculates the percentage of each color system (the percentage (area ratio) occupied by each color system) and stores this percentage of each color system in the storage unit 22 (ST203).

[0086] Next, we will explain the procedures for anomaly detection and alarm activation performed by the image analysis server 2. Figure 10 is a flowchart showing the procedures for anomaly detection and alarm activation.

[0087] In the image analysis server 2, first, the processor 23 obtains the current color analysis results (system-specific percentages) for the detection area to be processed from the storage unit 22 (ST301). Next, the processor 23 obtains the color analysis results for the detection area to be processed from the storage unit 22 for a predetermined past point in time (ST302). Next, the processor 23 compares the system-specific percentages for the predetermined past point in time with the current color analysis results (ST303). If an importance level for a color system is set, the color analysis results are compared with respect to the color system for which an importance level has been set.

[0088] At this time, the processor 23 obtains the color analysis results for the first, second, and third time points and compares the color analysis results for each of the first, second, and third time points with the color analysis result for the current time point. Here, the second time point is a time point before the first time point, and the third time point is a time point before the second time point. Specifically, for example, the first time point is one week ago, the second time point is one month ago, and the third time point is six months ago. The past time points are not limited to these and may be the previous day, two days ago, two months ago, three months ago, four months ago, five months ago, etc.

[0089] Furthermore, past color analysis results that are compared with the current color analysis result are selected from those that fall within the same time period as the current color analysis result. In particular, it is desirable to compare the current color analysis result with color analysis results from multiple times within the same time period as the current time. For example, if the current time is 10:00, the color analysis results from 9:55, 10:00, and 10:05 will be compared with the current color analysis result. This helps to reduce the impact of temporary environmental changes such as changes in weather.

[0090] Next, the processor 23 determines whether a change exceeding a predetermined limit has been observed between the color analysis result at a predetermined point in the past and the color analysis result at the present time (ST304). If an importance level for a color system has been set, the processor determines whether a change exceeding a predetermined limit has been observed for the color system for which the importance level has been set.

[0091] In this case, if there is a difference of a predetermined ratio (e.g., 10%) or more in the system-specific proportions of the color analysis results, it is determined that a change has occurred. Furthermore, when comparing the current color analysis result with the color analysis results of multiple past time points included in the same time period as the current time, if the comparison between the past color analysis results and the current color analysis result yields both determinations that indicate a change and determinations that indicate no change, the determination that receives the majority of votes is adopted. In addition, the current color analysis result may be compared with the statistically processed color analysis results of each past time point.

[0092] Here, if the current color analysis results do not show a change exceeding a predetermined limit, (ST 304 If the result is No, no alert will be issued. On the other hand, if a change exceeding the predetermined limit is observed in the current color analysis results, (ST 304 If the answer is Yes, then the processor 23 determines whether or not a change exceeding a predetermined limit was observed for any point in the past (ST305).

[0093] If a change exceeding a predetermined limit is observed in the current color analysis result compared to the color analysis result at the first time point (referred to as "the first time point" in ST305), the processor 23 issues an alarm with an alarm level (type of alarm) of "Alert" (ST306). Furthermore, if a change exceeding a predetermined limit is observed in the current color analysis result compared to the color analysis result at the second time point (referred to as "the second time point" in ST305), the processor 23 issues an alarm with an alarm level of "Warning" (ST307). Furthermore, if a change exceeding a predetermined limit is observed in the current color analysis result compared to the color analysis result at the third time point (referred to as "the third time point" in ST305), the processor 23 issues an alarm with an alarm level of "Caution" (ST308).

[0094] Furthermore, if an importance level is set for a color scheme, an alert will be issued based on the alarm level (warning, caution, caution) set for that importance level. For example, if the importance level for the green scheme is set to "warning" for a specific preset position of camera 1 in #1 (PTZ A), an alert will be issued with the alarm level set to "warning" when the proportion (area ratio) of the green scheme exceeds a predetermined threshold (e.g., 10%).

[0095] Here, if a significant change is observed in the current color analysis results compared to the analysis results from a point in time that is closer to the present, a high-level warning is issued, assuming a high level of urgency. Specifically, if a significant change is observed in the current color analysis results compared to the analysis results from the first point in time (e.g., one week ago), a high-level "caution" warning is issued. On the other hand, if a significant change is observed in the current color analysis results compared to the analysis results from a third point in time (e.g., six months ago), a low-level "warning" warning is issued. Furthermore, if a significant change is observed in the current color analysis results compared to the analysis results from a second point in time (e.g., one month ago), which is between the first and third points in time, a mid-level "caution" warning is issued. In addition, if current weather information indicates that there is a weather anomaly (typhoon, heavy rain, heavy snow, etc.), the color analysis processing for multiple past points in time may be temporarily suspended, and the color analysis processing may be resumed after the weather improves.

[0096] Next, we will explain the process when relationships are established between multiple cameras 1. Figure 11 is a block diagram illustrating the overview of the process when relationships are established between multiple cameras 1.

[0097] For example, if there is a continuous series of slopes with the same structure, the monitoring areas captured by multiple cameras 1 installed there will be similar to each other. Therefore, by comparing the current color analysis results based on the images captured by multiple cameras 1, it is possible to detect abnormal events occurring in some monitoring areas. That is, when comparing the current color analysis results based on the images captured by multiple cameras 1, normally no changes exceeding a predetermined limit are observed between the two. On the other hand, if some abnormal event occurs in some monitoring areas, when comparing the current color analysis results based on the images captured by multiple cameras 1, changes exceeding a predetermined limit will be observed between the two, and the abnormal event will be detected.

[0098] Therefore, in this embodiment, when the monitoring areas captured by multiple cameras 1 are similar to each other, the multiple cameras 1 are set to have a relationship, and an anomaly detection process is performed by comparing the current color analysis results based on the multiple cameras 1 that have that relationship.

[0099] For example, if a relationship is established between the first preset position of camera 1 #1 (PTZ A) and the first preset position of camera 1 #2 (PTZ B), the color analysis results based on the image captured by camera 1 #1 (PTZ A) at the first preset position are compared with the color analysis results based on the image captured by camera 1 #2 (PTZ B) at the first preset position.

[0100] Furthermore, the current color analysis results from multiple related cameras 1 may be compared with each other, as well as past color analysis results from multiple related cameras 1 at the same point in time. Anomaly detection processing may then be performed by comparing the comparison results of the current color analysis results with the comparison results of the past color analysis results.

[0101] Next, we will explain the list display screen 101 in setting mode, which is displayed on the monitoring terminal 4. Figure 12 is an explanatory diagram showing the list display screen 101 when the settings related to anomaly detection are not yet completed. Figure 13 is an explanatory diagram showing the list display screen 101 while the operation related to anomaly detection settings is being performed.

[0102] As shown in Figure 12, the list display screen 101 is provided with a mode selection unit 102. The mode selection unit 102 allows the user to select one of two modes: the setting mode, the list display mode, or the map display mode, which are operation modes. In the list display screen 101 shown in Figure 12, the setting mode is selected. If the user selects the list display mode, the screen transitions to the list display screen 101 in operation mode (see Figure 17). If the user selects the map display mode, the screen transitions to the map display screen 221 (see Figure 24).

[0103] Furthermore, the list display screen 101 is provided with a monitoring area list display section 103. In the monitoring area list display section 103, the user can select the target monitoring area (route). In this example, multiple monitoring areas are set, namely road cameras #1 to #5.

[0104] Furthermore, the list display screen 101 is equipped with a camera image list display unit 104. The camera image list display unit 104 displays the images (camera images) captured by each of the multiple cameras 1 installed in the monitoring area selected in the monitoring area list display unit 103. This allows the user (monitoring officer) to visually check the images captured by all cameras 1 that are photographing the target monitoring area and confirm the current status of the monitoring area by each camera 1.

[0105] Furthermore, the camera image list display unit 104 is provided with a camera name display field 105 and a setting details display field 106. The camera name display field 105 displays the name of camera 1. The setting details display field 106 displays the name of the preset position. In the example shown in Figure 12, cameras #1 to #3 are PTZ cameras whose shooting angle can be changed. Cameras #1 to #3 have two preset positions, #1 and #2. Cameras #4 and #5 are fixed-type cameras whose shooting angle cannot be changed. In this case, the name of the preset position is not displayed in the setting details display field 106.

[0106] Furthermore, the list display screen 101 is provided with a time display section 108. The time display section 108 displays the current time and the image time. The image time represents the time the captured image was taken, as displayed on the camera image list display section 104. When multiple preset positions are set, shooting for each preset position is performed alternately, so the shooting times of the captured images differ between different preset positions. That is, the shooting time of the image taken for the currently being shot preset position matches the current time, but the image taken for a preset position that is not currently being shot was taken during the previous shooting period, and its shooting time differs from the current time. For this reason, the image time is displayed in addition to the current time.

[0107] Furthermore, the list display screen 101 is provided with a playback operation unit 111. The playback operation unit 111 is provided with a playback time display unit 112, first, second, and third skip forward buttons 113-115, and first, second, and third skip backward buttons 116-118. The playback time display unit 112 displays the shooting time of the captured image displayed on the camera image list display unit 104. When the user operates the first, second, and third skip forward buttons 113-115, the playback time can be advanced by the respective forward time (15 minutes, 1 hour, and 1 day). When the user operates the first, second, and third skip backward buttons 116-118, the playback time can be reversed by the respective backward time (15 minutes, 1 hour, and 1 day).

[0108] As shown in Figure 13, the camera image list display section 104 of the list display screen 101 allows the user to set whether or not each camera 1 is subject to the anomaly detection processing performed by the image analysis server 2. Specifically, when the user performs a predetermined operation (right-click) on the camera name display field 105, a menu 109 is displayed, and the user can select either "subject to processing" or "exclude from processing" in the menu 109. If "subject to processing" is selected, the corresponding camera 1 is set as subject to the anomaly detection processing, and if "exclude from processing" is selected, the corresponding camera 1 is excluded from the anomaly detection processing. In the example shown in Figure 13, the preset positions #1 and #2 of camera 1 in #1 (PTZ A) and the preset position #2 of camera 1 in #2 (PTZ B) are set as subjects to processing, while the other cameras 1 are set as excludes from processing.

[0109] Furthermore, when a user selects a target camera 1 by operating one of the camera name display fields 105 for each camera 1 in the camera image list display unit 104, the system transitions to a camera settings screen 121 (see Figure 14) targeting the selected camera 1.

[0110] Next, we will explain the camera settings screen 121 displayed on the monitoring terminal 4. Figure 14 is an explanatory diagram showing the camera settings screen 121.

[0111] The camera settings screen 121 is equipped with a camera attribute information display unit 122. The camera attribute information display unit 122 displays the attribute information of the target camera 1, specifically the installation date and time, name, and IP address of camera 1. This allows the user to check the attribute information of camera 1.

[0112] Furthermore, the camera settings screen 121 is provided with a camera image display unit 123. The camera image display unit 123 displays the image (camera image) captured by the target camera 1. In the example shown in Figure 14, images captured using the preset positions #1 and #2 are displayed. This allows the user to visually check the shooting status of the target camera 1 by looking at the image captured by camera 1.

[0113] Furthermore, the camera settings screen 121 is equipped with a camera settings information display unit 124. The camera settings information display unit 124 displays setting information related to the shooting conditions (preset positions) of the target camera 1, specifically the pan, tilt, and zoom settings of camera 1. In this example, setting information related to the shooting conditions for preset positions #1 and #2 is displayed. The user can also change the pan, tilt, and zoom settings on the camera settings information display unit 124. Note that if camera 1 is a fixed type camera whose shooting angle cannot be changed, rather than a PTZ camera whose shooting angle can be changed, there are no settings related to shooting conditions (preset positions).

[0114] Furthermore, the camera settings screen 121 is provided with a "Get Camera Information" button 125. When the user operates the "Get Camera Information" button 125, the image analysis server 2 retrieves the attribute information of camera 1 from the storage unit 22 and displays it on the camera attribute information display unit 122, retrieves the captured image from camera 1 and displays it on the camera image display unit 123, and retrieves the setting information of camera 1 from the storage unit 22 and displays it on the camera setting information display unit 124.

[0115] Furthermore, the camera settings screen 121 includes a "Camera Settings" tab 126, an "Area Settings" tab 127, and a "Color Analysis Settings" tab 128. In the camera settings screen 121, the "Camera Settings" tab 126 is selected. When the user operates the "Area Settings" tab 127, the screen transitions to the Area Settings screen 131 (see Figure 15). Also, when the user operates the "Color Analysis Settings" tab 128, the screen transitions to the Color Analysis Settings screen 151 (see Figure 16).

[0116] Next, we will explain the area setting screen 131 displayed on the monitoring terminal 4. Figure 15 is an explanatory diagram showing the area setting screen 131.

[0117] The area setting screen 131 is provided with a camera image display unit 132. The camera image display unit 132 displays the image captured by camera 1 (camera image). When the detection area and mask area are set, the detection area and mask area are displayed on the captured image in the camera image display unit 132 (see Figure 3). It is preferable that if multiple detection areas are set, each be displayed in a different color (for example, orange, purple, etc.). Similarly, if both a detection area and a mask area are set, it is preferable that each be displayed in a different color.

[0118] Furthermore, the area setting screen 131 is provided with a detection area setting section 133. In the detection area setting section 133, the user can specify the detection area to be subject to anomaly detection processing for each of the images captured using the preset positions #1 and #2. The detection area setting section 133 is provided with a detection area selection field 141, a coordinate display field 142, an "Add" button 143, a "Delete" button 144, and a "Set" button 145.

[0119] When the user operates the detection area selection field 141, a pull-down menu is displayed, allowing the user to select a detection area from the pull-down menu. At this time, the system transitions to the detection area input mode, and the user can use the input device 42 to specify the positions of multiple endpoints of the polygon representing the detection area on the captured image displayed on the camera image display unit 132 (see Figure 3).

[0120] The coordinate display area 142 shows the coordinates of multiple endpoints of the polygon representing the detection area.

[0121] When a user presses the "Add" button 143, they can add a new detection area. When a user presses the "Settings" button 144, the detection area specified by the user is set. When a user presses the "Delete" button 145, they can delete a previously set detection area.

[0122] Furthermore, the area setting screen 131 is provided with a mask area setting section 134. In the mask area setting section 134, the user can specify mask areas to be excluded from the abnormality detection process for each of the images captured using the preset positions #1 and #2. The mask area setting section 134 is provided with a mask area selection field 146, a coordinate display field 147, an "Add" button 148, a "Delete" button 149, and a "Set" button 150.

[0123] When the user operates the mask area selection field 146, a pull-down menu is displayed, allowing the user to select a mask area from the pull-down menu. At this time, the system transitions to mask area input mode, and the user can use the input device 42 to specify the positions of multiple endpoints of the polygon representing the mask area on the captured image displayed on the camera image display unit 132 (see Figure 3).

[0124] The coordinate display area 147 shows the coordinates of multiple endpoints of the polygon representing the mask area.

[0125] When a user clicks the "Add" button 148, they can add a new mask area. When a user clicks the "Set" button 149, the mask area specified by the user is set. When a user clicks the "Delete" button 150, they can delete a set mask area.

[0126] Additionally, the area setting screen 131 is equipped with a "Register" button 135. When a user operates the "Register" button 135, the information entered on this screen is registered as setting information.

[0127] Next, we will explain the color analysis settings screen 151 displayed on the monitoring terminal 4. Figure 16 is an explanatory diagram showing the color analysis settings screen 151.

[0128] The color analysis settings screen 151 is equipped with a camera image display unit 152. The camera image display unit 152 displays the image captured by camera 1 (camera image). When the detection area and mask area are set, the detection area and mask area are displayed on the captured image in the camera image display unit 152 (see Figure 3).

[0129] Furthermore, the color analysis settings screen 151 is provided with a color system setting section 153. The color system setting section 153 includes a color system selection field 161, an attribute input field 162, and an importance specification field 163. In the color system selection field 161, the user can select the color system (hue) to be set using a pull-down menu. In the attribute input field 162, the user can input the attributes to be assigned to the color system to be set. The attributes specify the type of object that appeared in the captured image for each color system. In the importance specification field 163, the user can specify the importance to be assigned to the color system to be set. The importance can be specified in three stages, for example, caution, warning, and attention.

[0130] Furthermore, the color analysis settings screen 151 is provided with a relationship setting section 154. The relationship setting section 154 includes a camera specification field 164, a preset position specification field 165, and a color system specification field 166. In the camera specification field 164, the user can specify another camera 1 that is related to the camera 1 to be set. In the preset position specification field 165, if a preset position is set for the camera 1 to be set, the user can specify a preset position of another camera 1 that is related to that preset position. In the color system specification field 166, the user can specify a color system to focus on in the anomaly detection process.

[0131] In the example shown in Figure 16, for each of the preset positions #1 and #2 in camera 1 of camera #1 (PTZ A) that is the target of the settings, the preset positions #1 and #2 in camera 1 of camera #2 (PTZ B) are specified as other related cameras 1. In addition, here, the color system representing vegetation (green system) is specified as the color system to be focused on in the anomaly detection process.

[0132] Additionally, the color analysis settings screen 151 includes a "Register" button 155. When the user operates the "Register" button 155, the information entered on this screen is registered as setting information.

[0133] Next, we will explain the list display screen 101 shown on the monitoring terminal 4 when the operation mode is functioning normally. Figure 17 is an explanatory diagram showing the list display screen 101 when the operation mode is functioning normally.

[0134] The list display screen 101 in operation mode is displayed when the user selects the list display mode as the operation mode in the mode selection unit 102.

[0135] In the list display screen 101 in operation mode, the setting details display field 106 of camera 1 that has been set as the processing target (target for abnormality detection processing) is highlighted. For example, the background of the setting details display field 106 is displayed in gray. In the example shown in Figure 17, the preset positions #1 and #2 of camera 1 of #1 (PTZ A) and the preset position #2 of camera 1 of #2 (PTZ B) are set as processing targets, and their setting details display fields 106 are highlighted.

[0136] Next, we will explain the notification screen 171 displayed on the monitoring terminal 4 and the list display screen 101 for abnormal situations in operation mode. Figure 18 is an explanatory diagram showing the notification screen 171 and the list display screen 101 for abnormal situations in operation mode.

[0137] When an abnormal event is detected, notification screen 171 is displayed as a pop-up on the abnormal event list display screen 101 in operation mode.

[0138] On the notification screen 171, the name of camera 1 that captured the monitoring area where the abnormal event was detected (PTZ A), a description of the monitoring area where the abnormal event was detected (preset #1), and text indicating that an abnormal event has occurred are displayed.

[0139] Furthermore, the notification screen 171 is equipped with a "Confirm" button 172. When the user operates the "Confirm" button 172, the notification screen 171 closes and transitions to the abnormal situation list display screen 101.

[0140] In the operation mode, the abnormal event list display screen 101 highlights the camera name display field 105, which represents the camera 1 that took the image in which the abnormal event was detected, and the setting details display field 106, which shows the preset position at the time the image in which the abnormal event was detected was taken. For example, the background of the camera name display field 105 and the setting details display field 106 is displayed in red. In the operation mode, the abnormal event list display screen 101 allows the user to visually check the current status of the monitoring area where the abnormal event was detected by looking at the images displayed on the camera image list display field 104.

[0141] In the abnormal situation display screen 101 in operation mode, if the user operates the camera name display field 105 or the setting details display field 106, the system transitions to the status confirmation screen 181 (see Figure 19).

[0142] Next, we will explain the status confirmation screen 181 displayed on the monitoring terminal 4. Figure 19 is an explanatory diagram showing the status confirmation screen 181.

[0143] The status confirmation screen 181 is equipped with a camera image display unit 182. The camera image display unit 182 displays the image captured by camera 1 at the time of the alarm. In addition, the detection area where the abnormal event occurred is displayed in a predetermined color (for example, red) on the captured image (see Figure 3). This allows the user (monitoring officer) to visually check the status of the monitoring area at the time of the alarm by looking at the captured image, and in particular, the user can quickly confirm the occurrence of the abnormal event by visually checking the image of the detection area.

[0144] Furthermore, the status confirmation screen 181 is equipped with an analysis results display unit 183. The analysis results display unit 183 displays a graph showing the percentage of each color system as a result of the color analysis. The graph showing the percentage of each color system represents the trend of the percentage of each color system over time (see Figure 5).

[0145] The status confirmation screen 181 also includes a "No Problems" button 184 and a "Request Dispatch" button 185. When the user visually checks the captured image and confirms that there are no problems, they operate the "No Problems" button 184. In this case, the user returns to the operation mode list display screen 101 (see Figure 17). Alternatively, when the user visually checks the captured image and determines that on-site action is necessary, they operate the "Request Dispatch" button 185. In this case, a notification requesting dispatch is sent to the image analysis server 2 to the department responsible for taking the necessary action on-site.

[0146] Next, we will explain the time-series confirmation screens 191, 201, and 211 displayed on the monitoring terminal 4. Figures 20, 21, 22, and 23 are explanatory diagrams showing the time-series confirmation screens 191, 201, and 211.

[0147] On the operation mode list display screen 101 (see Figure 17), selecting the target camera 1, for example, by operating the camera name display field 105 on the camera image list display unit 104, transitions to the time series confirmation screen 191 shown in Figure 18.

[0148] The time-series confirmation screen 191 shown in Figure 18 is provided with a display period specification section 192. In the display period specification section 192, the user can select one of the following as the length of the display period: year, month, week, or day. In addition, the user can input the start date and time of the display period in the display period specification section 192.

[0149] Furthermore, the time-series confirmation screen 191 is provided with a camera selection section 193. In the camera selection section 193, the user can select the target camera 1.

[0150] Furthermore, the time-series confirmation screen 191 is provided with a display mode selection section 194. The display mode selection section 194 allows the user to select one of the following display modes: "Images only," "Graph," or "Heatmap." In the time-series confirmation screen 191 shown in Figure 20, the "Images only" display mode is selected.

[0151] Furthermore, the time-series confirmation screen 191 is provided with a camera image display unit 195. The camera image display unit 195 displays the captured images (camera images) from camera 1. In the example shown in Figure 20, camera 1 of #1 (PTZ A) is selected in the camera selection unit 193, and the captured images of camera 1 of #1 at the #1 and #2 preset positions are displayed in the camera image display unit 195.

[0152] Furthermore, the time-series confirmation screen 191 is equipped with a playback operation unit 111, similar to the list display screen 101 (see Figure 12). By operating the playback operation unit 111, the user can display images (camera images) taken at any date and time included in the display period specified by the display period specification unit 192 on the camera image display unit 195.

[0153] Furthermore, the time-series confirmation screen 191 is provided with a "Display" button 196 and a "Download" button 197. When the user operates the "Display" button 196, the captured image (camera image) is displayed on the camera image display unit 195 based on the display conditions specified by the user in the display period specification unit 192, the camera selection unit 193, and the display mode selection unit 194. When the user operates the "Download" button 197, a download is performed in which the captured image based on the display conditions specified by the user is sent from the image management server 3 to the monitoring terminal 4 and stored on the monitoring terminal 4.

[0154] The time-series confirmation screen 201 shown in Figures 21 and 22 is displayed when the "Graph" display mode is selected in the display mode selection unit 194.

[0155] The time-series confirmation screen 201 is equipped with a camera image display unit 195 and a color analysis result display unit 202. The camera image display unit 195 displays the image captured by camera 1, similar to the time-series confirmation screen 191 (see Figure 20). The color analysis result display unit 202 displays a color system percentage graph as the color analysis result (see Figure 5).

[0156] Furthermore, the color analysis result display unit 202 displays a mark 203, which represents the shooting time (playback time) of the captured image displayed on the camera image display unit 195, superimposed on the color system percentage graph. This allows the user to easily check the color system percentages as a result of the color analysis for the displayed captured image.

[0157] Furthermore, the color analysis result display unit 202 displays the color analysis results (color system percentage graph) for each detection area when multiple detection areas are set in the captured image. In the example shown in Figure 21, detection areas #1 and #2 are set for each captured image of the preset positions #1 and #2, respectively (see Figure 3), and for each of the preset positions #1 and #2, a color system percentage graph for each detection area of ​​#1 and #2 is displayed.

[0158] Furthermore, as shown in Figure 21, the camera selection unit 193 displays a "+" button if multiple preset positions are set for camera 1, that is, if it is a PTZ camera whose shooting angle can be changed. In the example shown in Figure 21, the "+" button is displayed in the display fields for camera 1 of #1 (PTZ A), camera 1 of #2 (PTZ B), and camera 1 of #3 (PTZ C).

[0159] When the user operates the "+" button, the display area for Camera 1 expands, as shown in Figure 22, and the preset positions #1 and #2 are displayed. The user can select either preset position #1 or #2. In the example shown in Figure 20, preset position #2 for Camera 1 in position #1 and preset position #2 for Camera 1 in position #2 are selected. At this time, the "-" button is displayed instead of the "+" button. When the user operates the "-" button, the display area for Camera 1 collapses, returning to the state shown in Figure 21.

[0160] In the example shown in Figure 21, the camera image display unit 195 and the color analysis result display unit 202 display side-by-side the captured images and their color analysis results (color system ratio graphs) for the same camera 1 of preset positions #1 and #2. This allows the user to compare captured images and their color analysis results from different preset positions of a single camera 1.

[0161] On the other hand, in the example shown in Figure 22, the camera image display unit 195 and the color analysis result display unit 202 display side by side the captured image and its color analysis result (color system ratio graph) for the preset position #2 of camera 1 #1 and the preset position #2 of camera 1 #2. This allows the user to compare captured images and their color analysis results for specific preset positions of different cameras 1.

[0162] Furthermore, in the camera selection unit 193, the user can select a specific preset position of camera 1, which has multiple preset positions set, and a fixed camera 1, thereby allowing the user to compare the captured images and their color analysis results for one preset position of one camera 1 and the fixed camera 1.

[0163] Furthermore, Figures 21 and 22 show the case when the "Graph" display mode is selected in the display mode selection unit 194. However, the expansion of the display field for camera 1 in the camera selection unit 193 is not limited to the "Graph" display mode, but is the same for the "Image Only" display mode and the "Heatmap" display mode.

[0164] The time-series confirmation screen 211 shown in Figure 23 is displayed when the "heatmap" display mode is selected in the display mode selection unit 194.

[0165] The time-series confirmation screen 211 is equipped with a camera image display unit 195 and a color analysis result display unit 212. The camera image display unit 195 displays the image captured by camera 1, similar to the time-series confirmation screen 191 (see Figure 20). The color analysis result display unit 212 displays a heatmap superimposed image (see Figure 6). The heatmap superimposed image is a heatmap representing the distribution of color systems as a result of color analysis, superimposed on the image captured by camera 1. This allows the user to easily check the state of color systems in the monitoring area.

[0166] Next, we will explain the map display screen 221 that is displayed on the monitoring terminal 4. Figure 24 is an explanatory diagram showing the map display screen 221.

[0167] The map display screen 221 is provided with a mode selection unit 102 and a monitoring area list display unit 103, similar to the list display screen 101 (see Figure 12). The map display screen 221 is also provided with a map display unit 222.

[0168] In the map display unit 222, a thumbnail image 224 is displayed on the map 223 (route map).

[0169] Map 223 displays the routes (#1 to #5). Node markers 225 and camera markers 226 are drawn on each route. Node markers 225 represent facilities that serve as nodes (junctions) of the route. Specifically, node markers 225 represent junctions and interchanges if the route is a highway, and stations if the route is a railway. Camera markers 226 represent the placement of multiple cameras 1. Here, camera markers 226 are drawn on each route (#1 to #5) to correspond to each camera 1 installed on that route.

[0170] The thumbnail images 224 are reduced-size displays of images taken by each of the multiple cameras 1, and are displayed to correspond to each of the multiple camera marks 226. If multiple preset positions (preset positions #1, #2, etc.) are set for camera 1, the thumbnail images 224, which are reduced-size displays of images taken at each preset position, will be displayed alternately at predetermined intervals.

[0171] As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to create new embodiments by combining the components described in the above embodiments. [Industrial applicability]

[0172] The railway line monitoring system and method according to the present invention allow for various settings regarding processing conditions when detecting abnormal events and issuing alerts based on color changes appearing in images captured by a camera, and have the effect of achieving highly accurate abnormality detection and alerting. They are useful as railway line monitoring systems and methods that detect abnormal events occurring in a monitoring area by image analysis of images taken of a monitoring area set along a railway line and issue alerts. [Explanation of Symbols]

[0173] 1: Camera 2: Image analysis server (server device) 3: Image management server 4: Monitoring terminal (terminal device) 23: Processor 41: Display 42: Input Devices 45: Processor 101: List display screen (monitoring screen) 121: Camera settings screen 131: Area setting screen 151: Color analysis setting screen 171: Notification screen 191,201,211: Time series confirmation screen 221: Map display screen

Claims

1. A railway line monitoring system that detects abnormal events occurring in a designated monitoring area by image analysis of images taken of the monitoring area along the railway line and issues an alert, At least one camera to film the surveillance area, A server device that performs a color analysis process that sorts multiple measurement units into color systems based on the color information for each of the multiple measurement units contained in the image captured by the camera; an anomaly detection process that compares the current color analysis result obtained from the color analysis process with the color analysis results from multiple past points in time that are different from the present, and determines that an anomaly has occurred if a change exceeding a predetermined limit is found between the two; and an alerting process that notifies the occurrence of an anomaly. A railway line monitoring system characterized by comprising: a terminal device that displays a setting screen for processing conditions related to the color analysis process, the anomaly determination process, and the alarm generation process, and displays a notification screen that notifies of the occurrence of an abnormal event according to the anomaly determination result from the anomaly determination process.

2. The server device is The railway line monitoring system according to claim 1, characterized in that if a change exceeding a predetermined limit is observed between the current color analysis result and the color analysis result at any point in the past, an alert of a type corresponding to the time when the change was observed is issued.

3. The server device is The railway line monitoring system according to claim 1, characterized in that the color analysis process and the anomaly determination process are performed for each of the multiple detection areas set within the monitoring area.

4. The aforementioned terminal device is The railway line monitoring system according to claim 1, characterized in that it displays a confirmation screen including a captured image of the monitoring area and a graph showing the progress of the color analysis results over time.

5. The aforementioned terminal device is The railway line monitoring system according to claim 1, characterized in that it displays a monitoring screen including a map showing the route and the arrangement of a plurality of the cameras, and thumbnail images for each camera generated from images taken in the monitoring area.

6. A railway line monitoring method in which a processor performs image analysis on images taken of a monitoring area set along the railway line to detect abnormal events occurring in the monitoring area and issue an alert, The server device Based on the color information for each of the multiple measurement units contained in the image captured by at least one camera that photographs the monitoring area, a color analysis process is performed to sort the multiple measurement units by color system; an abnormality detection process is performed to compare the current color analysis result from the color analysis process with the color analysis results from multiple past points in time that are different from the present, and if a change exceeding a predetermined limit is found between the two, an abnormal event is determined to have occurred; and an alarm notification process is performed to notify the occurrence of the abnormal event. The terminal device, Prior to this, a setting screen for processing conditions related to the color analysis process, the anomaly detection process, and the alarm generation process is displayed on the terminal device, A railway line monitoring method characterized by displaying a notification screen that notifies of the occurrence of an abnormal event in accordance with the abnormality determination result obtained by the abnormality determination process.

Images