Dynamic event detection device, dynamic event detection method, and program

Abstract

This invention provides a dynamic event detection device, a dynamic event detection method, and a program that can improve the failure to detect dynamic events. [Solution] One embodiment of the dynamic event detection device of the present disclosure includes: an input receiving unit that receives video footage captured by a camera; a time-series change video generation unit that generates a time-series change video showing the time-series changes of a moving object in the video; a contrast enhancement unit that enhances the contrast of the motion region in which the moving object is shown on the video based on the time-series change video; and a dynamic event detection unit that detects whether or not a target dynamic event has occurred based on the contrast-enhanced video.

Description

Technical Field

[0001] The present disclosure relates to a dynamic event detection device, a dynamic event detection method, and a program.

Background Art

[0002] Conventionally, there has been known a technique for detecting dynamic events such as smoke and landslides by analyzing a video captured by a photographing device such as a surveillance camera.

[0003] For example, Non-Patent Document 1 discloses a technique for detecting smoke using a video recognition deep learning (DL) model that takes as inputs an RGB video and an inter-frame difference video showing the difference between frames of the RGB video.

[0004] Also, for example, Patent Document 1 discloses a technique for generating a motion video in which only motion is extracted by performing temporal and spatial filtering processing on a video, and performing object tracking within the motion video. According to the technique disclosed in Patent Document 1, components that cause false detection of motion can be removed as noise, and thus, for example, it is said that the motion of earth and rock that is the initial motion of a landslide and the motion of earth and rock that causes false detection can be distinguished and detected.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

Summary of the Invention

[0007] However, in the technology disclosed in Non-Patent Document 1 mentioned above, if the contrast of the video is low, the RGB video and the inter-frame difference video cannot complement each other's information. Therefore, in the technology disclosed in Non-Patent Document 1 mentioned above, there is a risk that dynamic events will not be detected.

[0008] Furthermore, the technology disclosed in Patent Document 1 generates motion videos by extracting only motion through temporal and spatial filtering. As a result, the technology disclosed in Patent Document 1 removes small movements of moving objects that should be detected as dynamic events, as well as backgrounds within the video that contribute to the detection of dynamic events, as noise, which may lead to the failure to detect dynamic events.

[0009] This disclosure is made in view of the above circumstances and aims to provide a dynamic event detection device, a dynamic event detection method, and a program that can improve the failure to detect dynamic events. [Means for solving the problem]

[0010] One embodiment of the dynamic event detection device of the present disclosure includes: an input receiving unit that receives video input captured by a camera; a time-series change video generation unit that generates a time-series change video showing the time-series change of a moving object in the video; a contrast enhancement unit that enhances the contrast of the motion region in which the moving object is shown on the video based on the time-series change video; and a dynamic event detection unit that detects whether or not a target dynamic event has occurred based on the contrast-enhanced video.

[0011] One aspect of the dynamic event detection method of the present disclosure includes: an input receiving step in which an input receiving unit receives input of a video captured by a shooting device; a time-series change video generation step in which a time-series change video generation unit generates a time-series change video showing the time-series change of a moving object in the video; a contrast enhancement step in which a contrast enhancement unit enhances the contrast of the motion region in which the moving object is shown on the video based on the time-series change video; and a dynamic event detection step in which a dynamic event detection unit detects whether or not a target dynamic event has occurred based on the contrast-enhanced video.

[0012] One aspect of the program of this disclosure causes a computer to execute the following steps: an input receiving step of receiving a video captured by a camera; a time-series change video generation step of generating a time-series change video showing the time-series changes of a moving object in the video; a contrast enhancement step of enhancing the contrast of the motion region in which the moving object is shown in the video based on the time-series change video; and a dynamic event detection step of detecting whether or not a target dynamic event has occurred based on the contrast-enhanced video. [Effects of the Invention]

[0013] This disclosure provides a dynamic event detection device, a dynamic event detection method, and a program that can improve the failure to detect dynamic events. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 is a block diagram showing an example of the configuration of a dynamic event detection system including the dynamic event detection device of this embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the functional configuration of the dynamic event detection device of this embodiment. [Figure 3] Figure 3 shows an example of a video received by the input receiving unit of this embodiment. [Figure 4]FIG. 4 is a diagram showing an example of a time-series change video generated by the time-series change video generation unit of the present embodiment. [Figure 5] FIG. 5 is a diagram showing an example of a contrast enhancement filter generated by the filter generation unit of the present embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a video in which the contrast of a moving region is enhanced by the contrast enhancement unit of the present embodiment. [Figure 7] FIG. 7 is a diagram showing a comparison example between a video before contrast enhancement and a video after contrast enhancement in the present embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of the processing performed by the dynamic event detection device of the present embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the dynamic event detection process of the present embodiment.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present disclosure (hereinafter simply referred to as "the present embodiment") will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the following embodiments. Also, the following embodiments and modification examples can be combined as appropriate.

[0016] Hereinafter, the present disclosure will be described by taking smoke as an example of a dynamic event. However, the dynamic event is not limited to smoke, and may be fog, steam, gas, or landslide (rockfall), etc.

[0017] First, the configuration of the dynamic event detection device of the present embodiment will be described.

[0018] Figure 1 is a block diagram showing an example of the configuration of a dynamic event detection system 1 including the dynamic event detection device 100 of this embodiment. As shown in Figure 1, the dynamic event detection system 1 comprises an imaging device 10 and a dynamic event detection device 100. The imaging device 10 and the dynamic event detection device 100 are connected via a network 2. Network 2 can be implemented by at least one of the following, for example, the Internet and a LAN (Local Area Network). Network 2 may be a wired network, a wireless network, or a mixture of wired and wireless networks.

[0019] The recording device 10 captures video of the detection target area where events to be detected as dynamic events may occur, and examples include cameras. In this embodiment, an RGB camera having an RGB sensor capable of recording visible light as RGB colors is used as an example of the recording device 10, but it is not limited to this. The recording device 10 may be fixedly installed indoors or outdoors as a surveillance camera, or it may be a handheld camcorder used by a photographer to record.

[0020] The dynamic event detection device 100 uses video (RGB video) captured by the camera 10 to detect whether or not a dynamic event has occurred within the detection target area described above. In this embodiment, as mentioned above, the dynamic event is smoke, and the dynamic event detection device 100 detects, for example, whether or not smoke is generated. This allows the dynamic event detection device 100 to detect whether or not a fire has occurred, etc.

[0021] The dynamic event detection device 100 may be implemented as software using an information processing device such as a PC (Personal Computer), tablet terminal, or smartphone. The information processing device can be any computer with a general hardware configuration that includes at least a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and HDD (Hard Disk Drive). Alternatively, the dynamic event detection device 100 may be implemented as hardware using hardwired circuits such as an IC (Integrated Circuit), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array).

[0022] Figure 2 is a block diagram showing an example of the functional configuration of the dynamic event detection device 100 of this embodiment. As shown in Figure 2, the dynamic event detection device 100 includes an input receiving unit 101, a time-series change video generation unit 103, a filter generation unit 105, a contrast enhancement unit 107, a dynamic event detection unit 111, and a false detection suppression unit 121. The dynamic event detection unit 111 includes a first deep learning processing unit 113, a second deep learning processing unit 115, a synthesis unit 117, and a third deep learning processing unit 119.

[0023] When each of the above-mentioned functional units is implemented as software, the CPU reads the dynamic event detection program from ROM or HDD into RAM and executes it. This ensures that each of the above-mentioned functional units is implemented on the computer of the dynamic event detection device 100. When each of the above-mentioned functional units is implemented as hardware, each of the above-mentioned functional units can be implemented using the above-mentioned hardwired circuits. Alternatively, each of the above-mentioned functional units may be implemented through the cooperation of software and hardware, or any one functional unit may be implemented through the cooperation of software and hardware.

[0024] It should be noted that the dynamic event detection device 100 of this embodiment does not necessarily require all of the above-described functional units to be part of the configuration; at least some of the functional units can be omitted. For example, the false detection suppression unit 121 may be omitted.

[0025] The input receiving unit 101 receives input of video captured by the shooting device 10. In this embodiment, the input receiving unit 101 receives input of RGB video captured by the shooting device 10 of the detection target area described above.

[0026] Figure 3 shows an example of a video 201 received by the input receiving unit 101 of this embodiment. In the video 201 shown in Figure 3, smoke 206 is shown as a moving object. A moving object can be any moving object, and includes dynamic events such as smoke that are the target of detection by the dynamic event detection device 100.

[0027] The time-series change video generation unit 103 generates a time-series change video showing the time-series changes of moving objects in the video received by the input reception unit 101. The time-series change video generation unit 103 generates the time-series change video by, for example, the difference between adjacent frames of the video, the background difference of the video, or optical flow. However, the method of generating the time-series change video is not limited to these, and the time-series change video generation unit 103 may use any generation method to generate the time-series change video.

[0028] When using the difference between adjacent frames of a video, the time-series change video generation unit 103 acquires the difference in the intensity of each channel between adjacent frames of the RGB video received by the input reception unit 101. As a result, the time-series change video generation unit 103 generates an inter-frame difference video corresponding to the time-series change video of this embodiment.

[0029] For example, the time-series video generation unit 103 sets the first frame of the time-series video as a grayscale image ((R,G,B)=(128,128,128)). The time-series video generation unit 103 also obtains an inter-frame difference video of 35 frames by taking the difference in the channel intensity of adjacent frames for 36 frames of RGB video. The time-series video generation unit 103 obtains a time-series video (36 frames, vertical size, horizontal size, RGB channels) by combining the time-series video and the inter-frame difference video in the time-series direction.

[0030] When using background subtraction for a video, the time-series video generation unit 103 applies background subtraction to the RGB video received by the input reception unit 101 to obtain a background mask and a foreground mask. The time-series video generation unit 103 applies the background mask to the RGB video to generate a video in which moving objects corresponding to the time-series video of this embodiment are extracted.

[0031] When using optical flow, the time-series change video generation unit 103 calculates the movement vector of each pixel of the RGB video received by the input reception unit 101 and colors it according to its direction and velocity. As a result, the time-series change video generation unit 103 generates a video that represents the movement of a moving object corresponding to the time-series change video of this embodiment.

[0032] Figure 4 shows an example of a time-series change video 211 generated by the time-series change video generation unit 103 of this embodiment. The time-series change video 211 shown in Figure 4 is a time-series change video of the video 201 shown in Figure 3, and the time-series change of the moving object, smoke 206 shown in Figure 3, is shown as smoke 216.

[0033] The filter generation unit 105 generates a contrast enhancement filter based on the time-series change video generated by the time-series change video generation unit 103, for enhancing the contrast of the motion region where moving objects are visible in the video received by the input reception unit 101.

[0034] Figure 5 shows an example of a contrast enhancement filter 221 generated by the filter generation unit 105 of this embodiment. The contrast enhancement filter 221 shown in Figure 5 was generated based on the time-series change video 211 shown in Figure 4, and is a filter whose intensity increases as the motion of the moving object increases. As shown by the filter bar 229 in Figure 5, the contrast enhancement filter 221 is represented such that the higher the intensity (larger the intensity value), the brighter the image, and the lower the intensity (smaller the intensity value), the darker the image. For this reason, in the contrast enhancement filter 221 shown in Figure 5, the area of ​​the moving object, smoke 226, is represented as bright, and the other areas are represented as dark.

[0035] The filter generation unit 105, for example, if the time-series video is a frame-by-frame difference video, averages a predetermined number of frame images of the time-series video and applies a smoothing filter in the spatial direction. As a result, the filter generation unit 105 generates a contrast enhancement filter as shown in Figure 5. Averaging a predetermined number of frame images means, for example, taking the average value of the pixel values ​​for a specified number of frames for each pixel. A Gaussian filter can be used as an example of a smoothing filter.

[0036] More specifically, time-series videos are heavily influenced by noise and local variations, and only include the motion region, with the background removed. Therefore, the filter generation unit 105 obtains the motion region for each input video to the video recognition deep learning model by absolute-valuing the positive / negative values ​​of the frame images of the time-series video and averaging them in the time-series direction. The filter generation unit 105 then applies a spatial smoothing filter (such as a Gaussian filter) to further blur the image (vertical size, horizontal size, RGB channels). This results in a contrast enhancement filter.

[0037] The filter generation unit 105 may generate a contrast enhancement filter for each channel of the RGB video. Alternatively, the filter generation unit 105 may integrate the generated contrast enhancement filters (each channel intensity) for each channel to obtain a single (single-channel) contrast enhancement filter. Specifically, the filter generation unit 105 can average the generated contrast enhancement filters for each channel in the channel direction (vertical size, horizontal size). Single-channel contrast enhancement filters have the effect of improving processing efficiency and robustness to noise. Note that single-channel contrast enhancement filters are unnecessary if color information needs to be preserved.

[0038] Furthermore, the filter generation unit 105 may normalize the pixel values ​​of the generated contrast enhancement filter to a value between a predetermined minimum value and a predetermined maximum value. This is because the range of input values ​​of the contrast enhancement filter needs to be controlled for input to a video recognition deep learning model. An example of the predetermined minimum value is a specified minimum value (e.g., 1.0), and an example of the predetermined maximum value is a specified maximum value (e.g., 1.5).

[0039] The contrast enhancement unit 107 enhances the contrast of the motion region in which moving objects are visible on the video received by the input receiving unit 101, based on the time-series changing video generated by the time-series changing video generation unit 103.

[0040] Specifically, the contrast enhancement unit 107 applies a contrast enhancement filter generated by the filter generation unit 105 to the video received by the input reception unit 101 to enhance the contrast of the motion regions in the video. This increases the contrast of the moving objects and background included in the motion regions compared to the video before the contrast enhancement filter was applied.

[0041] More specifically, the contrast enhancement unit 107 multiplies the pixel value of each frame of the RGB video received by the input reception unit 101 by the pixel value of the contrast enhancement filter generated by the filter generation unit 105, for each pixel. In this case, if the filter generation unit 105 has not performed averaging in the time series direction when generating the contrast enhancement filter, the contrast enhancement unit 107 applies a different contrast enhancement filter to each frame. Also, if the filter generation unit 105 has not performed single-channelization of the contrast enhancement filter, the contrast enhancement unit 107 applies the contrast enhancement filter to each RGB channel. If the filter generation unit 105 has performed single-channelization of the contrast enhancement filter, the contrast enhancement unit 107 applies the single-channelized contrast enhancement filter to each RGB channel. The RGB video after contrast enhancement has a value obtained by multiplying each pixel by the minimum to maximum value determined by the normalization process of the contrast enhancement filter. After this, if necessary, standardization (scaling of mean and variance) is performed as is normally done for input to a video recognition deep learning model.

[0042] Figure 6 shows an example of a video 231 in which the contrast of the motion region has been enhanced by the contrast enhancement unit 107 of this embodiment. Figure 7 shows a comparative example of a video 201 before contrast enhancement and a video 231 after contrast enhancement of this embodiment. Note that the video 201 before contrast enhancement shown in Figure 7 is the video 201 received by the input receiving unit 101 shown in Figure 3.

[0043] As is clear from Figures 6 and 7, in video 231 after contrast enhancement, the contrast of the moving smoke 236 is enhanced compared to the smoke 206 in video 201 before contrast enhancement.

[0044] The dynamic event detection unit 111 detects whether or not a target dynamic event has occurred based on the contrast-enhanced video enhanced by the contrast enhancement unit 107. For example, the dynamic event detection unit 111 detects and outputs the probability of a dynamic event occurring for each frame as an indicator of whether or not a dynamic event has occurred.

[0045] In this embodiment, the dynamic event detection unit 111 further detects whether or not a dynamic event has occurred based on the time-series change video generated by the time-series change video generation unit 103. Thus, in this embodiment, a so-called two-stream method is used as an example, where both the contrast-enhanced video and the time-series change video are processed as input, but the embodiment is not limited to this. For example, a so-called one-stream method may be used, where only the contrast-enhanced video is processed as input, without using the time-series change video as input.

[0046] In this implementation, we will explain using the example where the dynamic event detection unit 111 performs detection using deep learning (DL) processing. Specifically, the dynamic event detection unit 111 will perform detection using, for example, a video recognition deep learning model that has been machine-learned using deep learning on a neural network. Hereafter, the video recognition deep learning model may be referred to as the DL model. Also, the process of performing detection using the DL model may be referred to as DL processing.

[0047] The DL model is generated by machine learning using deep learning on a dataset, which is a collection of training data in which videos to be trained are assigned a ground truth label indicating whether or not a dynamic event occurred within the video. The dynamic event detection unit 111 inputs the contrast-enhanced video into such a DL model, and from the DL model, it obtains the probability (likelihood) that a dynamic event has occurred within the video. Based on this likelihood, the dynamic event detection unit 111 detects whether or not a dynamic event has occurred.

[0048] Furthermore, the dynamic event detection unit 111 may also detect frame images in which dynamic events occur within the video. In this case, the ground truth labels of the training data will indicate not only whether or not a dynamic event has occurred, but also, for example, the frame number that identifies the frame image in which the dynamic event has occurred. The DL model can be generated by machine learning on a dataset which is a collection of such training data.

[0049] Furthermore, the dynamic event detection unit 111 may also detect regions within a frame image where dynamic events are occurring. In this case, the correct labels of the training data will include, in addition to whether or not a dynamic event has occurred and the frame number, coordinate information indicating, for example, the region within the frame image where the dynamic event is occurring. The DL model can be generated by machine learning a dataset that is a collection of such training data.

[0050] Furthermore, the dynamic event detection unit 111 may use deep learning processing to detect whether or not a dynamic event has occurred and the region in which the dynamic event has occurred, and may also use an object detection algorithm different from the deep learning processing to detect the region in the frame image where the dynamic event has occurred. In this case, the dynamic event detection unit 111 performs DL processing using the DL model described in the example of further detecting frame images in which a dynamic event has occurred. The dynamic event detection unit 111 can then use an object (smoke) detection algorithm on the frame image detected by the DL processing to detect the region in the frame image where the dynamic event has occurred.

[0051] The following describes the first deep learning processing unit 113, the second deep learning processing unit 115, the synthesis unit 117, and the third deep learning processing unit 119 included in the dynamic event detection unit 111.

[0052] The first deep learning processing unit 113 applies the preceding DL processing to the contrast-enhanced video enhanced by the contrast enhancement unit 107, and outputs a first feature map, which is an intermediate output of the DL model. The first feature map may be, for example, a multidimensional vector representing the motion in the contrast-enhanced video.

[0053] The second deep learning processing unit 115 applies the DL processing from the previous stage to the time-series change video generated by the time-series change video generation unit 103, and outputs a second feature map, which is an intermediate output of the DL model. The first feature map may be, for example, a multidimensional vector representing the movement within the time-series change video.

[0054] The synthesis unit 117 combines the first feature map generated by the first deep learning processing unit 113 and the second feature map generated by the second deep learning processing unit 115. Through this process, the synthesis unit 117 generates a combined feature map.

[0055] The third deep learning processing unit 119 applies subsequent DL processing to the composite feature map generated by the synthesis unit 117 and outputs the dynamic event detection results, such as the probability that a dynamic event has occurred, which is the final output of the DL model.

[0056] The false detection suppression unit 121 performs control to suppress false detections by the dynamic event detection unit 111. The false detection suppression unit 121 suppresses false detections by the dynamic event detection unit 111 by, for example, score smoothing processing, detection likelihood threshold processing, and alarm suppression processing.

[0057] For example, with the DL model described above, the probability of a processing unit being present may increase due to instantaneous movements or pixel changes other than the dynamic event being detected, potentially leading to false detections. In this case, the false detection suppression unit 121 can suppress such false detections through score smoothing processing. Specifically, as part of the score smoothing process, the false detection suppression unit 121 smooths the time-series output probability, which is the detection result of the dynamic event detection unit 111, by backward moving average processing. By increasing the step width (stride width) and window size (kernel size) of the moving average processing, a greater false detection suppression effect can be obtained.

[0058] Furthermore, depending on the installation conditions of the imaging device 10, it may be more important to avoid false detections than to detect dynamic events early. In this case, the false detection suppression unit 121 can suppress false detections by raising the detection likelihood threshold and performing threshold processing through detection likelihood threshold processing. Specifically, the false detection suppression unit 121 applies threshold processing to the output probability, which is the detection result of the dynamic event detection unit 111, and rejects processing results with low probabilities.

[0059] Furthermore, for example, if the imaging device 10 is fixed in place, false alarm factors may occur repeatedly. In this case, the false detection suppression unit 121 can suppress subsequent false alarms by suppressing alarm suppression processing after the first alarm is triggered. Specifically, if the false detection suppression unit 121 triggers an alarm because the output probability, which is the detection result of the dynamic event detection unit 111, exceeds a threshold, it controls the system so that it does not trigger an alarm for a specified number of seconds even if the threshold is exceeded.

[0060] Next, the operation of the dynamic event detection device of this embodiment will be described.

[0061] Figure 8 is a flowchart showing an example of the processing performed by the dynamic event detection device 100 of this embodiment.

[0062] First, the input receiving unit 101 receives input of RGB video captured by the shooting device 10 (step S101).

[0063] Next, the time-series change video generation unit 103 generates a time-series change video showing the time-series changes of moving objects in the RGB video received by the input reception unit 101 (step S103).

[0064] Next, the filter generation unit 105 generates a contrast enhancement filter based on the time-series change video generated by the time-series change video generation unit 103, to enhance the contrast of the motion region where moving objects are visible on the RGB video received by the input reception unit 101 (step S105).

[0065] Next, the contrast enhancement unit 107 applies a contrast enhancement filter generated by the filter generation unit 105 to the RGB video received by the input reception unit 101 to enhance the contrast of the motion regions in the RGB video (step S107).

[0066] Next, the dynamic event detection unit 111 performs a dynamic event detection process to detect whether or not a target dynamic event has occurred based on the contrast-enhanced video enhanced by the contrast enhancement unit 107 (step S109). After that, the false detection suppression unit 121 performs control to suppress false detections by the dynamic event detection unit 111 as necessary.

[0067] Figure 9 is a flowchart showing an example of the dynamic event detection process in this embodiment.

[0068] First, the first deep learning processing unit 113 applies the preceding DL processing to the RGB video after contrast enhancement by the contrast enhancement unit 107, and generates a first feature map, which is an intermediate output of the DL model (step S201).

[0069] Next, the second deep learning processing unit 115 applies the DL processing from the previous stage to the time-series change video generated by the time-series change video generation unit 103, and generates a second feature map, which is an intermediate output of the DL model (step S203).

[0070] Next, the synthesis unit 117 synthesizes the first feature map generated by the first deep learning processing unit 113 and the second feature map generated by the second deep learning processing unit 115 (step S205).

[0071] Next, the third deep learning processing unit 119 applies subsequent DL processing to the synthesized feature map synthesized by the synthesis unit 117, estimates the probability of a dynamic event occurring, which is the final output of the DL model, and outputs it as a dynamic event detection result (step S207).

[0072] As described above, in this embodiment, a contrast enhancement filter obtained from a time-series video is applied to the video captured by the shooting device 10 to enhance the contrast of moving objects and the background, thereby detecting dynamic events. Therefore, according to this embodiment, dynamic events can be efficiently detected from the start of detection, taking into account time-series changes, and it is expected that the number of undetected dynamic events will be improved.

[0073] For example, in the technology disclosed in Non-Patent Document 1 mentioned above, if the contrast of the RGB video is low, the RGB video and the inter-frame difference video cannot complement each other's information, which may result in the failure to detect dynamic events. In contrast, in this embodiment, even if the contrast of the RGB video is low, the contrast of moving objects and the background is enhanced by the contrast enhancement filter. Therefore, according to this embodiment, it is possible to complement each other's information with the time-series changing video, and it is expected that the failure to detect dynamic events will be improved.

[0074] Furthermore, in the technology disclosed in Patent Document 1 mentioned above, small movements of moving objects that should be detected as dynamic events, as well as the background in the video that contributes to the detection of dynamic events, are removed as noise, which may result in the failure to detect dynamic events. In contrast, in this embodiment, the contrast of moving objects and the background is enhanced by a contrast enhancement filter, so that small movements of moving objects and the background in the video can be taken into consideration with good sensitivity when detecting dynamic events, and detection can be performed with attention paid to small movements of moving objects and the background in the video. For this reason, it is expected that the failure to detect dynamic events will be improved according to this embodiment.

[0075] Thus, according to this embodiment, an effect of improving undetected areas can be expected by emphasizing the target area within the RGB video. Furthermore, according to this embodiment, an effect of improving false detections can be expected by making areas other than the target area relatively less noticeable. Moreover, according to this embodiment, an effect of improving false detections can be expected by raising the detection threshold.

[0076] (modified version) Next, modifications of the above embodiment will be described. In each of the following modifications, the parts that differ from the above embodiment will be mainly described, and the parts that are the same as the above embodiment will not be described.

[0077] (Variation 1) In the above embodiment, the case where the dynamic event is smoke was used as an example. However, the dynamic event is not limited to this and may be fog, steam, gas, or landslide (rockfall), etc. Even in this case, as in the above embodiment, the risk of undetected dynamic events is reduced and the event can be detected accurately from its initial state.

[0078] (Modification 2) In the above embodiment, the dynamic event detection unit 111 was described using DL processing as an example, but it is not limited to this. For example, the dynamic event detection unit 111 may be implemented using a machine learning algorithm other than deep learning, such as a support vector machine. Alternatively, the dynamic event detection unit 111 may be implemented using a rule-based algorithm that does not use a machine learning algorithm. An example of detection using a rule-based algorithm is to extract the region where a dynamic event occurs from a motion-enhanced video image by edge detection and binarization, calculate a judgment score, and compare it with a reference value. The dynamic event detection unit 111 may also be implemented using a combination of a machine learning algorithm other than deep learning and a rule-based algorithm.

[0079] (program) The programs executed in the above embodiments and each of the above modified devices are provided as installable or executable files stored on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk (FD).

[0080] Furthermore, the programs executed in each of the above embodiments and modified devices may be stored on a computer connected to a network such as the Internet and provided by allowing download via the network. Alternatively, the programs executed in each of the above embodiments and modified devices may be provided or distributed via a network such as the Internet. Furthermore, the programs executed in each of the above embodiments and modified devices may be pre-installed in ROM or the like and provided.

[0081] The programs executed in the above embodiments and each of the above modified devices are configured as modules for realizing the above-described parts on a computer. In actual hardware, for example, the CPU reads the learning program from the HDD into RAM and executes it, thereby realizing the above-described parts on the computer.

[0082] The above embodiments and their respective modifications are merely examples of how this disclosure may be implemented, and they do not restrict the technical scope of this disclosure. Therefore, this disclosure can be implemented in various ways without departing from its essence or its main features. For example, the above embodiments and their respective modifications may be combined as appropriate on a component basis. Also, for example, some components may be removed from the total components in the above embodiments and their respective modifications. [Explanation of Symbols]

[0083] 1. Dynamic Event Detection System 2 Network 10. Imaging device 100 Dynamic Event Detection Device 101 Input Reception Section 103 Time-series video generation unit 105 Filter generation unit 107 Contrast Enhancement Section 111 Dynamic Event Detection Unit 113 First Deep Learning Processing Unit 115 Second Deep Learning Processing Unit 117 Synthesis section 119 Third Deep Learning Processing Unit 121 False detection suppression unit

Claims

1. An input receiving unit that receives video input captured by a shooting device, A time-series change video generation unit generates a time-series change video showing the time-series changes of a moving object shown in the aforementioned video, Based on the aforementioned time-series video, a contrast enhancement unit enhances the contrast of the motion region in which the moving object is visible on the video, A dynamic event detection unit detects whether or not a target dynamic event has occurred based on the video after contrast enhancement, A dynamic event detection device equipped with the following features.

2. The system further comprises a filter generation unit that generates a contrast enhancement filter for enhancing the contrast of the motion region based on the aforementioned time-series video, The contrast enhancement unit applies the contrast enhancement filter to the video to enhance the contrast of the motion region. The dynamic event detection device according to claim 1.

3. The aforementioned contrast enhancement filter is a filter whose intensity increases as the motion of the moving object increases. The dynamic event detection device according to claim 2.

4. The filter generation unit averages a predetermined number of frame images of the time-series video, applies a smoothing filter in the spatial direction, and generates the contrast enhancement filter. The dynamic event detection device according to claim 3.

5. The filter generation unit normalizes the pixel values ​​of the generated contrast enhancement filter to values ​​between a predetermined minimum value and a predetermined maximum value. The dynamic event detection device according to claim 4.

6. The filter generation unit generates or generates and integrates the contrast enhancement filters for each channel of the video. The dynamic event detection device according to claim 2.

7. The dynamic event detection unit further detects a frame image in which the dynamic event occurs within the video. The dynamic event detection device according to claim 1.

8. The dynamic event detection unit further detects the region in the frame image where the dynamic event is occurring. The dynamic event detection device according to claim 7.

9. The dynamic event detection unit further detects whether or not the dynamic event has occurred based on the time-series video. The dynamic event detection device according to claim 1.

10. The dynamic event detection unit performs detection using deep learning processing. A dynamic event detection device according to claim 1, 7, 8, or 9.

11. The dynamic event detection unit uses deep learning processing to detect whether or not a dynamic event has occurred and the frame image in which the dynamic event has occurred, and uses an object detection algorithm different from the deep learning processing to detect the region in the frame image where the dynamic event has occurred. The dynamic event detection device according to claim 8.

12. The time-series video generation unit generates the time-series video by the difference between adjacent frames of the video, the background difference for the video, or optical flow. The dynamic event detection device according to claim 1.

13. The aforementioned imaging device is a camera that is fixedly positioned indoors or outdoors. The dynamic event detection device according to claim 1.

14. The input receiving unit includes an input receiving step that receives video input captured by the shooting device, The time-series change video generation unit generates a time-series change video that shows the time-series changes of moving objects in the video, The contrast enhancement unit performs a contrast enhancement step, which enhances the contrast of the motion region in which the moving object is shown on the video, based on the time-series video. A dynamic event detection step in which a dynamic event detection unit detects whether or not a target dynamic event has occurred based on the video after contrast enhancement, A dynamic event detection method that includes this.

15. An input receiving step that receives video input captured by a shooting device, A time-series change video generation step generates a time-series change video showing the time-series changes of a moving object shown in the aforementioned video, Based on the aforementioned time-series video, a contrast enhancement step is performed to enhance the contrast of the motion region in which the moving object is visible on the video, A dynamic event detection step that detects whether or not a target dynamic event has occurred based on the video after contrast enhancement, A program that causes a computer to execute something.

Images