Face stability evaluation device, control method, and program

Abstract

To provide a newly developed technology for stability evaluation on a working face.SOLUTION: A stability evaluation device 2000 on a working face acquires sound data 22 acquired through recording sound around the working face 10 and video data 32 acquiring through imaging the working face 10. The stability evaluation device 2000 on a working face identifies a type of sound representing by the sound data 22. The stability evaluation device 2000 on a working face calculates collapsed area of collapsed region on the working face 10 using the video data 32. The stability evaluation device 2000 on a working face determines stability of the working face 10 based on the identified type of the sound and the calculated collapsed area.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , , , , , , , ,

[0005] , , , ,

[0001] The present disclosure relates to a technique for evaluating the stability of an excavation site.

Background Art

[0002] At the site of excavation work such as tunnel construction, in order to ensure the safety of the work and implement appropriate measures as necessary, it is required to carry out the work while checking the stability of the face. In this regard, manually evaluating the stability of the face involves a lot of subjective elements and there are variations in the evaluation, so it largely depends on the judgment of experienced skilled technicians. However, it is often difficult to have such skilled technicians present at the excavation site for a long time. Therefore, techniques for evaluating the stability of the face using information processing technology have been developed.

[0003] Patent Document 1 discloses a technique for specifying the stability degree of a face by using data related to the blasting holes of the face. Patent Document 2 discloses a technique for specifying the stability degree of a face by analyzing the weathering state of the face, the frequency of face collapse, the state of cracks generated on the face, the rock type constituting the face, and the state of water gushing generated on the face from an image of the face.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Non-Patent Documents

[0005]

Non-Patent Document 1

[0006] Patent Document 1 assumes that blasting excavation is performed. Therefore, the technology described in Patent Document 1 cannot be used in cases where the tunnel face is excavated using a hydraulic breaker or the like. Patent Document 2 evaluates the stability of the tunnel face using only images of the tunnel face. Therefore, information other than visual information is not considered in the evaluation of the stability of the tunnel face. The present invention has been made in view of these problems, and one of its objectives is to provide a new technology for evaluating the stability of a tunnel face. [Means for solving the problem]

[0007] The face stability evaluation apparatus 2000 of this disclosure includes an acquisition unit that acquires audio data obtained by recording sounds around the face to be evaluated, and video data obtained by imaging the face to be evaluated, and a sound type identification step that identifies the type of sound represented by the audio data, The system includes a collapse area calculation unit that uses the video data to calculate the collapse area, which is the area of ​​the collapsed region at the tunnel face to be evaluated, and a stability determination unit that determines the stability of the tunnel face to be evaluated based on the identified sound type and the calculated collapse area.

[0008] The control method of the present disclosure is performed by a computer. The control method includes: an acquisition step of acquiring audio data obtained by recording sounds around the tunnel face to be evaluated and video data obtained by imaging the tunnel face to be evaluated; a sound type identification step of identifying the type of sound represented by the audio data; a collapse area calculation step of calculating the collapse area, which is the area of ​​the collapse region at the tunnel face to be evaluated, using the video data; and a stability identification step of identifying the stability of the tunnel face to be evaluated based on the identified sound type and the calculated collapse area.

[0009] The program in this disclosure causes a computer to execute the control method in this disclosure. [Effects of the Invention]

[0010] This disclosure provides a new technique for evaluating the stability of the working face. [Brief explanation of the drawing]

[0011] [Figure 1] This figure illustrates an overview of the operation of the face stability evaluation device of Embodiment 1. [Figure 2] This is a block diagram illustrating the functional configuration of the face stability evaluation device of Embodiment 1. [Figure 3] This is a block diagram illustrating the hardware configuration of a computer that implements a tunnel face stability evaluation device. [Figure 4] This is a flowchart illustrating the processing flow performed by the face stability evaluation device of Embodiment 1. [Figure 5] This diagram illustrates a method for classifying different types of sounds. [Figure 6] This is a diagram illustrating a sedimentary region. [Figure 7] This flowchart illustrates the process for determining the stability of the tunnel face according to the judgment rules. [Figure 8] This flowchart illustrates the process for determining the stability of the tunnel face according to the judgment rules. [Figure 9]It is a block diagram illustrating the functional configuration of a face stability evaluation device having an output unit. [Figure 10] It is a diagram illustrating stability information.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and duplicate descriptions are omitted as necessary for clarity of explanation. Also, unless otherwise specified, predetermined values such as predetermined values and threshold values are stored in advance in a storage unit accessible from the device that uses the values. Furthermore, unless otherwise specified, the storage unit is composed of one or more storage devices of any number.

[0013] FIG. 1 is a diagram illustrating an outline of the operation of the face stability evaluation device 2000 according to Embodiment 1. Here, FIG. 1 is a diagram for facilitating understanding of the outline of the face stability evaluation device 2000, and the operation of the face stability evaluation device 2000 is not limited to that shown in FIG. 1.

[0014] The face stability evaluation device 2000 evaluates the stability of the face 10, which is the face to be evaluated. Here, the face as used herein means the excavation face at the excavation site (such as a tunnel construction site) using an excavation machine.

[0015] The evaluation of the stability of the face 10 is performed using the audio data 22 and the video data 32. The audio data 22 is audio data generated by recording the sounds around the face 10 using the microphone 20. The video data 32 is video data obtained by photographing the face 10 using the camera 30. Although the microphone 20 and the camera 30 are shown separately in FIG. 1, they may be housed in the same housing.

[0016] The face stability evaluation device 2000 identifies the type of sound represented by the audio data 22 from among a plurality of specific types. For example, as the types of sound, three types can be adopted: 1) no excavation sound, 2) light excavation sound, and 3) heavy excavation sound. The sound classified as "no excavation sound" is a sound that contains little or no excavation sound. Such a sound is the sound in a situation where excavation by an excavation machine is not being performed.

[0017] The sound classified as "light excavation sound" is mainly the sound of the excavation machine, and has a large high-frequency component. Such a sound is generated when excavating relatively soft ground. The sound classified as "heavy excavation sound" is the sound generated when the excavation machine and the rock mass are integrated, and compared with the light excavation sound, the low-frequency component is larger. Such a sound is generated when excavating relatively hard ground. Thus, the difference between the light excavation sound and the heavy excavation sound reflects the difference in the hardness of the face 10 being excavated.

[0018] Furthermore, the face stability evaluation device 2000 uses the video data 32 to calculate the area of the collapsed area, which is the area that has collapsed from the face 10. Then, the face stability evaluation device 2000 identifies the stability of the face 10 based on the type of sound identified using the audio data 22 and the collapsed area calculated using the video data 32.

[0019] <An example of the function and effect> According to the face stability evaluation device 2000 of this embodiment, the stability of the tunnel face 10 is determined by utilizing audio data 22 obtained by recording sounds around the tunnel face 10 and video data 32 obtained by imaging the tunnel face 10. Thus, the face stability evaluation device 2000 provides a new technology for evaluating the stability of the tunnel face. Furthermore, the face stability evaluation device 2000 determines the stability of the tunnel face 10 using not only images but also audio. Therefore, it is possible to determine the stability of the tunnel face 10 more accurately compared to cases where only images are used to determine the stability of the tunnel face 10. Moreover, the face stability evaluation device 2000 can determine the stability of the tunnel face 10 during excavation work using excavation machinery.

[0020] The face stability evaluation device 2000 of this embodiment will be described in more detail below.

[0021] <Example of functional configuration> Figure 2 is a block diagram illustrating the functional configuration of the tunnel face stability evaluation device 2000 of Embodiment 1. The tunnel face stability evaluation device 2000 includes an acquisition unit 2020, a sound type identification unit 2040, a collapse area calculation unit 2060, and a stability determination unit 2080. The acquisition unit 2020 acquires audio data 22 and video data 32. The sound type identification unit 2040 identifies the type of sound represented by the audio data 22. The collapse area calculation unit 2060 calculates the collapse area using the video data 32. The stability determination unit 2080 determines the stability of the tunnel face 10 based on the type of sound and the collapse area.

[0022] <Example of hardware configuration> Each functional component of the tunnel face stability evaluation device 2000 may be implemented by hardware (e.g., hardwired electronic circuits) or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program to control it). The following will further explain the case where each functional component of the tunnel face stability evaluation device 2000 is implemented by a combination of hardware and software.

[0023] Figure 3 is a block diagram illustrating the hardware configuration of the computer 500 that implements the tunnel face stability evaluation device 2000. The computer 500 is any computer. For example, the computer 500 is a stationary computer such as a PC (Personal Computer) or a server machine. Alternatively, the computer 500 is a portable computer such as a smartphone or tablet terminal. The computer 500 may be a dedicated computer designed to implement the tunnel face stability evaluation device 2000, or it may be a general-purpose computer.

[0024] For example, by installing a predetermined application on computer 500, the various functions of the face stability evaluation device 2000 are realized on computer 500. The above application consists of programs for realizing each functional component of the face stability evaluation device 2000. The method of obtaining the above program is arbitrary. For example, the program can be obtained from a storage medium (such as a DVD disk or USB memory) on which it is stored. Alternatively, the program can be obtained by downloading it from a server device that manages the storage device on which it is stored.

[0025] Computer 500 includes a bus 502, a processor 504, memory 506, a storage device 508, an input / output interface 510, and a network interface 512. The bus 502 is a data transmission path for the processor 504, memory 506, storage device 508, input / output interface 510, and network interface 512 to send and receive data to and from each other. However, the method of connecting the processor 504 and other components to each other is not limited to bus connection.

[0026] The processor 504 is a variety of processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an FPGA (Field-Programmable Gate Array). The memory 506 is the main memory, implemented using RAM (Random Access Memory), etc. The storage device 508 is the auxiliary storage, implemented using a hard disk, SSD (Solid State Drive), memory card, or ROM (Read Only Memory), etc.

[0027] The input / output interface 510 is an interface for connecting the computer 500 with input / output devices. For example, input devices such as keyboards and output devices such as display devices are connected to the input / output interface 510.

[0028] The network interface 512 is an interface for connecting the computer 500 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0029] The storage device 508 stores programs that implement each functional component of the tunnel face stability evaluation device 2000 (programs that implement the aforementioned applications). The processor 504 reads these programs into memory 506 and executes them to implement each functional component of the tunnel face stability evaluation device 2000.

[0030] The tunnel face stability evaluation device 2000 may be implemented with one computer 500 or with multiple computers 500. In the latter case, the configuration of each computer 500 does not need to be the same and can be different.

[0031] <<Regarding Microphone 20 and Camera 30>> The microphone 20 is any microphone capable of recording sounds around the tunnel face 10 and generating audio data. The audio data 22 generated by the microphone 20 is stored in any memory unit. For example, the microphone 20 is connected to the computer 500 (tunnel face stability evaluation device 2000) via an input / output interface 510 or a network interface 512. However, it is sufficient that the audio data 22 generated by the microphone 20 can be acquired by the tunnel face stability evaluation device 2000, and the microphone 20 does not need to be connected to the tunnel face stability evaluation device 2000 for communication.

[0032] Camera 30 is any camera capable of imaging the tunnel face 10 and generating video data. The video data 32 generated by camera 30 is stored in any memory unit. For example, camera 30 is connected to computer 500 (tunnel face stability evaluation device 2000) via input / output interface 510 or network interface 512. However, camera 30 does not need to be connected to the tunnel face stability evaluation device 2000 for communication purposes, as long as the video data 32 generated by camera 30 can be acquired by the tunnel face stability evaluation device 2000.

[0033] <Processing flow> Figure 4 is a flowchart illustrating the processing flow performed by the tunnel face stability evaluation device 2000 of Embodiment 1. The acquisition unit 2020 acquires audio data 22 and video data 32 (S102). The sound type identification unit 2040 identifies the type of sound represented by the audio data 22 (S104). The collapse area calculation unit 2060 calculates the collapse area using the video data 32 (S106). The stability identification unit 2080 identifies the stability of the tunnel face 10 based on the identified sound type and the calculated collapse area (S108).

[0034] The processing flow performed by the face stability evaluation device 2000 is not limited to the flow shown in Figure 4. For example, the identification of the type of sound (S104) and the acquisition of video data 32 (S106) may be performed in the reverse order of the order shown in Figure 4, or they may be performed in parallel.

[0035] The tunnel face stability evaluation device 2000 is preferable to repeatedly determine the stability of the tunnel face 10 by repeatedly performing the process shown in Figure 4. For example, the tunnel face stability evaluation device 2000 determines the stability of the tunnel face 10 at specific time intervals by performing the process shown in Figure 4 at specific time intervals.

[0036] <Acquisition of audio data 22 and video data 32: S102> The acquisition unit 2020 acquires audio data 22 and video data 32 (S102). The method by which the acquisition unit 2020 acquires the audio data 22 is arbitrary. For example, the acquisition unit 2020 acquires the audio data 22 by accessing the storage unit where the audio data 22 is stored and reading the audio data 22. Alternatively, for example, the acquisition unit 2020 acquires the audio data 22 by receiving the audio data 22 transmitted from another device (e.g., microphone 20).

[0037] The same applies to how the acquisition unit 2020 acquires the video data 32. That is, for example, the acquisition unit 2020 acquires the video data 32 by reading the video data 32 from the storage unit where the video data 32 is stored, or by receiving the video data 32 transmitted from another device (e.g., camera 30).

[0038] Alternatively, the acquisition unit 2020 may acquire data containing both audio data 22 and video data 32, and extract the audio data 22 and video data 32 from that data. For example, suppose the camera 30 is a video camera with a built-in microphone 20. In this case, the camera 30 can be used to generate a video file in which both audio and video are recorded. The acquisition unit 2020 acquires the video file and extracts the data representing the video from the data contained in the video file as video data 32. The acquisition unit 2020 also extracts the data representing the audio from the data contained in the video file as audio data 22.

[0039] <Identification of sound type: S104> The sound type identification unit 2040 identifies the type of sound represented by the audio data 22 (S104). For example, as mentioned above, three types of sound are used: 1) no excavation sound, 2) light excavation sound, and 3) heavy excavation sound.

[0040] There are various methods for classifying the type of sound represented by the audio data 22. For example, the sound type identification unit 2040 classifies the type of sound by processing according to the flow shown in Figure 5. Figure 5 is a diagram illustrating a method for classifying the type of sound. The sound type identification unit 2040 obtains multiple partial audio data by dividing the audio data 22 into predetermined time lengths (for example, every second) (S202). The sound type identification unit 2040 binarizes each partial audio data by comparing it with a threshold (S204). Hereinafter, the value obtained by this binarization will be called the audio flag. An audio flag of 1 means that there is a high probability that the audio represented by the partial audio data includes excavation sounds.

[0041] For example, the sound type identification unit 2040 calculates statistical values ​​of sound pressure (such as the mean, maximum, or mode) from the time-series data of sound pressure represented by the partial speech data. If the calculated statistical value is above a threshold, the sound type identification unit 2040 converts the partial speech data into a speech flag with a value of 1. On the other hand, if the calculated statistical value is below the threshold, the sound type identification unit 2040 converts the partial speech data into a speech flag with a value of 0.

[0042] The sound type identification unit 2040 calculates statistical values ​​(such as the mean and mode) of the speech flags obtained from each partial speech data (S206). For example, suppose the length of the speech data 22 is N seconds, the length of the partial speech data is 1 second, and the mean is used as the type of statistical value. In this case, the sound type identification unit 2040 calculates the mean of the N speech flags obtained from the N partial speech data.

[0043] The sound type identification unit 2040 identifies the type of sound represented by the audio data 22 based on the statistical value of the audio flag (S208). Here, a larger statistical value of the audio flag means that the frequency of excavation noise is high in the audio data 22. Also, the harder the face 10, the more intense the excavation required, and therefore the higher the frequency of excavation noise. For this reason, when the audio data 22 represents heavy excavation noise, the statistical value of the audio flag is larger than when the audio data 22 represents light excavation noise. Also, when the audio data 22 represents light excavation noise, the statistical value of the audio flag is larger than when the audio data 22 does not contain any excavation noise.

[0044] For example, the numerical range of the statistical value of the audio flag is divided in advance into three ranges: a range representing heavy excavation sounds, a range representing light excavation sounds, and a range representing no excavation sounds. To this end, a threshold (hereinafter referred to as the first flag threshold) that separates the range of heavy excavation sounds from the range of light excavation sounds, and a threshold (hereinafter referred to as the second flag threshold) that separates the range of light excavation sounds from the range of no excavation sounds are defined. The sound type identification unit 2040 determines which of the three numerical ranges the statistical value of the audio flag calculated for the audio data 22 falls into.

[0045] If the statistical value of the audio flag is greater than or equal to the first flag threshold, the sound type identification unit 2040 determines that the type of sound represented by the audio data 22 is a heavy excavation sound. If the statistical value of the audio flag is greater than or equal to the second flag threshold but less than the first flag threshold, the sound type identification unit 2040 determines that the type of sound represented by the audio data 22 is a light excavation sound. If the statistical value of the audio flag is less than the second flag threshold, the sound type identification unit 2040 determines that the type of sound represented by the audio data 22 is no excavation sound.

[0046] Here, the sound type identification unit 2040 may perform any preprocessing on the audio data 22 before performing the processing shown in Figure 5. For example, preprocessing may involve removing noise in the high-frequency or low-frequency range. Existing techniques such as using high-pass filters or low-pass filters can be used to remove noise contained in a specific frequency range.

[0047] The method for classifying sound types is not limited to the method shown in Figure 5. For example, the sound type identification unit 2040 may use a pre-trained machine learning model to classify the sound types represented by the speech data 22. Hereinafter, this model will be referred to as the sound type classification model. The sound type classification model is pre-trained to output a label indicating the sound type represented by the speech data 22 in response to the input speech data 22. Any machine learning model capable of handling time-series data (e.g., an RNN (recurrent neural network)) can be used for such a model.

[0048] Training a sound classification model can be done using multiple training datasets, each consisting of a training input dataset and a training ground truth dataset. The training input dataset is audio data. The training ground truth dataset is a label indicating the type of sound represented by the corresponding audio data. Various existing techniques can be used to train machine learning models using such training datasets.

[0049] Here, the time scale used for identifying the type of sound is arbitrary. For example, the sound type identification unit 2040 identifies one type of sound for the entire sound represented by the audio data 22. Alternatively, for example, the sound type identification unit 2040 may divide the audio data 22 into audio data at predetermined time intervals (e.g., every second) and identify the type of sound represented by each audio data.

[0050] <Calculation of collapse area: S106> The collapse area calculation unit 2060 uses the video data 32 to calculate the collapse area (S106). The calculation of the collapse area can be achieved by detecting the collapsed area, which is the area that has collapsed, from the image area of ​​the tunnel face 10, and calculating the area of ​​the detected collapsed area.

[0051] Here, we will explain how to detect the collapse area from the tunnel face 10. For example, the collapse area calculation unit 2060 detects the motion area, which represents a moving object, from the image area representing the tunnel face 10 by comparing multiple video frames that make up the video data 32. Here, various techniques such as background subtraction, Lucas-Kanade method, or Gunnar-Farnback method can be used to detect motion.

[0052] Furthermore, the collapse area calculation unit 2060 identifies collapse areas by classifying the moving regions according to the type of object contained within them. For example, a collapse area can be defined as a moving region whose contained object is soil and sand. Therefore, the collapse area calculation unit 2060 identifies moving regions whose contained object is soil and sand as collapse areas.

[0053] For detecting collapse areas, semantic segmentation can be used, for example. Specifically, the collapse area calculation unit 2060 detects motion regions using video data 32, and then marks the motion regions for each video frame of the video data 32. The motion regions are marked, for example, by superimposing a specific color on the motion regions or by replacing the motion regions with a specific color. Then, the collapse area calculation unit 2060 detects collapse areas by performing semantic segmentation on the video frames on which the motion regions have been marked.

[0054] Semantic segmentation identifies the type of object represented by each pixel in a video frame. This allows for the detection of each pixel classified as a collapsed area, thus enabling the detection of the collapsed area.

[0055] The specific method for performing semantic segmentation is arbitrary. For example, a classifier trained to perform semantic segmentation can be used. Existing techniques can be used to train a classifier that performs semantic segmentation. For example, the techniques disclosed in Non-Patent Document 1 can be used.

[0056] The collapse area calculation unit 2060 calculates the collapse area by calculating the area of ​​the detected collapse region. If multiple collapse regions are detected, for example, the collapse area calculation unit 2060 calculates the collapse area by summing the areas of each collapse region.

[0057] Here, the tunnel face 10 may be divided into multiple sub-regions in advance, and the collapse area may be calculated for each sub-region. In this case, the collapse area is calculated for each sub-region. Specifically, the collapse area calculation unit 2060 detects each sub-region from each video frame that makes up the video data 32, and detects the collapse area for each sub-region. Then, the collapse area calculation unit 2060 calculates the collapse area for each sub-region by summing the areas of the collapse areas detected within that sub-region.

[0058] The area of ​​the collapsed region is represented, for example, by the area of ​​that region on the image. That is, the collapsed area calculation unit 2060 calculates the area of ​​the collapsed region as the number of pixels that make up the collapsed region (i.e., the area on the image). Alternatively, for example, the collapsed area calculation unit 2060 may calculate the area of ​​the region in the real world that corresponds to the collapsed region as the area of ​​the collapsed region. In this case, the ratio of the unit area in the real world to the unit area on the image in the video data 32 is predetermined. Then, the collapsed area calculation unit 2060 calculates the area of ​​the collapsed region on the image and calculates the area of ​​the collapsed region represented by the area of ​​the region in the real world by multiplying the calculated value by the above ratio. For example, if 1 pixel corresponds to N [cm^2] in the real world, the area of ​​the collapsed region can be calculated as the area in the real world (unit cm^2) by multiplying the number of pixels that make up the collapsed region by N.

[0059] In addition, for example, the area of ​​the collapsed region may be expressed as the ratio of the area of ​​the collapsed region to the total area. If one collapsed area is calculated for the entire tunnel face 10, the total area is the area of ​​tunnel face 10. On the other hand, if the collapsed area is calculated for each sub-region, the total area is the area of ​​the sub-region.

[0060] When the tunnel face is excavated with an excavating machine, the rock mass crushed by the machine is deposited as soil and sediment. This soil and sediment are not what we want to detect as part of a collapse. Therefore, it is preferable not to include the area of ​​soil and sediment from the rock mass crushed by the excavating machine in the collapse area. However, when detecting a collapse area as an area where soil and sediment are moving, the area representing the soil and sediment from the rock mass crushed by the excavating machine (hereinafter referred to as the deposition area) will also be detected as a collapse area.

[0061] For example, the collapse area calculation unit 2060 subtracts the area of ​​the deposition area from the collapse area to prevent the deposition area from being treated as a collapse area. Figure 6 is an example of a deposition area. In the figure, reference numeral 100 represents the coordinates of the point where the excavation machine contacts the face 10 (hereinafter referred to as the excavation position 100). Reference numeral 110 represents the deposition area. In Figure 6, the deposition area 110 is defined as a rectangle whose upper edge is centered at the excavation position 100, whose upper edge has a predetermined length L, and whose lower edge is the lower edge of the face 10 (i.e., the ground). Note that the shape of the deposition area 110 is not limited to a rectangle and can be any shape.

[0062] The deposition area 110 is identified, for example, as follows. First, the collapse area calculation unit 2060 detects the part of the excavating machine that is excavating the face 10 (hereinafter referred to as the excavation part) from each video frame. The excavation part is, for example, the tip of a hydraulic breaker. Here, various existing object recognition technologies can be used to detect the excavation part. For example, a method can be adopted that uses a machine learning model trained to detect the excavation part.

[0063] The collapse area calculation unit 2060 detects the coordinates of a specific part of the excavated area (for example, the center point or the centroid point) as the excavation position 100. Furthermore, the collapse area calculation unit 2060 determines the deposition area 110 based on the excavation position 100.

[0064] The collapse area calculation unit 2060 subtracts the area of ​​the collapse region that is included in the sedimentation region 110 from the collapse area. Alternatively, instead of subtracting, the collapse area calculation unit 2060 may exclude the collapse region included in the sedimentation region 110 from the calculation when totaling the areas of the collapse regions to calculate the collapse area.

[0065] Alternatively, the collapse area calculation unit 2060 may detect the sedimentation area 110 before detecting the collapse area, and exclude the sedimentation area 110 from the area targeted for collapse area detection. In this way, since the collapse area will not be detected from the sedimentation area 110, the area of ​​the sedimentation area 110 where sediment is moving will not be included in the collapse area.

[0066] The time scale used to calculate the collapse area is arbitrary. For example, the collapse area calculation unit 2060 calculates the collapse area for each video frame included in the video data 32, based on the state represented by that video frame. Alternatively, the collapse area calculation unit 2060 may calculate the collapse area for one video frame at a predetermined time interval (for example, one video frame every second or every minute) from among the video frames included in the video data 32.

[0067] In addition, the collapse area calculation unit 2060 may also calculate statistical values ​​(such as average or maximum values) of the collapse area at predetermined intervals. For example, suppose that for 30 fps (frames per second) video data 32, the collapse area is calculated for each video frame, and statistical values ​​of the collapse area are calculated every second. In this case, the collapse area calculation unit 2060 calculates statistical values ​​of the collapse area calculated from each of the 30 video frames.

[0068] <Determination of the stability of face 10: S108> The stability determination unit 2080 determines the stability of the tunnel face 10 based on the type of sound represented by the audio data 22 and the area of ​​the collapsed region (S108). Here, if the audio data 22 does not contain excavation sounds and the collapsed area is large, it means that a large collapse is occurring at the tunnel face 10 even if no vibrations are being applied due to excavation. Therefore, in such a situation, the stability of the tunnel face 10 is considered to be low. On the other hand, if the audio data 22 contains excavation sounds and the collapsed area is small, it means that even if vibrations are being applied at the tunnel face 10 due to excavation, the collapse is small. Therefore, in such a situation, the stability of the tunnel face 10 is considered to be high.

[0069] For example, the stability determination unit 2080 determines the stability of the tunnel face 10 based on predetermined determination rules. Figures 7 and 8 are flowcharts illustrating the process of determining the stability of the tunnel face 10 according to the determination rules.

[0070] In the processes shown in Figures 7 and 8, the stability of the working face 10 is represented by a risk level. A higher risk level for the working face 10 indicates lower stability (higher risk). More specifically, five risk levels are set: risk level 1, risk level 2, risk level 3A, risk level 3B, and risk level 4. Risk level 1 represents the highest level of stability, and risk level 4 represents the lowest level of stability. Both risk level 3A and risk level 3B are less stable than risk level 2 and more stable than risk level 4. However, for risk level 3A and risk level 3B, it is not specified which one is more stable; for example, they may be of similar stability.

[0071] The stability determination unit 2080 determines whether or not the audio data 22 contains excavation noise (S302). Here, if the type of sound represented by the audio data 22 is no excavation noise, it is determined that the audio data 22 does not contain excavation noise; otherwise, it is determined that the audio data 22 contains excavation noise.

[0072] If the audio data 22 contains excavation sounds (S302: YES), the stability determination unit 2080 determines whether the collapsed area is greater than or equal to the first area threshold (S304). If the collapsed area is less than the first area threshold (S304: NO), the stability determination unit 2080 determines that the stability of the tunnel face 10 is at danger level 1 (S306). The situation in which the stability of the tunnel face 10 is at danger level 1 is that excavation is being carried out (the audio data 22 contains excavation sounds), but the collapse is small (the collapsed area is less than the first area threshold).

[0073] If the collapse area is greater than or equal to the first area threshold (S304: YES), the stability determination unit 2080 determines whether the type of sound represented by the audio data 22 is a heavy excavation sound (S308). If the type of sound represented by the audio data 22 is a heavy excavation sound (S308: YES), the stability determination unit 2080 determines that the stability of the tunnel face 10 is at danger level 2 (S310). If the type of sound represented by the audio data 22 is not a heavy excavation sound (S308: NO), that is, if the type of sound is a light excavation sound, the stability determination unit 2080 determines that the stability of the tunnel face 10 is at danger level 3A (S312).

[0074] Here, the situations in which the stability of the tunnel face 10 is classified as danger level 2 or danger level 3A share the following characteristics: excavation is taking place (excavation sounds are included in the audio data 22), and the collapse is of a certain magnitude (the collapsed area is greater than or equal to the first area threshold). Therefore, the stability is distinguished by the intensity of the excavation. More specifically, as mentioned above, when the excavation sounds are intense, the rock being excavated is harder than when the excavation sounds are light. For this reason, the stability of the tunnel face 10 is considered higher when the excavation sounds are intense than when the excavation sounds are light. Thus, when the type of sound represented by the audio data 22 is intense excavation sound, the stability of the tunnel face 10 is judged to be higher than when the type of sound represented by the audio data 22 is light excavation sound (the former is danger level 2, and the latter is danger level 3A).

[0075] If the audio data 22 does not contain excavation sounds (S302: NO), the stability determination unit 2080 determines whether the collapsed area is greater than or equal to the second area threshold (S314). If the collapsed area is greater than or equal to the second area threshold (S314: YES), the stability determination unit 2080 determines that the stability of the tunnel face 10 is at danger level 4 (S316). A situation in which the stability of the tunnel face 10 is at danger level 4 is one in which there is a large collapse (collapsed area is greater than or equal to the second area threshold) even though no excavation is taking place (the audio data 22 does not contain excavation sounds).

[0076] If the collapsed area is less than the second area threshold (S314: NO), the stability determination unit 2080 determines whether the collapsed area is greater than or equal to the third area threshold (S318). Here, the second area threshold is greater than or equal to the third area threshold. If the collapsed area is greater than or equal to the third area threshold (S318: YES), the stability determination unit 2080 determines that the stability of the tunnel face 10 is at danger level 3B (S320). The situation in which the stability of the tunnel face 10 is at danger level 3B is when no excavation is being carried out (no excavation sounds are included in the audio data 22), and there is some collapse (the collapsed area is greater than or equal to the third area threshold but less than the second area threshold).

[0077] If the collapsed area is less than the third area threshold (S318: YES), the stability determination unit 2080 reserves judgment on the stability of the tunnel face 10 (S318). Here, the situation in which the audio data 22 does not contain excavation sounds and the collapsed area is less than the third area threshold means that no excavation has been performed and the collapse is small. For a tunnel face 10 in such a situation, it is difficult to determine the stability without verifying the magnitude of the collapse if excavation had been performed. Therefore, the stability determination unit 2080 reserves judgment. Note that the reserved judgment may also be expressed as "no judgment result".

[0078] Note that Figures 7 and 8 show only examples of judgment rules, and the judgment rules are not limited to those shown in Figure 7.

[0079] Here, if the collapse area is calculated for each sub-region, the stability of the tunnel face 10 is determined for each sub-region. For example, the stability determination unit 2080 determines the risk level for each sub-region based on the determination rules shown in Figures 7 and 8.

[0080] <Output of results> The tunnel face stability evaluation device 2000 outputs information indicating the stability of the tunnel face 10, as identified by the stability identification unit 2080 (hereinafter referred to as stability information). The functional component that outputs the stability information is called the output unit. Figure 9 is a block diagram illustrating the functional configuration of the tunnel face stability evaluation device 2000 having an output unit 2100. In Figure 9, stability information 200 is output from the output unit 2100.

[0081] The information contained in the stability information 200 is diverse. For example, the stability information 200 shows time-series data of the stability of the tunnel face 10 as identified by the stability identification unit 2080. Figure 10 is an example of the stability information 200. The stability information 200 in Figure 10 shows a display 210 representing the division pattern of the tunnel face 10, and a risk graph 220 which is a graph of the time-series data of the stability.

[0082] Display 210 shows how the tunnel face 10 is divided into sub-regions. In the example in Figure 10, the tunnel face 10 is divided into four sub-regions R1 to R4. The tunnel face stability evaluation device 2000 identifies the risk level for each of these four divided regions at predetermined time intervals.

[0083] Risk Graph 220 shows time-series data of risk levels for each sub-region as a line graph. In Risk Graph 220, the horizontal axis represents time, and the vertical axis represents the risk level. Here, risk levels 3A and 3B are both treated as risk level 3.

[0084] In Figure 10, the stability information 200 shows two hazard graphs 220. Hazard graph 220-1 shows the hazard level of the tunnel face 10 in real time. On the other hand, hazard graph 220-2 shows the hazard level of the tunnel face 10 in the past (for example, the previous day).

[0085] The timing at which the output unit 2100 generates the stability information 200 is not limited to real time. For example, the output unit 2100 may generate and output stability information 200 for the working face 10 for a specific period at regular intervals. As a concrete example, the output unit 2100 may generate stability information 200 for the working face 10 for that day once a day.

[0086] The time axis scale of the risk graph 220 may be coarser than the time scale used by the face stability evaluation device 2000 to determine the stability of the face 10. In this case, for example, the face stability evaluation device 2000 calculates statistical values ​​of the stability of the face 10 and plots them on the risk graph 220. For example, suppose the frame rate of camera 30 is 30fps, and the stability of the face 10 is determined for each video frame of video data 32. Also, suppose the time axis scale of the risk graph 220 is 1 second. In this case, the output unit 2100 calculates 30 stability statistics for the face 10 every second and plots these statistics on the risk graph 220.

[0087] The information contained in the stability information 200 is not limited to the risk graph 220. For example, the output unit 2100 may generate stability information 200 indicating a warning when a risk level above a predetermined level is identified. For example, a warning may be issued when a risk level above a predetermined level is identified at least once. Alternatively, for example, a warning may be issued when a risk level above a predetermined level is identified at a predetermined frequency or higher.

[0088] Here, the method of warning is arbitrary. For example, the stability information 200 indicating a warning includes a message indicating that the condition of the cutting face 10 is dangerous. This message may be a text message, an image message, or an audio message. Alternatively, for example, the stability information 200 indicating a warning may also be a warning sound.

[0089] The output method of the stability information 200 is arbitrary. For example, the output unit 2100 stores the stability information 200 in any storage unit. Alternatively, the output unit 2100 may display the stability information 200 on any display device (such as a display device installed at the excavation site) or play it back on any speaker (such as a speaker installed at the excavation site). Alternatively, the output unit 2100 may transmit the stability information 200 to any terminal (for example, a terminal used by workers or managers at the excavation site).

[0090] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications to the structure and details of the present invention can be made, which can be understood by those skilled in the art within the scope of the present invention.

[0091] In the above examples, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrical, optical, acoustic or other forms of propagating signals.

[0092] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) An acquisition unit that acquires audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification unit that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation unit calculates the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, using the aforementioned video data. A face stability evaluation device having a stability determination unit that determines the stability of the face to be evaluated based on the type of sound identified and the calculated collapse area. (Note 2) The sound type identification unit identifies whether the sound represented by the audio data is of one of the following types: a sound that does not contain excavation noise, a light excavation noise, or a heavy excavation noise. A heavy excavation sound is a sound with a larger high-frequency component compared to a light excavation sound, according to the face stability evaluation device described in Appendix 1. (Note 3) The collapse area calculation unit detects a dynamic region representing a moving object from the video frame of the video data, and performs semantic segmentation on the video frame in which the dynamic region is marked to detect the dynamic region representing the collapse area, as described in Appendix 1 or 2 of the face stability evaluation device. (Note 4) The collapse area calculation unit uses the video data to identify an excavation point, which is the location where the excavation machine is excavating; based on the excavation point, it identifies a deposit area, which is the area where soil from the rock mass crushed by the excavation machine is deposited; and excludes the deposit area from the collapse area, as described in any one of the appendices 1 to 3. (Note 5) The collapse area calculation unit calculates the area of ​​the collapse area included in each of the multiple sub-regions included in the face to be evaluated as the collapse area, according to any one of the appendices 1 to 4, in the face stability evaluation device. (Note 6) The face stability evaluation device according to any one of the appendices 1 to 5, wherein the stability determination unit determines that the stability of the face to be evaluated is the highest of multiple stability levels if the type of sound represented by the audio data is not a sound that does not include excavation noise, and the collapse area is greater than or equal to a threshold. (Note 7) The face stability evaluation device according to Appendix 6, wherein the stability determination unit makes the stability of the face to be evaluated higher when the type of sound represented by the audio data is a heavy excavation sound, in the case where the area of ​​the collapse region is less than the threshold, than the stability of the face to be evaluated when the type of sound represented by the audio data is a light excavation sound. (Note 8) It has an output unit that outputs stability information, which is information regarding the stability of the tunnel face to be evaluated, The face stability evaluation device according to any one of the appendices 1 to 7, wherein the stability information includes a graph showing the change in the stability of the face to be evaluated over time. (Note 9) A control method performed by a computer, Acquisition step: Acquisition step of acquiring audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification step that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation step, which uses the aforementioned video data to calculate the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, A control method comprising: a stability determination step of determining the stability of the tunnel face to be evaluated based on the type of sound identified and the calculated collapse area. (Note 10) In the sound type identification step, it is determined whether the sound represented by the audio data is of one of the following types: a sound that does not include excavation noise, a light excavation noise, or a heavy excavation noise. A heavy digging sound is a sound with a larger high-frequency component compared to a light digging sound, as described in Appendix 9 of the control method. (Note 11) The control method according to Appendix 9 or 10, wherein in the step of calculating the collapse area, a region representing a moving object is detected from the video frame of the video data, and the region representing the collapse area is detected by performing semantic segmentation on the video frame in which the region representing the moving object is marked. (Note 12) The control method according to any one of the appendices 9 to 11, wherein in the step of calculating the collapse area, the video data is used to identify the drilling point, which is the location where drilling is being performed by the drilling machine; based on the drilling point, the sedimentation area, which is the area where soil from the rock mass crushed by the drilling machine is deposited; and the sedimentation area is excluded from the collapse area. (Note 13) The control method according to any one of the appendices 9 to 12, wherein in the step of calculating the collapse area, for each of the multiple sub-regions included in the face to be evaluated, the area of ​​the collapse area included in that sub-region is calculated as the collapse area. (Note 14) The control method according to any one of the appendices 9 to 13, wherein in the stability determination step, if the type of sound represented by the audio data is not a sound that does not include excavation noise, and the collapse area is greater than or equal to a threshold, the stability of the face to be evaluated is determined to be the highest of multiple stability levels. (Note 15) The control method according to Appendix 14, wherein, in the stability determination step, when the area of ​​the collapse region is less than the threshold, the stability of the face to be evaluated when the type of sound represented by the audio data is a heavy excavation sound is set higher than the stability of the face to be evaluated when the type of sound represented by the audio data is a light excavation sound. (Note 16) The system has an output step that outputs stability information, which is information regarding the stability of the tunnel face being evaluated. The control method described in any one of Appendix 9 to 15, wherein the stability information includes a graph showing the change in stability of the working face to be evaluated over time. (Note 17) On the computer, Acquisition step: Acquisition step of acquiring audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification step that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation step, which uses the aforementioned video data to calculate the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, A program that performs a stability determination step, which determines the stability of the tunnel face to be evaluated based on the type of sound identified and the calculated collapse area. (Note 18) In the sound type identification step, it is determined whether the sound represented by the audio data is of one of the following types: a sound that does not include excavation noise, a light excavation noise, or a heavy excavation noise. A heavy drilling sound is a sound with a larger high-frequency component compared to a light drilling sound, as described in Appendix 17 of the program. (Note 19) The program according to Appendix 17 or 18, wherein in the step of calculating the collapse area, it detects a region representing a moving object from the video frame of the video data, and detects the region representing the collapse area by performing semantic segmentation on the video frame in which the region representing the moving object is marked. (Note 20) A program according to any one of the appendices 17 to 19, wherein in the step of calculating the collapse area, the program uses the video data to identify the drilling point, which is the location where drilling is being performed by the drilling machine, identifies the deposition area, which is the area where soil from the rock mass crushed by the drilling machine is deposited, based on the drilling point, and excludes the deposition area from the collapse area. (Note 21) The program described in any one of the appendices 17 to 20, wherein in the step of calculating the collapse area, for each of the multiple sub-regions included in the face to be evaluated, the area of ​​the collapse area included in that sub-region is calculated as the collapse area. (Note 22) The program described in any one of the appendices 17 to 21, wherein, in the stability determination step, if the type of sound represented by the audio data is not a sound that does not include excavation noise, and the collapse area is greater than or equal to a threshold, the program determines that the stability of the face to be evaluated is the highest of multiple stability levels. (Note 23) The program as described in Appendix 22, wherein, in the stability determination step, when the area of ​​the collapse region is less than the threshold, the stability of the face to be evaluated when the type of sound represented by the audio data is a heavy excavation sound is set higher than the stability of the face to be evaluated when the type of sound represented by the audio data is a light excavation sound. (Note 24) The system has an output step that outputs stability information, which is information regarding the stability of the tunnel face being evaluated. The stability information is a program described in any one of the appendices 17 to 23, which includes a graph showing the change in stability of the tunnel face under evaluation over time. [Explanation of Symbols]

[0093] 10. Segment 20 Microphones 22 Audio data 30 Cameras 32 video data 100 Excavation location 110 Deposition area 200 Stability Information 210 displays 220 Risk Level Graph 500 Computers Bus 502 504 Processors 506 memory 508 Storage Devices 510 Input / Output Interfaces 512 Network Interfaces 2000 Face Stability Evaluation Device 2020 Acquisition Department 2040 Sound type identification section 2060 Collapse area calculation part 2080 Stability identification part 2100 Output section

Claims

1. An acquisition unit that acquires audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification unit that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation unit calculates the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, using the aforementioned video data. It has a stability determination unit that determines the stability of the tunnel face to be evaluated based on the type of sound identified and the calculated collapse area, The collapse area calculation unit detects motion regions representing motion from video frames of the video data, and performs semantic segmentation on the video frames on which the motion regions are marked to detect the motion regions representing the collapse area, thereby enabling the detection of the motion regions representing the collapse area.

2. An acquisition unit that acquires audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification unit that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation unit calculates the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, using the aforementioned video data. It has a stability determination unit that determines the stability of the tunnel face to be evaluated based on the type of sound identified and the calculated collapse area, The collapse area calculation unit uses the video data to identify the excavation point, which is the location where the excavation machine is excavating, and based on the excavation point, identifies the deposition area, which is the area where the soil from the rock mass crushed by the excavation machine is deposited, and excludes the deposition area from the collapse area, thereby providing a face stability evaluation device.

3. An acquisition unit that acquires audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification unit that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation unit calculates the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, using the aforementioned video data. It has a stability determination unit that determines the stability of the tunnel face to be evaluated based on the type of sound identified and the calculated collapse area, The stability determination unit determines that, if the type of sound represented by the audio data is not a sound that does not include excavation noise, and the collapse area is less than a threshold, the stability of the face to be evaluated is the highest of several stability levels, in this face stability evaluation device.

4. An acquisition unit that acquires audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification unit identifies which type of sound represented by the aforementioned audio data is, among the following: a sound that does not contain excavation noise, a first type of excavation noise, and a second type of excavation noise that has a larger low-frequency component than the first type of excavation noise. A collapse area calculation unit calculates the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, using the aforementioned video data. It has a stability determination unit that determines the stability of the tunnel face to be evaluated based on the type of sound identified and the calculated collapse area, The stability determination unit is a face stability evaluation device that, when the area of ​​the collapse region is greater than or equal to a threshold, sets the stability of the face to be evaluated higher when the type of sound represented by the audio data is the second type of excavation sound than the stability of the face to be evaluated when the type of sound represented by the audio data is the first type of excavation sound.

5. The collapse area calculation unit calculates the area of ​​the collapse area included in each of the multiple sub-regions included in the face to be evaluated as the collapse area, according to any one of claims 1 to 4.

6. It has an output unit that outputs stability information, which is information regarding the stability of the tunnel face being evaluated, The face stability evaluation device according to any one of claims 1 to 5, wherein the stability information includes a graph showing the change in the stability of the face to be evaluated over time.

7. A control method performed by a computer, Acquisition step: Acquisition step of acquiring audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification step that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation step, which uses the aforementioned video data to calculate the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, A stability determination step of determining the stability of the face to be evaluated based on the specified sound type and the calculated collapse area, In the collapse area calculation step, a moving body area representing a moving body is detected from a video frame of the video data, and the collapse area is detected by performing semantic segmentation on the video frame in which the moving body area is marked. Control method.

8. A control method executed by a computer, An acquisition step of acquiring audio data obtained by recording sounds around the face to be evaluated, and video data obtained by imaging the face to be evaluated, A sound type identification step of identifying the type of sound represented by the audio data, A collapse area calculation step of calculating a collapse area, which is the area of the collapse area in the face to be evaluated, using the video data, A stability determination step of determining the stability of the face to be evaluated based on the specified sound type and the calculated collapse area, In the collapse area calculation step, using the video data, identify the excavation point, which is the position being excavated by the excavation machine, and based on the excavation point, identify the deposition area, which is the area where the earth and sand of the rock mass crushed by the excavation machine are deposited, and exclude the deposition area from the collapse area. Control method.

9. A control method executed by a computer, An acquisition step of acquiring audio data obtained by recording sounds around the face to be evaluated, and video data obtained by imaging the face to be evaluated, A sound type identification step of identifying the type of sound represented by the audio data, A collapse area calculation step of calculating a collapse area, which is the area of the collapse area in the face to be evaluated, using the video data, A stability determination step of determining the stability of the face to be evaluated based on the specified sound type and the calculated collapse area, In the stability determination step, when the type of sound represented by the audio data is not a sound that does not include excavation sound and the collapse area is less than a threshold value, it is determined that the stability of the face to be evaluated is the highest stability among multiple levels of stability. Control method.

10. On the computer, Acquisition step: Acquisition step of acquiring audio data obtained by recording sounds around the tunnel face to be evaluated, and video data obtained by imaging the tunnel face to be evaluated, A sound type identification step that identifies the type of sound represented by the aforementioned audio data, A collapse area calculation step, which uses the aforementioned video data to calculate the collapse area, which is the area of ​​the collapsed region at the face of the tunnel being evaluated, A stability determination step is performed to determine the stability of the tunnel face to be evaluated based on the type of sound identified and the calculated collapse area. A program that, in the step of calculating the collapse area, detects motion regions representing motion from video frames of the video data, and performs semantic segmentation on the video frames on which the motion regions are marked, thereby detecting the motion regions representing the collapse area.

Images