Classification device, classification method, and program

Abstract

The classification device (2000) acquires waterfall data (10) indicating the amplitude of vibration at each time point and each detection location of a vibration sensor (30) arranged along a target object (40), performs semantic segmentation on the waterfall data (10), generates class data indicating a normal class or an abnormal class for each element of the waterfall data (10), and classifies the detection locations into monitored locations and non-monitored locations. The normal class is assigned to elements for which the detection location is presumed to be a monitored location. The abnormal class is assigned to elements for which the detection location is presumed to be a non-monitored location. A monitored location is a detection location arranged along the target object (40). A non-monitored location is a detection location not arranged along the target object (40).

Description

Technical Field

[0001] The present disclosure relates generally to a classification device, a classification method, and a non - transient computer - readable storage medium.

Background Art

[0002] There is a technology for monitoring objects such as roads using vibration sensors. In Patent Document 1, a technology for obtaining waterfall data indicating the amplitude of vibrations detected by a vibration sensor for each of a plurality of positions and each of a plurality of time points using a distributed acoustic sensing (DAS) system as the vibration sensor is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 does not teach the case where some locations of the vibration sensor are not arranged along the monitoring object. The object of the present disclosure is to provide a new technology for processing data obtained from vibration sensors installed for monitoring an object.

Means for Solving the Problems

[0005] The present disclosure provides a classification device including at least one memory configured to store instructions and at least one processor. At least one processor is configured to execute instructions to acquire waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors arranged along the target object, perform semantic segmentation on the waterfall data to generate class data indicating a normal class or an abnormal class for each element of the waterfall data, assign the normal class to the element in which the detection point is presumed to be a monitored location, assign the abnormal class to the element in which the detection point is presumed to be an unmonitored location, assign the monitored location to the detection point arranged along the target object, and assign the unmonitored location to the detection point not arranged along the target object, and classify the detection points into monitored locations and unmonitored locations based on the class data.

[0006] This disclosure further provides a classification method performed by a computer. The classification method includes: acquiring waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors arranged along the target object; performing semantic segmentation on the waterfall data to generate class data indicating a normal class or an abnormal class for each element of the waterfall data; assigning the normal class to the element in which the detection point is predicted to be a monitored location; assigning the abnormal class to the element in which the detection point is presumed to be a non-monitored location; assigning the monitored location to the detection point arranged along the target object; and assigning the non-monitored location to the detection point not arranged along the target object; and classifying the detection points into monitored locations and non-monitored locations based on the class data.

[0007] This disclosure further provides a non-temporary computer-readable storage medium for storing programs. The program causes the computer to perform the following actions: acquire waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors arranged along the target object; perform semantic segmentation on the waterfall data to generate class data indicating a normal class or an abnormal class for each element of the waterfall data; assign the normal class to the element in which the detection point is predicted to be a monitored location; assign the abnormal class to the element in which the detection point is presumed to be a non-monitored location; the monitored locations are the detection points arranged along the target object; and the non-monitored locations are the detection points not arranged along the target object; and classify the detection points into monitored locations and non-monitored locations based on the class data. [Effects of the Invention]

[0008] This disclosure provides a novel technique for processing data obtained from vibration sensors installed to monitor an object. [Brief explanation of the drawing]

[0009] [Figure 1] An overview of the classification device of Embodiment 1 is shown. [Figure 2] This block diagram shows an example of the functional configuration of the classification device of Embodiment 1. [Figure 3] This block diagram shows an example of the hardware configuration of the classification device of Embodiment 1. [Figure 4] This flowchart shows an exemplary flow of processing performed by the classification device of Embodiment 1. [Figure 5] This document demonstrates a method for generating monitoring data based on waterfall data. [Figure 6] This flowchart shows an example of the processing flow performed by a classification device that utilizes the detection location information in subsequent processing. [Figure 7] An overview of the classification device of Embodiment 2 is shown. [Figure 8]This shows the object trajectories detected from the waterfall data. [Figure 9] This shows the object trajectory when the detection location was not correctly classified. [Figure 10] This shows the object trajectory when the detection location is correctly classified. [Figure 11] This block diagram shows an example of the functional configuration of the classification device of Embodiment 2. [Figure 12] This flowchart shows an exemplary flow of processing performed by the classification device of Embodiment 2. [Modes for carrying out the invention]

[0010] Embodiments of the present disclosure will be described below with reference to the drawings. The same elements are assigned the same reference numerals throughout the drawings, and redundant descriptions are omitted where necessary. Furthermore, unless otherwise specified, certain information (e.g., certain values ​​or thresholds) is pre-stored in a storage device accessible to the computer using that information.

[0011] Embodiment 1 <Overview> Figure 1 shows an overview of the classification device 2000 of Embodiment 1. Please note that the overview shown in Figure 1 is an example of the operation of the classification device 2000 in order to make it easier to understand, and does not limit or narrow the range of operations that the classification device 2000 can perform.

[0012] The classification device 2000 is configured to process waterfall data 10, which shows the amplitude of vibration detected by the vibration sensor 30, for each of two or more locations within the vibration sensor 30 and each of two or more time points. In some embodiments, the waterfall data 10 may be two or more time-series detection data 20. The detection data 20 is generated by the vibration sensor 30 and shows the amplitude of vibration detected at each of two or more locations within the vibration sensor 30 at a given time point.

[0013] The vibration sensor 30 is installed (arranged) along a monitoring target such as a road. Hereinafter, the monitoring target by the vibration sensor 30 is referred to as "target object 40".

[0014] An example of the vibration sensor 30 is a DAS system including a DAS device and an optical fiber cable. When a DAS system is adopted as the vibration sensor 30, the optical fiber cable is arranged along the target object 40 and attached to the DAS device. The DAS device is configured to transmit a laser pulse via the optical fiber cable and receive the reflection of the transmitted laser pulse.

[0015] Vibration generated at a specific location of the target object 40 affects the laser pulse traveling through that location of the target object 40 at that time. Therefore, the DAS device can measure the amplitude of the vibration generated at that location of the target object 40 by analyzing the reflection of the transmitted laser pulse. Accordingly, the DAS device can generate detection data 20 indicating the amplitudes of the vibrations detected at each of two or more locations of the optical fiber cable. Hereinafter, the location where the amplitude of the vibration is detected in the vibration sensor 30 (for example, the optical fiber cable) is referred to as the "detection location".

[0016] In some implementations, the waterfall data 10 may be formed as matrix data denoted as W. The rows of the matrix data W may represent time points, and the columns thereof may represent the detection locations of the vibration sensor 30. In this case, the element at the i-th row and j-th column of the waterfall data 10 (i.e., W[i][j]) represents the amplitude of the vibration detected at the j-th detection location of the vibration sensor 30 at the i-th time point.

[0017] Furthermore, the sequence of elements in the i-th row of the waterfall data 10 (i.e., {W[i][0], W[i][1],..., W[i][M]} (where M is the total number of detection locations indicated by the waterfall data 10)) represents the detection data 20 generated at the i-th time point. On the other hand, the sequence of elements in the j-th column of the waterfall data 10 (i.e., {W[0][j], W[0][j],..., W[N][j]} (where N is the total number of time points indicated by the waterfall data 10)) represents the time series of the amplitude of the vibration detected at the j-th detection location.

[0018] The waterfall data 10, formed as matrix data, can be processed as image data referred to as a "waterfall image." In this case, W[i][j] corresponds to the pixel (i,j) value of the waterfall image. Assume that the amplitude of the vibration detected by the vibration sensor 30 is quantized and normalized to a range of 0 to 255. In this case, the waterfall data 10 can be formed as a grayscale image. However, the waterfall data 10 is not necessarily processed as image data.

[0019] The vibration sensor 30 may include one or more detection points that are not positioned along the target object 40 (in other words, not located in a suitable place to monitor the vibration of the target object 40). For example, the vibration sensor 30 may have several additional segments 32, as shown in Figure 1. The amplitude of vibration detected at detection points within these additional segments 32 does not accurately represent the amplitude of vibration of the target object 40. Hereinafter, detection points positioned along the target object 40 will be referred to as "monitoring points," and detection points not positioned along the target object 40 (e.g., detection points included in the additional segments 32) will be referred to as "non-monitoring points."

[0020] Taking into account the presence of unmonitored areas, the classification device 2000 is configured to detect monitored areas of the vibration sensor 30 using waterfall data 10. Specifically, the classification device 2000 acquires the waterfall data 10 and performs semantic segmentation on the waterfall data 10. Semantic segmentation is a technique that analyzes sets of two or more data and classifies them into two or more classes. Through semantic segmentation, each element of the waterfall data 10 is assigned one of two or more classes (in other words, types).

[0021] The classes may include "Normal" and "Abnormal". Elements of the waterfall data 10 that are presumed to represent the amplitude of vibration detected at the monitoring points of the vibration sensor 30 are assigned the "Normal" class. On the other hand, elements of the waterfall data 10 that are presumed to represent the amplitude of vibration detected at the non-monitoring points of the vibration sensor 30 are assigned the "Abnormal" class.

[0022] As a result of semantic segmentation of the waterfall data 10, the classifier 2000 generates class data indicating the class of each element of the waterfall data 10. The class data can be formed in the same way as the waterfall data 10. Assume that the waterfall data 10 is formed as an NxM matrix denoted as W. In this case, the class data can be formed as an NxM matrix denoted as C (where C[i][j] indicates the class assigned to W[i][j]).

[0023] Based on the class data, the classification device 2000 determines which detection locations are monitored locations. That is, the classification device 2000 classifies the detection locations into monitored locations and unmonitored locations based on the class data. If the waterfall data 10 is matrix data in which detection locations are represented as columns, this can be rephrased as, "The classification device 2000 determines, based on the class data, which columns of the waterfall data 10 indicate the amplitude of vibrations detected at the monitored locations." That is, the classification device 2000 classifies the columns of the waterfall data 10 into columns corresponding to monitored locations and columns corresponding to unmonitored locations based on the class data. Some other examples of classification devices that determine monitored and unmonitored locations, or parts of monitored and unmonitored locations, may be analytical techniques based on statistical measures of waterfall data.

[0024] <Examples of effects and benefits> According to the classification device 2000 of Embodiment 1, elements of the waterfall data 10 are classified into normal and abnormal classes by semantic segmentation, and the detection locations of the vibration sensor 30 are classified into monitored and unmonitored locations based on the classification results of the elements of the waterfall data 10. This is a novel method for processing waterfall data 10 acquired from a vibration sensor that monitors an object.

[0025] As will be detailed later, classifying detection locations into monitored and unmonitored areas is useful in various ways. For example, the classification device 2000 can remove the influence of unmonitored areas from the waterfall data 10, which cannot accurately measure the amplitude of vibration of the target object 40, by removing the unmonitored areas from the waterfall data 10.

[0026] The following provides a more detailed explanation of the classification device 2000.

[0027] <Example of functional configuration> Figure 2 is a block diagram showing an example of the functional configuration of the classification device 2000 of Embodiment 1. The classification device 2000 includes an acquisition unit 2020, a segmentation unit 2040, and a classification unit 2060. The acquisition unit 2020 acquires waterfall data 10. The segmentation unit 2040 performs semantic segmentation on the waterfall data 10 to generate class data indicating one of two or more classes for each element of the waterfall data 10. Based on the class data, the classification unit 2060 classifies the detection locations of the vibration sensor 30 into monitored locations and non-monitored locations.

[0028] <Example hardware configuration> The classification device 2000 can be implemented by one or more computers. Each of these computers may be a dedicated computer manufactured to implement the classification device 2000, or it may be a general-purpose computer such as a personal computer (PC), server machine, or mobile device.

[0029] The classification device 2000 can be realized by installing an application on a computer. The application is implemented by a program that makes the computer function as the classification device 2000. In other words, the program is an implementation of the functional part of the classification device 2000 as illustrated in Figure 2.

[0030] Figure 3 is a block diagram showing an example of the hardware configuration of a computer 1000 that implements the classification device 2000 of Embodiment 1. In Figure 3, the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120.

[0031] Bus 1020 is a data transmission channel for the processor 1040, memory 1060, storage device 1080, input / output interface 1100, and network interface 1120 to send and receive data to and from each other. The processor 1040 is a processor such as a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processor (DAP), or Field-Programmable Gate Array (FPGA). Memory 1060 is a main memory element such as Random Access Memory (RAM) or Read Only Memory (ROM). Storage device 1080 is an auxiliary storage element such as a hard disk, Solid State Drive (SSD), or memory card. The input / output interface 1100 is an interface between the computer 1000 and peripheral devices such as a keyboard, mouse, or display device. The network interface 1120 is an interface between the computer 1000 and a network. The network may be a Local Area Network (LAN) or a Wide Area Network (WAN).

[0032] The hardware configuration of computer 1000 is not limited to that shown in Figure 3. For example, as described above, the classification device 2000 may be implemented as a combination of multiple computers. In this case, these computers may be connected to each other via a network.

[0033] <Processing flow> Figure 4 is a flowchart illustrating an exemplary processing flow by the classification device 2000 of Embodiment 1. The acquisition unit 2020 acquires waterfall data 10 (S102). The segmentation unit 2040 performs semantic segmentation on the waterfall data 10 to generate class data (S104). The classification unit 2060 classifies the detected locations into monitored locations and non-monitored locations (S106).

[0034] <Acquisition of waterfall data 10: S102> The acquisition unit 2020 acquires waterfall data 10 (S102). As described above, the waterfall data 10 represents the time series of detection data 20. In some embodiments, the acquisition unit 2020 can acquire two or more detection data 20 at different points in time, thereby acquiring the time series of those detection data 20 as waterfall data 10. In other words, the acquisition unit 2020 converts the two or more acquired detection data 20 into waterfall data 10.

[0035] There are various ways to acquire the detection data 20. In some embodiments, the vibration sensor 30 stores the detection data 20 in a storage device accessible by the classification device 2000. In this case, the acquisition unit 2020 can access the storage device to acquire the detection data 20. In other embodiments, the vibration sensor 30 transmits the detection data 20 to the classification device 2000. In this case, the acquisition unit 2020 can acquire the detection data 20 by receiving the detection data 20 transmitted by the vibration sensor 30. The acquisition unit 2020 may acquire two or more detection data 20 one by one, or simultaneously.

[0036] The conversion of two or more detection data 20 into waterfall data 10 may be performed in advance by another computer. In this case, the acquisition unit 2020 can acquire the waterfall data 10 all at once.

[0037] <Semantic Segmentation: S104> The segmentation unit 2040 performs semantic segmentation on the waterfall data 10 (S104). As described above, the classification device 2000 can handle two classes called "normal" and "abnormal". In this case, as a result of semantic segmentation, one of these two classes is assigned to each element of the waterfall data 10.

[0038] There are various methods for performing semantic segmentation on waterfall data 10. In some embodiments, a machine learning-based model called a "segmentation model" is used to perform semantic segmentation on waterfall data 10. A segmentation model can be configured to take waterfall data 10 as input, analyze the waterfall data 10 to determine the class of each element, and output class data. The analysis of waterfall data 10 may include extracting features from the waterfall data 10 and upsampling the extracted features to the same size as the input data to generate class data.

[0039] Segmentation models can be implemented as one of various types of machine learning-based models, such as neural networks. Some examples of neural networks suitable for implementing segmentation models include U-net, region-based convolutional neural network (R-CNN), Fast R-CNN, and Faster R-CNN.

[0040] The segmentation model is trained prior to the operational phase (in other words, the testing phase) of the classification device 2000. Hereafter, the computer used to train the segmentation model will be referred to as the "training device." The training device may be the classification device 2000 or another device.

[0041] To train a segmentation model, the training device uses a training dataset containing two or more training data sets. The training data can be formed as a combination of training input data and ground truth data. The training input data represents waterfall data, and the ground truth data represents class data corresponding to the training input data. Specifically, the ground truth data is class data, where each element indicates the class to which the corresponding element of the training input data should be assigned.

[0042] There are various well-known techniques for training machine learning-based models using training datasets, and any one of these techniques can be applied to a training device to train a segmented model. For example, a training device inputs training input data into a segmented model and retrieves class data from the segmented model. The training device then updates the trainable parameters of the segmented model (e.g., edge weights and biases of the neural network) based on ground truth data and loss, which represents the magnitude of the difference between the ground truth data and the class data output from the segmented model. The training device trains the segmented model by repeatedly updating it with multiple training data points from the training dataset.

[0043] Note that the size of data that the segmentation model can process at one time may be smaller than the size of the waterfall data 10. In this case, the segmentation unit 2040 divides the waterfall data 10 into two or more partial data (called "patches") that are the same size as the input size of the segmentation model. Next, the segmentation unit 2040 inputs the patches into the segmentation model to obtain the class data for each patch. The segmentation unit 2040 can obtain the class data for the entire waterfall data 10 by concatenating the class data of each patch.

[0044] In some embodiments, the segmentation unit 2040 may further take into account one or more measurement conditions in order to perform semantic segmentation on the waterfall data 10. For example, the measurement conditions may include the period during which the vibration sensor 30 performs measurements to generate the waterfall data 10. In another example, the measurement conditions may include one or more weather conditions such as the weather type (e.g., sunny, cloudy, or rainy), temperature, or humidity during the period in which the vibration sensor 30 performs measurements. In yet another example, the measurement conditions may include parameters relating to the vibration sensor 30, such as the sensitivity of the vibration sensor 30.

[0045] If one or more measurement conditions are used for semantic segmentation, the segmentation model may be configured to take one or more measurement conditions as input. Furthermore, the segmentation model may be further configured to extract features from each of the waterfall data 10 and measurement conditions, compute a combined feature from these extracted features, and upsample the combined feature to the same size as the input data to generate class data.

[0046] The segmentation model needs to be trained using not only waterfall data but also measurement conditions. Therefore, the training input data includes not only waterfall data but also measurement conditions so that the training device can train the segmentation model using the measurement conditions.

[0047] To use the measurement conditions for semantic segmentation, the acquisition unit 2020 acquires the measurement conditions. For example, the acquisition unit 2020 may acquire the measurement conditions from a storage device, which pre-stores the measurement conditions and is accessible to the classification device 2000. In another example, the measurement conditions may be transmitted to the classification device 2000 from another computer, and the acquisition unit 2020 may receive those measurement conditions.

[0048] <Detection location classification: S106> The classification unit 2060 classifies the detected locations into monitored locations and non-monitored locations based on the class data (S106). Conceptually, the more the elements of the waterfall data 10 corresponding to the detected location are classified into the "normal" class, the more likely it is that the detected location is a monitored location.

[0049] Since the waterfall data 10 includes two or more elements for each detection location, both the "normal" class and the "abnormal" class can be assigned to the elements of the waterfall data 10 corresponding to the same detection location. To address this situation, the classification unit 2060 may determine whether a detection location is a monitoring location based on the number of elements of the waterfall data 10 corresponding to the detection locations assigned the "normal" class for each detection location. Specifically, the classification unit 2060 may use the class data to identify the number of elements of the waterfall data 10 corresponding to the detection locations assigned the "normal" class for each detection location, and determine whether the identified number is greater than or equal to a predetermined threshold value.

[0050] If the identified number is greater than or equal to the threshold value, the classification unit 2060 determines that the detection location is a monitoring location. On the other hand, if the identified number is less than the threshold value, the classification unit 2060 determines that it is a non-monitoring location.

[0051] Assume that the waterfall data 10 is NxM matrix data where the columns represent detection locations and the rows represent time points. Further, the above-mentioned threshold value is set to T. In this case, the classification unit 2060 identifies the number of elements (denoted as C[j]) assigned the "normal" class in column j for each column j (1 <= j <= M) of the waterfall data 10. If C[j] >= T (that is, in column j, the "normal" class is assigned to T or more elements), it is determined that the detection location corresponding to column j is a monitoring location. If C[j] < T (that is, in column j, the "normal" class is assigned to less than T elements), it is determined that the detection location corresponding to column j is a non-monitoring location.

[0052] In other embodiments, instead of using the number of elements in column j of the waterfall data 10 assigned the "normal" class, monitoring locations may be detected using the proportion of elements in column j of the waterfall data 10 assigned the "normal" class (denoted as P[j]). P[j] can be defined as "P[j] = C[j] / N". In this case, the classification unit 2060 may compare P[j] with a predetermined threshold for each column j to determine whether the detected location corresponding to column j is a monitoring location or a non-monitoring location.

[0053] In other embodiments, the classification unit 2060 may consider the total number of monitoring points in the vibration sensor 30. If the length of the target object 40 is known in advance, the total number of monitoring points in the vibration sensor 30 can be estimated, and this total number is denoted as Ns. Assume that the length of the target object 40 is L [m] and the interval between monitoring points is defined in advance as a [m]. In this case, the total number of monitoring points can be estimated as Ns = L / a.

[0054] The classification unit 2060 may also sort the detected locations in order of their likelihood of being monitored, taking into account the total number of monitored locations, and determine that the first to (L / a) detected locations are monitored locations. The remaining detected locations are determined to be non-monitored locations. The likelihood that a detected location is a monitored location can be represented by the number of elements in the waterfall data 10 corresponding to the detected location assigned the "normal" class, for example, in the example given above, it can be represented by C[j].

[0055] The classification unit 2060 can generate information called "detection location information" that shows the results of the classification of detection locations. Specifically, the detection location information may show two lists called "monitoring location list" and "non-monitoring location list". The monitoring location list shows identifiers of detection locations classified as monitoring locations. On the other hand, the non-monitoring location list shows identifiers of detection locations classified as non-monitoring locations.

[0056] <Examples of using detection location information> The detection location information can be used in various ways. For example, the classification device 2000 may use the detection location information to remove the influence of unmonitored locations from the waterfall data 10. Specifically, by removing the elements of the waterfall data 10 corresponding to unmonitored locations, the classification device 2000 can obtain time-series data representing the amplitude of vibration at each monitored location, in other words, the amplitude of vibration at each point of the target object 40. Hereinafter, this time-series data will be referred to as "monitoring data".

[0057] Figure 5 shows how monitoring data is generated based on waterfall data 10. In this example, waterfall data 10 is matrix data where columns represent detection locations and rows represent time points. The classification device 2000 performs steps S102 to S106 to classify the detection locations into monitored locations and unmonitored locations. In Figure 5, the columns for unmonitored locations are colored with diagonal stripes.

[0058] The classification device 2000 removes columns from the waterfall data 10 that are not being monitored, and concatenates the remaining columns into a single matrix data. This matrix data is treated as the monitored data 50.

[0059] The classification device 2000 can use detection location information to identify one or more locations of the target object 40. Specifically, the classification device 2000 can determine the interval between monitoring locations by dividing the length of the target object 40 by the number of monitoring locations. If the interval between monitoring locations is determined to be I [m], the location of the target object 40 corresponding to the k-th monitoring location can be determined to be I*k [m] from the starting point of the target object 40. By applying the location identification result to the monitoring data 50, the classification device 2000 can modify the monitoring data 50 to show the time series of vibration amplitudes at each location of the target object 40 corresponding to the monitoring locations.

[0060] The detection location information may be used not only for the current waterfall data 10 but also for waterfall data 10 to be acquired in the future. This allows the classification device 2000 to avoid frequently performing the classification of detection locations, thereby reducing the computer resources used by the classification device 2000.

[0061] Figure 6 is a flowchart illustrating an example of the processing flow performed by the classification device 2000, which utilizes the detection location information in subsequent processing.

[0062] The classification device 2000 acquires the waterfall data 10 generated by the vibration sensor 30 (S202) and determines whether or not the detection location information is stored in the memory device (S204).

[0063] If it is determined that detection location information is not stored in the storage device (S204: NO), the classification device 2000 performs semantic segmentation on the waterfall data 10 (S206) and classifies the detection locations (S208). Next, the classification device 2000 generates detection location information and saves it to the storage device (S210). Based on the detection location information, the classification device 2000 generates monitoring data 50 from the waterfall data 10 (S212).

[0064] In step S204, if it is determined that detection location information is stored (S204: YES), the classification device 2000 obtains detection location information from the storage device (S214) and generates monitoring data 50 from the waterfall data 10 based on the detection location information (S212).

[0065] Here, in order to regenerate the detection location information at some point, an expiration date can be set for the detection location information. In this case, the classification device 2000 also determines whether the detection location information in the storage device is valid or not based on the expiration date. Next, the classification device 2000 generates monitoring data 50 using the detection location information only if valid detection location information is stored in the storage device. Otherwise, the classification device 2000 executes steps S206 to S210 to generate new detection location information.

[0066] <Output of results> The classification device 2000 may be configured to output one or more pieces of information (collectively referred to as "output information") related to the results of the classification of the detection locations. The output information may include detection location information, monitoring data 50, or both.

[0067] There are various ways to output the output information. In some implementations, the output information may be stored in a memory device, displayed on a display device, or transmitted to another computer (such as the user's PC or smartphone of the classification device 2000).

[0068] Embodiment 2 <Overview> Figure 7 shows an overview of the classification device 2000 of Embodiment 2. It should be noted that the overview shown in Figure 7 is an example of the operation of the classification device 2000 of Embodiment 2, intended to facilitate understanding of the device, and does not limit or restrict the range of operations that the classification device 2000 of Embodiment 2 can perform.

[0069] In this embodiment, it is assumed that one or more moving objects 70 that vibrate the target object 40 are on the target object 40. For example, if the target object 40 is a road, the moving objects 70 may be vehicles traveling on the road (e.g., cars or motorcycles). Furthermore, it is assumed that the waterfall data 10 is formed as matrix data where columns represent detection locations and rows represent time points (or vice versa). Unless otherwise specified, the columns of the waterfall data 10 represent detection locations and their rows represent time points.

[0070] Under the above assumptions, the classification device 2000 detects the trajectories (e.g., time series of positions) of one or more moving objects 70 from the waterfall data 10 and uses the detected trajectories to modify the detection location information (i.e., the result of the classification of detection locations performed by the classification unit 2060). That is, the classification unit 2060 may reclassify some of the detection locations it has classified as monitored locations as unmonitored locations, or it may reclassify some of the detection locations it has classified as unmonitored locations as monitored locations, or both. Hereinafter, the trajectories of the moving objects 70 will be referred to as "object trajectories".

[0071] The closer the detection point is to the position of the moving object 70, the larger the amplitude of the vibration detected at that point is expected to be. This allows the object's trajectory to be detected based on the vibration amplitude shown in the waterfall data 10.

[0072] Figure 8 shows the object trajectory detected from the waterfall data 10. In this example, the waterfall data 10 is formed as a waterfall image 60, where the X-axis represents the detection location and the Y-axis represents the time. The waterfall image 60 is shown as a grayscale image where the larger the amplitude of the vibration corresponding to the pixel, the larger the pixel value. For illustrative purposes, darker colors are drawn with denser and larger dots. The object trajectory 80 is drawn as a white line superimposed on the waterfall image 60.

[0073] As shown in Figure 8, the object trajectory 80 may be discontinuous (in other words, disconnected) due to the presence of unmonitored areas as shown in Figure 8. If the detected areas are not correctly classified as monitored or unmonitored, the object trajectory 80 becomes discontinuous when the areas corresponding to the unmonitored areas are removed from the waterfall data 10.

[0074] Figure 9 shows the object trajectory 80 when the detected area is not correctly classified. The waterfall image 60 includes anomaly sections 90, which are regions of one or more continuous unmonitored sections. As shown in Figure 9, when the classification device 2000 removes the anomaly sections 90 from the waterfall image 60 to generate monitoring data 50, the object trajectory 80 becomes discontinuous.

[0075] On the other hand, if the detected locations are correctly classified into monitored and unmonitored locations, the object trajectory 80 becomes continuous when the area corresponding to the unmonitored location is removed from the waterfall data 10. Figure 10 shows the object trajectory 80 when the detected locations are correctly classified. In the case shown in Figure 10, when the classification device 2000 removes the abnormal section 90 from the waterfall image 60 to generate monitored data 50, the object trajectory 80 becomes continuous within the monitored data 50.

[0076] Taking the above into consideration, the classification device 2000 of Embodiment 2 corrects the detection location information based on the object trajectory 80 detected from the waterfall data 10. Specifically, the start point, end point, or both of the abnormal section 90 are corrected so that the object trajectory 80 is almost continuous before and after the abnormal section 90 (in other words, so that the object trajectory 80 becomes almost continuous when the abnormal section 90 is removed from the waterfall data 10).

[0077] <Examples of effects and benefits> According to the classification device 2000 of Embodiment 2, the detection location information is corrected based on the object trajectory 80, which is the trajectory of the moving object 70 moving over the target object 40. As a result, errors in the detection location information due to misclassification can be reduced, and the detection location information can be made more accurate.

[0078] The following provides a more detailed explanation of the classification device 2000.

[0079] <Example of functional configuration> Figure 11 is a block diagram showing an example of the functional configuration of the classification device 2000 of Embodiment 2. Similar to the classification device 2000 of Embodiment 1, the classification device 2000 of Embodiment 2 includes an acquisition unit 2020, a segmentation unit 2040, and a classification unit 2060. Furthermore, the classification device 2000 of Embodiment 2 further includes a detection unit 2080 and a correction unit 2100. The detection unit 2080 detects one or more object trajectories 80 from the waterfall data 10. The correction unit 2100 corrects the detection location information based on the object trajectories 80.

[0080] <Example hardware configuration> The classification device 2000 of Embodiment 2 may be implemented in the same way as the classification device 2000 of Embodiment 1. For example, the classification device 2000 of Embodiment 2 is implemented by the computer 1000 shown in Figure 3. However, the storage device 1080 of Embodiment 2 includes a program that implements the functions of the classification device 2000 of Embodiment 2.

[0081] <Processing flow> Figure 12 is a flowchart illustrating an exemplary flow of processing performed by the classification device 2000 of Embodiment 2. The classification device 2000 of Embodiment 2 can perform steps S102 to S106 in the same manner as in Embodiment 1. After the execution of step S106, the detection unit 2080 detects the object trajectory 80 from the waterfall data 10 (S302) and corrects the detection location information based on the object trajectory 80 (S304).

[0082] <Detection of object trajectory 80: S302> The detection unit 2080 detects one or more object trajectories 80 from the waterfall data 10 (S302). Methods for detecting the trajectory of a moving object 70 from time-series data showing the amplitude of vibration at two or more locations are well known, and one of these methods can be applied to the detection unit 2080. For example, the detection unit 2080 can detect one or more locations (i.e., detection points) where the moving object 70 is presumed to be located for each detection data 20 in the waterfall data 10. Specifically, the detection data 20 may have one or more local maximums, and the detection points corresponding to these local maximums are presumed to be the locations of the moving object 70. Then, by connecting the detected locations of the moving object 70 over time, the object trajectory 80 is detected.

[0083] <Correction of detection location information: S304> The correction unit 2100 corrects the detection location information generated by the classification unit 2060 (S304). To do this, the classification device 2000 identifies one or more abnormal sections 90 from the waterfall data 10 based on the detection location information. Next, the correction unit 2100 corrects the detection location information for each abnormal section 90 by correcting the start point, end point, or both of the abnormal section 90 so that the object trajectory 80 is substantially continuous between the abnormal sections 90.

[0084] The correction unit 2100 may, if there is a single object trajectory 80 traversing the abnormal section 90, identify the target width Wt based on the object trajectory 80 and correct the detection location information by changing the width of the abnormal section 90 to Wt. Changing the width of the abnormal section 90 includes reclassifying the detection locations near the boundary of the abnormal section 90.

[0085] Assume that the start and end points of the anomaly section 90 are detection points Ss and Se, respectively. Furthermore, the target width Wt of the anomaly section 90 is 6 less than its current width. In this case, the correction unit 2100 can shift the start point by +3 and the end point by -3. To do this, detection points Ss, Ss+1, Se+2, Se, Se-1, and Se-2 are reclassified as monitoring points.

[0086] The target width Wt of the abnormal section is determined by line detection. link The endpoint of the object trajectory 80 can be identified by using techniques such as detection or vehicle tracking algorithms to find the endpoint. The width Wt represents the length or distance of the unmonitored section. If Wt is estimated correctly, the object trajectory is made continuous after removing this anomalous section, as shown in Figure 10. If the object trajectory is not continuous, the endpoint coordinates of the vehicle trajectory are corrected and the width Wt is estimated again.

[0087] If there are two or more object trajectories 80 that cross the same abnormal section 90, the correction unit 2100 may determine the target width Wt of the abnormal section 90 based on the object trajectories 80 that cross the abnormal section 90, and change the width of the abnormal section 90 to Wt by shifting the start and end points of the abnormal section 90 by the same distance from each other. Specifically, for each object trajectory 80 that crosses the same abnormal section 90, the correction unit 2100 identifies a candidate width Wc of the abnormal section 90, and calculates a statistical value (e.g., the mean) of the candidate width Wc as Wt. The candidate width Wc corresponding to object trajectory OT1 can be identified in the same way as the method for identifying the target width Wt when there are no object trajectories 80 other than OT1 that cross the abnormal section 90 as described above.

[0088] When determining the target width Wt of the abnormal interval 90, the correction unit 2100 may exclude one or more outliers (called "outlier trajectories") from the object trajectories 80 from which the candidate width Wc has been calculated. Assume there are four object trajectories OT1, OT2, OT3, and OT4 that traverse the abnormal interval A1. Furthermore, object trajectory OT2 is determined to be an outlier trajectory. In this case, the correction unit 2100 calculates the candidate widths Wc1, Wc3, and Wc4 for OT1, OT3, and OT4, respectively. Since OT2 is determined to be an outlier, no candidate width Wc is calculated for OT2. Next, the correction unit 2100 calculates the statistical value SV1 of Wc1, Wc2, and Wc3 as the target width Wt of the abnormal interval A1, and corrects the start and end points of A1 included in the detection location information so that the width of A1 is changed to Wt (=SV1).

[0089] To determine whether the object trajectory 80 is an outlier trajectory, the correction unit 2100 may calculate the degree of irregularity of the object trajectory 80. The correction unit 2100 determines whether the degree of irregularity of the object trajectory 80 is less than a predetermined threshold. If the correction unit 2100 determines that the degree of irregularity of the object trajectory 80 is less than the predetermined threshold, it calculates a candidate width for an abnormal interval based on the object trajectory 80. On the other hand, if the correction unit 2100 determines that the degree of irregularity of the object trajectory 80 is greater than or equal to the predetermined threshold, it does not calculate a candidate width for an abnormal interval based on the object trajectory 80.

[0090] In some embodiments, the degree of irregularity of the object trajectory 80 can be expressed using the degree of linearity (in other words, uniformity) of the object trajectory 80. Specifically, the lower the degree of linearity of the object trajectory 80, the higher the degree of irregularity of the object trajectory 80 is determined to be. There are well-known methods for measuring the degree of linearity of a curve, and one of these methods can be applied to the correction unit 2100 to calculate the degree of linearity of the object trajectory 80.

[0091] In other embodiments, the degree of irregularity may be measured based on direction, changes in velocity, overall motion behavior, or two or more of these, calculated using the object trajectory 80. Specifically, for driving behavior such as low speed, excessive speed, or abrupt changes in velocity, the object trajectory is tracked for the measurements described above. Irregular trajectories indicate the measured values ​​of these behaviors as outliers compared to adjacent vehicle trajectories in this measurement section.

[0092] <Uses of the results> In some embodiments, the classification device 2000 generates monitoring data 50 from the waterfall data 10 using corrected detection location information. The monitoring data 50 and object trajectories 80 detected from the waterfall data 10 can then be used in a traffic flow monitoring application. The traffic flow monitoring application can calculate traffic flow characteristics such as vehicle speed and vehicle count. These characteristics can be used to monitor traffic flow.

[0093] Programs can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs). Programs may also be provided to a computer using various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0094] While the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications that will be understood by those skilled in the art can be made to the structure and details of the present disclosure within the scope of the present invention.

[0095] All or part of the embodiments described above may be described as follows, but are not limited to: <Note> (Note 1) At least one memory configured to store instructions, Execute the aforementioned instruction, Waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors placed along the target object is acquired. Semantic segmentation is performed on the waterfall data to generate class data indicating a normal class or an abnormal class for each element of the waterfall data, the normal class is assigned to the element in which the detection location is presumed to be a monitoring location, the abnormal class is assigned to the element in which the detection location is presumed to be a non-monitoring location, the monitoring location is the detection location located along the target object, and the non-monitoring location is the detection location not located along the target object. Based on the class data, the detection locations are classified into monitored locations and non-monitored locations. A classification device comprising at least one processor configured as follows: (Note 2) The classification of the detection locations is as follows for each detection location: The number of elements in the waterfall data to which the normal class is assigned corresponds to the detection location is calculated, Based on the number calculated above, it is determined whether the detection location is a monitored location or a non-monitored location. A classification device as described in Appendix 1, which includes performing the following: (Note 3) The aforementioned at least one processor is For each detection location, detection location information is generated indicating whether the detection location is a monitored location or a non-monitored location. From the waterfall data, one or more trajectories of moving objects, which are objects moving on the target object, are detected. A classification device according to Appendix 1 or 2, configured to correct the detection location information based on the detected trajectory. (Note 4) The correction of the detection location information is performed for each of the abnormal intervals, which are areas of one or more consecutive unmonitored locations in the waterfall data. For each of the trajectories that cross the aforementioned abnormal section, a candidate width of the abnormal section is identified based on the trajectory, The statistical value of the candidate width calculated above is used as the target width for the abnormal interval. A classification device according to Appendix 3, which includes correcting the width of the abnormal interval to the target width. (Note 5) Identifying the candidate width of the abnormal section of the trajectory is: To calculate the degree of irregularity of the aforementioned trajectory, The classification device according to Appendix 4, which includes calculating the candidate width of the abnormal interval based on the trajectory if the degree of irregularity of the trajectory is less than a predetermined threshold. (Note 6) The classification device according to Appendix 5, wherein the degree of irregularity of the trajectory is determined based on the degree of linearity of the trajectory. (Note 7) A classification method calculated by a computer, This involves acquiring waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors placed along the target object, Semantic segmentation is performed on the waterfall data to generate class data indicating a normal class or an abnormal class for each element of the waterfall data, the normal class is assigned to the element in which the detection location is predicted to be a monitoring location, the abnormal class is assigned to the element in which the detection location is presumed to be a non-monitoring location, the monitoring location is the detection location located along the target object, and the non-monitoring location is the detection location not located along the target object. Based on the class data, the detection locations are classified into monitored locations and non-monitored locations. A classification method that includes this. (Note 8) Classifying the detection locations means that for each detection location, The number of elements in the waterfall data to which the normal class is assigned corresponds to the detection location is calculated, Based on the number calculated above, it is determined whether the detection location is a monitored location or a non-monitored location. The classification method described in Appendix 7, which includes performing the following. (Note 9) For each detection location, detection location information is generated indicating whether the detection location is a monitored location or a non-monitored location. From the waterfall data, one or more trajectories of moving objects, which are objects moving on the target object, are detected. The classification method according to Appendix 7 or 8, further comprising correcting the detection location information based on the detected trajectory. (Note 10) The correction of the detection location information is performed for each of the abnormal intervals, which are areas of one or more consecutive unmonitored locations in the waterfall data. The correction of the detection location information is performed for each of the abnormal intervals, which are areas of one or more consecutive unmonitored locations in the waterfall data. For each of the trajectories that cross the aforementioned abnormal section, a candidate width of the abnormal section is identified based on the trajectory, The statistical value of the candidate width calculated above is used as the target width for the abnormal interval. Correcting the width of the abnormal interval to the target width, The classification method described in Appendix 9, which includes performing the following. (Note 11) Identifying the candidate width of the abnormal section of the trajectory is: To calculate the degree of irregularity of the aforementioned trajectory, If the degree of irregularity of the trajectory is less than a predetermined threshold, the candidate width of the abnormal interval is calculated based on the trajectory. The classification method described in Appendix 10, including the method described in Appendix 10. (Note 12) The degree of irregularity of the trajectory is determined based on the degree of linearity of the trajectory, according to the classification method described in Appendix 11. (Note 13) On the computer, This involves acquiring waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors placed along the target object, Semantic segmentation is performed on the waterfall data to generate class data indicating a normal class or an abnormal class for each element of the waterfall data, the normal class is assigned to the element in which the detection location is predicted to be a monitoring location, the abnormal class is assigned to the element in which the detection location is presumed to be a non-monitoring location, the monitoring location is the detection location located along the target object, and the non-monitoring location is the detection location not located along the target object. Based on the class data, the detection locations are classified into monitored locations and non-monitored locations. A non-temporary, computer-readable storage medium that stores a program to execute. (Note 14) Classifying the detection locations means that for each detection location, The number of elements in the waterfall data to which the normal class is assigned corresponds to the detection location is calculated, Based on the number calculated above, it is determined whether the detection location is a monitored location or a non-monitored location. A storage medium as described in Appendix 13, which includes performing the following actions. (Note 15) The program is installed on the computer. For each detection location, detection location information is generated indicating whether the detection location is a monitored location or a non-monitored location. From the waterfall data, one or more trajectories of moving objects, which are objects moving on the target object, are detected. Correcting the detection location information based on the detected trajectory, A storage medium as described in Appendix 13 or 14, which further enables the execution of the above. (Note 16) The correction of the detection location information is performed for each of the abnormal intervals, which are areas of one or more consecutive unmonitored locations in the waterfall data. For each of the trajectories that cross the aforementioned abnormal section, a candidate width of the abnormal section is identified based on the trajectory, The statistical value of the candidate width calculated above is used as the target width for the abnormal interval. Correcting the width of the abnormal interval to the target width, A storage medium as described in Appendix 15, which includes performing the following actions. (Note 17) Identifying the candidate width of the abnormal section of the trajectory is: To calculate the degree of irregularity of the aforementioned trajectory, If the degree of irregularity of the trajectory is less than a predetermined threshold, the candidate width of the abnormal interval is calculated based on the trajectory. The storage media described in Appendix 16, including the storage media described in Appendix 16. (Note 18) The degree of irregularity of the trajectory is determined based on the degree of linearity of the trajectory, according to the storage medium described in Appendix 17. [Explanation of symbols]

[0096] 10 Waterfall Data 20 detection data 30 Vibration Sensor 40 Target Objects 50 Monitoring Data 60 Waterfall Images 70 Moving objects 80 Object trajectory 90 Abnormal interval 1000 computers 1020 Bus 1040 processor 1060 memory 1080 storage device 1100 Input / Output Interface 1120 Network Interface 2000 classification device 2020 Acquisition Department 2040 Segmentation Department 2060 Classification Department 2080 Detection Unit 2100 Correction Unit

Claims

1. An acquisition unit acquires waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors arranged along the target object, The system includes a segmentation unit that inputs the waterfall data into a segmentation model, performs semantic segmentation on the waterfall data, and generates class data indicating a normal class or an abnormal class for each element of the waterfall data. The normal class is assigned to the element in which the detection location is presumed to be a monitoring location. The aforementioned anomaly class is assigned to the element in which the detection location is presumed to be an unmonitored location. The aforementioned monitoring location is the detection location where the vibration sensor is positioned along the target object. The non-monitored locations are the detection locations where the vibration sensors are not positioned along the target object. The segmentation model is pre-trained based on a loss derived from the error between the training waterfall data and ground truth data, so that when the waterfall data is input, it outputs class data indicating the normal class or the abnormal class for each element of the waterfall data. The ground truth data is class data that indicates the normal class for elements corresponding to the detection locations where the vibration sensor is positioned along the target object, and indicates the abnormal class for elements corresponding to the detection locations where the vibration sensor is not positioned along the target object. A classification device having a classification unit that classifies the detection locations into monitored locations and non-monitored locations based on the class data.

2. The classification of the detection locations is as follows for each detection location: The number of elements in the waterfall data to which the normal class is assigned corresponds to the detection location is calculated, The classification device according to claim 1, comprising determining whether the detection location is a monitoring location or a non-monitoring location based on the number calculated above.

3. The classification unit generates detection location information for each detection location, indicating whether the detection location is a monitored location or a non-monitored location. A detection unit that detects one or more trajectories of moving objects, which are objects moving on the target object, from the waterfall data, A classification device according to claim 1 or 2, further comprising a correction unit that corrects the detection location information based on the detected trajectory.

4. The correction of the detection location information is performed for each of the abnormal intervals, which are one or more consecutive areas of the unmonitored locations in the waterfall data. For each of the trajectories that cross the aforementioned abnormal section, a candidate width of the abnormal section is identified based on the trajectory, The statistical value of the identified candidate width is calculated as the target width of the abnormal interval, The classification device according to claim 3, further comprising correcting the width of the abnormal interval to the target width.

5. Identifying the candidate width of the abnormal section of the trajectory is: To calculate the degree of irregularity of the aforementioned trajectory, The classification device according to claim 4, further comprising: calculating the candidate width of the abnormal interval based on the trajectory if the degree of irregularity of the trajectory is less than a predetermined threshold.

6. The classification device according to claim 5, wherein the degree of irregularity of the trajectory is determined based on the degree of linearity of the trajectory.

7. An acquisition step to acquire waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors arranged along the target object, The system includes a segmentation step of inputting the waterfall data into a segmentation model, performing semantic segmentation on the waterfall data, and generating class data indicating a normal class or an abnormal class for each element of the waterfall data. The normal class is assigned to the element in which the detection location is predicted to be a monitoring location. The aforementioned anomaly class is assigned to the element in which the detection location is presumed to be an unmonitored location. The aforementioned monitoring location is the detection location where the vibration sensor is positioned along the target object. The non-monitored locations are the detection locations where the vibration sensors are not positioned along the target object. The segmentation model is pre-trained based on a loss derived from the error between the training waterfall data and ground truth data, so that when the waterfall data is input, it outputs class data indicating the normal class or the abnormal class for each element of the waterfall data. The ground truth data is class data that indicates the normal class for elements corresponding to the detection locations where the vibration sensor is positioned along the target object, and indicates the abnormal class for elements corresponding to the detection locations where the vibration sensor is not positioned along the target object. A computer-based classification method comprising a classification step of classifying the detection locations into monitored locations and non-monitored locations based on the class data.

8. Classifying the detection locations means that for each detection location, The number of elements in the waterfall data to which the normal class is assigned corresponds to the detection location is calculated, The classification method according to claim 7, comprising determining whether the detection location is a monitoring location or a non-monitoring location based on the number calculated above.

9. An acquisition step to acquire waterfall data showing the amplitude of vibration at each time point and each detection point of vibration sensors arranged along the target object, The computer is made to perform a segmentation step, which involves inputting the waterfall data into a segmentation model, performing semantic segmentation on the waterfall data, and generating class data indicating whether each element of the waterfall data is a normal class or an abnormal class. The normal class is assigned to the element in which the detection location is predicted to be a monitoring location. The aforementioned anomaly class is assigned to the element in which the detection location is presumed to be an unmonitored location. The aforementioned monitoring location is the detection location where the vibration sensor is positioned along the target object. The non-monitored locations are the detection locations where the vibration sensors are not positioned along the target object. The segmentation model is pre-trained based on a loss derived from the error between the training waterfall data and ground truth data, so that when the waterfall data is input, it outputs class data indicating the normal class or the abnormal class for each element of the waterfall data. The ground truth data is class data that indicates the normal class for elements corresponding to the detection locations where the vibration sensor is positioned along the target object, and indicates the abnormal class for elements corresponding to the detection locations where the vibration sensor is not positioned along the target object. A program that causes the computer to perform a classification step of classifying the detection locations into monitored locations and non-monitored locations based on the class data.

10. Classifying the detection locations means that for each detection location, The number of elements in the waterfall data to which the normal class is assigned corresponds to the detection location is calculated, The program according to claim 9, comprising determining whether the detection location is a monitoring location or a non-monitoring location based on the number calculated above.

Images