Classification device, classification method, and program

Abstract

In order to perform labeling suitably even in a case where an ambiguous event boundary is present, a classification device (1) comprises: an acquisition means (11) for acquiring sequential data; and a class label assignment means (12) for assigning class labels to element data items included in the sequential data by repeatedly applying change point detection to the sequential data.

Description

[Technical Field]

[0001] The present invention relates to a classification device, a classification method, and a program. [Background technology]

[0002] There is a known technique for referencing sequence data such as time-series data and labeling what events occur at what points (times) in the sequence. This technique is also called temporal localization (TL). For example, Non-Patent Document 1 discloses a technique for performing temporal localization on untrimmed video. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Shou, Zheng, Dongang Wang, and Shih-Fu Chang, "Temporal action localization in untrimmed videos via multi-stage cnns," Proceedings of the IEEE conference on computer vision and pattern recognition. 2016 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in sequence data, ambiguity can occur at the boundaries of events (for example, the start point (start time) and end point (end time) of an event). With the technology described in Non-Patent Document 1, it was difficult to assign appropriate labels in such a situation.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of its purpose is to provide a technology that can perform appropriate labeling even when ambiguous event boundaries exist. [Means for solving the problem]

[0006] A classification device according to one aspect of the present invention includes an acquisition means for acquiring sequence data, and a class label assignment means for repeatedly applying change-point detection to the sequence data to assign a class label to each element data included in the sequence data.

[0007] A classification method according to one aspect of the present invention includes acquiring sequence data and assigning a class label to each element data included in the sequence data by repeatedly applying change-point detection to the sequence data.

[0008] A program according to one aspect of the present invention causes a computer to acquire sequence data and assign class labels to each element data included in the sequence data by repeatedly applying change-point detection to the sequence data. [Effects of the Invention]

[0009] According to one aspect of the present invention, labeling can be performed appropriately even when an ambiguous event boundary exists. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a classification device according to a first exemplary embodiment. [Figure 2] 1 is a flow chart showing the flow of a classification method according to the first exemplary embodiment. [Figure 3] FIG. 10 is a block diagram showing the configuration of a classification device according to a second exemplary embodiment. [Figure 4] FIG. 10 is a diagram showing a specific example of sequence data according to the second exemplary embodiment. [Figure 5]FIG. 10 is a flow chart showing the flow of a classification method according to the second exemplary embodiment. [Figure 6] FIG. 10 is a block diagram showing the configuration of a classification device according to an exemplary embodiment 3. [Figure 7] FIG. 10 is a block diagram showing the configuration of a classification device according to an exemplary embodiment 4. [Figure 8] 10A and 10B are diagrams illustrating examples of displays according to the fourth exemplary embodiment. [Figure 9] FIG. 1 is a block diagram illustrating a configuration of a computer that functions as a classification device according to exemplary embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0011] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0012] <Overview of the classification device> The classification device 1 according to this exemplary embodiment is a device that classifies sequential data into multiple events. The sequential data is ordered data consisting of one or more element data. In other words, the sequential data is composed of each element data. An example of the sequential data is video data or audio data. The sequential data may also be data that represents the transition of power consumption. The sequential data is not limited to time series data in which element data are arranged in chronological order, but may also be data in which element data are arranged in other order. The element data is data that becomes an element that makes up the sequential data. An example of the element data is a frame in a video.

[0013] The classification device 1 classifies the element data into a plurality of events by assigning a class label to each element data included in the sequence data. A class label is a label that indicates an event. Examples of class labels are labels that indicate the behavior of a subject (jumping, etc.), the work of a worker, information that identifies a speaker, or an event related to power consumption. An event is the behavior of a target, and examples of such an event include the behavior of a subject (jumping, etc.), the work of a worker, the speech of a speaker, and an action related to power consumption.

[0014] As an example, the classification device 1 is a device that logs the work content of workers from video footage captured by surveillance cameras at a construction site during work and creates a work journal. In this case, the series data is video data representing the video captured by the surveillance cameras. The class labels are labels that indicate the work content of the workers, and examples include "background class," "task w1," "task w2," "task w3," and so on. In this case, as an example, "background class" is a class label that is assigned when element data cannot be classified into any of "task w1," "task w2," and so on.

[0015] Moreover, as an example, the classification device 1 is a device that performs behavior analysis to determine what was done when and where from surveillance camera footage. In this case, the sequence data is video data representing the footage captured by the surveillance camera. The class labels are labels that indicate the behavior of the subject captured by the surveillance camera (e.g., jumping, etc.), and examples include "background class," "behavior a1," "behavior a2," "behavior a3," etc. In this case, as an example, the "background class" is a class label that is assigned when element data cannot be classified into any of "behavior a1," "behavior a2," etc.

[0016] Moreover, as an example, the classification device 1 is a device that logs who is speaking and when from audio data of a conference. In this case, the sequence data is audio data recorded from the conference. The class labels are labels that identify speakers, and examples include "background class," "speaker s1," "speaker s2," and "speaker s3," etc. In this case, as an example, the "background class" is a class label that is assigned when element data cannot be classified into any of "speaker s1," "speaker s2," etc.

[0017] Moreover, as an example, the classification device 1 is a device that analyzes the emotion of a specific speaker from the speech data of the specific speaker. In this case, the sequence data is audio recording data of the speech of the specific speaker. The class labels are labels that indicate the classification results of the speaker's emotion (e.g., emotion m1, emotion m2, ..., etc.), and examples include "background class," "emotion m1," "emotion m2," "emotion m3," .... In this case, as an example, the "background class" is a class label that is assigned when element data cannot be classified into any of "emotion m1," "emotion m2," ....

[0018] The classification device 1 is, for example, a device that analyzes what was done and when based on power consumption data for each time period. In this case, the sequence data is data that indicates the transition of power consumption, and the class labels are labels that indicate behavior related to power consumption.

[0019] <Configuration of classification device> The configuration of the classification device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the classification device 1. The classification device 1 includes an acquisition unit 11 and a class label assignment unit 12.

[0020] (Acquisition part 11) The acquiring unit 11 acquires the sequence data. The acquiring unit 11 may acquire the sequence data all at once, or may acquire the sequence data sequentially.

[0021] (Class label assignment unit 12) The class label assignment unit 12 assigns a class label to each element data included in the sequential data by repeatedly applying change-point detection to the sequential data. In other words, the class label assignment unit 12 assigns a class label to each element data included in the sequential data according to the result of change-point detection for the sequential data. Here, change-point detection (quickest change detection: QCD) is a method for detecting points (change points) where the feature quantities of sequential data change. The change-point detection method is not limited, but an example is the D-CUSUM (dynamic cumulative sum test) algorithm. An example of the D-CUSUM technique is the method described in the following document: G. Rovatsos, X. Jiang, AD Dominguez-Garcia and VV Veeravalli, "Statistical Power System Line Outage Detection Under Transient Dynamics," in IEEE Transactions on Signal Processing, vol. 65, no. 11, pp. 2787-2797, 1 June 2017, doi: 10.1109 / TSP.2017.2673802.

[0022] For example, the class label assignment unit 12 repeatedly applies change-point detection to the sequence data and assigns the class label to element data from the point where a change-point to a certain class is detected to the point where the next change-point is detected. More specifically, for example, the class label assignment unit 12 sequentially calculates statistics obtained by referring to each element data included in the sequence data and determines class transitions by comparing the calculated statistics with a threshold.

[0023] For example, the class label assignment unit 12 applies change-point detection to the video data and determines the class transition at the point where a change-point is detected. As an example, if the class label assignment unit 12 determines that a transition has occurred from a "background class" to "work w1" through change-point detection, the class label assignment unit 12 assigns the class label "work w1" to the element data up to the time the next change-point is detected. Also, if the class label assignment unit 12 determines that a transition has occurred from "work w1" to "work w2" through change-point detection, the class label assignment unit 12 assigns the class label "work w2" to the element data from the element data determined to have transitioned up to the time the next change-point is detected.

[0024] As described above, the classification device 1 according to this exemplary embodiment acquires sequence data and repeatedly applies change-point detection to the acquired sequence data to assign class labels to each element data included in the sequence data. Because change-point detection is repeatedly applied to sequence data, the classification device 1 according to this exemplary embodiment has the advantage of being able to appropriately label even when ambiguous event boundaries exist.

[0025] <Classification method flow> The flow of the classification method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the classification method S1. In step S11, the acquisition unit 11 acquires sequence data. In step S12, the class label assignment unit 12 repeatedly applies change-point detection to the sequence data, thereby assigning a class label to each element data included in the sequence data.

[0026] As described above, the classification method S1 according to this exemplary embodiment acquires sequence data, and then repeatedly applies change-point detection to the acquired sequence data to assign class labels to each element data included in the sequence data. Therefore, the classification method S1 according to this exemplary embodiment has the advantage of being able to appropriately label data even when ambiguous event boundaries exist.

[0027] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will not be repeated.

[0028] <Configuration of Classification Device 1A> 3 is a block diagram showing the configuration of a classification device 1A according to this exemplary embodiment. The classification device 1A includes a control unit 10A, a storage unit 20A, a communication unit 30A, and an input / output unit 40A.

[0029] (Communication unit 30A) The communication unit 30A communicates with devices external to the classification device 1A via a communication line. While the specific configuration of the communication line does not limit the present exemplary embodiment, examples of the communication line include a wireless local area network (LAN), a wired LAN, a wide area network (WAN), a public line network, a mobile data communication network, or a combination thereof. The communication unit 30A transmits data supplied from the control unit 10A to other devices, and supplies data received from other devices to the control unit 10A.

[0030] (Input / output section 40A) Input / output devices such as a keyboard, a mouse, a display, a printer, and a touch panel are connected to the input / output unit 40A. The input / output unit 40A receives various types of information input to the classification device 1A from the connected input devices. Furthermore, the input / output unit 40A outputs various types of information to the connected output devices under the control of the control unit 10A. An example of the input / output unit 40A is an interface such as a USB (Universal Serial Bus).

[0031] (Control unit 10A) As shown in FIG. 3, the control unit 10A includes an acquisition unit 11 and a class label assignment unit 12.

[0032] (Acquisition part 11) The acquisition unit 11 acquires the sequence data SD. For example, the acquisition unit 11 acquires the sequence data SD from another device via the communication unit 30A. Alternatively, the acquisition unit 11 may acquire the sequence data SD input via the input / output unit 40A. Alternatively, the acquisition unit 11 may acquire the sequence data SD by reading it from the storage unit 20A or an externally connected storage device.

[0033] The sequence data SD acquired by the acquisition unit 11 is, for example, time-series data having a time sequence. The sequence data SD includes one or more element data Δ^θ[k]. Here, index k is an index indicating the order of the element data Δ^θ[k]. If the sequence data SD is time-series data, index k indicates time. However, index k is not limited to indicating time. For example, if the sequence data SD is data including multiple regions, index k may be an index that identifies the regions included in the sequence data SD.

[0034] (Class label assignment unit 12) The class label assigning unit 12 assigns a class label l (l=0, 1, . . . , or L) to each element data Δ̂θ[k] included in the sequence data SD by repeatedly applying change point detection to the sequence data SD.

[0035] (Storage unit 20A) The storage unit 20A stores the sequence data SD acquired by the acquisition unit 11, as well as classified data CD in which a class label l is assigned to each element data Δ^θ[k] of the sequence data. The storage unit 20A also stores an estimation model M1. Here, storing the estimation model M1 in the storage unit 20A means that parameters defining the estimation model M1 are stored in the storage unit 20A.

[0036] (Estimated model M1) The estimation model M1 is a model used when the class label assignment unit 12 assigns a class label l to the element data Δ̂θ[k]. More specifically, the estimation model M1 is, for example, a model using a statistic W obtained by referring to the element data Δ̂θ[k]. D l The density function f used to calculate [k] (i) l (Δ^θ[k]), where index i (i=0,1,...,T; T is an integer greater than or equal to 0) is the transition phase p i It is an index to identify the

[0037] Transition phase p i indicates a section in which the feature quantity of the sequence data SD changes. The feature quantity of the sequence data SD may change suddenly in a short period, or may change slowly and eventually stabilize. The above-mentioned D-CUSUM method, for example, is used as a method for detecting such a slow change. Here, the transition phase p i A specific example of this will be described with reference to FIG.

[0038] 4 is a diagram showing a specific example of sequence data SD. In the graph 51 shown in the upper part of FIG. 4, the horizontal axis represents index k, and the vertical axis represents the feature amount of element data Δ^θ[k] included in the sequence data SD. In the example of FIG. 4, index k st ~k end In the section TP1, the feature value of the element data Δ^θ[k] changes gradually, and the index k end After that, the feature value of the element data Δ^θ[k] becomes stable.

[0039] In this exemplary embodiment, the class labeling unit 12 assigns one or more transition phases p in which the intervals in which the feature quantities of the sequence data SD change are identified by index i (i=0, 1, . . . , T). i Here, the calculation is performed assuming that the transition phase p i Among them, transition phases p0 to p (T-1) The feature of the sequence data SD changes in the final phase pT is a stable phase where the change has ended. For example, when T=3, the class label assignment unit 12 performs calculations assuming transition phases p0 to p2 where the feature values ​​of the sequence data SD change and transition phase p3 where the feature values ​​of the sequence data SD stabilize.

[0040] The density function f estimated by the estimation model M1 (i) l (Δ^θ[k]) is the element data Δ^θ[k] whose index k is the transition phase p i (i=0,1,...,T) is the density function of the class label l. In other words, the density function f (i) l (Δ^θ[k]) is a function estimated for each index i and for each class label l for one index k.

[0041] For example, the estimation model M1 takes element data Δ^θ[k] as input and calculates the density function f (i) l The estimation model M1 is a trained model constructed by machine learning to output (Δ^θ[k]). The learning of the estimation model M1 may be performed by the control unit 10A of the classification device 1A, or may be performed by another device. The machine learning method of the estimation model M1 is not limited, and, for example, a decision tree-based, linear regression, or neural network method may be used, or two or more of these methods may be used. Examples of decision tree-based methods include LightGBM (Light Gradient Boosting Machine), random forest, and XGBoost. Examples of linear regression methods include Bayesian regression, support vector regression, Ridge regression, Lasso regression, and ElasticNet. Examples of neural networks include deep learning.

[0042] However, the estimation model M1 is not limited to a trained model constructed by machine learning, and may be other models. For example, the estimation model M1 may be a rule-based system or a system that references an external database.

[0043] <Classification method flow by classification device 1A> The flow of the classification method executed by the classification device 1A configured as above will be described with reference to the drawings. Fig. 5 is a flow diagram showing the flow of classification method S1A, which is an example of the classification method executed by the classification device 1A. Note that the contents that have already been explained will not be explained again.

[0044] (Step S101) In step S101, the acquisition unit 11 acquires the sequence data SD. For example, the acquisition unit 11 may receive the sequence data SD from another device via the communication unit 30A, or may acquire the sequence data SD input via the input / output unit 40A. Alternatively, the acquisition unit 11 may acquire the sequence data SD by reading it from the storage unit 20A or an external storage device.

[0045] (Steps S102 to S110) The class label assigning unit 12 executes the processes of steps S102 to S110 to assign a class label y[k] to each element data Δ^θ[k] included in the sequence data SD. As an example, the class label assigning unit 12 sequentially calculates statistics obtained by referring to each element data Δ^θ[k] included in the sequence data SD, and determines class transitions by comparing the calculated statistics with a threshold. In this exemplary embodiment, the statistics are calculated using the density function f estimated by the estimation model M1. (i) l The statistics W for each class label l calculated using (Δ^θ[k]) D l Use [k].

[0046] (Step S102) First, in step S102, the class label assignment unit 12 executes various initialization processes. For example, the class label assignment unit 12 sets a counter k, which is a loop counter, to "0" and sets a predetermined class label (for example, "background class") to the class label y(0). In addition, the class label assignment unit 12 calculates the statistic W D l An initial value (for example, "0") is set to [0]. The class label assignment unit 12 also assigns a predetermined initial distribution as the initial distribution f0(Δ̂θ[k]) of the density function.

[0047] (Step S103) In step S103, the class label assignment unit 12 increments the counter k.

[0048] (Step S104) In step S104, the class label assignment unit 12 determines whether the counter k is greater than a predetermined maximum value maxk. The maximum value maxk is set in advance, for example, by an administrator of the classification device 1. If the counter k is equal to or less than the maximum value maxk (NO in step S104), the class label assignment unit 12 proceeds to step S105. On the other hand, if the counter k is greater than the maximum value maxk (YES in step S104), the class label assignment unit 12 proceeds to the processing of step S110.

[0049] (Steps S105 and S106) In step S105, the class label assignment unit 12 acquires element data Δ̂θ[k]. In step S106, the class label assignment unit 12 calculates the statistics {W D l [k]} l Calculate the statistic {W D l [k]} l is the statistic W calculated for each class label l (l=0, 1, , or L). D l [k] is the set of

[0050] Statistics WD l [k] is, for example, the cumulative density function ratio Ω (i) l It is expressed using [k]. Cumulative density function ratio Ω (i) l [k] is, for example, (i) Index i (i = 0, 1, , T) to identify the transition phase; (ii) an index k indicating the order of the element data Δ^θ[k]; (iii) class label l (l = 0, 1, ···, or L), (iv) Density function f (i) l (Δ^θ[k]), and (v) Initial distribution of density function f0(Δ^θ[k]) By

number

[0051] Also, the statistic W D l [k] is, for example, the cumulative density function ratio Ω (i) l Using [k],

number

[0052] The statistic W in step S106 D l In the process of calculating [k], the class label assignment unit 12, for example, calculates the statistic W D l Density function f to calculate [k] (i) l The density function f(Δ̂θ[k]) is calculated by the estimation model M1, which has been trained in advance. In this case, the class label assignment unit 12 calculates the density function f(Δ̂θ[k]) obtained by inputting Δ̂θ[k] to the estimation model M1. (i) l(Δ^θ[k]) is used to calculate the statistic W D l Calculate [k].

[0053] density function f (i) l Methods for estimating (Δ^θ[k]) include, for example, (i) parametric estimation and (ii) non-parametric estimation. (i) Parametric estimation is a method of estimating parameters that define a distribution (for example, a Gaussian distribution) so that the distribution fits the input (mean and covariance matrix in the case of a Gaussian distribution). An example of parametric estimation is maximum likelihood estimation.

[0054] (ii) In the case of nonparametric estimation, for example, the density function f of the class label l (i) l (Δ^θ[k]) and the density function f of the class label l´ after the transition (i) l´To calculate density ratios such as the ratio of (Δ^θ[k]) to (Δ^θ[k]), the class label assignment unit 12 uses an estimation model M1 that learns density ratios using a deep neural network (DNN) with a loss function for density ratio estimation. In this case, for example, the technology described in "Akinori F. Ebihara et. al., SEQUENTIAL DENSITY RATIO ESTIMATION FOR SIMULTANEOUS OPTIMIZATION OF SPEED AND ACCURACY, https: / / openreview.net / forum?id=Rhsu5qD36cL" or the technology described in "Taiki Miyagawa et. al., "The Power of Log-Sum-Exp: Sequential Density Ratio Matrix Estimation for Speed-Accuracy Optimization," https: / / proceedings.mlr.press / v139 / miyagawa21a.html" is used. However, the density function used by the class label assignment unit 12 is not limited to the above examples. The class label assignment unit 12 calculates the statistics W using a density function estimated by another method. D l [k] may be calculated.

[0055] The class label assignment unit 12 assigns a statistic W for each class label l for each element data Δ^θ[k]. D l Calculate [k]. Statistic W D l It can also be said that [k] is a value indicating the confidence of transition to the class of class label l at index k.

[0056] (Step S107) In step S107, the class label assignment unit 12 calculates the statistic W D l [k] is the threshold A l Determine whether there is a class label l that satisfies the condition that it is greater than the threshold A lis a threshold value set in advance for each class label l. If a class label l that satisfies the above condition exists (YES in step S107), the class label assignment unit 12 proceeds to step S108. On the other hand, if a class label l that satisfies the above condition does not exist (NO in step S107), the class label assignment unit 12 proceeds to step S109.

[0057] (Step S108) In step S108, the class label assignment unit 12 stores the class label y[k]=y[k−1]. In other words, for all class labels l, the statistic W D l [k] is the threshold A l If it is equal to or less than this, the class label assignment unit 12 assigns the class label assigned to the element data Δ^θ[k-1] (i.e., the class label assigned to the immediately previous element data) to the element data Δ^θ[k]. After completing the processing of step S108, the class label assignment unit 12 returns to the processing of step S103 and determines the class transition for the next index k.

[0058] (Step S109) In step S109, the class label assignment unit 12 stores the class label y[k]=l. In other words, the class label assignment unit 12 assigns the statistic W D l [k] is the threshold A l The class label l is assigned when the statistic W D l [k] is the threshold A l If there are multiple class labels l that exceed the threshold, the class label assignment unit 12 may, for example, select the class label l that has the largest difference from the threshold and assign it to the element data Δ̂θ[k].

[0059] (Steps S110 and S111) In step S110, the class label assignment unit 12 calculates the statistic W D lIn step S111, the class label assignment unit 12 redefines the initial distribution f0. In other words, the class label assignment unit 12 initializes the statistic W D l [k] is the threshold A l Each time the statistic W D l [k] (step S110) and the statistic W D l The initial distribution f0 of the density function used to calculate [k] is redefined (step S111).

[0060] As an example, the class label assignment unit 12 assigns the initial distribution f0 of the density function to the density function f in the transition phase T of the class label l set to the class label y[k] in step S109. (T) l (Δ^θ[k]). The transition phase T is a phase in which the sequence data SD has finished changing and is stable. Therefore, the class label assignment unit 12 redefines the density function f (T) l Let (Δ^θ[k]) be the initial distribution for subsequent change point detection.

[0061] After completing the process of step S111, the class label assignment unit 12 returns to the process of step S103 and determines the class transition for the next counter k.

[0062] (Step S112) In step S112, the class label assignment unit 12 outputs the result of assigning the class label l to each element data Δ^θ[k]. As an example, the class label assignment unit 12 outputs classified data CD in which the class label l has been assigned to each element data Δ^θ[k]. The class label assignment unit 12 may output the classified data CD to an output device (display device, printer, etc.) connected via the input / output unit 40A, or may transmit the classified data CD to another device via the communication unit 30A. The class label assignment unit 12 may also output the classified data CD by writing the classified data CD to the storage unit 20A or an external storage device.

[0063] Here, a specific example of the class label assignment process performed by the class label assignment unit 12 will be described with reference to Fig. 4. A graph 52 shown in the upper part of Fig. 4 is a graph of the statistical quantity W D l In the graph 52, the horizontal axis represents the index k, and the vertical axis represents the statistic W D l In the example in Figure 4, the statistic W D l [k] increases and at index k' the statistic W D l [k´] is the threshold A l The class label assignment unit 12 determines that there is a class transition at index k' (YES in step S107), and calculates the statistic W D l [k´] is the threshold A l The class label l' that has reached this point is set as the class label of the element data Δ^θ[k'] (step S109). In other words, in this example, the class label assigning unit 12 determines that the index k' is an event boundary.

[0064] Furthermore, the class label assignment unit 12 assigns the statistic W D lThe initial distribution f0(Δ^θ[k]) of the density function is redefined (steps S110 and S111). The lower part of FIG. 4 shows the statistics W D l Graph 52 of [k] and the statistical value W after initialization D l The graph 53 of [k] is shown. From index k′ onwards, the class label assignment unit 12 assigns the statistics W D l [k] is again the threshold A l The class label l' assigned to the element data Δ^θ[k'] is assigned to the element data Δ^θ[k] until

[0065] The class label assigning unit 12 repeatedly executes the processes of steps S103 to S111 in FIG. 5, thereby assigning a class label y[k] to each of the element data Δ̂θ[k].

[0066] <Effects of Classification Device 1A> In a technology for classifying and labeling sequence data SD, the start point (e.g., start time) and end point (e.g., end time) of an event may be ambiguous. For example, does the event "jump" begin when the subject begins to bend their knees, or when the subject's feet leave the ground? Therefore, when labeling is performed based on machine learning, performance may be degraded if ambiguous boundaries are not explicitly modeled. In contrast, according to this exemplary embodiment, the class labeling unit 12 repeatedly applies change-point detection to the sequence data SD, thereby enabling appropriate labeling even when ambiguous event boundaries exist.

[0067] As described above, in the classification device 1A according to this exemplary embodiment, the class label assignment unit 12 calculates the statistic W D l [k] is calculated sequentially, and the calculated statistics are compared with a threshold to determine the transition of classes. D l [k] is the threshold Al Each time it reaches the statistic W D l Initialization of [k] and the statistic W D l The classification device 1A according to this exemplary embodiment is configured to redefine the initial distribution of the density function used to calculate [k]. Therefore, in addition to the effects of the classification device 1 according to the first exemplary embodiment, the classification device 1A according to this exemplary embodiment can appropriately identify and label event boundaries even when the changes in the sequence data SD are not rapid but gradual.

[0068] In addition, in the classification device 1A according to this exemplary embodiment, the statistic W expressed using the cumulative density function ratio is D l [k] to determine class transitions. Therefore, the classification device 1A according to this exemplary embodiment can more appropriately determine and label event boundaries in addition to the effects of the classification device 1 according to the first exemplary embodiment.

[0069] Exemplary Embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are denoted by the same reference numerals, and their description will not be repeated.

[0070] <Configuration of Classification Device 1B> 6 is a block diagram showing the configuration of the classification device 1B. The classification device 1B includes a control unit 10A, a storage unit 20A, a communication unit 30A, and an input / output unit 40A. The control unit 10A includes an acquisition unit 11, a class label assignment unit 12, and a learning unit 13. In this exemplary embodiment, the estimation model M1 is a trained model constructed by machine learning using training data.

[0071] The acquisition unit 11 acquires training data S used for training the estimation model M1. The training data S is a sequence data {x j [k]} k and the sequence data {x j [k]}k Each element data x contained in j Correct label y assigned to [k] j Here, index j (1≦j≦N) is an index for identifying a sample, and N is the number of samples.

[0072] The training data S may be, for example, a number of samples N, an index j for identifying the samples, and a number m of intervals included in the samples identified by the index j. j , and is expressed as follows:

number

[0073] Here, the start point s of the interval j , the end point of the interval e j , class label y j , respectively.

number

[0074] The learning unit 13 learns the estimation model M1 by referring to the training data. For example, the learning unit 13 uses a decision tree-based, linear regression, or neural network technique to learn the density function f (i) l An estimation model M1 that estimates (Δ^θ[k]) is trained.

[0075] As described above, the classification device 1B according to this exemplary embodiment is configured to train the estimation model M1 by referring to training data including sequence data and ground truth labels assigned to each element data included in the sequence data. Therefore, the classification device 1B according to this exemplary embodiment has the advantage of being able to estimate a density function using the trained estimation model M1, in addition to the advantages of the classification device 1 according to the first exemplary embodiment.

[0076] Exemplary Embodiment 4 A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to third exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0077] 7 is a block diagram showing the configuration of a classification device 1C according to this exemplary embodiment. The classification device 1C includes a control unit 10A, a storage unit 20A, a communication unit 30A, an input / output unit 40A, and a display unit 50C. The display unit 50C displays various information based on data supplied by the control unit 10A. As an example, the display unit 50C displays the class label y[k] assigned by the class label assignment unit 12 together with at least some of the element data Δ^θ[k] included in the sequence data SD.

[0078] 8 is a diagram showing a screen SC11 that is a display example of the display unit 50C. On the screen SC11, the display unit 50C displays a class label LBL11 together with an image Img11. The display unit 50C also displays a class label LBL12 together with the image Img12. The display unit 50C also displays a class label LBL13 together with the image Img13. The images Img11 to Img13 are, for example, frames included in video data, and are an example of the element data Δ^θ[k] according to this specification. The class labels LBL11 to LBL13 are labels that the class label assignment unit 12 assigns to the images Img11 to Img13, respectively.

[0079] A user of the classification device 1C can understand the class label y[k] assigned to the element data Δ^θ[k] included in the sequence data SD from the screen displayed by the display unit 50C. In addition, in this exemplary embodiment, the user may be able to change the class label y[k] displayed by the display unit 50C using an input device connected via the input / output unit 40A.

[0080] [Software implementation example] Some or all of the functions of the classification devices 1, 1A, and 1B may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0081] In the latter case, the classification devices 1, 1A, and 1B are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 9. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the classification devices 1, 1A, and 1B. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the classification devices 1, 1A, and 1B.

[0082] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0083] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0084] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0085] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0086] [Appendix 2] Some or all of the above-described embodiments can also be described as follows: However, the present invention is not limited to the following described aspects.

[0087] (Appendix 1) an acquisition means for acquiring sequence data; a class label assignment means for repeatedly applying change point detection to the sequence data to assign a class label to each element data included in the sequence data; A classification device comprising:

[0088] According to the above configuration, labeling can be performed appropriately even when an ambiguous event boundary exists.

[0089] (Appendix 2) The class label assignment means a statistical value obtained by referring to each element data included in the sequence data is calculated sequentially, and the calculated statistical value is compared with a threshold value to determine a transition of classes; Each time the statistic reaches the threshold, Initialization of the statistics; Redefining the initial distribution of the density function used to calculate the statistic; 2. The classification apparatus of claim 1,

[0090] According to the above configuration, even when the change in the sequence data is not sudden but gradual, it is possible to appropriately determine the event boundary and perform labeling.

[0091] (Appendix 3) The class label assignment means index i (i = 0, 1, ,T) to identify the transition phase, An index k indicates the order of the element data Δ^θ[k]. class label l (l=0, 1, , or L), density function f (i) l (Δ^θ[k]), and The initial distribution of the density function f0(Δ^θ[k]), The cumulative density function ratio expressed by

number

number

[0092] According to the above configuration, by determining class transitions with reference to statistics expressed using cumulative density function ratios, it is possible to more appropriately determine event boundaries and perform labeling.

[0093] (Appendix 4) The class label assignment means a density function estimated by a pre-trained estimation model is used as the density function for calculating the statistics; 4. The classification apparatus of claim 2 or 3.

[0094] According to the above configuration, by calculating statistics using the density function estimated by the estimation model, it is possible to more appropriately determine and label event boundaries.

[0095] (Appendix 5) the acquiring means acquires training data including sequence data and correct labels assigned to each element data included in the sequence data; The classification device further comprising a learning means for learning the estimation model by referring to the training data; 10. The classification apparatus of claim 4.

[0096] According to the above configuration, it is possible to estimate a density function using a trained estimation model.

[0097] (Appendix 6) The sequence data is time-series data having a sequence with respect to time. 6. The classification device of any one of appendices 1 to 5.

[0098] According to the above configuration, labeling can be performed appropriately even when there is an ambiguous event boundary in the time series data.

[0099] (Appendix 7) a display means for displaying the class labels assigned by the class label assigning means together with at least some of the element data included in the sequence data; 7. The classification device of any one of appendices 1 to 6, comprising:

[0100] According to the above configuration, a user of the classification device or the like can understand the class labels assigned to element data included in the sequence data.

[0101] (Appendix 8) Obtaining sequence data; assigning a class label to each element data included in the sequence data by repeatedly applying change point detection to the sequence data; Classification methods including.

[0102] The above classification method provides the same effects as the above classification device.

[0103] (Appendix 9) On the computer, A process for acquiring sequence data; a process of assigning a class label to each element data included in the sequence data by repeatedly applying change point detection to the sequence data; A program that executes the following.

[0104] According to the above configuration, the same effects as those of the above-mentioned classification device can be achieved.

[0105] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows.

[0106] A classification device comprising at least one processor that executes an acquisition process for acquiring sequence data, and a class label assignment process for assigning a class label to each element data included in the sequence data by repeatedly applying change point detection to the sequence data.

[0107] The classification device may further include a memory that stores a program for causing the processor to execute the acquisition process and the class label assignment process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0108] 1, 1A, 1B, 1C classification device 10A Control unit 11 Acquisition Department 12 Class label assignment part 13 Learning Department 20A storage section 30A Communications Department 40A input / output section 50C Display section C1 processor C2 Memory M1 estimation model S1, S1A classification method

Claims

1. an acquisition means for acquiring sequence data; a class label assignment means for repeatedly applying change point detection to the sequence data to assign a class label to each element data included in the sequence data; Equipped with The class label assignment means a statistical value obtained by referring to each element data included in the sequence data is calculated sequentially, and the calculated statistical value is compared with a threshold value to determine a transition of classes; Each time the statistic reaches the threshold, Initialization of the statistics; Redefinition of the initial distribution of the density function used to calculate the statistics Run Classification device.

2. The class label assignment means an index i (i=0, 1, . . . , T) for identifying the transition phase; An index k indicating the order of the element data Δ^θ[k], Class label l (l = 0, 1, ..., or L), density function f （ｉ） ｌ (Δ^θ[k]), and Initial distribution of density function f ０ (Δ^θ[k]) The cumulative density function ratio expressed by [Number 1] Using [Number 2] The statistical quantity W expressed as Ｄ ｌ Refer to [k] to determine class transition The classification device of claim 1 .

3. The class label assignment means A density function estimated by a pre-trained estimation model is used as the density function for calculating the statistics. The classification device according to claim 1 or 2.

4. the acquiring means acquires training data including sequence data and correct labels assigned to each element data included in the sequence data; The classification device The method further includes a learning means for learning the estimation model by referring to the training data. The classification device according to claim 3 .

5. The sequence data is time-series data having a sequence characteristic with respect to time. A classification device according to any one of claims 1 to 4.

6. a display means for displaying the class labels assigned by the class label assigning means together with at least some of the element data included in the sequence data; 6. The classification device according to claim 1, further comprising:

7. A computer comprising: Obtaining sequence data; assigning a class label to each element data included in the sequence data by repeatedly applying change point detection to the sequence data; The assigning of the class label includes: a statistical value obtained by referring to each element data included in the sequence data is calculated sequentially, and the calculated statistical value is compared with a threshold value to determine a transition of classes; Each time the statistic reaches the threshold, Initialization of the statistics; Redefinition of the initial distribution of the density function used to calculate the statistics Run Classification method.

8. On the computer, A process for acquiring sequence data; and assigning a class label to each element data included in the sequence data by repeatedly applying change-point detection to the sequence data; The process of assigning class labels includes: a statistical value obtained by referring to each element data included in the sequence data is calculated sequentially, and the calculated statistical value is compared with a threshold value to determine a transition of classes; Each time the statistic reaches the threshold, Initialization of the statistics; Redefinition of the initial distribution of the density function used to calculate the statistics Run program.

Images