Information processing system, information processing method, and computer program

Abstract

This information processing system (1) comprises: an acquisition means (50) that acquires a plurality of elements included in series data; a calculation means (100) that calculates a likelihood ratio indicating the likelihood of a class to which the series data belongs, on the basis of at least two consecutive elements of the plurality of elements; a classification means (200) that classifies, on the basis of the likelihood ratio, the series data into at least one of a plurality of classes which are classification candidates; and a learning means (300) that performs learning relating to calculation of the likelihood ratio by using a loss function of a log-sum-exp type. This information processing system makes it possible to appropriately select a class to which series data belongs, from among a plurality of classes that are classification candidates.

Description

[Technical field]

[0001] The present disclosure relates to the technical fields of an information processing system, an information processing method, and a computer program for processing information relating to, for example, class classification. [Background technology]

[0002] As this type of system, a system that performs data classification is known. For example, Patent Document 1 discloses a technology for classifying sequence data into one of a number of predetermined classes by sequentially acquiring and analyzing a number of elements contained in the sequence data. Patent Document 2 discloses a technology for classifying trajectories contained in an image subset into subclasses, assigning the same subclass label to those with a high shared ratio of the subclasses, and classifying each subclass.

[0003] As another related technique, for example, Patent Document 3 describes an evaluation value G=sum{(c-exp(a×log(X))+by) 2 ×wp} is minimized by repeating the process to optimize the coefficients. Patent Document 4 discloses optimizing the parameters of the classification device by updating the parameters so that a loss function including a log-likelihood ratio is reduced. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 194497 [Patent Document 2] International Publication No. 2012 / 127815 [Patent Document 3] JP 2017-049674 A [Patent Document 4] JP 2007-114413 A Summary of the Invention [Problem to be solved by the invention]

[0005] This disclosure aims to improve upon the related art mentioned above. [Means for solving the problem]

[0006] One aspect of the information processing system disclosed herein includes an acquisition means for acquiring a plurality of elements included in sequence data, a calculation means for calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements, a classification means for classifying the sequence data into at least one class among a plurality of candidate classes based on the likelihood ratio, and a learning means for learning regarding the calculation of the likelihood ratio using a log-sum-exp type loss function.

[0007] One aspect of the information processing method disclosed herein includes obtaining a plurality of elements included in sequence data, calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements, classifying the sequence data into at least one class among a plurality of candidate classes based on the likelihood ratio, and performing learning related to the calculation of the likelihood ratio using a log-sum-exp type loss function.

[0008] One aspect of the computer program of the present disclosure operates a computer to obtain multiple elements included in sequence data, calculate a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the multiple elements, classify the sequence data into at least one class among multiple candidate classes based on the likelihood ratio, and perform learning related to the calculation of the likelihood ratio using a log-sum-exp type loss function. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a functional configuration of an information processing system according to a first embodiment. [Diagram 3] 5 is a flowchart showing the flow of operations of the classification device in the information processing system according to the first embodiment. [Figure 4] 5 is a flowchart showing a flow of operations of a learning unit in the information processing system according to the first embodiment. [Diagram 5] 10 is a flowchart showing the flow of operations of a learning unit in the information processing system according to the second embodiment. [Figure 6] 13 is a matrix diagram showing an example of likelihood ratios taken into account by a learning unit in the information processing system according to the second embodiment. FIG. [Figure 7] 13 is a flowchart showing the flow of operations of a learning unit in the information processing system according to the third embodiment. [Figure 8] 13 is a flowchart showing the flow of operations of a learning unit in the information processing system according to the fourth embodiment. [Figure 9] FIG. 13 is a matrix diagram showing an example of likelihood ratios taken into account by a learning unit in the information processing system according to the fourth embodiment. [Figure 10] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to a seventh embodiment. [Figure 11] 20 is a flowchart showing the flow of operations of the classification device in the information processing system according to the seventh embodiment. [Figure 12] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to an eighth embodiment. [Figure 13] 20 is a flowchart showing a flow of operations of a likelihood ratio calculation unit in the information processing system according to the eighth embodiment. [Figure 14] 13 is a flowchart showing the flow of operations of the classification device in the information processing system according to the ninth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of an information processing system, an information processing method, and a computer program will be described with reference to the drawings.

[0011] First Embodiment An information processing system according to a first embodiment will be described with reference to FIGS. 1 to 4. FIG.

[0012] (Hardware configuration) First, the hardware configuration of the information processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0013] 1, the information processing system 1 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 1 may further include an input device 15 and an output device 16. The processor 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, and the output device 16 are connected via a data bus 17.

[0014] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable recording medium using a recording medium reading device (not shown). The processor 11 may acquire (i.e., read) a computer program from a device (not shown) disposed outside the information processing system 1 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the computer program that has been read. In particular, in this embodiment, when the processor 11 executes the computer program that has been read, a functional block for executing classification using a likelihood ratio and a learning process related thereto is realized in the processor 11. Examples of the processor 11 include a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a demand-side platform (DSP), and an application specific integrated circuit (ASIC). The processor 11 may be one of the above-mentioned examples, or a plurality of processors may be used in parallel.

[0015] The RAM 12 temporarily stores a computer program executed by the processor 11. The RAM 12 temporarily stores data that is temporarily used by the processor 11 when the processor 11 is executing the computer program. The RAM 12 may be, for example, a D-RAM (Dynamic RAM).

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store other fixed data. The ROM 13 may be, for example, a programmable ROM (P-ROM).

[0017] The storage device 14 stores data that is to be stored long-term by the information processing system 1. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the information processing system 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be a dedicated controller (operation terminal). The input device 15 may also include a terminal owned by a user (for example, a smartphone, a tablet terminal, etc.). The input device 15 may be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the information processing system 1 to the outside. For example, the output device 16 may be a display device (for example, a display) that can display information related to the information processing system 1. The display device here may be a television monitor, a personal computer monitor, a smartphone monitor, a tablet terminal monitor, or a monitor of another mobile terminal. In addition, the display device may be a large monitor or digital signage installed in various facilities such as a store. In addition, the output device 16 may be a device that outputs information in a format other than an image. For example, the output device 16 may be a speaker that outputs information related to the information processing system 1 as sound.

[0020] (Functional configuration) Next, the functional configuration of the information processing system 1 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0021] As shown in FIG. 2, the information processing system 1 according to the first embodiment includes a classification device 10 and a learning unit 300. The classification device 10 is a device that performs class classification of input sequence data, and includes a data acquisition unit 50, a likelihood ratio calculation unit 100, and a class classification unit 200 as processing blocks for realizing the function. The learning unit 300 is configured to be able to execute a learning process related to the classification device 10. Note that, although an example in which the learning unit 300 is provided separately from the classification device 10 is given here, the classification device 10 may be configured to include the learning unit 300. Each of the data acquisition unit 50, the likelihood ratio calculation unit 100, the class classification unit 200, and the learning unit 300 may be realized by the above-mentioned processor 11 (see FIG. 1).

[0022] The data acquisition unit 50 is configured to be able to acquire a plurality of elements included in the sequence data. The data acquisition unit 50 may acquire data directly from any data acquisition device (for example, a camera, a microphone, etc.), or may read data acquired in advance by a data acquisition device and stored in a storage or the like. When acquiring data from a camera, the data acquisition unit 50 may be configured to acquire data from each of a plurality of cameras. The elements of the sequence data acquired by the data acquisition unit 50 are configured to be output to the likelihood ratio calculation unit 100. Note that sequence data is data including a plurality of elements arranged in a predetermined order, and one example is time-series data. More specific examples of sequence data include, but are not limited to, video data and audio data.

[0023] The likelihood ratio calculation unit 100 is configured to be able to calculate a likelihood ratio based on at least two consecutive elements among the multiple elements acquired by the data acquisition unit 50. Note that the "likelihood ratio" here is an index indicating the likelihood of the class to which the sequence data belongs. Specific examples of the likelihood ratio and a specific calculation method will be described in detail in other embodiments described later.

[0024] The classifying unit 200 is configured to classify the sequence data based on the likelihood ratio calculated by the likelihood ratio calculating unit 100. The classifying unit 200 selects at least one class to which the sequence data belongs from among a plurality of classes that are classification candidates. The plurality of classes that are classification candidates may be preset. Alternatively, the plurality of classes that are classification candidates may be appropriately set by a user, or may be appropriately set based on the type of sequence data to be handled, etc.

[0025] The learning unit 300 performs learning related to the calculation of the likelihood ratio using a loss function. Specifically, learning related to the calculation of the likelihood ratio is executed so that class classification based on the likelihood ratio is performed accurately. The loss function used by the learning unit 300 according to this embodiment is a log-sum-exp type loss function, more specifically, a function in which sum and exp are included in log. The loss function may be set in advance as a function that satisfies such a definition. Specific examples of the loss function will be described in detail in other embodiments described later.

[0026] (Classification operation flow) Next, a flow of operation of the classification device 10 in the information processing system 1 according to the first embodiment (specifically, a class classification operation after learning) will be described with reference to Fig. 3. Fig. 3 is a flowchart showing a flow of operation of the classification device in the information processing system according to the first embodiment.

[0027] 3, when the operation of the classification device 10 is started, first, the data acquiring unit 50 acquires elements included in the sequence data (step S11). The data acquiring unit 50 outputs the acquired elements of the sequence data to the likelihood ratio calculating unit 100. Then, the likelihood ratio calculating unit 100 calculates a likelihood ratio based on two or more acquired elements (step S12).

[0028] Next, the classifying unit 200 performs classifying based on the calculated likelihood ratio (step S13). The classifying may determine one class to which the sequence data belongs, or may determine multiple classes to which the sequence data is likely to belong. The classifying unit 200 may output the result of the classifying to a display or the like. The classifying unit 200 may also output the result of the classifying by voice via a speaker or the like.

[0029] (Learning procedure) Next, a flow of operation of the learning unit 300 in the information processing system 1 according to the first embodiment (i.e., a learning operation related to the calculation of a likelihood ratio) will be described with reference to Fig. 4. Fig. 4 is a flowchart showing a flow of operation of the learning unit in the information processing system according to the first embodiment.

[0030] 4, when the learning operation is started, first, training data is input to the learning unit 300 (step S101). The training data may be configured as a set of, for example, sequence data and information on the correct class to which the sequence data belongs (i.e., correct data).

[0031] Next, the learning unit 300 calculates a loss function using the input training data (step S102). The loss function here is a log-sum-exp type loss function, as already described. The log-sum-exp type loss function is a loss function that includes sum (sum) and exponential function (exponential function) in log, such as a function log(Σexp(x)).

[0032] Next, the learning unit 300 adjusts the parameters (specifically, the parameters of the model for calculating the likelihood ratio) so as to reduce the calculated loss function (step S103). That is, the learning unit 300 optimizes the parameters of the model for calculating the likelihood ratio. Note that as a parameter optimization method using the loss function, it is possible to appropriately adopt existing technology. An example of the optimization method is the backpropagation method, but other methods may also be used.

[0033] Thereafter, the learning unit 300 determines whether all learning has been completed (step S104). The learning unit 300 may determine whether all learning has been completed, for example, based on whether all training data has been input. Alternatively, the learning unit 300 may determine whether all learning has been completed, based on whether a predetermined period of time has elapsed since the start of learning. Alternatively, the learning unit 300 may determine whether all learning has been completed, based on whether the processes from steps S101 to S103 described above have been looped a predetermined number of times.

[0034] If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101. This repeats the learning process using the training data, and the parameters are adjusted to be more optimal.

[0035] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the first embodiment will be described.

[0036] As described in FIG. 1 to FIG. 4, in the information processing system 1 according to the first embodiment, the learning unit 300 executes learning related to the calculation of the likelihood ratio used for class classification. In this embodiment, learning is performed using a log-sum-exp type loss function. According to the research of the present inventor, it has been found that the convergence in the stochastic gradient descent method is improved by using a log-sum-exp type loss function for learning the likelihood ratio. More specifically, since a larger gradient can be assigned to an object that is relatively difficult to classify using the likelihood ratio (for example, a hard class, a hard frame, a hard example), it is possible to speed up the convergence and perform efficient learning. For example, since it takes a relatively long time to learn a DNN (Deep Neural Network), extremely efficient learning can be performed by improving the convergence as described above.

[0037] As an existing technique that can be used when learning difficult-to-classify objects, a method of weighting by multiplying a loss function by an appropriate coefficient (so-called loss weighting) is known, but this method requires empirical rules and tuning when determining the coefficient. In addition, a method of learning by inputting difficult-to-classify data many times while allowing overlaps (so-called oversampling) is also known, but simple data is less likely to appear in a mini-batch, and it takes many steps to see all the data, slowing down the convergence. Alternatively, a method of relatively emphasizing difficult data by deleting easy-to-classify data (so-called undersampling) is also known, but since part of the data is deleted, the accuracy of learning is inevitably deteriorated. However, if learning is performed using the log-sum-exp type loss function described in this embodiment, it becomes possible to perform efficient learning while solving the above problems.

[0038] <Second embodiment> The information processing system 1 according to the second embodiment will be described with reference to Figures 5 and 6. The second embodiment differs from the first embodiment only in some operations, and may be similar to the first embodiment in, for example, the device configuration (see Figures 1 and 2) and the operation of the classification device 10 (see Figure 3). Therefore, hereinafter, only the parts that differ from the first embodiment will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0039] (Learning procedure) First, the flow of operations of the learning unit 300 in the information processing system 1 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of operations of the information processing system according to the second embodiment. In Fig. 5, the same processes as those shown in Fig. 4 are denoted by the same reference numerals.

[0040] As shown in FIG. 5, when the operation of the information processing system 1 according to the second embodiment is started, training data is first input to the learning unit 300 (step S101).

[0041] Next, the learning unit 300 calculates a loss function using the input training data. In particular, in the second embodiment, the learning unit 300 calculates a loss function that takes into account the likelihood ratio of an N×(N−1) pattern, in which the likelihood indicating the likelihood of belonging to one class among N classes (where N is a natural number) that are classification candidates for sequence data is used as the denominator and the likelihood indicating the likelihood of belonging to another class is used as the numerator (step S201). This loss function is a function that increases the likelihood ratio when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and decreases the likelihood ratio when the correct class is in the denominator of the likelihood ratio. This loss function is also a log-sum-exp type loss function, as in the first embodiment. The likelihood ratio taken into account in the loss function will be described in detail later with a specific example.

[0042] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). That is, the learning unit 300 optimizes the parameters of the model for calculating the likelihood ratio. After that, the learning unit 300 judges whether or not all learning has been completed (step S104). If it is judged that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is judged that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0043] (Examples of likelihood ratios to consider) Next, the likelihood ratios taken into account in the learning operation by the learning unit 300 described above (i.e., the likelihood ratios taken into account in the calculation of the loss function) will be specifically described with reference to Fig. 6. Fig. 6 is a matrix diagram showing an example of the likelihood ratios taken into account by the learning unit in the information processing system according to the second embodiment.

[0044] As shown in Figure 6, the likelihood is considered in the form of a matrix. For ease of explanation, it is assumed that there are three classes, which are classification candidates: "Class 0", "Class 1", and "Class 2". p(X|y=0) is the likelihood that the sequence data is "Class 0". p(X|y=1) is the likelihood that the sequence data is "Class 1". p(X|y=2) is the likelihood that the sequence data is "Class 2".

[0045] In the first row from the top of the matrix, all numerators of the log-likelihood ratios (hereinafter simply referred to as "likelihood ratios") are p(X|y=0). In the second row from the top of the matrix, all numerators of the likelihood ratios are p(X|y=1). In the third row from the top of the matrix, all numerators of the likelihood ratios are p(X|y=2). Meanwhile, in the first column from the left of the matrix, all denominators of the likelihood ratios are p(X|y=0). In the second column from the left of the matrix, all denominators of the likelihood ratios are p(X|y=1). In the third column from the left of the matrix, all denominators of the likelihood ratios are p(X|y=2).

[0046] The likelihood ratios on the diagonal of the matrix (likelihood ratios shaded in gray in FIG. 6) have the same likelihood in the denominator and numerator. Specifically, log{p(X|y=0) / p(X|y=0)} in the first row from the top and first column from the left, log{p(X|y=1) / p(X|y=1)} in the second row from the top and second column from the left, and log{p(X|y=2) / p(X|y=2)} in the third row from the top and third column from the left each have the same denominator and numerator. In addition, the likelihood ratios on opposite sides of the diagonal likelihood ratios have reversed denominators and numerators. Specifically, the numerator and denominator are reversed between log{p(X|y=0) / p(X|y=1)} in the first row from the top and second column from the left and log{p(X|y=1) / p(X|y=0)} in the second row from the top and first column from the left. Similarly, the numerator and denominator are reversed between log{p(X|y=0) / p(X|y=2)} in the first row from the top and third column from the left and log{p(X|y=2) / p(X|y=0)} in the third row from the top and first column from the left. The numerator and denominator are reversed between log{p(X|y=1) / p(X|y=2)} in the second row from the top and third column from the left and log{p(X|y=2) / p(X|y=1)} in the third row from the top and second column from the left. Therefore, likelihood ratios located on opposite sides of the diagonal line have values ​​with opposite signs. In this way, the likelihood ratios represented in the matrix are arranged like an alternating matrix.

[0047] Here, in particular, the likelihood ratios in which the denominator and numerator on the diagonal are the same are all log1, and the value is zero. Therefore, the likelihood ratios in which the denominator and numerator on the diagonal are the same are essentially meaningless even if they are considered in the loss function. Therefore, the likelihood ratios in which the denominator and numerator on the diagonal are the same are not considered in the loss function. The number of remaining likelihood ratios excluding the diagonal likelihood ratios is N×(N-1), where N is the number of classes. In this embodiment, the likelihood ratios of these N×(N-1) patterns (i.e., the likelihood ratios excluding the diagonal likelihood ratios in the matrix) are considered in the loss function. Note that a specific example of a loss function in which the likelihood ratios of N×(N-1) patterns are considered will be described in detail in another embodiment described later.

[0048] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the second embodiment will be described.

[0049] As described in Fig. 5 and Fig. 6, in the information processing system 1 according to the second embodiment, learning is performed using a loss function that takes into account the likelihood ratio of an N × (N-1) pattern, with the likelihood indicating the likelihood of belonging to one class as the denominator and the likelihood indicating the likelihood of belonging to another class as the numerator. By using such a loss function, learning can be performed in which the penalty is large when the class is incorrect and the penalty is small when the class is correct, as in the first embodiment. As a result, it becomes possible to appropriately select at least one class to which the sequence data belongs from among multiple classes that are classification candidates.

[0050] When there are multiple classes as classification candidates (so-called multi-class classification), it is not easy to determine what likelihood ratio should be considered during learning (for example, what ratio should be taken). However, when the above-mentioned loss function is used, the magnitude of the likelihood ratio changes depending on whether the correct class is in the numerator or denominator of the likelihood ratio, and the influence on the loss function changes. When such a loss function is used, learning regarding the calculation of the likelihood ratio in multi-class classification can be appropriately performed. As a result, it is possible to realize appropriate class classification. It is particularly difficult to determine what likelihood ratio should be considered during learning when there are three or more classes as classification candidates. Therefore, the technical effect of this embodiment is prominently exhibited when there are three or more classes as classification candidates.

[0051] In addition, the loss function used in the second embodiment is a log-sum-exp type loss function, as in the first embodiment. Therefore, it is possible to improve the convergence in the stochastic gradient descent method, and as a result, it becomes possible to perform efficient learning.

[0052] <Third embodiment> The information processing system 1 according to the third embodiment will be described with reference to Fig. 7. The third embodiment differs from the second embodiment in some operations, and other parts may be the same as the second embodiment. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit descriptions of other overlapping parts as appropriate.

[0053] (Learning procedure) First, the flow of operations of the learning unit 300 in the information processing system 1 according to the third embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of operations of the information processing system according to the third embodiment. In Fig. 7, the same processes as those shown in Fig. 4 are denoted by the same reference numerals.

[0054] As shown in FIG. 7, when the operation of the information processing system 1 according to the third embodiment is started, first, training data is input to the learning unit 300 (step S101).

[0055] Next, the learning unit 300 calculates a loss function using the input training data. In particular, in the third embodiment, the loss function is calculated by taking into account a part of the likelihood ratio of an N×(N-1) pattern, in which the denominator is a likelihood indicating the likelihood of belonging to one class among N classes that are classification candidates of the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class (step S301). That is, the learning unit 300 according to the third embodiment does not take into account all the likelihood ratios of the N×(N-1) patterns described in the second embodiment, but only a part of them. Note that, like the first embodiment, this loss function is a function in which the likelihood ratio becomes large when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and the likelihood ratio becomes small when the correct class is in the denominator of the likelihood ratio. Also, like the first embodiment, this loss function is a log-sum-exp type loss function.

[0056] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). After that, the learning unit 300 judges whether or not all learning has been completed (step S104). If it is judged that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is judged that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0057] (Example of selection of likelihood ratio to consider) Next, a selection example of the likelihood ratios taken into account in the loss function (that is, a selection example of some of the likelihood ratios of the N×(N−1) patterns) will be specifically described.

[0058] Of the likelihood ratios of the N×(N−1) patterns, some of the likelihood ratios to be considered in the loss function may be selected in advance by a user or the like, or may be automatically selected by the learning unit 300. When the learning unit 300 selects some of the likelihood ratios to be considered in the loss function, the learning unit 300 may select the likelihood ratios according to a predetermined rule set in advance. Alternatively, the learning unit 300 may determine whether or not to make a selection based on the value of the calculated likelihood ratio.

[0059] An example of selecting a portion of the likelihood ratios to be considered in the loss function is, for example, selecting only the likelihood ratios in one row or one column in the matrix shown in Fig. 6. For example, as the likelihood ratios to be considered in the loss function, only the likelihood ratios in the first row, only the likelihood ratios in the second row, or only the likelihood ratios in the third row of the matrix shown in Fig. 6 may be selected. Alternatively, only the likelihood ratios in the first column, only the likelihood ratios in the second column, or only the likelihood ratios in the third column of the matrix may be selected.

[0060] Also, only the likelihood ratios of some rows or some columns in the matrix may be selected. Specifically, only the likelihood ratios of the first and second rows of the matrix may be selected, only the likelihood ratios of the second and third rows may be selected, or only the likelihood ratios of the third and first rows may be selected. Alternatively, only the likelihood ratios of the first and second columns of the matrix may be selected, only the likelihood ratios of the second and third columns may be selected, or only the likelihood ratios of the third and first columns may be selected.

[0061] Note that the above-mentioned selection example of the likelihood ratio is merely an example, and other likelihood ratios may be selected as the likelihood ratios taken into account in the loss function. For example, the likelihood ratios taken into account in the loss function may be selected randomly regardless of rows or columns.

[0062] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the third embodiment will be described.

[0063] As described in FIG. 7, in the information processing system 1 according to the third embodiment, learning is performed using a loss function that takes into account some likelihood ratios among N×(N-1) patterns, in which the likelihood indicating the likelihood of belonging to one class is used as the denominator and the likelihood indicating the likelihood of belonging to another class is used as the numerator. By using such a loss function, learning can be performed such that the penalty when the class is incorrect is large and the penalty when the class is correct is small, as in the above-mentioned second embodiment. As a result, it becomes possible to appropriately select at least one class to which the sequence data belongs from among multiple classes that are classification candidates. Furthermore, particularly in the third embodiment, by appropriately selecting the likelihood ratios to be taken into account in the loss function from among N×(N-1) patterns, learning can be performed more efficiently than when all likelihood ratios of the N×(N-1) patterns are taken into account. For example, by selecting only likelihood ratios that have a relatively large effect on the loss function and not selecting likelihood ratios that have a relatively small effect on the loss function, learning efficiency can be improved.

[0064] In addition, the loss function used in the third embodiment is a log-sum-exp type loss function, as in the above-mentioned embodiments. Therefore, it is possible to improve the convergence in the stochastic gradient descent method, and as a result, it is possible to perform more efficient learning.

[0065] <Fourth embodiment> The information processing system 1 according to the fourth embodiment will be described with reference to Fig. 8 and Fig. 9. The fourth embodiment describes a specific selection example of the above-mentioned third embodiment (i.e., a selection example of a part of the likelihood ratios to be considered in the loss function), and other parts may be similar to the third embodiment. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0066] (Learning procedure) First, the flow of operations of the learning unit 300 in the information processing system 1 according to the fourth embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of operations of the information processing system according to the fourth embodiment. In Fig. 8, the same processes as those shown in Fig. 4 are denoted by the same reference numerals.

[0067] As shown in FIG. 8, when the operation of the information processing system 1 according to the fourth embodiment is started, training data is first input to the learning unit 300 (step S101).

[0068] Next, the learning unit 300 calculates a loss function using the input training data. In particular, in the fourth embodiment, the learning unit 300 calculates a loss function that takes into account the likelihood ratio in which the correct class is in the numerator, among the likelihood ratios of the N×(N-1) patterns already described (step S401). That is, the learning unit 300 according to the fourth embodiment selects a likelihood ratio in which the correct class is in the numerator, as a part of the likelihood ratios of the N×(N-1) patterns described in the third embodiment. Note that, like the second and third embodiments, this loss function is a function in which the likelihood ratio becomes large when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and becomes small when the correct class is in the denominator of the likelihood ratio. Also, like the first embodiment, this loss function is a log-sum-exp type loss function. A specific example of a loss function that takes into account the likelihood ratio in which the correct class is in the numerator will be described in detail in other embodiments described later.

[0069] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). After that, the learning unit 300 judges whether or not all learning has been completed (step S104). If it is judged that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is judged that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0070] (Examples of likelihood ratios to consider) Next, the likelihood ratios taken into account in the learning operation by the learning unit 300 described above (i.e., the likelihood ratios taken into account in the calculation of the loss function) will be specifically described with reference to Fig. 9. Fig. 9 is a matrix diagram showing an example of the likelihood ratios taken into account by the learning unit in the information processing system according to the fourth embodiment.

[0071] As already explained in the second embodiment (see FIG. 6), the likelihood ratios are arranged in a skew matrix in the matrix shown in FIG. 9. The learning unit 300 according to the fourth embodiment selects the likelihood ratio with the correct class in the numerator from the N×(N-1) patterns of likelihood ratios excluding the likelihood ratios on the diagonal line in such a matrix, and takes them into account in the loss function.

[0072] For example, assume that the correct class of sequence data input as training data is "class 1". In this case, the learning unit 300 selects the likelihood ratios with class 1 as the numerator from the likelihood ratios of N×(N-1) patterns and considers them in the loss function. Specifically, only the likelihood ratios in the second row from the top of FIG. 9 (excluding the likelihood ratios on the diagonal) are selected and considered in the loss function. In this case, log{p(X|y=1) / p(X|y=0)} in the second row from the top and the first column from the left and log{p(X|y=1) / p(X|y=2)} in the second row from the top and the third column from the left are considered in the loss function. That is, the likelihood ratios not shaded in gray in FIG. 9 are considered in the loss function.

[0073] If the correct class of the sequence data input as training data is "class 0", the learning unit 300 may select the likelihood ratios with class 0 as the numerator from the likelihood ratios of the N×(N-1) patterns and consider them in the loss function. Specifically, only the likelihood ratios in the first row from the top of FIG. 9 (excluding the likelihood ratios on the diagonal) may be selected and considered in the loss function. In this case, log{p(X|y=0) / p(X|y=1)} in the first row from the top and second column from the left and log{p(X|y=0) / p(X|y=2)} in the first row from the top and third column from the left are considered in the loss function.

[0074] Similarly, when the correct class of sequence data input as training data is "class 2", the learning unit 300 may select the likelihood ratios with class 2 as the numerator from the likelihood ratios of N×(N-1) patterns and consider them in the loss function. Specifically, only the likelihood ratios in the third row from the top of FIG. 9 (excluding the likelihood ratios on the diagonal) may be selected and considered in the loss function. In this case, log{p(X|y=2) / p(X|y=0)} in the third row from the top and first column from the left and log{p(X|y=2) / p(X|y=1)} in the third row from the top and second column from the left are considered in the loss function.

[0075] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the fourth embodiment will be described.

[0076] As described in FIG. 8 and FIG. 9, in the information processing system 1 according to the fourth embodiment, learning is performed using a loss function that takes into account the likelihood ratio in which the correct class is in the numerator of the N×(N-1) pattern. By using such a loss function, appropriate learning can be performed as in the above-described embodiments, and it is possible to appropriately select at least one class to which sequence data belongs from among multiple classes that are classification candidates. Furthermore, in particular, in the fourth embodiment, the loss function takes into account the likelihood ratio in which the correct class is in the numerator (in other words, the likelihood ratio that may have a large effect on the loss function), and therefore learning can be performed more efficiently than when all likelihood ratios of the N×(N-1) pattern are taken into account.

[0077] In addition, the loss function used in the fourth embodiment is a log-sum-exp type loss function, as in the above-mentioned embodiments. Therefore, it is possible to improve the convergence in the stochastic gradient descent method, and as a result, it is possible to perform more efficient learning.

[0078] <Fifth embodiment> An information processing system 1 according to the fifth embodiment will be described. The fifth embodiment describes a specific example of the loss function used in the first to fourth embodiments described above, and the device configuration and operation flow may be similar to those of the first to fourth embodiments. Therefore, in the following, only the parts that differ from the embodiments already described will be described in detail, and the explanation of the other overlapping parts will be omitted as appropriate.

[0079] (Example of loss function) An example of a log-sum-exp type loss function used in the information processing system 1 according to the fifth embodiment is given by the following formula (1). Note that the input data set (a set of data and labels) is {X i ,y i} N i=1 Let us assume that.

[0080]

number

[0081] The above formula (1) is a loss function corresponding to the configuration described in the fourth embodiment that considers the likelihood ratio in which the correct class is in the numerator. In formula (1), K is the number of classes, M is the number of data, and T is the time series length. Also, i is a subscript in the row direction and l is a subscript in the column direction (i.e., the subscripts indicating the row number and column number in the matrix shown in FIG. 6, etc.). λ is a likelihood ratio, and in the above formula (1), it represents the log-likelihood ratio of the row l-th column of label yk at time t.

[0082] The above formula (1) is in the form of "log(Σexp(x))" where the sum is inserted into the log, and a large gradient is created for the dominant sum in the log. Therefore, compared to loss functions such as "Σlog(1+exp(x))" or "Σlog(x)", the convergence in the stochastic gradient descent method is faster.

[0083] (Modification of loss function) The loss function of the above formula (1) can be transformed into the following formula (2).

[0084]

number

[0085] In the above formula (2), two of the three sums that were in log in the above formula (1) are taken out of log. In this way, when there are multiple sums in a loss function, there needs to be at least one sum in log, and the remaining sums can be taken out of log.

[0086] When transforming the loss function as in the above formula (2), multiple variations arise depending on which sums are put into log. For example, in formula (2), only the sums for K are put into log, but only the sums for M may be put into log, or only the sums for T may be put into log. Alternatively, two sums for M and T may be put into log, two sums for M and K may be put into log, or two sums for T and K may be put into log.

[0087] In the above-mentioned variations, the influence on convergence differs depending on which sum is put into log. Therefore, the sum to be put into log may be determined taking into consideration the influence of each item. Note that it is sufficient that the loss function with which sum is put into log is set in advance. However, the loss function with which sum is put into log may be configured to be appropriately selected by the user.

[0088] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the fifth embodiment will be described.

[0089] As described in FIG. 10, in the information processing system 1 according to the fifth embodiment, the learning unit 300 uses loss functions such as the above formulas (1) and (2). This improves the convergence in the stochastic gradient descent method, and as a result, makes it possible to perform efficient learning. In addition, for a loss function including multiple sums, such as formula (2), the influence on the convergence can be changed by selecting at least one sum to be put in log and taking the remaining sums out of log. As a result, by appropriately setting which sum among multiple sums is put in log to use as the loss function, more efficient learning can be performed.

[0090] Sixth Embodiment An information processing system 1 according to the sixth embodiment will be described. As with the fifth embodiment, the sixth embodiment describes a specific example of the loss function used in the first to fourth embodiments, and the device configuration and operation flow may be the same as those of the first to fourth embodiments. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and the explanation of the other overlapping parts will be omitted as appropriate.

[0091] (Example of loss function) An example of a log-sum-exp type loss function used in the information processing system 1 according to the sixth embodiment is given by the following formula (3). Note that the input data set (a set of data and labels) is {X i ,y i} N i=1 Let us assume that.

[0092]

number

[0093] The above formula (3) is a loss function corresponding to the configuration that considers all likelihood ratios of the N×(N-1) patterns described in the second embodiment. In formula (3), K is the number of classes, M is the number of data, and T is the time series length. Also, i is a subscript in the row direction, and l is a subscript in the column direction (i.e., a subscript indicating the row number and column number in the matrix shown in FIG. 6, etc.). δ is the Kronecker delta, which is "1" when the subscripts match, and "0" otherwise. λ is a likelihood ratio, and in the above formula (3), it represents the log likelihood ratio of the row / lth column of label yk at time t.

[0094] The above formula (3) is in the form of "log(Σexp(x))" with sum inside log, and a large gradient is created for the dominant sum inside log. Therefore, compared to loss functions such as "Σlog(1+exp(x))" or "Σlog(x)", the convergence in stochastic gradient descent is faster.

[0095] (Modification of loss function) The loss function of the above formula (3) can be transformed into the following formula (4).

[0096]

number

[0097] In the above formula (4), two of the four sums included in log in the above formula (3) are taken out of log. As explained in the fifth embodiment, when there are multiple sums in a loss function, it is sufficient that at least one sum is included in log, and the remaining sums may be taken out of log.

[0098] (Example of weighting) The loss function may be weighted. For example, weighting the above formula (4) results in the following formula (5).

[0099]

number

[0100] In the above formula (5), w it and w' itkl are weighting coefficients. These weighting coefficients may be values ​​determined by, for example, experience or tuning. it and w' itkl The weighting in the above-mentioned formula (5) is merely an example, and the weighting may be performed by, for example, multiplying a term other than that in the formula (5) by a weighting coefficient.

[0101] Although an example of weighting a loss function such as formula (4) has been described here, weighting can be similarly performed on other log-sum-exp type loss functions. For example, weighting may be performed on formula (3) before transformation, or weighting may be performed on formulas (1) and (2) described in the fifth embodiment.

[0102] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the sixth embodiment will be described.

[0103] As described in FIG. 11, in the information processing system 1 according to the sixth embodiment, the learning unit 300 uses loss functions such as the above formulas (3), (4), and (5). This improves the convergence in the stochastic gradient descent method, and as a result, makes it possible to perform efficient learning. Furthermore, by performing weighting as in formula (5), even more efficient learning can be performed.

[0104] Seventh embodiment An information processing system 1 according to the seventh embodiment will be described with reference to Fig. 10 and Fig. 11. The seventh embodiment differs from the first to sixth embodiments described above only in part of the configuration and operation (specifically, the configuration and operation of the classification device 10), and other parts may be similar to the first to sixth embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0105] (Functional configuration) First, the functional configuration of the information processing system 1 according to the seventh embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the functional configuration of the information processing system according to the seventh embodiment. In Fig. 10, the same reference numerals are used to designate the same elements as those shown in Fig. 2.

[0106] 10, in the information processing system 1 according to the seventh embodiment, the likelihood ratio calculation unit 100 in the classification device 10 includes a first calculation unit 110 and a second calculation unit 120. Each of the first calculation unit 110 and the second calculation unit 120 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0107] The first calculation unit 110 is configured to be able to calculate an individual likelihood ratio based on two consecutive elements included in the sequence data. The individual likelihood ratio is calculated as a likelihood ratio indicating the likelihood of a class to which the two consecutive elements belong. The first calculation unit 110 may, for example, sequentially acquire elements included in the sequence data from the data acquisition unit 50, and sequentially calculate individual likelihood ratios based on the two consecutive elements. The individual likelihood ratios calculated by the first calculation unit 110 are configured to be output to the second calculation unit 120.

[0108] The second calculation unit 120 is configured to be able to calculate an integrated likelihood ratio based on the multiple individual likelihood ratios calculated by the first calculation unit 110. The integrated likelihood ratio is calculated as a likelihood ratio indicating the likelihood of a class to which the multiple elements considered in each of the multiple individual likelihood ratios belong. In other words, the integrated likelihood ratio is calculated as a likelihood ratio indicating the likelihood of a class to which sequence data including multiple elements belongs. The integrated likelihood ratio calculated by the second calculation unit 120 is output to the class classification unit 200. The class classification unit 200 performs class classification of the sequence data based on the integrated likelihood ratio.

[0109] The learning unit 300 according to the fifth embodiment may perform learning on the likelihood ratio calculation unit 100 as a whole (i.e., the first calculation unit 110 and the second calculation unit 120 together), or may perform learning separately on the first calculation unit 110 and the second calculation unit 120. Alternatively, the learning unit 300 may be provided separately as a first learning unit that performs learning only on the first calculation unit 110, and a second learning unit that performs learning only on the second calculation unit 120. In this case, only one of the first learning unit and the second learning unit may be provided.

[0110] (Classification operation flow) Next, a flow of operation of the classification device 10 in the information processing system 1 according to the seventh embodiment (specifically, a class classification operation after learning) will be described with reference to Fig. 11. Fig. 11 is a flowchart showing a flow of operation of the classification device in the information processing system according to the seventh embodiment.

[0111] 11, when the operation of the classification device 10 is started, first, the data acquiring unit 50 acquires elements included in the sequence data (step S21). The data acquiring unit 50 outputs the acquired elements of the sequence data to the first calculation unit 110.

[0112] Then, the first calculation unit 110 calculates an individual likelihood ratio based on the two consecutive elements obtained (step S22). After that, the second calculation unit 120 calculates an integrated likelihood ratio based on the multiple individual likelihood ratios calculated by the first calculation unit 110 (step S23).

[0113] Next, the classifying unit 200 performs classifying based on the calculated integrated likelihood ratio (step S24). The classifying may determine one class to which the sequence data belongs, or may determine multiple classes to which the sequence data is likely to belong. The classifying unit 200 may output the result of the classifying to a display or the like. The classifying unit 200 may also output the result of the classifying by voice via a speaker or the like.

[0114] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the seventh embodiment will be described.

[0115] As described in FIG. 10 and FIG. 11, in the information processing system 1 according to the seventh embodiment, first, individual likelihood ratios are calculated based on two elements, and then, an integrated likelihood ratio is calculated based on a plurality of individual likelihood ratios. By using the integrated likelihood ratio calculated in this way, it becomes possible to appropriately select the class to which the sequence data belongs. Also, in the classification device 10 that calculates the individual likelihood ratios and the integrated likelihood ratio, the convergence in the stochastic gradient descent method can be improved by using the log-sum-exp type loss function described in each of the above-mentioned embodiments. Therefore, efficient learning can be performed.

[0116] Eighth embodiment The information processing system 1 according to the eighth embodiment will be described with reference to Fig. 12 and Fig. 13. The eighth embodiment differs from the seventh embodiment described above only in part of the configuration and operation (specifically, the configuration and operation of the likelihood ratio calculation unit 100), and other parts may be similar to the seventh embodiment. Therefore, hereinafter, the parts that differ from the embodiments already described will be described in detail, and the description of other overlapping parts will be omitted as appropriate.

[0117] (Functional configuration) First, the functional configuration of the information processing system 1 according to the eighth embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram showing the functional configuration of the information processing system according to the eighth embodiment. In Fig. 12, the same reference numerals are used to designate the same elements as those shown in Figs. 2 and 10.

[0118] As shown in FIG. 12, in the information processing system 1 according to the eighth embodiment, the likelihood ratio calculation unit 100 in the classification device 10 includes a first calculation unit 110 and a second calculation unit 120. The first calculation unit 110 includes an individual likelihood ratio calculation unit 111 and a first storage unit. The second calculation unit 120 includes an integrated likelihood ratio calculation unit 121 and a second storage unit 122. Each of the individual likelihood ratio calculation unit 111 and the integrated likelihood ratio calculation unit 121 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1). Each of the first storage unit 112 and the second storage unit 122 may be realized by, for example, the above-mentioned storage device 14 (see FIG. 1).

[0119] The individual likelihood ratio calculation unit 111 is configured to be able to calculate an individual likelihood ratio based on two consecutive elements among the elements sequentially acquired by the data acquisition unit 50. More specifically, the individual likelihood ratio calculation unit 111 calculates an individual likelihood ratio based on a newly acquired element and past data stored in the first storage unit 112. The information stored in the first storage unit 112 is configured to be readable by the individual likelihood ratio calculation unit 111. When the first storage unit 112 stores past individual likelihood ratios, the individual likelihood ratio calculation unit 111 may read out the stored past individual likelihood ratios and calculate a new individual likelihood ratio taking the acquired element into consideration. On the other hand, when the first storage unit 112 stores the elements themselves acquired in the past, the individual likelihood ratio calculation unit 111 may calculate the past individual likelihood ratios from the stored past elements and calculate the likelihood ratio for the newly acquired element.

[0120] The integrated likelihood ratio calculation unit 121 is configured to be able to calculate an integrated likelihood ratio based on a plurality of individual likelihood ratios. The integrated likelihood ratio calculation unit 121 calculates a new integrated likelihood ratio using the individual likelihood ratios calculated by the individual likelihood ratio calculation unit 111 and past integrated likelihood ratios stored in the second storage unit 122. The information stored in the second storage unit 122 (i.e., past integrated likelihood ratios) is configured to be readable by the integrated likelihood ratio calculation unit 121.

[0121] <Flow of likelihood ratio calculation> Next, a flow of a likelihood ratio calculation operation (i.e., an operation of the likelihood ratio calculation unit 100) in the information processing system 1 according to the eighth embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing a flow of an operation of the likelihood ratio calculation unit in the information processing system according to the eighth embodiment.

[0122] 13, when the likelihood ratio calculation operation by the likelihood ratio calculation unit 100 is started, first, the individual likelihood ratio calculation unit 111 in the first calculation unit 110 reads out past data from the first storage unit 112 (step S31). The past data may be, for example, the processing result in the individual likelihood ratio calculation unit 111 of the element acquired immediately before the element currently acquired by the data acquisition unit 50 (in other words, the individual likelihood ratio calculated for the element immediately before). Alternatively, the past data may be the element itself acquired immediately before the element acquired in acquisition.

[0123] Next, the individual likelihood ratio calculation unit 111 calculates a new individual likelihood ratio (i.e., an individual likelihood ratio for the element currently acquired by the data acquisition unit 50) based on the element acquired by the data acquisition unit 50 and the past data read out from the first storage unit 112 (step S32). The individual likelihood ratio calculation unit 111 outputs the calculated individual likelihood ratio to the second calculation unit 120. The individual likelihood ratio calculation unit 111 may store the calculated individual likelihood ratio in the first storage unit 112.

[0124] Next, the integrated likelihood ratio calculation unit 121 in the second calculation unit 120 reads out a past integrated likelihood ratio from the second storage unit 122 (step S33). The past integrated likelihood ratio may be, for example, a processing result by the integrated likelihood ratio calculation unit 121 for an element acquired immediately before the element currently acquired by the data acquisition unit 50 (in other words, an integrated likelihood ratio calculated for the immediately previous element).

[0125] Next, the integrated likelihood ratio calculation unit 121 calculates a new integrated likelihood ratio (i.e., an integrated likelihood ratio for the element currently acquired by the data acquisition unit 50) based on the likelihood ratio calculated by the individual likelihood ratio calculation unit 111 and the past integrated likelihood ratio read out from the second storage unit 122 (step S34). The integrated likelihood ratio calculation unit 121 outputs the calculated integrated likelihood ratio to the classifying unit 200. The integrated likelihood ratio calculation unit 121 may store the calculated integrated likelihood ratio in the second storage unit 122.

[0126] (Technical effect) Next, technical effects obtained by the information processing system 1 according to the eighth embodiment will be described.

[0127] As described in FIG. 12 and FIG. 13, in the information processing system 1 according to the eighth embodiment, after individual likelihood ratios are calculated using past individual likelihood ratios, an integrated likelihood ratio is calculated using past integrated likelihood ratios. By using the integrated likelihood ratio calculated in this way, it becomes possible to appropriately select the class to which the sequence data belongs. Also, in the classification device 10 that calculates individual likelihood ratios and integrated likelihood ratios using past data, the convergence in the stochastic gradient descent method can be improved by using the log-sum-exp type loss function described in each of the above-mentioned embodiments. Therefore, efficient learning can be performed.

[0128] <Ninth embodiment> The information processing system 1 according to the ninth embodiment will be described with reference to Fig. 14. The ninth embodiment differs from the first to eighth embodiments described above only in some operations (specifically, the operation of the classifier 200), and other parts may be similar to the first to eighth embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0129] (Classification operation flow) First, the flow of operation of the classification device 10 in the information processing system 1 according to the ninth embodiment (specifically, class classification operation after learning) will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of operation of the classification device in the information processing system according to the ninth embodiment. In Fig. 14, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0130] 14, when the operation of the classification device 10 is started, first, the data acquiring unit 50 acquires elements included in the sequence data (step S11). The data acquiring unit 50 outputs the acquired elements of the sequence data to the likelihood ratio calculating unit 100. Then, the likelihood ratio calculating unit 100 calculates a likelihood ratio based on two or more acquired elements (step S12).

[0131] Next, the classifying unit 200 performs classifying based on the calculated likelihood ratio. Particularly in the ninth embodiment, the classifying unit 200 selects and outputs multiple classes to which the sequence data may belong (step S41). That is, the classifying unit 200 does not determine one class to which the sequence data belongs, but determines multiple classes to which the sequence data is likely to belong. More specifically, the classifying unit 200 executes a process of selecting k classes (where k is a natural number equal to or less than n) from n classes (where n is a natural number) prepared as classification candidates.

[0132] The classifying unit 200 may output information about the k classes to which the sequence data may belong to a display or the like. Also, the classifying unit 200 may output information about the k classes to which the sequence data may belong by voice via a speaker or the like.

[0133] When outputting information about k classes to which sequence data may possibly belong, the class classification unit 200 may rearrange the information and output it. For example, the class classification unit 200 may rearrange the information about the k classes in descending order of likelihood ratio and output it. Alternatively, the class classification unit 200 may output information about the k classes in a different manner for each class. For example, the class classification unit 200 may output the information about the k classes in a display manner in which classes with high likelihood ratios are emphasized, while outputting the information about classes with low likelihood ratios in a display manner in which they are not emphasized. When highlighting, for example, the size or color of the display may be changed, or the displayed object may be given a movement.

[0134] (Specific examples of use) The configuration for outputting k classes from among n classes described above will be described below by giving some specific application examples.

[0135] (Product proposal) The information processing system 1 according to the ninth embodiment may be used to suggest products that a user may be interested in on a web shopping site. Specifically, the information processing system 1 may select k products (i.e., k classes) that the user may be interested in from n products (i.e., n classes) that are handled products, and output them to the user (note that k is a number smaller than n). In this case, examples of the sequence data to be input include past purchase history, browsing history, etc.

[0136] Similarly, it may be used to suggest products and stores in digital signage, etc. Digital signage may be capable of capturing an image of a user using a camera mounted thereon. In this case, the user's emotions may be estimated from the user's image, and stores and products may be suggested according to the emotions. In addition, the user's line of sight may be estimated from the user's image (i.e., the part the user is looking at may be estimated), and stores and products that the user may be interested in may be suggested. Alternatively, the user's attributes (e.g., gender, age, etc.) may be estimated from the user's image, and stores and products that the user may be interested in may be suggested. When estimating information about a user as described above, weighting may be applied to the n classes according to the estimated information.

[0137] (Criminal Investigation) The information processing system 1 according to the ninth embodiment can also be used in criminal investigations. For example, when finding the real culprit from among multiple suspects, if only one person who is most likely to be the culprit is selected from among them, a major problem will occur if the selection is incorrect. However, the information processing system 1 according to this embodiment can select and output the top K suspects who are most likely to be the culprit. Specifically, it is sufficient to select and output a class corresponding to the top k suspects who are most likely to be the culprit from sequence data that includes information about each of the multiple suspects as elements. In this way, it is possible to appropriately find the real culprit by investigating, for example, multiple suspects who are most likely to be the culprit.

[0138] (Radar image analysis) The information processing system 1 according to the ninth embodiment can also be applied to the analysis of radar images. Radar images, by their nature, tend to have low clarity, so it is difficult for a machine alone to accurately determine, for example, what is in the image. However, the information processing system 1 according to this embodiment can select and output k candidates that are likely to be in the radar image. Therefore, it is possible to first output k candidates and have the user make a judgment from among them. For example, if "dogs," "cats," "ships," and "tanks" are presented as candidates that appear in a radar image of a port, the user can easily determine that a "ship," which is highly related to the port, appears in the radar image.

[0139] Note that the above-mentioned application example is just one example, and in a situation where it is required to select k items from n candidates, beneficial effects can be achieved by applying the information processing system 1 of this embodiment.

[0140] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-mentioned embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. That is, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0141] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the recording media, but also those that execute processing by operating on an OS in cooperation with other software and functions of an extension board.

[0142] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing systems, information processing methods, and computer programs involving such modifications are also included in the technical idea of ​​this disclosure.

[0143] <Additional Notes> The above-described embodiment may be further described as follows, but is not limited to the following.

[0144] (Appendix 1) The information processing system described in Supplementary Note 1 is an information processing system characterized by comprising: an acquisition means for acquiring a plurality of elements included in sequence data; a calculation means for calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; a classification means for classifying the sequence data into at least one class among a plurality of candidate classes based on the likelihood ratio; and a learning means for performing learning regarding the calculation of the likelihood ratio by using a log-sum-exp type loss function.

[0145] (Appendix 2) The information processing system described in Supplementary Note 2 is the information processing system described in Supplementary Note 1, characterized in that the learning means performs the learning using a loss function that takes into account the likelihood ratio of an N×(N-1) pattern, where the denominator is a likelihood indicating the likelihood of belonging to one class out of N classes (where N is a natural number) that are classification candidates for the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class.

[0146] (Appendix 3) The information processing system described in Appendix 3 is the information processing system described in Appendix 2, characterized in that the learning means performs the learning using a loss function that takes into account a portion of the likelihood ratios of the N×(N-1) patterns.

[0147] (Appendix 4) The information processing system described in Appendix 4 is the information processing system described in Appendix 3, characterized in that the learning means performs the learning using a loss function that takes into account a likelihood ratio in which the correct class is in the numerator among the N×(N-1) patterns.

[0148] (Appendix 5) The information processing system described in Appendix 5 is the information processing system described in any one of Appendixes 1 to 4, characterized in that the loss function includes multiple sums, and at least one of the multiple sums is included in the log-sum-exp type.

[0149] (Appendix 6) The information processing system described in Supplementary Note 6 is the information processing system described in any one of Supplementary Notes 1 to 5, characterized in that the loss function includes a weighting coefficient according to the difficulty of classifying the sequence data.

[0150] (Appendix 7) The information processing system described in Supplementary Note 7 is the information processing system described in any one of Supplementary Notes 1 to 6, characterized in that the likelihood ratio is an integrated likelihood ratio calculated by taking into account multiple individual likelihood ratios calculated based on two consecutive elements included in the sequence data.

[0151] (Appendix 8) The information processing system described in Appendix 8 is the information processing system described in Appendix 7, characterized in that the acquisition means sequentially acquires a plurality of elements included in the sequence data, and the calculation means calculates a new integrated likelihood ratio using the individual likelihood ratio calculated based on the newly acquired elements and the integrated likelihood ratio calculated in the past.

[0152] (Appendix 9) The information processing method described in Supplementary Note 9 is an information processing method comprising: acquiring a plurality of elements included in sequence data; calculating a likelihood ratio indicating a likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; classifying the sequence data into at least one class among a plurality of candidate classes based on the likelihood ratio; and performing learning related to the calculation of the likelihood ratio using a log-sum-exp type loss function.

[0153] (Appendix 10) The computer program described in Supplementary Note 10 is a computer program that causes a computer to obtain a plurality of elements included in sequence data, calculate a likelihood ratio indicating a likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements, classify the sequence data into at least one class among a plurality of candidate classes based on the likelihood ratio, and perform learning related to the calculation of the likelihood ratio by using a log-sum-exp type loss function.

[0154] (Appendix 11) The recording medium according to claim 11 is a recording medium having the computer program according to claim 10 recorded thereon. [Explanation of symbols]

[0155] 1. Information Processing Systems 11 Processors 14 Storage device 10 Classification device 50 Data Acquisition Section 100 Likelihood ratio calculation section 110 First Calculation Section 111 Individual likelihood ratio calculation unit 112 1st memory section 120 Second Calculation Section 121 Integrated Likelihood Ratio Calculation Unit 122 2nd memory section 200 Classification Section 300 Learning Department

Claims

1. An acquisition means for acquiring a plurality of elements included in the sequence data; a calculation means for calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; a classification means for classifying the sequence data into at least one class out of a plurality of classes that are classification candidates based on the likelihood ratio; A learning means for performing learning regarding the calculation of the likelihood ratio using a log-sum-exponential loss function; Equipped with The learning means performs the learning using a loss function that takes into account the likelihood ratio in which the correct class is in the numerator among the likelihood ratios of N×(N−1) patterns, in which the denominator is a likelihood indicating the likelihood of belonging to one of N classes (where N is a natural number) that are classification candidates of the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class. An information processing system comprising:

2. 2. The information processing system according to claim 1, wherein the loss function includes a plurality of sums, and at least one of the plurality of sums is included in the log-sum-exponential type.

3. 3. The information processing system according to claim 1, wherein the likelihood ratio is an integrated likelihood ratio calculated by taking into account a plurality of individual likelihood ratios calculated based on two consecutive elements included in the sequence data.

4. The acquiring means sequentially acquires a plurality of elements included in the sequence data, The calculation means calculates a new integrated likelihood ratio by using the individual likelihood ratio calculated based on a newly acquired element and the integrated likelihood ratio calculated in the past.

4. The information processing system according to claim 3.

5. Obtain multiple elements contained in the sequence data, calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; classifying the sequence data into at least one class among a plurality of candidate classes based on the likelihood ratio; A log-sum-exponential loss function is used to learn the likelihood ratio. The learning is performed using a loss function that takes into account the likelihood ratio in which the correct class is in the numerator among the likelihood ratios of N×(N−1) patterns, in which the denominator is a likelihood indicating the likelihood of belonging to one of N classes (where N is a natural number) that are classification candidates of the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class.

23. An information processing method comprising:

6. Obtain multiple elements contained in the sequence data, calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; classifying the sequence data into at least one class among a plurality of candidate classes based on the likelihood ratio; A log-sum-exponential loss function is used to learn the likelihood ratio. The learning is performed using a loss function that takes into account the likelihood ratio in which the correct class is in the numerator among the likelihood ratios of N×(N−1) patterns, in which the denominator is a likelihood indicating the likelihood of belonging to one of N classes (where N is a natural number) that are classification candidates of the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class. A computer program for causing a computer to operate in such a manner.

Images