Information processing system, information processing method, and computer program

Abstract

To appropriately adjust model parameters.SOLUTION: An information processing system comprises: acquisition means for acquiring a plurality of elements contained in sequential data; calculation means for calculating a likelihood ratio indicative of the plausibility of the class to which the sequential data belongs on the basis of at least two consecutive elements among the plurality of elements; classification means for classifying the sequential data into at least one class among three or more classes, which are classification candidates, on the basis of the likelihood ratio; and learning means for adjusting the parameters of a model for calculating the likelihood ratio so as to minimize the value of a loss function calculated using the likelihood ratio. The loss function does not take into account cases where the numerator and the denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes small in cases where the correct class to which the sequential data belongs is included in the numerator but not in the denominator of the likelihood ratio, or where it is included in the denominator but not in the numerator.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of an information processing system, an information processing method, and a computer program for processing information relating to, for example, class classification. [Background technology]

[0002] Known systems of this type perform learning processes related to likelihood. For example, Patent Document 1 discloses training a support vector machine used to determine likelihood. Patent Document 2 discloses that a support vector machine or logistic regression may be used to train a classifier that uses likelihood.

[0003] As another related technology, for example, Patent Document 3 discloses a technology in which a device determines whether a person included in an image is a registered person, and performs folder classification processing on image files based on predetermined recognition conditions and predominance conditions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-139618 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-174054 [Patent Document 3] Japanese Patent Application Laid-Open No. 2010-250730 Summary of the Invention [Problem to be solved by the invention]

[0005] This disclosure aims to improve upon the related art discussed above. [Means for solving the problem]

[0006] One aspect of the information processing system disclosed herein includes an acquisition means for acquiring multiple elements included in sequential data; a calculation means for calculating a likelihood ratio indicating the likelihood of a class to which the sequential data belongs based on at least two consecutive elements among the multiple elements; a classification means for classifying the sequential data into at least one of three or more candidate classes based on the likelihood ratio; and a learning means for adjusting parameters of a model for calculating the likelihood ratio so as to reduce the value of a loss function calculated using the likelihood ratio. The loss function does not take into account cases where the numerator and denominator of the likelihood ratio have the same likelihood, and reduces the value of the loss function when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio, and when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio, and when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio.

[0007] One aspect of the information processing method disclosed herein includes: acquiring multiple elements included in sequential data; calculating a likelihood ratio indicating the likelihood of a class to which the sequential data belongs based on at least two consecutive elements among the multiple elements; classifying the sequential data into at least one of three or more candidate classes based on the likelihood ratio; adjusting parameters of a model for calculating the likelihood ratio so that the value of a loss function calculated using the likelihood ratio is small; the loss function does not take into account cases where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function is small when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio, and when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio, and when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio.

[0008] One aspect of the computer program disclosed herein acquires multiple elements included in sequential data, calculates a likelihood ratio indicating the likelihood of a class to which the sequential data belongs based on at least two consecutive elements among the multiple elements, classifies the sequential data into at least one of three or more candidate classes based on the likelihood ratio, adjusts model parameters for calculating the likelihood ratio so that the value of a loss function calculated using the likelihood ratio is small, and operates the computer so that the loss function does not take into account cases where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function is small when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio, and when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio, and when the correct class to which the sequential data belongs is in the numerator but not the denominator of the likelihood ratio. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration of an information processing system according to a first embodiment. [Figure 2] 1 is a block diagram showing a functional configuration of an information processing system according to a first embodiment. [Figure 3] 4 is a flowchart showing the flow of operations of the classification device in the information processing system according to the first embodiment. [Figure 4] 5 is a flowchart showing the flow of operations of a learning unit in the information processing system according to the first embodiment. [Figure 5] 10 is a flowchart showing the flow of operations of a learning unit in the information processing system according to the second embodiment. [Figure 6] FIG. 11 is a matrix diagram showing an example of likelihood ratios taken into account by a learning unit in the information processing system according to the second embodiment. [Figure 7] 11 is a flowchart showing the flow of operations of a learning unit in the information processing system according to the third embodiment. [Figure 8] 10 is a flowchart showing the flow of operations of a learning unit in the information processing system according to the fourth embodiment. [Figure 9]FIG. 13 is a matrix diagram showing an example of likelihood ratios taken into account by a learning unit in the information processing system according to the fourth embodiment. [Figure 10] 13 is a graph showing an example of a sigmoid function used in the information processing system according to the fifth embodiment. [Figure 11] 13 is a graph showing an example of a logistic function used in the information processing system according to the sixth embodiment. [Figure 12] FIG. 13 is a block diagram showing the functional configuration of an information processing system according to a seventh embodiment. [Figure 13] 13 is a flowchart showing the flow of operations of the classification device in the information processing system according to the seventh embodiment. [Figure 14] FIG. 13 is a block diagram showing the functional configuration of an information processing system according to an eighth embodiment. [Figure 15] 13 is a flowchart showing the flow of operations of a likelihood ratio calculation unit in the information processing system according to the eighth embodiment. [Figure 16] 13 is a flowchart showing the flow of operations of the classification device in the information processing system according to the ninth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an information processing system, an information processing method, and a computer program will be described with reference to the drawings.

[0011] First Embodiment An information processing system according to a first embodiment will be described with reference to FIGS.

[0012] (Hardware configuration) First, the hardware configuration of the information processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0013] 1, the information processing system 1 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 1 may further include an input device 15 and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected via a data bus 17.

[0014] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing system 1 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, functional blocks for performing classification using likelihood ratios and related learning processing are realized within the processor 11. Examples of the processor 11 include a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a demand-side platform (DSP), and an application-specific integrated circuit (ASIC). The processor 11 may be one of the above examples, or a plurality of processors may be used in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that is temporarily used by the processor 11 while the processor 11 is executing the computer programs. The RAM 12 may be, for example, a D-RAM (Dynamic RAM).

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable ROM (P-ROM).

[0017] The storage device 14 stores data that is to be saved long-term by the information processing system 1. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the information processing system 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be a dedicated controller (operation terminal). The input device 15 may also include a terminal owned by the user (for example, a smartphone, a tablet terminal, etc.). The input device 15 may also be a device that allows voice input, for example, including a microphone.

[0019] The output device 16 is a device that outputs information related to the information processing system 1 to the outside. For example, the output device 16 may be a display device (e.g., a display) that can display information related to the information processing system 1. The display device here may be a television monitor, a personal computer monitor, a smartphone monitor, a tablet terminal monitor, or a monitor of another mobile terminal. The display device may also be a large monitor or digital signage installed in various facilities such as a store. The output device 16 may also be a device that outputs information in a format other than an image. For example, the output device 16 may be a speaker that outputs information related to the information processing system 1 as audio.

[0020] (Functional configuration) Next, the functional configuration of the information processing system 1 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0021] As shown in FIG. 2, the information processing system 1 according to the first embodiment includes a classification device 10 and a learning unit 300. The classification device 10 is a device that performs class classification of input sequence data, and includes a data acquisition unit 50, a likelihood ratio calculation unit 100, and a classification unit 200 as processing blocks for realizing the function. The learning unit 300 is configured to be able to execute a learning process for the classification device 10. Note that, although an example is given here in which the learning unit 300 is provided separately from the classification device 10, the classification device 10 may also be configured to include the learning unit 300. Each of the data acquisition unit 50, the likelihood ratio calculation unit 100, the classification unit 200, and the learning unit 300 may be realized by the above-described processor 11 (see FIG. 1).

[0022] The data acquisition unit 50 is configured to acquire multiple elements included in the sequence data. The data acquisition unit 50 may acquire data directly from any data acquisition device (e.g., a camera, a microphone, etc.), or may read data that has been acquired in advance by a data acquisition device and stored in storage, etc. When acquiring data from a camera, the data acquisition unit 50 may be configured to acquire data from each of multiple cameras. The elements of the sequence data acquired by the data acquisition unit 50 are configured to be output to the likelihood ratio calculation unit 100. Note that sequence data is data including multiple elements arranged in a predetermined order, and an example of this is time-series data. More specific examples of sequence data include, but are not limited to, video data and audio data.

[0023] The likelihood ratio calculation unit 100 is configured to be able to calculate a likelihood ratio based on at least two consecutive elements among the multiple elements acquired by the data acquisition unit 50. Note that the "likelihood ratio" here is an index indicating the likelihood of the class to which the sequence data belongs. Specific examples of likelihood ratios and specific calculation methods will be described in detail in other embodiments described later.

[0024] The classification unit 200 is configured to classify sequential data based on the likelihood ratio calculated by the likelihood ratio calculation unit 100. The classification unit 200 selects at least one class to which the sequential data belongs from among a plurality of candidate classes. The candidate classes may be preset. Alternatively, the candidate classes may be appropriately set by a user, or may be appropriately set based on the type of sequential data to be handled.

[0025] The learning unit 300 performs learning related to the calculation of likelihood ratios using a loss function. Specifically, learning related to the calculation of likelihood ratios is performed so that class classification based on likelihood ratios is performed accurately. The loss function used by the learning unit 300 is defined as a function that increases the likelihood ratio when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and decreases the likelihood ratio when the correct class is in the denominator of the likelihood ratio. The loss function may be set in advance as a function that satisfies this definition. Specific examples of loss functions will be described in detail in other embodiments described later.

[0026] (Classification operation flow) Next, the flow of operation of the classification device 10 in the information processing system 1 according to the first embodiment (specifically, the class classification operation after learning) will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operation of the classification device in the information processing system according to the first embodiment.

[0027] 3, when the operation of the classification device 10 starts, the data acquiring unit 50 first acquires elements included in the sequence data (step S11). The data acquiring unit 50 outputs the acquired elements of the sequence data to the likelihood ratio calculating unit 100. Then, the likelihood ratio calculating unit 100 calculates a likelihood ratio based on the two or more acquired elements (step S12).

[0028] Next, the classification unit 200 performs classification based on the calculated likelihood ratio (step S13). Classification may determine one class to which the sequence data belongs, or may determine multiple classes to which the sequence data is likely to belong. The classification unit 200 may output the classification result to a display or the like. Alternatively, the classification unit 200 may output the classification result as sound via a speaker or the like.

[0029] (Learning procedure) Next, the flow of operation of the learning unit 300 in the information processing system 1 according to the first embodiment (i.e., learning operation related to calculation of likelihood ratios) will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of operation of the learning unit in the information processing system according to the first embodiment.

[0030] 4, when the learning operation is started, training data is first input to the learning unit 300 (step S101). The training data may be configured as a set of, for example, sequence data and information on the correct class to which the sequence data belongs (i.e., correct data).

[0031] Next, the learning unit 300 calculates a loss function using the input training data (step S102). As already explained, the loss function here is a function that increases the likelihood ratio when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and decreases the likelihood ratio when the correct class is in the denominator of the likelihood ratio.

[0032] Next, the learning unit 300 adjusts the parameters (specifically, the parameters of the model that calculates the likelihood ratio) so as to reduce the calculated loss function (step S103). That is, the learning unit 300 optimizes the parameters of the model that calculates the likelihood ratio. Note that existing techniques can be appropriately adopted as a parameter optimization method using the loss function. An example of the optimization method is the backpropagation method, but other methods may also be used.

[0033] Thereafter, the learning unit 300 determines whether all learning has been completed (step S104). The learning unit 300 may determine whether all learning has been completed, for example, based on whether all training data has been input. Alternatively, the learning unit 300 may determine whether all learning has been completed based on whether a predetermined period of time has elapsed since the start of learning. Alternatively, the learning unit 300 may determine whether all learning has been completed based on whether the processes from steps S101 to S103 described above have been looped a predetermined number of times.

[0034] If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101. This repeats the learning process using the training data, and the parameters are adjusted to be more optimal.

[0035] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the first embodiment will be described.

[0036] As described with reference to FIGS. 1 to 4, in the information processing system 1 according to the first embodiment, the learning unit 300 performs learning related to the calculation of likelihood ratios used for class classification. In this embodiment, learning is performed using a loss function that increases the likelihood ratio when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio and decreases the likelihood ratio when the correct class is in the denominator of the likelihood ratio. Using such a loss function enables learning to increase the penalty when the class is incorrect and decrease the penalty when the class is correct. As a result, it becomes possible to appropriately select at least one class to which the sequence data belongs from among multiple candidate classes.

[0037] When multiple classes exist as classification candidates (so-called multi-class classification), it is not easy to determine what likelihood ratios to consider during training (e.g., what ratios should be taken). However, by using the loss function described above, the magnitude of the likelihood ratio changes depending on whether the correct class is in the numerator or denominator of the likelihood ratio, and the impact on the loss function changes. By using such a loss function, it is possible to appropriately perform training related to the calculation of likelihood ratios in multi-class classification. As a result, it is possible to achieve appropriate class classification. Note that it is particularly difficult to determine what likelihood ratios to consider during training when there are three or more classification candidates. Therefore, the technical effect of this embodiment is particularly pronounced when there are three or more classification candidates.

[0038] Second Embodiment An information processing system 1 according to the second embodiment will be described with reference to Figures 5 and 6. The second embodiment differs from the first embodiment only in some of its operations, and may be similar to the first embodiment in terms of, for example, the device configuration (see Figures 1 and 2) and the operation of the classification device 10 (see Figure 3). Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit a description of other overlapping parts as appropriate.

[0039] (Learning procedure) First, the flow of operations of the learning unit 300 in the information processing system 1 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of operations of the information processing system according to the second embodiment. Note that in Fig. 5, the same processes as those shown in Fig. 4 are denoted by the same reference numerals.

[0040] As shown in FIG. 5, when the operation of the information processing system 1 according to the second embodiment starts, training data is first input to the learning unit 300 (step S101).

[0041] Next, the learning unit 300 calculates a loss function using the input training data. In particular, in the second embodiment, the loss function takes into account likelihood ratios of N×(N−1) patterns, where the denominator is a likelihood indicating the likelihood of belonging to one of N classes (where N is a natural number) that are classification candidates for the sequential data, and the numerator is a likelihood indicating the likelihood of belonging to another class (step S201). As in the first embodiment, this loss function is a function in which the likelihood ratio increases when the correct class to which the sequential data belongs is in the numerator, and decreases when the correct class is in the denominator. The likelihood ratios taken into account in the loss function will be described in detail later with specific examples.

[0042] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). That is, the learning unit 300 optimizes the parameters of the model that calculates the likelihood ratio. Thereafter, the learning unit 300 determines whether all learning has been completed (step S104). If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0043] (Examples of likelihood ratios to consider) Next, the likelihood ratios taken into account in the learning operation by the learning unit 300 (i.e., the likelihood ratios taken into account in calculating the loss function) will be specifically described with reference to Fig. 6. Fig. 6 is a matrix diagram showing an example of likelihood ratios taken into account by the learning unit in the information processing system according to the second embodiment.

[0044] As shown in Figure 6, the likelihoods are considered in the form of a matrix. For ease of explanation, let us assume that there are three candidate classes: "Class 0," "Class 1," and "Class 2." p(X|y=0) is the likelihood that the sequence data is "Class 0." p(X|y=1) is the likelihood that the sequence data is "Class 1." p(X|y=2) is the likelihood that the sequence data is "Class 2."

[0045] In the first row from the top of the matrix, the numerators of the log-likelihood ratios (hereinafter simply referred to as "likelihood ratios") are all p(X|y=0). In the second row from the top of the matrix, the numerators of the likelihood ratios are all p(X|y=1). In the third row from the top of the matrix, the numerators of the likelihood ratios are all p(X|y=2). Meanwhile, in the first column from the left of the matrix, the denominators of the likelihood ratios are all p(X|y=0). In the second column from the left of the matrix, the denominators of the likelihood ratios are all p(X|y=1). In the third column from the left of the matrix, the denominators of the likelihood ratios are all p(X|y=2).

[0046] The likelihood ratios on the diagonal line of the matrix (likelihood ratios shaded in gray in FIG. 6) have the same numerator and denominator. Specifically, log{p(X|y=0) / p(X|y=0)} in the first row from the top and first column from the left, log{p(X|y=1) / p(X|y=1)} in the second row from the top and second column from the left, and log{p(X|y=2) / p(X|y=2)} in the third row from the top and third column from the left each have the same numerator and denominator. Furthermore, the likelihood ratios on opposite sides of the diagonal likelihood ratios have reversed numerators and denominators. Specifically, the numerator and denominator are reversed between log{p(X|y=0) / p(X|y=1)} in the first row from the top, second column from the left and log{p(X|y=1) / p(X|y=0)} in the second row from the top, first column from the left. Similarly, the numerator and denominator are reversed between log{p(X|y=0) / p(X|y=2)} in the first row from the top, third column from the left and log{p(X|y=2) / p(X|y=0)} in the third row from the top, first column from the left. The numerator and denominator are reversed between log{p(X|y=1) / p(X|y=2)} in the second row from the top, third column from the left and log{p(X|y=2) / p(X|y=1)} in the third row from the top, second column from the left. Therefore, likelihood ratios at opposite positions across the diagonal line have values with opposite signs. In this way, the likelihood ratios represented by the matrix are arranged like a skew matrix.

[0047] In particular, all likelihood ratios on the diagonal where the denominator and numerator are the same are log1, and their value is zero. Therefore, likelihood ratios on the diagonal where the denominator and numerator are the same are essentially meaningless even if they are taken into account in the loss function. Therefore, likelihood ratios on the diagonal where the denominator and numerator are the same are not taken into account in the loss function. The number of likelihood ratios remaining after excluding the likelihood ratios on the diagonal is N×(N−1), where N is the number of classes. In this embodiment, the likelihood ratios of these N×(N−1) patterns (i.e., likelihood ratios excluding the likelihood ratios on the diagonal in the matrix) are taken into account in the loss function. Note that specific examples of loss functions that take into account likelihood ratios of N×(N−1) patterns will be described in detail in other embodiments described later.

[0048] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the second embodiment will be described.

[0049] As described with reference to Figures 5 and 6, the information processing system 1 according to the second embodiment performs learning using a loss function that takes into account the likelihood ratio of an N × (N-1) pattern, where the denominator is the likelihood indicating the likelihood of belonging to one class and the numerator is the likelihood indicating the likelihood of belonging to another class. Using such a loss function, as in the first embodiment, learning can be performed such that the penalty for an incorrect class is large and the penalty for a correct class is small. As a result, it becomes possible to appropriately select at least one class to which sequential data belongs from among multiple candidate classes.

[0050] <Third embodiment> An information processing system 1 according to the third embodiment will be described with reference to Fig. 7. Note that the third embodiment differs from the first and second embodiments described above only in some of its operations, and other parts may be similar to the first and second embodiments. Therefore, the following will describe in detail only the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0051] (Learning procedure) First, the flow of operations of the learning unit 300 in the information processing system 1 according to the third embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of operations of the information processing system according to the third embodiment. Note that in Fig. 7, the same processes as those shown in Fig. 4 are denoted by the same reference numerals.

[0052] As shown in FIG. 7, when the operation of the information processing system 1 according to the third embodiment is started, training data is first input to the learning unit 300 (step S101).

[0053] Next, the learning unit 300 calculates a loss function using the input training data. In particular, in the third embodiment, the loss function takes into account a portion of the likelihood ratios of N×(N−1) patterns, where the denominator is the likelihood indicating the likelihood of belonging to one of N classes that are classification candidates for the sequential data, and the numerator is the likelihood indicating the likelihood of belonging to another class (step S301). That is, the learning unit 300 in the third embodiment does not take into account all of the likelihood ratios of the N×(N−1) patterns described in the second embodiment, but only a portion of them. Note that, like the first embodiment, this loss function is a function in which the likelihood ratio increases when the correct class to which the sequential data belongs is in the numerator of the likelihood ratio and decreases when the correct class is in the denominator of the likelihood ratio. Specific examples of loss functions that take into account a portion of the likelihood ratios of the N×(N−1) patterns will be described in detail in other embodiments below.

[0054] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). After that, the learning unit 300 determines whether all learning has been completed (step S104). If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0055] (Example of selecting likelihood ratios to consider) Next, a specific example of selecting likelihood ratios to be taken into account in the loss function (that is, an example of selecting some likelihood ratios of N×(N−1) patterns) will be described.

[0056] Of the N×(N−1) patterns of likelihood ratios, some of the likelihood ratios to be considered in the loss function may be selected in advance by a user or the like, or may be automatically selected by the learning unit 300. When the learning unit 300 selects some of the likelihood ratios to be considered in the loss function, the learning unit 300 may select the likelihood ratios according to a predetermined rule. Alternatively, the learning unit 300 may determine whether to make a selection based on the calculated likelihood ratio values.

[0057] An example of selecting some of the likelihood ratios to be considered in the loss function is to select only the likelihood ratios in one row or one column in the matrix shown in Fig. 6. For example, as the likelihood ratios to be considered in the loss function, only the likelihood ratios in the first row, only the likelihood ratios in the second row, or only the likelihood ratios in the third row of the matrix shown in Fig. 6 may be selected. Alternatively, only the likelihood ratios in the first column, only the likelihood ratios in the second column, or only the likelihood ratios in the third column of the matrix may be selected.

[0058] Also, only the likelihood ratios of some rows or some columns in the matrix may be selected. Specifically, only the likelihood ratios of the first and second rows of the matrix may be selected, only the likelihood ratios of the second and third rows may be selected, or only the likelihood ratios of the third and first rows may be selected. Alternatively, only the likelihood ratios of the first and second columns of the matrix may be selected, only the likelihood ratios of the second and third columns may be selected, or only the likelihood ratios of the third and first columns may be selected.

[0059] The above-described example of selecting likelihood ratios is merely an example, and other likelihood ratios may be selected as likelihood ratios to be taken into account in the loss function. For example, the likelihood ratios to be taken into account in the loss function may be selected randomly regardless of rows or columns.

[0060] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the third embodiment will be described.

[0061] As described with reference to FIG. 7 , the information processing system 1 according to the third embodiment performs learning using a loss function that takes into account some likelihood ratios among N×(N−1) patterns, where the denominator is the likelihood indicating the likelihood of belonging to one class and the numerator is the likelihood indicating the likelihood of belonging to another class. Using such a loss function, as in the above-described embodiments, learning can be performed such that the penalty for an incorrect class is large and the penalty for a correct class is small. As a result, it becomes possible to appropriately select at least one class to which sequential data belongs from among multiple candidate classes. Furthermore, particularly in the third embodiment, by appropriately selecting likelihood ratios to be considered in the loss function from among N×(N−1) patterns, learning can be performed more efficiently than when all likelihood ratios of N×(N−1) patterns are considered. For example, by selecting only likelihood ratios that have a relatively large effect on the loss function and avoiding selecting likelihood ratios that have a relatively small effect on the loss function, learning efficiency can be improved.

[0062] <Fourth embodiment> An information processing system 1 according to the fourth embodiment will be described with reference to Fig. 8 and Fig. 9. The fourth embodiment describes a specific selection example of the third embodiment described above (i.e., a selection example of some likelihood ratios to be taken into account in the loss function), and other parts may be the same as those of the third embodiment. Therefore, the following will describe in detail parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0063] (Learning procedure) First, the flow of operations of the learning unit 300 in the information processing system 1 according to the fourth embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of operations of the information processing system according to the fourth embodiment. Note that in Fig. 8, the same processes as those shown in Fig. 4 are denoted by the same reference numerals.

[0064] As shown in FIG. 8, when the operation of the information processing system 1 according to the fourth embodiment starts, training data is first input to the learning unit 300 (step S101).

[0065] Next, the learning unit 300 calculates a loss function using the input training data. In the fourth embodiment, the learning unit 300 particularly calculates a loss function that takes into account the likelihood ratios for the N×(N−1) patterns already described, the likelihood ratios for which the correct class is in the numerator, among the likelihood ratios for the N×(N−1) patterns described in the third embodiment (step S401). That is, the learning unit 300 according to the fourth embodiment selects likelihood ratios for which the correct class is in the numerator, as some of the likelihood ratios for the N×(N−1) patterns described in the third embodiment. Note that, like the first embodiment, this loss function is a function that increases the likelihood ratio when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and decreases when the correct class is in the denominator of the likelihood ratio. Specific examples of loss functions that take into account the likelihood ratios for which the correct class is in the numerator will be described in detail in other embodiments below.

[0066] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). After that, the learning unit 300 determines whether all learning has been completed (step S104). If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0067] (Examples of likelihood ratios to consider) Next, the likelihood ratios taken into account in the learning operation by the learning unit 300 (i.e., the likelihood ratios taken into account in calculating the loss function) will be specifically described with reference to Fig. 9. Fig. 9 is a matrix diagram showing an example of likelihood ratios taken into account by the learning unit in the information processing system according to the fourth embodiment.

[0068] As already explained in the second embodiment (see FIG. 6), the likelihood ratios are arranged in a skew matrix in the matrix shown in Fig. 9. The learning unit 300 according to the fourth embodiment selects the likelihood ratios with the correct class in the numerator from the N × (N-1) patterns of likelihood ratios excluding the likelihood ratios on the diagonal of such a matrix, and takes these into account in the loss function.

[0069] For example, suppose the correct class of the sequence data input as training data is "Class 1." In this case, the learning unit 300 selects likelihood ratios with Class 1 as the numerator from the N×(N-1) patterns of likelihood ratios and considers them in the loss function. Specifically, only the likelihood ratios in the second row from the top of FIG. 9 (excluding the likelihood ratios on the diagonal) are selected and considered in the loss function. In this case, the loss function considers log{p(X|y=1) / p(X|y=0)} in the second row from the top and first column from the left and log{p(X|y=1) / p(X|y=2)} in the second row from the top and third column from the left. In other words, the loss function considers likelihood ratios not shaded in gray in FIG. 9.

[0070] If the correct class of the sequence data input as training data is "Class 0," the learning unit 300 selects likelihood ratios with Class 0 as the numerator from the N×(N-1) patterns of likelihood ratios and considers them in the loss function. Specifically, only the likelihood ratios in the first row from the top of FIG. 9 (excluding the likelihood ratios on the diagonal) are selected and considered in the loss function. In this case, the loss function considers log{p(X|y=0) / p(X|y=1)} in the first row from the top and second column from the left and log{p(X|y=0) / p(X|y=2)} in the first row from the top and third column from the left.

[0071] Similarly, if the correct class of the sequence data input as training data is "Class 2," the learning unit 300 selects likelihood ratios with Class 2 as the numerator from the N×(N-1) patterns of likelihood ratios and considers them in the loss function. Specifically, only the likelihood ratios in the third row from the top of FIG. 9 (excluding the likelihood ratios on the diagonal) are selected and considered in the loss function. In this case, the loss function considers log{p(X|y=2) / p(X|y=0)} in the third row from the top and first column from the left and log{p(X|y=2) / p(X|y=1)} in the third row from the top and second column from the left.

[0072] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the fourth embodiment will be described.

[0073] As described with reference to FIGS. 8 and 9, the information processing system 1 according to the fourth embodiment performs learning using a loss function that takes into account the likelihood ratios of N×(N−1) patterns in which the correct class is in the numerator. Using such a loss function, as in the above-described embodiments, allows for appropriate learning, making it possible to appropriately select at least one class to which sequential data belongs from among multiple candidate classes. Furthermore, in the fourth embodiment, the loss function particularly takes into account the likelihood ratios in which the correct class is in the numerator (in other words, likelihood ratios that may have a significant impact on the loss function), allowing for more efficient learning than when all likelihood ratios of N×(N−1) patterns are taken into account.

[0074] Fifth Embodiment An information processing system 1 according to the fifth embodiment will be described with reference to Fig. 10. The fifth embodiment describes a specific example of the loss function used in the first to fourth embodiments described above, and the device configuration and operation flow may be similar to those of the first to fourth embodiments. Therefore, the following will describe in detail the parts that are different from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0075] (Loss functions including sigmoid functions) First, an overview of the loss function used in the information processing system 1 according to the fifth embodiment will be described with reference to Fig. 10. Fig. 10 is a graph showing an example of a sigmoid function used in the information processing system according to the fifth embodiment.

[0076] 10, in the information processing system 1 according to the fifth embodiment, the learning unit 300 performs learning using a loss function including a sigmoid function. More specifically, the learning unit 300 performs learning using a loss function including a sigmoid function as a nonlinear function acting on the likelihood ratio. The following description will be given with reference to a specific example of a loss function including a sigmoid function.

[0077] (Example considering all likelihood ratios) As a first example, a loss function will be described in which all likelihood ratios of N×(N−1) patterns are considered, as in the second embodiment. Note that the likelihood ratios in the following equations are assumed to be log likelihood ratios (LLRs).

[0078] An example of a loss function that takes into account all likelihood ratios of N×(N−1) patterns is the following equation (1).

[0079]

number

[0080] In the above formula (1), K is the number of classes, M is the number of data, and T is the time series length. Furthermore, k is a subscript in the row direction, and l is a subscript in the column direction (i.e., the subscript indicates the row number and column number in the matrix shown in Figure 6, etc.). δ is Kronecker's delta, which is "1" when the subscripts match, and "0" otherwise. λ is a likelihood ratio, and in the above formula (1), it represents the log-likelihood ratio of the kth row and lth column at time t. σ is a sigmoid function, and is included as a nonlinear function that acts on the likelihood ratio λ.

[0081] In equation (1), 1 / MT is the average across all data and time series. 1 / 2K is the product of 1 / K, which is the average of K rows, and 1 / 2, which is the average of a skew matrix. 1 / (K-1) is the average of (K-1) columns, which is the Kth column minus one diagonal column.

[0082] In the loss function L of equation (1), if row k is the correct class, Kronecker's δ is "1," and if row k is the incorrect class (i.e., a class other than the correct class), Kronecker's δ is "0." As a result, if row k is the correct class (in other words, if the correct class is in the numerator of the likelihood ratio), the value of the likelihood ratio will be large. On the other hand, if row k is the incorrect class (in other words, if the incorrect class is in the numerator of the likelihood ratio), the value of the likelihood ratio will be small.

[0083] (Example considering some likelihood ratios) Next, as a second example, a loss function will be described in which some likelihood ratios of N×(N−1) patterns are considered, as in the third and fourth embodiments. In particular, the following description will be given of an example in which only likelihood ratios of N×(N−1) patterns in which the correct class is in the numerator are considered, as in the fourth embodiment.

[0084] An example of a loss function that considers only the likelihood ratio of the correct class in the numerator among N×(N−1) patterns is the following formula (2).

[0085]

number

[0086] In the above formula (2), K is the number of classes, M is the number of data, and T is the time series length. Furthermore, k is a subscript in the row direction, and l is a subscript in the column direction (i.e., the subscript indicates the row number and column number in the matrix shown in FIG. 6, etc.). λ is a likelihood ratio, and in the above formula (1), it represents the log-likelihood ratio of the kth row and lth column at time t. σ is a sigmoid function, and is included as a nonlinear function acting on the likelihood ratio λ.

[0087] Since formula (2) only considers the rows of the correct class, it does not include the steps of summing the K rows and averaging the K rows and skew matrix by 1 / 2K, as compared to formula (1) already explained. Also, the Kronecker δ part in formula (1) is "1".

[0088] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the fifth embodiment will be described.

[0089] As described in Fig. 10, in the information processing system 1 according to the fifth embodiment, the learning unit 300 uses a loss function including a sigmoid function σ. By using such a loss function, it becomes possible to appropriately perform learning related to the calculation of likelihood ratios, as described in the first to fourth embodiments. Specifically, learning can be performed such that the penalty becomes large when the class is incorrect and the penalty becomes small when the class is correct.

[0090] The loss functions given in the fifth embodiment (i.e., formulas (1) and (2)) are merely examples, and a different loss function may be created using a sigmoid function. Also, instead of the sigmoid function, a loss function may be created using another nonlinear function. For example, as in the embodiments described below, a loss function including a logistic function may be used.

[0091] Sixth Embodiment An information processing system 1 according to the sixth embodiment will be described with reference to Fig. 11. Similar to the fifth embodiment, the sixth embodiment describes a specific example of the loss function used in the first to fourth embodiments, and the device configuration and operation flow may be the same as those of the first to fourth embodiments. Therefore, the following will describe in detail the parts that are different from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0092] (Loss functions including logistic functions) First, an overview of the loss function used in the information processing system 1 according to the sixth embodiment will be described with reference to Fig. 11. Fig. 11 is a graph showing an example of a logistic function used in the information processing system according to the sixth embodiment.

[0093] 11, in the information processing system 1 according to the sixth embodiment, the learning unit 300 performs learning using a loss function including a logistic function. More specifically, the learning unit 300 performs learning using a loss function including the logistic function as a nonlinear function acting on the likelihood ratio. The following description will be given with reference to specific examples of loss functions including the logistic function.

[0094] (Example considering all likelihood ratios) As a first example, a loss function will be described in which all likelihood ratios of N×(N−1) patterns are considered, as in the second embodiment. Note that the likelihood ratios in the following equations are assumed to be log likelihood ratios (LLRs).

[0095] An example of a loss function that takes into account all likelihood ratios of N×(N−1) patterns is the following equation (3).

[0096]

number

[0097] In the above formula (3), K is the number of classes, M is the number of data, and T is the time series length. Furthermore, k is the row-direction subscript, and l is the column-direction subscript (i.e., the subscript indicating the row number and column number in the matrix shown in Figure 6, etc.). δ is Kronecker's delta, which is "1" when the subscripts match, and "0" otherwise. λ is the likelihood ratio, and in the above formula (3), it represents the log-likelihood ratio of the kth row and lth column at time t. logistic is a logistic function, and is included as a nonlinear function that acts on the likelihood ratio λ.

[0098] In equation (1), 1 / MT is the average across all data and time series. 1 / 2K is the product of 1 / K, which is the average of K rows, and 1 / 2, which is the average of a skew matrix. 1 / (K-1) is the average of (K-1) columns, which is the Kth column minus one diagonal column.

[0099] In the loss function L of equation (3), if row k is the correct class, Kronecker's δ will be "1", and if row k is the incorrect class (i.e., a class other than the correct class), Kronecker's δ will be "0". As a result, if row k is the correct class (in other words, if the correct class is in the numerator of the likelihood ratio), of the two terms containing Kronecker's δ, the first term will remain and the latter term will be zero. On the other hand, if row k is the incorrect class (in other words, if the incorrect class is in the numerator of the likelihood ratio), of the two terms containing Kronecker's δ, the first term will be zero and the latter term will remain.

[0100] (Example considering some likelihood ratios) Next, as a second example, a loss function will be described in which some likelihood ratios of N×(N−1) patterns are considered, as in the third and fourth embodiments. In particular, the following description will be given of an example in which only likelihood ratios of N×(N−1) patterns in which the correct class is in the numerator are considered, as in the fourth embodiment.

[0101] An example of a loss function that considers only the likelihood ratio of the correct class in the numerator among N×(N−1) patterns is the following formula (4).

[0102]

number

[0103] In the above formula (4), K is the number of classes, M is the number of data, and T is the time series length. Furthermore, k is a subscript in the row direction, and l is a subscript in the column direction (i.e., the subscript indicates the row number and column number in the matrix shown in FIG. 6, etc.). λ is a likelihood ratio, and in the above formula (1), it represents the log-likelihood ratio of the kth row and lth column at time t. logistic is a logistic function, and is included as a nonlinear function that acts on the likelihood ratio λ.

[0104] Since equation (4) only considers the rows of the correct class, it does not include the steps of summing the K rows and averaging the K rows and skew matrix by 1 / 2K, as compared to equation (3) already explained. Also, of the two terms containing Kronecker's δ in equation (3), only the former term remains.

[0105] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the sixth embodiment will be described.

[0106] As described in FIG. 11 , in the information processing system 1 according to the sixth embodiment, the learning unit 300 uses a loss function including a logistic function. Using such a loss function enables appropriate learning related to likelihood ratio calculation, as described in the first to fourth embodiments. Specifically, learning can be performed such that the penalty for an incorrect class is large and the penalty for a correct class is small. Furthermore, while the sigmoid function σ used in the fifth embodiment varies so that the slope approaches zero (see FIG. 10 ), the logistic function varies while maintaining a constant slope (see FIG. 11 ). Therefore, when a loss function including a logistic function is used, components do not disappear in the process of calculating the slope of the loss function (i.e., the process equivalent to differentiation), enabling more appropriate learning.

[0107] The loss functions given in the sixth embodiment (i.e., formulas (3) and (4)) are merely examples, and a different loss function may be created using a logistic function. Also, instead of the logistic function, a loss function may be created using another nonlinear function. For example, a loss function including a function different from either the sigmoid function or the logistic function described above may be used.

[0108] Seventh Embodiment An information processing system 1 according to the seventh embodiment will be described with reference to Fig. 12 and Fig. 13. The third embodiment differs from the first to sixth embodiments described above only in part of its configuration and operation (specifically, the configuration and operation of the classification device 10), and other parts may be the same as the first to sixth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0109] (Functional configuration) First, the functional configuration of the information processing system 1 according to the seventh embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram showing the functional configuration of the information processing system according to the seventh embodiment. In Fig. 12, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0110] 12, in the information processing system 1 according to the seventh embodiment, the likelihood ratio calculation unit 100 in the classification device 10 includes a first calculation unit 110 and a second calculation unit 120. Note that each of the first calculation unit 110 and the second calculation unit 120 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0111] The first calculation unit 110 is configured to be able to calculate an individual likelihood ratio based on two consecutive elements included in the sequence data. The individual likelihood ratio is calculated as a likelihood ratio indicating the likelihood of a class to which the two consecutive elements belong. The first calculation unit 110 may, for example, sequentially acquire elements included in the sequence data from the data acquisition unit 50 and sequentially calculate individual likelihood ratios based on the two consecutive elements. The individual likelihood ratios calculated by the first calculation unit 110 are configured to be output to the second calculation unit 120.

[0112] The second calculation unit 120 is configured to be able to calculate an integrated likelihood ratio based on the multiple individual likelihood ratios calculated by the first calculation unit 110. The integrated likelihood ratio is calculated as a likelihood ratio indicating the likelihood of a class to which multiple elements considered in each of the multiple individual likelihood ratios belong. In other words, the integrated likelihood ratio is calculated as a likelihood ratio indicating the likelihood of a class to which sequence data consisting of multiple elements belongs. The integrated likelihood ratio calculated by the second calculation unit 120 is output to the classification unit 200. The classification unit 200 classifies the sequence data based on the integrated likelihood ratio.

[0113] The learning unit 300 according to the fifth embodiment may perform learning on the likelihood ratio calculation unit 100 as a whole (i.e., the first calculation unit 110 and the second calculation unit 120 together), or may perform learning separately on the first calculation unit 110 and the second calculation unit 120. Alternatively, the learning unit 300 may be provided separately as a first learning unit that performs learning only on the first calculation unit 110, and a second learning unit that performs learning only on the second calculation unit 120. In this case, only one of the first learning unit and the second learning unit may be provided.

[0114] (Classification operation flow) Next, the flow of operation of the classification device 10 in the information processing system 1 according to the seventh embodiment (specifically, the class classification operation after learning) will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the flow of operation of the classification device in the information processing system according to the seventh embodiment.

[0115] 13, when the operation of the classification device 10 starts, the data acquiring unit 50 first acquires elements included in the sequence data (step S21). The data acquiring unit 50 outputs the acquired elements of the sequence data to the first calculation unit 110.

[0116] Then, the first calculation unit 110 calculates an individual likelihood ratio based on the acquired two consecutive elements (step S22). After that, the second calculation unit 120 calculates an integrated likelihood ratio based on the multiple individual likelihood ratios calculated by the first calculation unit 110 (step S23).

[0117] Next, the classification unit 200 performs classification based on the calculated integrated likelihood ratio (step S24). Classification may determine one class to which the sequence data belongs, or may determine multiple classes to which the sequence data is likely to belong. The classification unit 200 may output the classification result to a display or the like. Alternatively, the classification unit 200 may output the classification result as sound via a speaker or the like.

[0118] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the seventh embodiment will be described.

[0119] As described with reference to FIGS. 12 and 13 , in the information processing system 1 according to the seventh embodiment, individual likelihood ratios are first calculated based on two elements, and then an integrated likelihood ratio is calculated based on multiple individual likelihood ratios. Using the integrated likelihood ratio calculated in this manner makes it possible to appropriately select the class to which the sequence data belongs. Furthermore, even in the classification device 10 that calculates the individual likelihood ratios and the integrated likelihood ratio, more appropriate class classification can be achieved by performing learning using the learning unit 300 described in each of the above-described embodiments. That is, more appropriate classification can be achieved by performing learning such that the penalty for an incorrect class is large and the penalty for a correct class is small.

[0120] Eighth Embodiment An information processing system 1 according to the eighth embodiment will be described with reference to Fig. 14 and Fig. 15. The eighth embodiment differs from the seventh embodiment described above only in part of its configuration and operation (specifically, the configuration and operation of the likelihood ratio calculation unit 100), and other parts may be the same as those of the seventh embodiment. Therefore, the following will describe in detail only the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0121] (Functional configuration) First, the functional configuration of an information processing system 1 according to the eighth embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing the functional configuration of the information processing system according to the eighth embodiment. Note that in Fig. 14, the same elements as those shown in Figs. 2 and 12 are denoted by the same reference numerals.

[0122] As shown in FIG. 14 , in the information processing system 1 according to the eighth embodiment, the likelihood ratio calculation unit 100 in the classification device 10 includes a first calculation unit 110 and a second calculation unit 120. The first calculation unit 110 includes an individual likelihood ratio calculation unit 111 and a first storage unit. The second calculation unit 120 includes an integrated likelihood ratio calculation unit 121 and a second storage unit 122. Note that each of the individual likelihood ratio calculation unit 111 and the integrated likelihood ratio calculation unit 121 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1 ). Also, each of the first storage unit 112 and the second storage unit 122 may be realized by, for example, the above-mentioned storage device 14 (see FIG. 1 ).

[0123] The individual likelihood ratio calculation unit 111 is configured to be able to calculate an individual likelihood ratio based on two consecutive elements among the elements sequentially acquired by the data acquisition unit 50. More specifically, the individual likelihood ratio calculation unit 111 calculates an individual likelihood ratio based on a newly acquired element and past data stored in the first storage unit 112. The information stored in the first storage unit 112 is configured to be readable by the individual likelihood ratio calculation unit 111. When the first storage unit 112 stores past individual likelihood ratios, the individual likelihood ratio calculation unit 111 may read the stored past individual likelihood ratios and calculate new individual likelihood ratios taking the acquired elements into consideration. On the other hand, when the first storage unit 112 stores the elements acquired in the past, the individual likelihood ratio calculation unit 111 may calculate the past individual likelihood ratios from the stored past elements and calculate the likelihood ratio for the newly acquired element.

[0124] The integrated likelihood ratio calculation unit 121 is configured to be able to calculate an integrated likelihood ratio based on a plurality of individual likelihood ratios. The integrated likelihood ratio calculation unit 121 calculates a new integrated likelihood ratio using the individual likelihood ratios calculated by the individual likelihood ratio calculation unit 111 and past integrated likelihood ratios stored in the second storage unit 122. The information stored in the second storage unit 122 (i.e., past integrated likelihood ratios) is configured to be readable by the integrated likelihood ratio calculation unit 121.

[0125] <Flow of likelihood ratio calculation> Next, the flow of the likelihood ratio calculation operation (i.e., the operation of the likelihood ratio calculation unit 100) in the information processing system 1 according to the eighth embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing the flow of the operation of the likelihood ratio calculation unit in the information processing system according to the eighth embodiment.

[0126] 15, when the likelihood ratio calculation operation by the likelihood ratio calculation unit 100 is started, first, the individual likelihood ratio calculation unit 111 in the first calculation unit 110 reads out past data from the first storage unit 112 (step S31). The past data may be, for example, the processing result by the individual likelihood ratio calculation unit 111 of the element acquired immediately before the element currently acquired by the data acquisition unit 50 (in other words, the individual likelihood ratio calculated for the immediately previous element). Alternatively, the past data may be the element itself acquired immediately before the element currently acquired.

[0127] Subsequently, the individual likelihood ratio calculation unit 111 calculates a new individual likelihood ratio (i.e., an individual likelihood ratio for the element currently acquired by the data acquisition unit 50) based on the element acquired by the data acquisition unit 50 and the past data read out from the first storage unit 112 (step S32). The individual likelihood ratio calculation unit 111 outputs the calculated individual likelihood ratio to the second calculation unit 120. The individual likelihood ratio calculation unit 111 may store the calculated individual likelihood ratio in the first storage unit 112.

[0128] Subsequently, the integrated likelihood ratio calculation unit 121 in the second calculation unit 120 reads out a past integrated likelihood ratio from the second storage unit 122 (step S33). The past integrated likelihood ratio may be, for example, a processing result by the integrated likelihood ratio calculation unit 121 for an element acquired immediately before the element currently acquired by the data acquisition unit 50 (in other words, an integrated likelihood ratio calculated for the immediately previous element).

[0129] Next, the integrated likelihood ratio calculation unit 121 calculates a new integrated likelihood ratio (i.e., an integrated likelihood ratio for the element currently acquired by the data acquisition unit 50) based on the likelihood ratio calculated by the individual likelihood ratio calculation unit 111 and the past integrated likelihood ratio read out from the second storage unit 122 (step S34). The integrated likelihood ratio calculation unit 121 outputs the calculated integrated likelihood ratio to the classifying unit 200. The integrated likelihood ratio calculation unit 121 may store the calculated integrated likelihood ratio in the second storage unit 122.

[0130] (Technical Effects) Next, the technical effects obtained by the information processing system 1 according to the eighth embodiment will be described.

[0131] As described with reference to FIGS. 14 and 15 , in the information processing system 1 according to the eighth embodiment, individual likelihood ratios are calculated using past individual likelihood ratios, and then an integrated likelihood ratio is calculated using past integrated likelihood ratios. Using the integrated likelihood ratio calculated in this manner makes it possible to appropriately select the class to which sequential data belongs. Furthermore, even in the classification device 10 that calculates individual likelihood ratios and integrated likelihood ratios using past data, more appropriate class classification can be achieved by performing learning using the learning unit 300 described in each of the above-described embodiments. That is, more appropriate classification can be achieved by performing learning such that the penalty for an incorrect class is large and the penalty for a correct class is small.

[0132] Ninth Embodiment An information processing system 1 according to the ninth embodiment will be described with reference to Fig. 16. The ninth embodiment differs from the first to seventh embodiments described above only in some operations (specifically, the operation of the classifying unit 200), and other parts may be the same as the first to seventh embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0133] (Classification operation flow) First, the flow of operation of the classification device 10 in the information processing system 1 according to the ninth embodiment (specifically, the class classification operation after learning) will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of operation of the classification device in the information processing system according to the ninth embodiment. Note that in Fig. 16, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0134] 16, when the operation of the classification device 10 starts, the data acquiring unit 50 first acquires elements included in the sequence data (step S11). The data acquiring unit 50 outputs the acquired elements of the sequence data to the likelihood ratio calculating unit 100. Then, the likelihood ratio calculating unit 100 calculates a likelihood ratio based on the two or more acquired elements (step S12).

[0135] Next, the classification unit 200 performs classification based on the calculated likelihood ratio. In particular, in the ninth embodiment, the classification unit 200 selects and outputs multiple classes to which the sequential data likely belongs (step S41). That is, the classification unit 200 does not determine a single class to which the sequential data likely belongs, but determines multiple classes to which the sequential data likely belongs. More specifically, the classification unit 200 executes a process of selecting k classes (where k is a natural number equal to or less than n) from n classes (where n is a natural number) prepared as classification candidates.

[0136] The classification unit 200 may output information about the k classes to which the sequence data may belong to on a display or the like. Alternatively, the classification unit 200 may output information about the k classes to which the sequence data may belong as audio via a speaker or the like.

[0137] When outputting information about k classes to which sequential data may possibly belong, the classifying unit 200 may rearrange the information and output it. For example, the classifying unit 200 may rearrange the information about the k classes in descending order of likelihood ratio and output it. Alternatively, the classifying unit 200 may output information about the k classes in a different manner for each class. For example, the classifying unit 200 may output information about classes with high likelihood ratios in a display manner that highlights classes, while outputting information about classes with low likelihood ratios in a display manner that does not highlight classes. When highlighting, for example, the display size or color may be changed, or the displayed object may be animated.

[0138] (Specific examples of use) The configuration for outputting k classes from among n classes described above will be described below by giving some specific application examples.

[0139] (Product proposal) The information processing system 1 according to the ninth embodiment may be used on a web shopping site to suggest products that a user may be interested in. Specifically, the information processing system 1 may select k products (i.e., k classes) that the user may be interested in from n products (i.e., n classes) that are available, and output the selected products to the user (note that k is a number smaller than n). In this case, examples of the input sequence data include past purchase history and browsing history.

[0140] Similarly, it may be used to suggest products and stores on digital signage, etc. Digital signage may be equipped with a camera capable of capturing an image of a user. In this case, the user's emotions may be estimated from the user's image, and stores and products may be suggested according to the emotions. Furthermore, the user's line of sight may be estimated from the user's image (i.e., the part the user is looking at) and stores and products that the user may be interested in may be suggested. Alternatively, the user's attributes (e.g., gender, age, etc.) may be estimated from the user's image, and stores and products that the user may be interested in may be suggested. When estimating information about a user as described above, weighting may be assigned to the n classes according to the estimated information.

[0141] (Criminal Investigation) The information processing system 1 according to the ninth embodiment can also be used in criminal investigations. For example, when trying to find the true culprit from multiple suspects, selecting only the one most likely to be the culprit can cause serious problems if the selection is incorrect. However, the information processing system 1 according to this embodiment can select and output the top K suspects most likely to be the culprit. Specifically, it is sufficient to select and output classes corresponding to the top K suspects most likely to be the culprit from sequence data containing information about each of the multiple suspects as elements. In this way, it is possible to appropriately find the true culprit by investigating, for example, multiple suspects who are likely to be the culprit.

[0142] (radar image analysis) The information processing system 1 according to the ninth embodiment can also be applied to the analysis of radar images. Radar images, by their nature, often have low clarity, making it difficult to accurately determine, for example, what is captured in the image using only a machine. However, the information processing system 1 according to this embodiment can select and output k candidates that are likely to be captured in the radar image. Therefore, it is possible to first output k candidates and have the user make a judgment from among them. For example, if "dog," "cat," "ship," and "tank" are presented as candidates that may be captured in a radar image of a port, the user can easily determine that a "ship," which is highly related to the port, is captured in the radar image.

[0143] The above-mentioned application example is just one example, and in a situation where it is required to select k items from n candidates, applying the information processing system 1 according to this embodiment can produce beneficial effects.

[0144] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0145] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. The scope of each embodiment is not limited to programs recorded on the recording media that execute processing by themselves, but also includes programs that run on an OS in cooperation with other software and functions of an expansion board to execute processing.

[0146] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of the invention that can be read from the claims and the entire specification, and information processing systems, information processing methods, and computer programs that involve such modifications are also included in the technical idea of this disclosure.

[0147] <Additional Notes> The above-described embodiment may be further described as follows, but is not limited to the following.

[0148] (Appendix 1) The information processing system described in Supplementary Note 1 comprises: an acquisition means for acquiring multiple elements included in sequential data; a calculation means for calculating a likelihood ratio indicating the likelihood of a class to which the sequential data belongs based on at least two consecutive elements among the multiple elements; a classification means for classifying the sequential data into at least one class among multiple candidate classes based on the likelihood ratio; and a learning means for performing learning related to the calculation of the likelihood ratio using a loss function such that the likelihood ratio is large when the true class to which the sequential data belongs is in the numerator of the likelihood ratio and is small when the true class is in the denominator of the likelihood ratio.

[0149] (Appendix 2) The information processing system described in Supplementary Note 2 is the information processing system described in Supplementary Note 1, characterized in that the learning means performs the learning using a loss function that takes into account the likelihood ratio of N×(N-1) patterns, where the denominator is a likelihood indicating the likelihood of belonging to one class out of N classes (where N is a natural number) that are classification candidates for the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class.

[0150] (Appendix 3) The information processing system described in Appendix 3 is the information processing system described in Appendix 2, characterized in that the learning means performs the learning using a loss function that takes into account some of the likelihood ratios of the N×(N-1) patterns.

[0151] (Appendix 4) The information processing system described in Appendix 4 is the information processing system described in Appendix 3, characterized in that the learning means performs the learning using a loss function that takes into account the likelihood ratio of the N×(N-1) patterns in which the correct class is in the numerator.

[0152] (Appendix 5) The information processing system described in Supplementary Note 5 is the information processing system described in any one of Supplementary Notes 1 to 4, characterized in that the loss function includes a sigmoid function as a nonlinear function acting on the likelihood ratio.

[0153] (Appendix 6) The information processing system described in Supplementary Note 6 is the information processing system described in any one of Supplementary Notes 1 to 4, characterized in that the loss function includes a logistic function as a nonlinear function acting on the likelihood ratio.

[0154] (Appendix 7) The information processing system described in Supplementary Note 7 is the information processing system described in any one of Supplementary Notes 1 to 6, characterized in that the likelihood ratio is an integrated likelihood ratio calculated by taking into account multiple individual likelihood ratios calculated based on two consecutive elements included in the sequence data.

[0155] (Appendix 8) The information processing system described in Appendix 8 is the information processing system described in Appendix 7, characterized in that the acquisition means sequentially acquires multiple elements included in the sequence data, and the calculation means calculates a new integrated likelihood ratio using the individual likelihood ratio calculated based on the newly acquired elements and the integrated likelihood ratio calculated in the past.

[0156] (Appendix 9) The information processing method described in Supplementary Note 9 is an information processing method characterized by acquiring multiple elements included in sequential data, calculating a likelihood ratio indicating the likelihood of a class to which the sequential data belongs based on at least two consecutive elements among the multiple elements, classifying the sequential data into at least one class among multiple candidate classes based on the likelihood ratio, and performing learning related to the calculation of the likelihood ratio using a loss function such that the likelihood ratio is large when the correct class to which the sequential data belongs is in the numerator of the likelihood ratio and is small when the correct class is in the denominator of the likelihood ratio.

[0157] (Appendix 10) The computer program described in Supplementary Note 10 is a computer program that operates a computer to acquire multiple elements included in sequential data, calculate a likelihood ratio indicating the likelihood of a class to which the sequential data belongs based on at least two consecutive elements among the multiple elements, classify the sequential data into at least one class among multiple candidate classes based on the likelihood ratio, and perform training related to the calculation of the likelihood ratio using a loss function that increases the likelihood ratio when the correct class to which the sequential data belongs is in the numerator of the likelihood ratio and decreases when the correct class is in the denominator of the likelihood ratio.

[0158] (Appendix 11) The recording medium described in Supplementary Note 11 is a recording medium having the computer program described in Supplementary Note 10 recorded thereon. [Explanation of symbols]

[0159] 1. Information Processing Systems 11 processors 14 Storage device 10 Classification device 50 Data Acquisition Section 100 Likelihood ratio calculation unit 110 First Calculation Unit 111 Individual likelihood ratio calculation unit 112 1st memory section 120 Second Calculation Unit 121 Integrated likelihood ratio calculation unit 122 2nd memory section 200 Classification Department 300 Learning Department

Claims

1. an acquisition means for acquiring a plurality of elements included in the sequence data; a calculation means for calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs, based on at least two consecutive elements among the plurality of elements; a classification means for classifying the sequence data into at least one class out of three or more candidate classes based on the likelihood ratio; a learning means for adjusting parameters of a model for calculating the likelihood ratio so that the value of a loss function calculated using the likelihood ratio becomes small; Equipped with The loss function does not take into account the case where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes small when the correct class to which the sequence data belongs is in the numerator but not in the denominator of the likelihood ratio, or when the correct class to which the sequence data belongs is in the denominator but not in the numerator of the likelihood ratio. An information processing system comprising:

2. 2. The information processing system according to claim 1, wherein the loss function used by the learning means is a loss function that takes into account all of the likelihood ratios of N × (N-1) patterns, where the denominator is a likelihood indicating the likelihood of belonging to one of N classes (where N is a natural number greater than or equal to 3) that are classification candidates for the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class.

3. 2. The information processing system according to claim 1, wherein the loss function used by the learning means is a loss function that takes into account some of the likelihood ratios of N × (N-1) patterns, where the denominator is a likelihood indicating the likelihood of belonging to one of N classes (where N is a natural number greater than or equal to 3) that are classification candidates for the sequence data, and the numerator is a likelihood indicating the likelihood of belonging to another class.

4. The information processing system according to claim 3, wherein the loss function used by the learning means is a loss function that takes into account the likelihood ratio of the correct class being in the numerator among the N×(N−1) patterns.

5. 5. The information processing system according to claim 1, wherein the loss function includes a sigmoid function as a nonlinear function that acts on the likelihood ratio.

6. 6. The information processing system according to claim 1, wherein the likelihood ratio is an integrated likelihood ratio calculated by taking into account multiple individual likelihood ratios calculated based on two consecutive elements included in the sequence data.

7. the acquiring means sequentially acquires a plurality of elements included in the sequence data; The calculation means calculates a new integrated likelihood ratio using the individual likelihood ratio calculated based on a newly acquired element and the integrated likelihood ratio calculated in the past.

7. The information processing system according to claim 6.

8. Obtain multiple elements contained in the sequence data, calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; classifying the sequence data into at least one class out of three or more candidate classes based on the likelihood ratio; adjusting parameters of a model for calculating the likelihood ratio so that the value of a loss function calculated using the likelihood ratio becomes small; The loss function does not take into account the case where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes small when the correct class to which the sequence data belongs is in the numerator but not in the denominator of the likelihood ratio, or when the correct class to which the sequence data belongs is in the denominator but not in the numerator of the likelihood ratio.

1. An information processing method comprising:

9. Obtain multiple elements contained in the sequence data, calculating a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; classifying the sequence data into at least one class out of three or more candidate classes based on the likelihood ratio; adjusting parameters of a model for calculating the likelihood ratio so that the value of a loss function calculated using the likelihood ratio becomes small; The loss function does not take into account the case where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes small when the correct class to which the sequence data belongs is in the numerator but not in the denominator of the likelihood ratio, or when the correct class to which the sequence data belongs is in the denominator but not in the numerator of the likelihood ratio. A computer program that causes a computer to operate in such a manner.

Images