Information processing system, information processing method, and computer program

Abstract

To appropriately adjust model parameters.SOLUTION: An information processing system comprises: acquisition means for acquiring a plurality of elements contained in sequential data; calculation means for calculating a likelihood ratio indicative of the plausibility of the class to which the sequential data belongs on the basis of at least two consecutive elements among the plurality of elements; classification means for classifying the sequential data into at least one class among three or more classes, which are classification candidates, on the basis of the likelihood ratio; and learning means for adjusting the parameters of a model for calculating the likelihood ratio so as to minimize the value of a loss function calculated using the likelihood ratio. The loss function does not take into account cases where the numerator and the denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes small in cases where the correct class to which the sequential data belongs is included in the numerator but not in the denominator of the likelihood ratio, or where it is included in the denominator but not in the numerator.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates to the technical field of information processing systems, information processing methods, and computer programs for processing information related to, for example, classification.

Background Art

[0002] As this type of system, those that perform learning processing related to likelihood are known. For example, in Patent Document 1, it is disclosed to learn a support vector machine used for likelihood determination. In Patent Document 2, it is disclosed that support vector machines or logistic regression may be used for learning an identification machine that utilizes likelihood.

[0003] As other related technologies, for example, in Patent Document 3, a technique for performing folder classification processing on an image file based on predetermined recognition conditions and dominant conditions in an apparatus for determining whether a person included in an image is a registered person is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] This disclosure aims to improve the related technologies described above.

Means for Solving the Problems

[0006] One aspect of the information processing system in this disclosure includes: acquisition means for acquiring a plurality of elements included in sequence data; calculation means for calculating a likelihood ratio indicating the likelihood of the class to which the sequence data belongs based on at least two consecutive elements among the plurality of elements; classification means for classifying the sequence data into at least one class from three or more classification candidate classes based on the likelihood ratio; and learning means for adjusting the parameters of a model for determining the likelihood ratio so that the value of the loss function calculated using the likelihood ratio becomes small, wherein the value of the loss function becomes small when the numerator and denominator of the likelihood ratio have the same likelihood, and when the correct class to which the sequence data belongs is in the numerator and not in the denominator of the likelihood ratio, and when the correct class to which the sequence data belongs is in the denominator and not in the numerator of the likelihood ratio.

[0007] One aspect of the information processing method of this disclosure involves obtaining a plurality of elements contained in a sequence of data, calculating a likelihood ratio indicating the likelihood of the class to which the sequence of data belongs based on at least two consecutive elements among the plurality of elements, classifying the sequence of data into at least one class from three or more candidate classes based on the likelihood ratio, adjusting the parameters of the model for determining the likelihood ratio so that the value of the loss function calculated using the likelihood ratio becomes small, wherein the value of the loss function becomes small when the numerator and denominator of the likelihood ratio have the same likelihood, and when the correct class to which the sequence of data belongs is in the numerator but not in the denominator of the likelihood ratio, and when the correct class to which the sequence of data belongs is in the denominator but not in the numerator of the likelihood ratio.

[0008] One aspect of the computer program of this disclosure involves taking a plurality of elements contained in a sequence of data, calculating a likelihood ratio indicating the likelihood of the class to which the sequence of data belongs based on at least two consecutive elements of the plurality of elements, classifying the sequence of data into at least one of three or more candidate classes based on the likelihood ratio, adjusting the parameters of a model for calculating the likelihood ratio so that the value of the loss function calculated using the likelihood ratio is small, and operating the computer such that the value of the loss function is small when the numerator and denominator of the likelihood ratio are the same likelihood, and when the ground truth class to which the sequence of data belongs is in the numerator and not in the denominator of the likelihood ratio, and when the ground truth class to which the sequence of data belongs is in the denominator and not in the numerator of the likelihood ratio. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the hardware configuration of the information processing system according to the first embodiment. [Figure 2] This is a block diagram showing the functional configuration of the information processing system according to the first embodiment. [Figure 3] This is a flowchart showing the operation flow of the classification device in the information processing system according to the first embodiment. [Figure 4] This is a flowchart showing the operation flow of the learning unit in the information processing system according to the first embodiment. [Figure 5] This is a flowchart showing the operation flow of the learning unit in the information processing system according to the second embodiment. [Figure 6] This matrix diagram shows an example of a likelihood ratio considered by the learning unit in the information processing system according to the second embodiment. [Figure 7] This is a flowchart showing the operation flow of the learning unit in the information processing system according to the third embodiment. [Figure 8] This is a flowchart showing the operation flow of the learning unit in the information processing system according to the fourth embodiment. [Figure 9]A matrix diagram showing an example of the likelihood ratio considered by the learning unit in the information processing system according to the fourth embodiment. [Figure 10] A graph showing an example of the sigmoid function used in the information processing system according to the fifth embodiment. [Figure 11] A graph showing an example of the logistic function used in the information processing system according to the sixth embodiment. [Figure 12] A block diagram showing the functional configuration of the information processing system according to the seventh embodiment. [Figure 13] A flowchart showing the operation flow of the classification device in the information processing system according to the seventh embodiment. [Figure 14] A block diagram showing the functional configuration of the information processing system according to the eighth embodiment. [Figure 15] A flowchart showing the operation flow of the likelihood ratio calculation unit in the information processing system according to the eighth embodiment. [Figure 16] A flowchart showing the operation flow of the classification device in the information processing system according to the ninth embodiment.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of an information processing system, an information processing method, and a computer program will be described with reference to the drawings.

[0011] <First Embodiment> The information processing system according to the first embodiment will be described with reference to FIGS. 1 to 4.

[0012] (Hardware Configuration) First, the hardware configuration of the information processing system according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0013] As shown in FIG. 1, an information processing system 1 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 1 may further include an input device 15 and an output device 16. The processor 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, and the output device 16 are connected via a data bus 17.

[0014] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable recording medium using a recording medium reader (not shown). The processor 11 may obtain (i.e., read) a computer program from a device (not shown) disposed outside the information processing system 1 via a network interface. By executing the read computer program, the processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16. In particular, in this embodiment, when the processor 11 executes the read computer program, a functional block for performing classification using likelihood ratios and learning processes related thereto is realized within the processor 11. Examples of the processor 11 include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a DSP (Demand-Side Platform), and an ASIC (Application Specific Integrated Circuit). The processor 11 may use one of the above examples or may use a plurality of them in parallel.

[0015] RAM12 temporarily stores computer programs executed by processor 11. RAM12 also temporarily stores data that processor 11 uses temporarily while it is executing computer programs. RAM12 may be, for example, D-RAM (Dynamic RAM).

[0016] ROM 13 stores computer programs executed by processor 11. ROM 13 may also store other static data. ROM 13 may be, for example, a P-ROM (Programmable ROM).

[0017] The storage device 14 stores data that the information processing system 1 saves for the long term. The storage device 14 may also operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk drive, a magneto-optical disk drive, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from the user of the information processing system 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may also be a dedicated controller (operating terminal). Alternatively, the input device 15 may include a terminal owned by the user (for example, a smartphone or tablet). The input device 15 may also be a device capable of voice input, for example, including a microphone.

[0019] The output device 16 is a device that outputs information related to the information processing system 1 to the outside. For example, the output device 16 may be a display device (e.g., a display) capable of displaying information related to the information processing system 1. The display device here may be a television monitor, a personal computer monitor, a smartphone monitor, a tablet monitor, or a monitor of another mobile device. Alternatively, the display device may be a large monitor or digital signage installed in various facilities such as stores. Furthermore, the output device 16 may be a device that outputs information in a format other than images. For example, the output device 16 may be a speaker that outputs information related to the information processing system 1 as sound.

[0020] (Functional configuration) Next, the functional configuration of the information processing system 1 according to the first embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0021] As shown in Figure 2, the information processing system 1 according to the first embodiment comprises a classification device 10 and a learning unit 300. The classification device 10 is a device that classifies input sequence data, and is configured as a processing block to realize this function, comprising a data acquisition unit 50, a likelihood ratio calculation unit 100, and a class classification unit 200. The learning unit 300 is configured to execute learning processing related to the classification device 10. Here, an example is given in which the learning unit 300 is provided separately from the classification device 10, but the classification device 10 may also be configured to include the learning unit 300. Each of the data acquisition unit 50, the likelihood ratio calculation unit 100, the class classification unit 200, and the learning unit 300 may be realized by the processor 11 (see Figure 1) described above.

[0022] The data acquisition unit 50 is configured to acquire multiple elements included in sequential data. The data acquisition unit 50 may acquire data directly from any data acquisition device (e.g., a camera or microphone), or it may read data that has been acquired in advance by a data acquisition device and stored in storage. When acquiring data from a camera, the data acquisition unit 50 may be configured to acquire data from each of multiple cameras. The elements of the sequential data acquired by the data acquisition unit 50 are output to the likelihood ratio calculation unit 100. Sequential data refers to data containing multiple elements arranged in a predetermined order; time-series data is one example. More specific examples of sequential data include, but are not limited to, video data and audio data.

[0023] The likelihood ratio calculation unit 100 is configured to calculate the likelihood ratio based on at least two consecutive elements from among the multiple elements acquired by the data acquisition unit 50. Here, "likelihood ratio" refers to an index indicating the likelihood of a given class to which a data series belongs. Specific examples of the likelihood ratio and its calculation method will be explained in detail in other embodiments described later.

[0024] The classification unit 200 is configured to classify series data based on the likelihood ratio calculated by the likelihood ratio calculation unit 100. The classification unit 200 selects at least one class to which the series data belongs from among a plurality of candidate classes. The plurality of candidate classes may be pre-set. Alternatively, the plurality of candidate classes may be set as appropriate by the user, or may be set as appropriate based on the type of series data being handled.

[0025] The learning unit 300 performs learning on the calculation of likelihood ratios using a loss function. Specifically, it performs learning on the calculation of likelihood ratios so that classification based on likelihood ratios is performed accurately. The loss function used by the learning unit 300 is defined as a function that increases the likelihood ratio when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and decreases the likelihood ratio when the correct class is in the denominator of the likelihood ratio. The loss function may be pre-set as a function that satisfies this definition. Specific examples of loss functions will be explained in detail in other embodiments described later.

[0026] (The flow of the classification process) Next, with reference to Figure 3, the operation flow of the classification device 10 in the information processing system 1 according to the first embodiment (specifically, the class classification operation after learning) will be described. Figure 3 is a flowchart showing the operation flow of the classification device in the information processing system according to the first embodiment.

[0027] As shown in Figure 3, when the classification device 10 starts operating, the data acquisition unit 50 first acquires elements included in the series data (step S11). The data acquisition unit 50 outputs the acquired elements of the series data to the likelihood ratio calculation unit 100. The likelihood ratio calculation unit 100 then calculates the likelihood ratio based on two or more acquired elements (step S12).

[0028] Next, the classification unit 200 performs classification based on the calculated likelihood ratio (step S13). The classification may determine one class to which the series data belongs, or it may determine multiple classes to which the series data is likely to belong. The classification unit 200 may output the classification results to a display or the like. Alternatively, the classification unit 200 may output the classification results as sound via a speaker or the like.

[0029] (The flow of the learning process) Next, with reference to Figure 4, the flow of operation of the learning unit 300 in the information processing system 1 according to the first embodiment (i.e., the learning operation related to the calculation of the likelihood ratio) will be explained. Figure 4 is a flowchart showing the flow of operation of the learning unit in the information processing system according to the first embodiment.

[0030] As shown in Figure 4, when the learning operation is started, training data is first input to the learning unit 300 (step S101). The training data may consist of, for example, sequence data and information about the correct answer class to which the sequence data belongs (i.e., correct answer data).

[0031] Next, the learning unit 300 calculates a loss function using the input training data (step S102). As explained earlier, the loss function here is a function in which the likelihood ratio increases when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and decreases when the correct class is in the denominator of the likelihood ratio.

[0032] Next, the learning unit 300 adjusts the parameters (specifically, the parameters of the model used to calculate the likelihood ratio) so that the calculated loss function becomes smaller (step S103). In other words, the learning unit 300 optimizes the parameters of the model used to calculate the likelihood ratio. As for the parameter optimization method using the loss function, existing techniques can be adopted as appropriate. One example of an optimization method is the backpropagation method, but other methods may also be used.

[0033] Subsequently, the learning unit 300 determines whether all learning has been completed (step S104). The learning unit 300 may determine whether all learning has been completed, for example, by whether all training data has been input. Alternatively, the learning unit 300 may determine whether all learning has been completed by whether a predetermined period of time has elapsed since the start of learning. Alternatively, the learning unit 300 may determine whether all learning has been completed by whether the processes described in steps S101 to S103 have been looped a predetermined number of times.

[0034] If it is determined that all learning is complete (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning is not complete (step S104: NO), the learning unit 300 starts the process again from step S101. In this way, the learning process using the training data is repeated, and the parameters are adjusted to become more optimal.

[0035] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the first embodiment will be described.

[0036] As explained in Figures 1 to 4, in the information processing system 1 according to the first embodiment, the learning unit 300 performs learning related to the calculation of likelihood ratios used for class classification. In this embodiment, learning is performed using a loss function in which the likelihood ratio becomes large when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and becomes small when the correct class is in the denominator of the likelihood ratio. By using such a loss function, learning can be performed in a way that the penalty for incorrect class selection is large and the penalty for correct class selection is small. As a result, it becomes possible to appropriately select at least one class to which the sequence data belongs from among multiple candidate classes for classification.

[0037] When there are multiple classes as classification candidates (so-called multi-class classification), it is not easy to decide what likelihood ratios to consider during training (for example, what ratios to use). However, by using the loss function described above, the magnitude of the likelihood ratio changes depending on whether the correct class is in the numerator or the denominator of the likelihood ratio, and this changes the influence on the loss function. Using such a loss function, it is possible to appropriately perform training regarding the calculation of likelihood ratios in multi-class classification. As a result, it becomes possible to achieve appropriate class classification. It should be noted that deciding what likelihood ratios to consider during training becomes particularly difficult when there are three or more classes as classification candidates. Therefore, the technical effect of this embodiment is particularly evident when there are three or more classification candidates.

[0038] <Second Embodiment> The information processing system 1 according to the second embodiment will be described with reference to Figures 5 and 6. Note that the second embodiment differs from the first embodiment in only a few operations; for example, the device configuration (see Figures 1 and 2) and the operation of the classification device 10 (see Figure 3) may be the same as in the first embodiment. Therefore, the following will explain in detail the parts that differ from the first embodiment, and will omit explanations of other overlapping parts as appropriate.

[0039] (The flow of the learning process) First, the operation flow of the learning unit 300 in the information processing system 1 according to the second embodiment will be explained with reference to Figure 5. Figure 5 is a flowchart showing the operation flow of the information processing system according to the second embodiment. Note that in Figure 5, the same reference numerals are used for the same processes as shown in Figure 4.

[0040] As shown in Figure 5, when the operation of the information processing system 1 according to the second embodiment is started, training data is first input to the learning unit 300 (step S101).

[0041] Next, the learning unit 300 calculates a loss function using the input training data. In the second embodiment in particular, it calculates a loss function that considers the likelihood ratio of N × (N-1) patterns, where the likelihood of belonging to one of the N (where N is a natural number) classes that are classification candidates for the sequence data is the denominator, and the likelihood of belonging to other classes is the numerator (step S201). Similar to the first embodiment, this loss function is such that the likelihood ratio is large when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and small when the correct class is in the denominator of the likelihood ratio. The likelihood ratios considered in the loss function will be explained in detail later with specific examples.

[0042] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). That is, the learning unit 300 optimizes the parameters of the model used to calculate the likelihood ratio. After that, the learning unit 300 determines whether all learning has been completed (step S104). If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0043] (Specific examples of likelihood ratios to consider) Next, with reference to Figure 6, we will specifically explain the likelihood ratios considered in the learning operation by the learning unit 300 described above (i.e., the likelihood ratios considered in calculating the loss function). Figure 6 is a matrix diagram showing an example of the likelihood ratios considered by the learning unit in the information processing system according to the second embodiment.

[0044] As shown in Figure 6, we consider the likelihoods in a matrix. For the sake of explanation, let's assume there are three candidate classes: "Class 0", "Class 1", and "Class 2". p(X|y=0) is the likelihood that the sequence data is "Class 0". p(X|y=1) is the likelihood that the sequence data is "Class 1". p(X|y=2) is the likelihood that the sequence data is "Class 2".

[0045] In the first row from the top of the matrix, the numerator of the log-likelihood ratio (hereinafter simply referred to as "likelihood ratio") is always p(X|y=0). In the second row from the top of the matrix, the numerator of the likelihood ratio is always p(X|y=1). In the third row from the top of the matrix, the numerator of the likelihood ratio is always p(X|y=2). On the other hand, in the first column from the left of the matrix, the denominator of the likelihood ratio is always p(X|y=0). In the second column from the left of the matrix, the denominator of the likelihood ratio is always p(X|y=1). In the third column from the left of the matrix, the denominator of the likelihood ratio is always p(X|y=2).

[0046] The likelihood ratios on the diagonal of the matrix (the likelihood ratios shaded in gray in Figure 6) have the same numerator and denominator. Specifically, log{p(X|y=0) / p(X|y=0)} in the first row from the top and first column from the left, log{p(X|y=1) / p(X|y=1)} in the second row from the top and second column from the left, and log{p(X|y=2) / p(X|y=2)} in the third row from the top and third column from the left have the same numerator and denominator. Furthermore, the likelihood ratios located opposite each other on the diagonal have their numerators and denominators reversed. Specifically, the numerator and denominator are reversed in the log{p(X|y=0) / p(X|y=1)} in the first row from the top and the second column from the left, and in the log{p(X|y=1) / p(X|y=0)} in the second row from the top and the first column from the left. Similarly, the numerator and denominator are reversed in the log{p(X|y=0) / p(X|y=2)} in the first row from the top and the third column from the left, and in the log{p(X|y=2) / p(X|y=0)} in the third row from the top and the first column from the left. The numerator and denominator are reversed in the log{p(X|y=1) / p(X|y=2)} in the second row from the top and the third column from the left, and in the log{p(X|y=2) / p(X|y=1)} in the third row from the top and the second column from the left. Therefore, likelihood ratios located on opposite sides of the diagonal will have opposite signs. In this way, the likelihood ratios shown in the matrix are arranged like an alternating matrix.

[0047] In particular, all likelihood ratios where the numerator and denominator are the same on the diagonal are log1, and their value is zero. Therefore, considering likelihood ratios where the numerator and denominator are the same on the diagonal is practically meaningless when included in the loss function. For this reason, likelihood ratios where the numerator and denominator are the same on the diagonal are not considered in the loss function. The number of remaining likelihood ratios after removing the likelihood ratios on the diagonal is N × (N-1), where N is the number of classes. In this embodiment, these N × (N-1) patterns of likelihood ratios (i.e., likelihood ratios in the matrix excluding the likelihood ratios on the diagonal) are considered in the loss function. A specific example of a loss function that considers N × (N-1) patterns of likelihood ratios will be explained in detail in other embodiments described later.

[0048] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the second embodiment will be described.

[0049] As explained in Figures 5 and 6, in the information processing system 1 according to the second embodiment, learning is performed using a loss function that considers the likelihood ratio of an N × (N-1) pattern, where the likelihood of belonging to one class is the denominator and the likelihood of belonging to another class is the numerator. Using such a loss function, learning can be performed in a way that, similar to the first embodiment, increases the penalty when the class is incorrect and decreases the penalty when the class is correct. As a result, it becomes possible to appropriately select at least one class to which the sequence data belongs from among multiple classes that are classification candidates.

[0050] <Third Embodiment> The information processing system 1 according to the third embodiment will be described with reference to Figure 7. Note that the third embodiment differs from the first and second embodiments described above only in some operations; other parts may be the same as those of the first and second embodiments. Therefore, the following will explain in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0051] (The flow of the learning process) First, the operation flow of the learning unit 300 in the information processing system 1 according to the third embodiment will be explained with reference to Figure 7. Figure 7 is a flowchart showing the operation flow of the information processing system according to the third embodiment. Note that in Figure 7, the same reference numerals are used for the same processes as shown in Figure 4.

[0052] As shown in Figure 7, when the operation of the information processing system 1 according to the third embodiment is started, training data is first input to the learning unit 300 (step S101).

[0053] Next, the learning unit 300 calculates a loss function using the input training data. In the third embodiment in particular, it calculates a loss function that considers a portion of the likelihood ratios of an N×(N-1) pattern, where the likelihood of belonging to one of the N classes that are classification candidates for the sequential data is the denominator, and the likelihood of belonging to other classes is the numerator (step S301). That is, the learning unit 300 according to the third embodiment does not consider all of the likelihood ratios of the N×(N-1) pattern described in the second embodiment, but only a portion of them. This loss function, like the first embodiment, is a function in which the likelihood ratio is large when the correct class to which the sequential data belongs is in the numerator of the likelihood ratio, and small when the correct class is in the denominator of the likelihood ratio. Specific examples of the loss function that considers a portion of the likelihood ratios of the N×(N-1) pattern will be explained in detail in other embodiments described later.

[0054] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). After that, the learning unit 300 determines whether all learning has been completed (step S104). If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0055] (Examples of selecting likelihood ratios to consider) Next, we will specifically explain examples of selecting likelihood ratios to consider in the loss function (i.e., examples of selecting some likelihood ratios for the N×(N-1) pattern).

[0056] Of the N×(N-1) pattern likelihood ratios, some of the likelihood ratios to be considered in the loss function may be selected in advance by the user or the like, or the learning unit 300 may be automatically selected. When the learning unit 300 selects some of the likelihood ratios to be considered in the loss function, the learning unit 300 may select the likelihood ratios according to predetermined rules set in advance. Alternatively, the learning unit 300 may decide whether or not to select based on the calculated likelihood ratio values.

[0057] One example of selecting only a subset of likelihood ratios to consider in the loss function is to select only the likelihood ratios from one row or one column of the matrix shown in Figure 6. For example, one could select only the likelihood ratios from the first row, the second row, or the third row of the matrix shown in Figure 6 as the likelihood ratios to consider in the loss function. Alternatively, one could select only the likelihood ratios from the first column, the second column, or the third column of the matrix.

[0058] Furthermore, only the likelihood ratios for some rows or some columns in the matrix may be selected. Specifically, only the likelihood ratios for the first and second rows of the matrix may be selected, or only the likelihood ratios for the second and third rows may be selected, or only the likelihood ratios for the third and first rows may be selected. Alternatively, only the likelihood ratios for the first and second columns of the matrix may be selected, or only the likelihood ratios for the second and third columns may be selected, or only the likelihood ratios for the third and first columns may be selected.

[0059] The above examples of likelihood ratio selection are merely illustrative, and other likelihood ratios may be selected as the likelihood ratios considered in the loss function. For example, the likelihood ratios considered in the loss function may be selected randomly, regardless of row or column.

[0060] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the third embodiment will be described.

[0061] As explained in Figure 7, in the information processing system 1 according to the third embodiment, learning is performed using a loss function that considers some of the likelihood ratios from an N×(N-1) pattern, where the likelihood of belonging to one class is the denominator and the likelihood of belonging to other classes is the numerator. Using such a loss function, learning can be performed in a way that increases the penalty when the class is incorrect and decreases the penalty when the class is correct, similar to the embodiments described above. As a result, it becomes possible to appropriately select at least one class to which the sequential data belongs from among multiple classes that are classification candidates. Furthermore, in the third embodiment in particular, by appropriately selecting the likelihood ratios to be considered in the loss function from the N×(N-1) pattern, more efficient learning can be performed compared to the case where all likelihood ratios of the N×(N-1) pattern are considered. For example, by selecting only the likelihood ratios that have a relatively large impact on the loss function and not selecting the likelihood ratios that have a relatively small impact on the loss function, learning efficiency can be increased.

[0062] <Fourth Embodiment> The information processing system 1 according to the fourth embodiment will be described with reference to Figures 8 and 9. The fourth embodiment describes a specific selection example of the third embodiment described above (i.e., a selection example of some likelihood ratios to be considered in the loss function), and other parts may be the same as those of the third embodiment. For this reason, the following will explain in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0063] (The flow of the learning process) First, the operation flow of the learning unit 300 in the information processing system 1 according to the fourth embodiment will be explained with reference to Figure 8. Figure 8 is a flowchart showing the operation flow of the information processing system according to the fourth embodiment. Note that in Figure 8, the same reference numerals are used for the same processes as shown in Figure 4.

[0064] As shown in Figure 8, when the operation of the information processing system 1 according to the fourth embodiment is started, training data is first input to the learning unit 300 (step S101).

[0065] Next, the learning unit 300 calculates a loss function using the input training data. In the fourth embodiment, in particular, it calculates a loss function that takes into account the likelihood ratios of the N×(N-1) patterns described earlier, where the correct class is in the numerator (step S401). That is, the learning unit 300 according to the fourth embodiment selects the likelihood ratios where the correct class is in the numerator as a subset of the N×(N-1) patterns described in the third embodiment. This loss function, like the first embodiment, is a function in which the likelihood ratio increases when the correct class to which the sequence data belongs is in the numerator of the likelihood ratio, and decreases when the correct class is in the denominator of the likelihood ratio. Specific examples of the loss function that takes into account the likelihood ratio where the correct class is in the numerator will be explained in detail in other embodiments described later.

[0066] Next, the learning unit 300 adjusts the parameters so that the calculated loss function becomes smaller (step S103). After that, the learning unit 300 determines whether all learning has been completed (step S104). If it is determined that all learning has been completed (step S104: YES), the series of processes ends. On the other hand, if it is determined that all learning has not been completed (step S104: NO), the learning unit 300 starts the process again from step S101.

[0067] (Specific examples of likelihood ratios to consider) Next, with reference to Figure 9, we will specifically explain the likelihood ratios considered in the learning operation by the learning unit 300 described above (i.e., the likelihood ratios considered in calculating the loss function). Figure 9 is a matrix diagram showing an example of the likelihood ratios considered by the learning unit in the information processing system according to the fourth embodiment.

[0068] As already explained in the second embodiment (see Figure 6), the matrix shown in Figure 9 has each likelihood ratio arranged in an alternating matrix. The learning unit 300 according to the fourth embodiment selects the likelihood ratios in the numerator of the correct class from among the N × (N-1) patterns of likelihood ratios obtained by excluding the likelihood ratios on the diagonal of such a matrix, and considers them in the loss function.

[0069] For example, suppose the correct class of the sequence data input as training data is "Class 1". In this case, the learning unit 300 selects the likelihood ratios in the numerator of Class 1 from among the N × (N-1) patterns of likelihood ratios and considers them in the loss function. Specifically, it selects only the likelihood ratios in the second row from the top of Figure 9 (excluding the likelihood ratios on the diagonal) and considers them in the loss function. In this case, log{p(X|y=1) / p(X|y=0)} in the second row from the top and the first column from the left, and log{p(X|y=1) / p(X|y=2)} in the second row from the top and the third column from the left, will be considered in the loss function. That is, the likelihood ratios in Figure 9 that are not shaded in gray will be considered in the loss function.

[0070] Furthermore, if the correct class of the sequence data input as training data is "Class 0", the learning unit 300 only needs to select the likelihood ratios in the numerator of Class 0 from the N × (N-1) pattern likelihood ratios and consider them in the loss function. Specifically, only the likelihood ratios in the first row from the top of Figure 9 (excluding the likelihood ratios on the diagonal) should be selected and considered in the loss function. In this case, log{p(X|y=0) / p(X|y=1)} in the first row from the top and second column from the left, and log{p(X|y=0) / p(X|y=2)} in the first row from the top and third column from the left will be considered in the loss function.

[0071] Similarly, if the correct class of the sequence data input as training data is "Class 2", the learning unit 300 only needs to select the likelihood ratios in the numerator of Class 2 from the N × (N-1) pattern likelihood ratios and consider them in the loss function. Specifically, only the likelihood ratios in the third row from the top of Figure 9 (excluding the likelihood ratios on the diagonal) should be selected and considered in the loss function. In this case, log{p(X|y=2) / p(X|y=0)} in the third row from the top and the first column from the left, and log{p(X|y=2) / p(X|y=1)} in the third row from the top and the second column from the left will be considered in the loss function.

[0072] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the fourth embodiment will be described.

[0073] As explained in Figures 8 and 9, in the information processing system 1 according to the fourth embodiment, learning is performed using a loss function that takes into account the likelihood ratio in the numerator of the N × (N-1) patterns in which the correct class is located. By using such a loss function, appropriate learning is performed, as in the embodiments described above, making it possible to appropriately select at least one class to which the sequential data belongs from among multiple classes that are classification candidates. Furthermore, in the fourth embodiment in particular, since the likelihood ratio in the numerator of the correct class (in other words, the likelihood ratio that can have a significant impact on the loss function) is taken into consideration in the loss function, learning can be performed more efficiently compared to the case where all likelihood ratios of the N × (N-1) patterns are taken into consideration.

[0074] <Fifth Embodiment> The information processing system 1 according to the fifth embodiment will be described with reference to Figure 10. The fifth embodiment describes a specific example of the loss function used in the first to fourth embodiments described above, and the device configuration and operation flow may be the same as in the first to fourth embodiments. Therefore, the following will explain in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0075] (Loss function including sigmoid function) First, with reference to Figure 10, an overview of the loss function used in the information processing system 1 according to the fifth embodiment will be described. Figure 10 is a graph showing an example of a sigmoid function used in the information processing system according to the fifth embodiment.

[0076] As shown in Figure 10, in the information processing system 1 according to the fifth embodiment, the learning unit 300 performs learning using a loss function that includes a sigmoid function. More specifically, the learning unit 300 performs learning using a loss function that includes a sigmoid function as a nonlinear function acting on the likelihood ratio. Below, we will explain with specific examples of loss functions that include a sigmoid function.

[0077] (Example considering all likelihood ratios) As a first example, we will explain the loss function when considering all likelihood ratios for N × (N-1) patterns, as in the second embodiment. Note that the likelihood ratios in the following equations are assumed to be log-likelihood ratios (LLR).

[0078] For example, equation (1) below can be used as a loss function that considers all likelihood ratios for N × (N-1) patterns.

[0079]

number

[0080] In equation (1) above, K is the number of classes, M is the number of data points, and T is the time series length. Also, k is the row index and l is the column index (i.e., the index indicating the row and column numbers in the matrix shown in Figure 6, etc.). δ is the Kronecker delta, which is "1" when the indices match, and "0" otherwise. λ is the likelihood ratio, and in equation (1) above, it represents the log-likelihood ratio of the k-th row, l-th column at time t. σ is the sigmoid function and is included as a nonlinear function acting on the likelihood ratio λ.

[0081] In equation (1), 1 / MT is the average taken over all data and time series directions. 1 / 2K is the product of 1 / K, which is the average of K rows, and 1 / 2, which is the average of the skew-symmetric matrix. 1 / (K-1) is the average of (K-1) columns, which is obtained by subtracting the diagonal column from the K columns.

[0082] In the loss function L of equation (1), if row k belongs to the correct class, Kronecker δ is "1", and if row k belongs to an incorrect class (i.e., a class other than the correct class), Kronecker δ is "0". As a result, if row k belongs to the correct class (in other words, if the correct class is in the numerator of the likelihood ratio), the value of the likelihood ratio will be large. On the other hand, if row k belongs to an incorrect class (in other words, if the incorrect class is in the numerator of the likelihood ratio), the value of the likelihood ratio will be small.

[0083] (Example of considering some likelihood ratios) Next, as a second example, we will explain the loss function when considering only some of the likelihood ratios among the N×(N-1) patterns, as in the third and fourth embodiments. In particular, we will explain the example in the fourth embodiment, which considers only the likelihood ratios in the numerator of the N×(N-1) patterns.

[0084] For example, equation (2) below can be given as a loss function that considers only the likelihood ratios in which the correct class is in the numerator out of N × (N-1) patterns.

[0085]

number

[0086] In equation (2) above, K is the number of classes, M is the number of data points, and T is the time series length. Also, k is the row index and l is the column index (i.e., the index indicating the row and column numbers in the matrix shown in Figure 6, etc.). λ is the likelihood ratio, and in equation (1) above, it represents the log-likelihood ratio of the k-th row and l-th column at time t. σ is the sigmoid function and is included as a nonlinear function acting on the likelihood ratio λ.

[0087] Equation (2) only considers the rows of the correct class, and therefore, compared to equation (1) which has already been explained, it does not include the step of taking the sum of K rows and taking the average of K rows and the skew-symmetric matrix by 1 / 2K. Also, the δ in the Kronecker matrix in equation (1) is "1".

[0088] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the fifth embodiment will be described.

[0089] As explained in Figure 10, in the information processing system 1 according to the fifth embodiment, the learning unit 300 utilizes a loss function that includes a sigmoid function σ. By using such a loss function, it becomes possible to appropriately learn how to calculate the likelihood ratio, as described in the first to fourth embodiments above. Specifically, learning can be performed such that the penalty for incorrect class selection is large, and the penalty for correct class selection is small.

[0090] The loss functions given in the fifth embodiment (i.e., equations (1) and (2)) are examples, and different loss functions may be created using the sigmoid function. Alternatively, a loss function may be created using other nonlinear functions instead of the sigmoid function. For example, a loss function including the logistic function may be used, as in the embodiment described later.

[0091] <Sixth Embodiment> The information processing system 1 according to the sixth embodiment will be described with reference to Figure 11. The sixth embodiment, like the fifth embodiment, describes a specific example of the loss function used in the first to fourth embodiments described above, and the device configuration and operation flow may be the same as in the first to fourth embodiments. Therefore, the following will explain in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0092] (Loss function including logistic function) First, with reference to Figure 11, an overview of the loss function used in the information processing system 1 according to the sixth embodiment will be described. Figure 11 is a graph showing an example of the logistic function used in the information processing system according to the sixth embodiment.

[0093] As shown in Figure 11, in the information processing system 1 according to the sixth embodiment, the learning unit 300 performs learning using a loss function that includes a logistic function. More specifically, the learning unit 300 performs learning using a loss function that includes the logistic function as a nonlinear function acting on the likelihood ratio. Below, we will explain with specific examples of loss functions that include a logistic function.

[0094] (Example considering all likelihood ratios) As a first example, we will explain the loss function when considering all likelihood ratios for N × (N-1) patterns, as in the second embodiment. Note that the likelihood ratios in the following equations are assumed to be log-likelihood ratios (LLR).

[0095] For example, equation (3) below can be used as a loss function that considers all likelihood ratios for N × (N-1) patterns.

[0096]

number

[0097] In equation (3) above, K is the number of classes, M is the number of data points, and T is the time series length. Also, k is the row index and l is the column index (i.e., the index indicating the row and column numbers in the matrix shown in Figure 6, etc.). δ is the Kronecker delta, which is "1" when the indices match, and "0" otherwise. λ is the likelihood ratio, and in equation (3) above, it represents the log-likelihood ratio of the k-th row, l-th column at time t. logistic is the logistic function and is included as a nonlinear function acting on the likelihood ratio λ.

[0098] In equation (1), 1 / MT is the average taken over all data and time series directions. 1 / 2K is the product of 1 / K, which is the average of K rows, and 1 / 2, which is the average of the skew-symmetric matrix. 1 / (K-1) is the average of (K-1) columns, which is obtained by subtracting the diagonal column from the K columns.

[0099] In the loss function L of equation (3), if row k is in the correct class, Kronecker δ is "1", and if row k is in the incorrect class (i.e., a class other than the correct class), Kronecker δ is "0". As a result, if row k is in the correct class (in other words, if the correct class is in the numerator of the likelihood ratio), of the two terms containing Kronecker δ, the first term remains and the second term becomes zero. On the other hand, if row k is in the incorrect class (in other words, if the incorrect class is in the numerator of the likelihood ratio), of the two terms containing Kronecker δ, the first term becomes zero and the second term remains.

[0100] (Example of considering some likelihood ratios) Next, as a second example, we will explain the loss function when considering only some of the likelihood ratios among the N×(N-1) patterns, as in the third and fourth embodiments. In particular, we will explain the example in the fourth embodiment, which considers only the likelihood ratios in the numerator of the N×(N-1) patterns.

[0101] For example, equation (4) below can be given as a loss function that considers only the likelihood ratios in which the correct class is in the numerator out of N × (N-1) patterns.

[0102]

number

[0103] In equation (4) above, K is the number of classes, M is the number of data points, and T is the time series length. Also, k is the row index and l is the column index (i.e., the index indicating the row and column numbers in the matrix shown in Figure 6, etc.). λ is the likelihood ratio, and in equation (1) above, it represents the log-likelihood ratio of the k-th row and l-th column at time t. logistic is the logistic function and is included as a nonlinear function acting on the likelihood ratio λ.

[0104] Equation (4) only considers the rows of the correct class, and therefore, compared to equation (3) which has already been explained, it does not include the step of taking the sum of K rows and taking the average of K rows and the skew-ascending matrix by 1 / 2K. Also, of the two terms in equation (3) that include Kronecker's delta, only the first term remains.

[0105] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the sixth embodiment will be described.

[0106] As explained in Figure 11, in the information processing system 1 according to the sixth embodiment, the learning unit 300 utilizes a loss function that includes a logistic function. By using such a loss function, it becomes possible to appropriately learn how to calculate the likelihood ratio, as explained in the first to fourth embodiments described above. Specifically, it is possible to perform learning such that the penalty for incorrect class selections is large, and the penalty for correct class selections is small. Furthermore, while the sigmoid function σ used in the fifth embodiment changes so that its slope asymptotically approaches zero (see Figure 10), the logistic function changes while its slope remains constant (see Figure 11). Therefore, when using a loss function that includes a logistic function, components do not disappear in the process of calculating the slope of the loss function (i.e., the process equivalent to differentiation), making it possible to perform more appropriate learning.

[0107] The loss functions given in the sixth embodiment (i.e., equations (3) and (4)) are examples, and different loss functions may be created using the logistic function. Alternatively, a loss function may be created using another nonlinear function instead of the logistic function. For example, a loss function that includes a function different from both the sigmoid function and the logistic function described above may be used.

[0108] <Seventh Embodiment> The information processing system 1 according to the seventh embodiment will be described with reference to Figures 12 and 13. The third embodiment differs from the first to sixth embodiments described above only in some configurations and operations (specifically, the configuration and operation of the classification device 10), and other parts may be the same as those of the first to sixth embodiments. Therefore, the following will explain in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0109] (Functional configuration) First, the functional configuration of the information processing system 1 according to the seventh embodiment will be described with reference to Figure 12. Figure 12 is a block diagram showing the functional configuration of the information processing system according to the seventh embodiment. In Figure 12, the same reference numerals are used for elements as in Figure 2.

[0110] As shown in Figure 12, in the information processing system 1 according to the seventh embodiment, the likelihood ratio calculation unit 100 in the classification device 10 includes a first calculation unit 110 and a second calculation unit 120. Note that each of the first calculation unit 110 and the second calculation unit 120 may be implemented, for example, by the processor 11 (see Figure 1) described above.

[0111] The first calculation unit 110 is configured to calculate individual likelihood ratios based on two consecutive elements included in the sequential data. The individual likelihood ratio is calculated as a likelihood ratio that indicates the likelihood of the class to which the two consecutive elements belong. The first calculation unit 110 may, for example, sequentially acquire elements included in the sequential data from the data acquisition unit 50 and sequentially calculate individual likelihood ratios based on two consecutive elements. The individual likelihood ratios calculated by the first calculation unit 110 are output to the second calculation unit 120.

[0112] The second calculation unit 120 is configured to calculate a combined likelihood ratio based on multiple individual likelihood ratios calculated by the first calculation unit 110. The combined likelihood ratio is calculated as a likelihood ratio indicating the likelihood of the class to which the multiple elements considered in each of the multiple individual likelihood ratios belong. In other words, the combined likelihood ratio is calculated as a likelihood ratio indicating the likelihood of the class to which the sequence data, which consists of multiple elements, belongs. The combined likelihood ratio calculated by the second calculation unit 120 is output to the classification unit 200. The classification unit 200 performs classification of the sequence data based on the combined likelihood ratio.

[0113] In the fifth embodiment, the learning unit 300 may perform learning on the likelihood ratio calculation unit 100 as a whole (i.e., the first calculation unit 110 and the second calculation unit 120 together), or it may perform learning on the first calculation unit 110 and the second calculation unit 120 separately. Alternatively, the learning unit 300 may be provided separately as a first learning unit that performs learning only on the first calculation unit 110, and a second learning unit that performs learning only on the second calculation unit 120. In this case, only one of the first learning unit or the second learning unit may be provided.

[0114] (The flow of the classification process) Next, with reference to Figure 13, the operation flow of the classification device 10 in the information processing system 1 according to the seventh embodiment (specifically, the class classification operation after learning) will be described. Figure 13 is a flowchart showing the operation flow of the classification device in the information processing system according to the seventh embodiment.

[0115] As shown in Figure 13, when the classification device 10 starts operating, the data acquisition unit 50 first acquires the elements included in the series data (step S21). The data acquisition unit 50 outputs the acquired elements of the series data to the first calculation unit 110.

[0116] Then, the first calculation unit 110 calculates individual likelihood ratios based on two consecutive elements obtained (step S22). Subsequently, the second calculation unit 120 calculates a combined likelihood ratio based on the multiple individual likelihood ratios calculated by the first calculation unit 110 (step S23).

[0117] Next, the classification unit 200 performs classification based on the calculated combined likelihood ratio (step S24). The classification may determine one class to which the series data belongs, or it may determine multiple classes to which the series data is likely to belong. The classification unit 200 may output the classification results to a display or the like. Alternatively, the classification unit 200 may output the classification results as sound via a speaker or the like.

[0118] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the seventh embodiment will be described.

[0119] As explained in Figures 12 and 13, in the information processing system 1 according to the seventh embodiment, individual likelihood ratios are first calculated based on two elements, and then a combined likelihood ratio is calculated based on multiple individual likelihood ratios. Using the combined likelihood ratio calculated in this way, it becomes possible to appropriately select the class to which the sequence data belongs. Furthermore, in the classification device 10 that calculates the individual likelihood ratios and the combined likelihood ratio, class classification can be made more appropriate by performing learning by the learning unit 300 as described in each of the embodiments described above. That is, by performing learning that increases the penalty when the class is incorrect and decreases the penalty when the class is correct, more appropriate classification can be achieved.

[0120] <Eighth Embodiment> The information processing system 1 according to the eighth embodiment will be described with reference to Figures 14 and 15. The eighth embodiment differs from the seventh embodiment described above only in some configurations and operations (specifically, the configuration and operation of the likelihood ratio calculation unit 100), and other parts may be the same as those of the seventh embodiment. Therefore, the following will explain in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0121] (Functional configuration) First, the functional configuration of the information processing system 1 according to the eighth embodiment will be described with reference to Figure 14. Figure 14 is a block diagram showing the functional configuration of the information processing system according to the eighth embodiment. In Figure 14, the same reference numerals are used for elements as in Figures 2 and 12.

[0122] As shown in Figure 14, in the information processing system 1 according to the eighth embodiment, the likelihood ratio calculation unit 100 in the classification device 10 includes a first calculation unit 110 and a second calculation unit 120. The first calculation unit 110 is configured to include an individual likelihood ratio calculation unit 111 and a first storage unit. The second calculation unit 120 is configured to include an integrated likelihood ratio calculation unit 121 and a second storage unit 122. Note that the individual likelihood ratio calculation unit 111 and the integrated likelihood ratio calculation unit 121 may each be implemented by, for example, the processor 11 (see Figure 1) described above. Also, the first storage unit 112 and the second storage unit 122 may each be implemented by, for example, the storage device 14 (see Figure 1) described above.

[0123] The individual likelihood ratio calculation unit 111 is configured to calculate an individual likelihood ratio based on two consecutive elements from among the elements sequentially acquired by the data acquisition unit 50. More specifically, the individual likelihood ratio calculation unit 111 calculates an individual likelihood ratio based on the newly acquired elements and past data stored in the first storage unit 112. The information stored in the first storage unit 112 is configured to be readable by the individual likelihood ratio calculation unit 111. If the first storage unit 112 stores past individual likelihood ratios, the individual likelihood ratio calculation unit 111 can read the stored past individual likelihood ratios and calculate a new individual likelihood ratio that takes into account the acquired elements. On the other hand, if the first storage unit 112 stores the elements themselves that were acquired in the past, the individual likelihood ratio calculation unit 111 can calculate the past individual likelihood ratios from the stored past elements and calculate the likelihood ratio for the newly acquired elements.

[0124] The integrated likelihood ratio calculation unit 121 is configured to calculate an integrated likelihood ratio based on multiple individual likelihood ratios. The integrated likelihood ratio calculation unit 121 calculates a new integrated likelihood ratio using the individual likelihood ratios calculated by the individual likelihood ratio calculation unit 111 and past integrated likelihood ratios stored in the second storage unit 122. The information stored in the second storage unit 122 (i.e., past integrated likelihood ratios) is configured to be readable by the integrated likelihood ratio calculation unit 121.

[0125] <Flowchart of Likelihood Ratio Calculation Process> Next, with reference to Figure 15, the flow of the likelihood ratio calculation operation (i.e., the operation of the likelihood ratio calculation unit 100) in the information processing system 1 according to the eighth embodiment will be described. Figure 15 is a flowchart showing the flow of the operation of the likelihood ratio calculation unit in the information processing system according to the eighth embodiment.

[0126] As shown in Figure 15, when the likelihood ratio calculation operation by the likelihood ratio calculation unit 100 is started, the individual likelihood ratio calculation unit 111 in the first calculation unit 110 first reads past data from the first storage unit 112 (step S31). The past data may be, for example, the processing result of the individual likelihood ratio calculation unit 111 of the element acquired by the data acquisition unit 50 immediately before the element acquired this time (in other words, the individual likelihood ratio calculated for the previous element). Alternatively, the past data may be the element acquired immediately before the element acquired this time.

[0127] Next, the individual likelihood ratio calculation unit 111 calculates a new individual likelihood ratio (i.e., the individual likelihood ratio for the element acquired this time by the data acquisition unit 50) based on the elements acquired by the data acquisition unit 50 and past data read from the first storage unit 112 (step S32). The individual likelihood ratio calculation unit 111 outputs the calculated individual likelihood ratio to the second calculation unit 120. The individual likelihood ratio calculation unit 111 may also store the calculated individual likelihood ratio in the first storage unit 112.

[0128] Next, the integrated likelihood ratio calculation unit 121 in the second calculation unit 120 reads past integrated likelihood ratios from the second storage unit 122 (step S33). Past integrated likelihood ratios may be, for example, the processing result of the integrated likelihood ratio calculation unit 121 for the element acquired by the data acquisition unit 50 immediately before the element acquired this time (in other words, the integrated likelihood ratio calculated for the previous element).

[0129] Next, the integrated likelihood ratio calculation unit 121 calculates a new integrated likelihood ratio (i.e., the integrated likelihood ratio for the elements acquired this time by the data acquisition unit 50) based on the likelihood ratios calculated by the individual likelihood ratio calculation unit 111 and past integrated likelihood ratios read from the second storage unit 122 (step S34). The integrated likelihood ratio calculation unit 121 outputs the calculated integrated likelihood ratio to the classification unit 200. The integrated likelihood ratio calculation unit 121 may also store the calculated integrated likelihood ratio in the second storage unit 122.

[0130] (Technical effects) Next, the technical effects obtained by the information processing system 1 according to the eighth embodiment will be described.

[0131] As explained in Figures 14 and 15, in the information processing system 1 according to the eighth embodiment, individual likelihood ratios are calculated using past individual likelihood ratios, and then a combined likelihood ratio is calculated using past combined likelihood ratios. By using the combined likelihood ratio calculated in this way, it becomes possible to appropriately select the class to which the sequence data belongs. Furthermore, in the classification device 10 that calculates individual likelihood ratios and combined likelihood ratios using past data, class classification can be made more appropriate by performing learning by the learning unit 300 as described in each of the embodiments described above. That is, by performing learning that increases the penalty when the class is incorrect and decreases the penalty when the class is correct, more appropriate classification can be achieved.

[0132] <Ninth Embodiment> The information processing system 1 according to the ninth embodiment will be described with reference to Figure 16. The ninth embodiment differs from the first to seventh embodiments described above only in some operations (specifically, the operation of the class classification unit 200), and other parts may be the same as those of the first to seventh embodiments. Therefore, the following will explain in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0133] (The flow of the classification process) First, with reference to Figure 16, the operation flow of the classification device 10 in the information processing system 1 according to the ninth embodiment (specifically, the class classification operation after learning) will be explained. Figure 16 is a flowchart showing the operation flow of the classification device in the information processing system according to the ninth embodiment. Note that in Figure 16, the same reference numerals are used for the same processes as those described in Figure 3.

[0134] As shown in Figure 16, when the classification device 10 starts operating, the data acquisition unit 50 first acquires elements included in the series data (step S11). The data acquisition unit 50 outputs the acquired elements of the series data to the likelihood ratio calculation unit 100. The likelihood ratio calculation unit 100 then calculates the likelihood ratio based on two or more acquired elements (step S12).

[0135] Next, the classification unit 200 performs classification based on the calculated likelihood ratio. In the ninth embodiment in particular, the classification unit 200 selects and outputs multiple classes to which the series data may belong (step S41). That is, the classification unit 200 does not determine a single class to which the series data belongs, but rather determines multiple classes to which the series data is likely to belong. More specifically, the classification unit 200 performs the process of selecting k classes (where k is a natural number less than or equal to n) from n classes (where n is a natural number) prepared as classification candidates.

[0136] The classification unit 200 may output information about the k possible classes to which the sequence data belongs to a display or the like. Alternatively, the classification unit 200 may output information about the k possible classes to which the sequence data belongs as audio via a speaker or the like.

[0137] The classification unit 200 may sort the information about the k classes to which the sequence data may belong when outputting the information. For example, the classification unit 200 may sort the information about the k classes in descending order of likelihood ratio. Alternatively, the classification unit 200 may output each of the information about the k classes in a different manner for each class. For example, the classification unit 200 may output the information about classes with high likelihood ratios in a display manner that emphasizes them, while outputting the information about classes with low likelihood ratios in a display manner that does not emphasize them. When highlighting, for example, the size or color of the display may be changed, or the display may be given animation.

[0138] (Specific examples of use) The above-mentioned configuration, which outputs k classes from n classes, will be explained with several specific application examples.

[0139] (Product proposal) The information processing system 1 according to the ninth embodiment may be used on a web-based shopping site to suggest products that a user might be interested in. Specifically, the information processing system 1 may select k products (i.e., k classes) that a user might be interested in from n products (i.e., n classes) that are available for sale, and output them to the user (where k is a number smaller than n). In this case, an example of sequential data to be input would be past purchase history or browsing history.

[0140] Similarly, this can be used to suggest products and stores on digital signage, etc. Digital signage may have cameras that can capture images of users. In this case, the system may estimate the user's emotions from the user's image and suggest stores or products that match those emotions. Alternatively, the system may estimate the user's gaze from the user's image (i.e., estimate what the user is looking at) and suggest stores or products that the user might be interested in. Or, the system may estimate the user's attributes (e.g., gender, age, etc.) from the user's image and suggest stores or products that the user might be interested in. When estimating information about the user as described above, the system may assign weights to n classes according to the estimated information.

[0141] (Criminal investigation) The information processing system 1 according to the ninth embodiment can also be used in criminal investigations. For example, when trying to find the real culprit from among several suspects, if only the one person most likely to be the culprit is selected, a major problem arises if that selection is incorrect. However, the information processing system 1 according to this embodiment can select and output the top K suspects who are most likely to be the culprit. Specifically, this can be achieved by selecting and outputting a class corresponding to the top k suspects most likely to be the culprit from sequential data that includes information about each of the multiple suspects as elements. In this way, it becomes possible to appropriately find the real culprit, for example, by investigating multiple suspects who are likely to be the culprit.

[0142] (Radar image analysis) The information processing system 1 according to the ninth embodiment can also be applied to the analysis of radar images. Due to their nature, radar images often have low clarity, making it difficult for a machine alone to accurately determine, for example, what is depicted in the image. However, the information processing system 1 according to this embodiment can select and output k candidates that are highly likely to be present in the radar image. Therefore, it is possible to first output k candidates and have the user make a decision from among them. For example, if "dog," "cat," "ship," and "tank" are listed as candidates for what may be present in a radar image of a port, the user can easily determine that a "ship," which is highly related to the port, is present in the radar image.

[0143] The above-mentioned application example is just one example; in situations where it is required to select k items from n candidates, applying the information processing system 1 according to this embodiment can yield beneficial results.

[0144] The processing method of recording a program that operates the configuration of each embodiment in order to realize the functions of each embodiment described above on a recording medium, reading the program recorded on the recording medium as code, and executing it on a computer is also included in the scope of each embodiment. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0145] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that perform processing on the recording media alone, but also includes programs that operate on the OS in cooperation with other software and the functions of expansion boards to perform processing.

[0146] This disclosure may be modified as appropriate, insofar as it does not contradict the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole, and such modifications to information processing systems, information processing methods, and computer programs are also included in the technical idea of ​​this disclosure.

[0147] <Note> The embodiments described above may also be described in the following appendix, but are not limited to these.

[0148] (Note 1) The information processing system described in Appendix 1 is characterized by comprising: acquisition means for acquiring a plurality of elements included in sequential data; calculation means for calculating a likelihood ratio indicating the likelihood of the class to which the sequential data belongs based on at least two consecutive elements among the plurality of elements; classification means for classifying the sequential data into at least one class among a plurality of classification candidates based on the likelihood ratio; and learning means for learning the calculation of the likelihood ratio using a loss function in which the likelihood ratio increases when the correct class to which the sequential data belongs is in the numerator of the likelihood ratio, and decreases when the correct class is in the denominator of the likelihood ratio.

[0149] (Note 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, characterized in that the learning means performs the learning using a loss function that takes into account the likelihood ratio of N × (N-1) patterns, where the denominator is the likelihood of belonging to one of the N classes (where N is a natural number) that are classification candidates for the sequence data, and the numerator is the likelihood of belonging to another class.

[0150] (Note 3) The information processing system described in Appendix 3 is the information processing system described in Appendix 2, characterized in that the learning means performs the learning using a loss function that takes into account the likelihood ratio of some of the N × (N-1) patterns.

[0151] (Note 4) The information processing system described in Appendix 4 is the information processing system described in Appendix 3, characterized in that the learning means performs the learning using a loss function that takes into account the likelihood ratio in which the correct answer class is in the numerator among the N × (N-1) patterns.

[0152] (Note 5) The information processing system described in Appendix 5 is the information processing system described in any one of Appendix 1 to 4, characterized in that the loss function includes a sigmoid function as a nonlinear function acting on the likelihood ratio.

[0153] (Note 6) The information processing system described in Appendix 6 is the information processing system described in any one of Appendix 1 to 4, characterized in that the loss function includes a logistic function as a nonlinear function acting on the likelihood ratio.

[0154] (Note 7) The information processing system described in Appendix 7 is the information processing system described in any one of Appendix 1 to 6, characterized in that the likelihood ratio is an integrated likelihood ratio calculated by considering multiple individual likelihood ratios calculated based on two consecutive elements included in the series data.

[0155] (Note 8) The information processing system described in Appendix 8 is the information processing system described in Appendix 7, characterized in that the acquisition means sequentially acquires a plurality of elements included in the series data, and the calculation means calculates a new integrated likelihood ratio using the individual likelihood ratio calculated based on the newly acquired elements and the integrated likelihood ratio calculated in the past.

[0156] (Note 9) The information processing method described in Appendix 9 is characterized by acquiring multiple elements included in sequential data, calculating a likelihood ratio indicating the likelihood of the class to which the sequential data belongs based on at least two consecutive elements among the multiple elements, classifying the sequential data into at least one of a plurality of candidate classes based on the likelihood ratio, and performing learning regarding the calculation of the likelihood ratio using a loss function in which the likelihood ratio increases when the correct class to which the sequential data belongs is in the numerator of the likelihood ratio, and decreases when the correct class is in the denominator of the likelihood ratio.

[0157] (Note 10) The computer program described in Appendix 10 is characterized by the following: obtaining multiple elements contained in sequential data; calculating a likelihood ratio indicating the likelihood of the class to which the sequential data belongs based on at least two consecutive elements among the multiple elements; classifying the sequential data into at least one of a plurality of candidate classes based on the likelihood ratio; and operating the computer to learn how to calculate the likelihood ratio using a loss function in which the likelihood ratio increases when the correct class to which the sequential data belongs is in the numerator of the likelihood ratio, and decreases when the correct class is in the denominator of the likelihood ratio.

[0158] (Note 11) The recording medium described in Appendix 11 is a recording medium characterized by having the computer program described in Appendix 10 recorded on it. [Explanation of Symbols]

[0159] 1. Information Processing System 11 processors 14 Storage device 10 Classification device 50 Data Acquisition Unit 100 Likelihood Ratio Calculation Unit 110 First Calculation Unit 111 Individual Likelihood Ratio Calculation Unit 112 1st memory section 120 Second Calculation Unit 121 Integrated Likelihood Ratio Calculation Unit 122 2nd memory section 200 Classification Section 300 Learning Department

Claims

1. A means for obtaining multiple elements included in sequential data, A calculation means for calculating a likelihood ratio that indicates the likelihood of the class to which the sequence data belongs, based on at least two consecutive elements among the plurality of elements, A classification means for classifying the sequence data into at least one of three or more candidate classes based on the likelihood ratio, A learning means that adjusts the parameters of the model for determining the likelihood ratio so that the value of the loss function calculated using the likelihood ratio becomes small. Equipped with, The loss function does not consider the case where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes smaller when the ground truth class to which the series data belongs is in the numerator and not in the denominator of the likelihood ratio, and when the ground truth class to which the series data belongs is in the denominator and not in the numerator of the likelihood ratio. An information processing system characterized by the following:

2. The information processing system according to claim 1, characterized in that the loss function used by the learning means is a loss function that takes into account all of the likelihood ratios of N × (N-1) patterns, where the denominator is the likelihood of belonging to one of the N classes (where N is a natural number of 3 or more) that are classification candidates for the sequence data, and the numerator is the likelihood of belonging to another class.

3. The information processing system according to claim 1, characterized in that the loss function used by the learning means is a loss function that takes into account some of the likelihood ratios among N × (N-1) patterns, where the denominator is the likelihood of belonging to one of the N classes (where N is a natural number of 3 or more) that are classification candidates for the sequence data, and the numerator is the likelihood of belonging to another class.

4. The information processing system according to claim 3, characterized in that the loss function used by the learning means is a loss function that takes into account the likelihood ratio in which the correct answer class is in the numerator among the N × (N-1) patterns.

5. The information processing system according to any one of claims 1 to 4, characterized in that the loss function includes a sigmoid function as a nonlinear function acting on the likelihood ratio.

6. The information processing system according to any one of claims 1 to 5, characterized in that the likelihood ratio is a combined likelihood ratio calculated by considering multiple individual likelihood ratios calculated based on two consecutive elements included in the series data.

7. The acquisition means sequentially acquires multiple elements included in the series data, The calculation means calculates a new integrated likelihood ratio using the individual likelihood ratios calculated based on the newly acquired elements and the integrated likelihood ratio calculated in the past. The information processing system according to feature 6.

8. Retrieve multiple elements contained in the series data, Based on at least two consecutive elements among the aforementioned plurality of elements, a likelihood ratio indicating the likelihood of the class to which the sequence data belongs is calculated. Based on the likelihood ratio, the series data is classified into at least one of the three or more candidate classes. The parameters of the model used to determine the likelihood ratio are adjusted so that the value of the loss function calculated using the likelihood ratio becomes small. The loss function does not consider the case where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes smaller when the ground truth class to which the series data belongs is in the numerator and not in the denominator of the likelihood ratio, and when the ground truth class to which the series data belongs is in the denominator and not in the numerator of the likelihood ratio. An information processing method characterized by the following:

9. Retrieve multiple elements contained in the series data, Based on at least two consecutive elements among the aforementioned plurality of elements, a likelihood ratio indicating the likelihood of the class to which the sequence data belongs is calculated. Based on the likelihood ratio, the series data is classified into at least one of the three or more candidate classes. The parameters of the model used to determine the likelihood ratio are adjusted so that the value of the loss function calculated using the likelihood ratio becomes small. The loss function does not consider the case where the numerator and denominator of the likelihood ratio have the same likelihood, and the value of the loss function becomes smaller when the ground truth class to which the series data belongs is in the numerator and not in the denominator of the likelihood ratio, and when the ground truth class to which the series data belongs is in the denominator and not in the numerator of the likelihood ratio. A computer program characterized by causing a computer to operate in a certain manner.

Images