Information processing device, information processing method, and recording medium

Abstract

An information processing device (10) comprises: an acquisition means (50) that sequentially acquires a plurality of elements included in sequential data; a likelihood ratio calculation means (100) that, each time an element is acquired, calculates, on the basis of two or more adjacent elements, a likelihood ratio indicating the likelihood of a class to which the sequential data may belong; a classification means (150) that classifies the sequential data into any of a plurality of classes on the basis of the likelihood ratio; and a learning means (200) that carries out training related to the calculation of the likelihood ratio by using a sampling cost loss function related to the cost of acquiring the element. Such an information processing device makes it possible to more appropriately classify sequential data because training can carried out with consideration of the cost when an element is acquired.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Known examples of this type of device include one that classifies sequential data using likelihood ratios. For example, Patent Document 1 discloses a device that sequentially acquires and analyzes multiple elements included in the sequential data to classify the sequential data into one of multiple predetermined classes.

[0003] International Publication No. 2020 / 194497

[0004] This disclosure aims to improve upon the related art discussed above.

[0005] One aspect of the information processing device disclosed herein includes an acquisition means for sequentially acquiring multiple elements included in sequence data; a likelihood ratio calculation means for calculating, each time an element is acquired, a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on two or more adjacent elements; a classification means for classifying the sequence data into one of multiple classes based on the likelihood ratio; and a learning means for performing learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

[0006] One aspect of the information processing method disclosed herein sequentially acquires multiple elements included in sequence data, and for each acquired element, calculates a likelihood ratio indicating the likelihood of the class to which the sequence data belongs based on two or more adjacent elements, classifies the sequence data into one of multiple classes based on the likelihood ratio, and performs learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

[0007] One aspect of the recording medium of this disclosure has recorded thereon a computer program that causes a computer to execute an information processing method, which includes sequentially acquiring multiple elements included in sequence data, calculating, for each acquired element, a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on two or more adjacent elements, classifying the sequence data into one of multiple classes based on the likelihood ratio, and performing learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

[0008] 1 is a block diagram showing the hardware configuration of a first information processing device; FIG. 2 is a block diagram showing the functional configuration of the first information processing device; FIG. 3 is a flowchart showing the flow of a classification operation in the first information processing device; FIG. 4 is a flowchart showing the flow of a learning operation in the first information processing device; FIG. 5 is a block diagram showing the functional configuration of a second information processing device; FIG. 6 is a flowchart showing the flow of a learning operation in the second information processing device; and FIG. 7 is a graph showing an example of decision-making time in the second information processing device.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first embodiment will be described with reference to FIGS. 1 to 4. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing apparatus.

[0012] 1, a first information processing device 10 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing device 10 may further include an input device 15 and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected to each other via a data bus 17. The data bus 17 may be an interface other than a data bus (for example, a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing device 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, functional blocks that execute various processes for classifying sequence data are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the information processing device 10.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that is to be saved long-term by the information processing device 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the information processing device 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as a mobile terminal such as a smartphone or a tablet. The input device 15 may also be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the information processing device 10 to the outside. For example, the output device 16 may be a display device (e.g., a display) that can display information related to the information processing device 10. The output device 16 may also be a speaker or the like that can output information related to the information processing device 10 as audio. The output device 16 may be configured as a mobile terminal such as a smartphone or a tablet.

[0020] 1 shows an example of an information processing device 10 including multiple devices, but all or some of the functions may be realized by a single device. Such an information processing device may be configured to include only the above-mentioned processor 11, RAM 12, and ROM 13, and the other components (i.e., the storage device 14, input device 15, output device 16, etc.) may be provided by an external device connected to the information processing device 10. Furthermore, some of the calculation functions of the information processing device 10 may be realized by an external device (e.g., an external server, a cloud, etc.).

[0021] (Functional Configuration) Next, the functional configuration of the first information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the first information processing device.

[0022] 2, the first information processing device 10 is configured as a device for classifying sequential data. For example, the first information processing device 10 may be configured as a device that acquires images of time-series data and classifies the types of objects included in the images. The first information processing device 10 is configured to include, as components for realizing its functions, an acquisition unit 50, a likelihood ratio calculation unit 100, a classification unit 150, and a learning unit 200. Each of the acquisition unit 50, the likelihood ratio calculation unit 100, the classification unit 150, and the learning unit 200 may be a processing block realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0023] The acquisition unit 50 is configured to acquire sequence data. Sequence data here refers to data including multiple elements arranged in a predetermined order, such as time-series data. More specific examples of sequence data include, but are not limited to, video data, audio data, or subdivided image data. The acquisition unit 50 is configured to sequentially acquire multiple elements included in the sequence data. For example, the acquisition unit 50 may be configured to acquire elements included in the sequence data one by one. The acquisition unit 50 may acquire data directly from any data acquisition device (e.g., a camera, a microphone, etc.), or may read data previously acquired by a data acquisition device and stored in storage, etc. When acquiring data from a camera, the acquisition unit 50 may be configured to acquire data from each of multiple cameras.

[0024] The likelihood ratio calculation unit 100 is configured to calculate a likelihood ratio from the sequence data acquired by the acquisition unit 50. The "likelihood ratio" here is an index indicating to which of multiple candidate classes the sequence data belongs (more specifically, an index indicating the likelihood of the class to which the sequence data belongs). When the acquisition unit 50 sequentially acquires elements, the likelihood ratio calculation unit 100 calculates a likelihood ratio each time an element is acquired. That is, the likelihood ratio calculation unit 100 calculates a likelihood ratio when the acquisition unit 50 acquires an element, and calculates a new likelihood ratio when the acquisition unit 50 acquires another element. The likelihood ratio calculation unit 100 calculates a likelihood ratio based on two or more consecutive elements among the elements sequentially acquired by the acquisition unit 50. For example, the likelihood ratio calculation unit 100 may calculate a likelihood ratio using a newly acquired element and a previously acquired element or a previously calculated likelihood ratio. In this case, the likelihood ratio calculation unit 100 may include a storage unit that stores previously acquired elements or previously calculated likelihood ratios. The likelihood ratio calculation unit 100 is configured, for example, by a trained neural network. The likelihood ratio calculation unit 100 is configured to be trained by a learning unit 200, which will be described later.

[0025] The classification unit 150 classifies the sequence data into one of a plurality of classes based on the likelihood ratio calculated by the likelihood ratio calculation unit 100. Specifically, the classification unit 150 classifies the sequence data into one of a plurality of classes by comparing the likelihood ratio calculated by the likelihood ratio calculation unit 100 with a preset threshold. For example, the classification unit 150 classifies the sequence data into class A when the likelihood ratio exceeds a threshold corresponding to class A. Furthermore, the classification unit 150 classifies the sequence data into class B when the likelihood ratio exceeds a threshold corresponding to class B. The classification unit 150 may classify, for example, whether a person's face in an image is a real face or a fake face (e.g., impersonation using a photograph or a 3D mask). The plurality of classes may also be three or more. In this case, the classification unit 150 may perform classification using a threshold set for each class.

[0026] The learning unit 200 is configured to be able to perform learning related to the calculation of likelihood ratios in the likelihood ratio calculation unit 100. That is, learning related to the calculation of likelihood ratios is performed so that class classification based on likelihood ratios can be performed quickly and accurately. The learning unit 200 performs learning using a loss function. Specifically, the learning unit 200 optimizes the parameters of a model used when calculating likelihood ratios using the loss function. Note that existing techniques can be appropriately adopted as a parameter optimization method using a loss function. An example of an optimization method is backpropagation, but other methods may also be used.

[0027] Furthermore, the learning unit 200 according to this embodiment particularly includes a sampling cost loss function calculation unit 210. The sampling cost loss function calculation unit 210 calculates a sampling cost loss function, which is a loss function used for learning. The sampling cost loss function is a loss function related to the cost of acquiring elements (hereinafter referred to as "sampling cost" as appropriate). As already explained, when the likelihood ratio calculation unit 100 calculates the likelihood ratios, the acquisition unit 50 sequentially acquires elements included in the sequence data, and the cost incurred in this process is the sampling cost. The sampling cost loss function may be a function including a preset cost parameter. Specific examples of the sampling cost loss function will be explained in other embodiments described later.

[0028] (Classification Operation) Next, the classification operation (i.e., the operation of classifying sequence data into one of a plurality of classes) in the first information processing device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the classification operation in the first information processing device.

[0029] 3, when the classification operation in the first information processing device 10 starts, the acquisition unit 50 first acquires one element included in the sequence data (step S101), and the likelihood ratio calculation unit 100 then calculates a likelihood ratio based on the element acquired by the acquisition unit 50 (step S102).

[0030] Next, the classification unit 150 determines whether there is a class for which the likelihood ratio calculated by the likelihood ratio calculation unit 100 exceeds a threshold value (step S103). That is, the classification unit 150 compares the likelihood ratio calculated by the likelihood ratio calculation unit 100 with the threshold value corresponding to each class that is a classification candidate, and determines whether the likelihood ratio exceeds the threshold value corresponding to any class.

[0031] If there is a class whose likelihood ratio exceeds the threshold (step S103: YES), the classifier 150 classifies the sequence data into the class whose likelihood ratio exceeds the threshold (step S104). On the other hand, if there is no class whose likelihood ratio exceeds the threshold (step S103: NO), the process is repeated from step S101. That is, the acquirer 50 acquires the next element included in the sequence data and repeats the same process as described above.

[0032] The likelihood ratio calculated by the likelihood ratio calculation unit 100 tends to approach the threshold value corresponding to the class to be classified each time an element is acquired. Therefore, by repeating the process while acquiring elements one by one as described above, sequential data can be classified into appropriate classes. In this case, for sequential data that is easy to classify, the threshold value is exceeded at an early stage, allowing the sequential data to be classified early. On the other hand, for sequential data that is difficult to classify, elements continue to be acquired until the threshold value is exceeded, allowing the sequential data to be classified accurately over a period of time.

[0033] (Learning Operation) Next, the learning operation in the first information processing device 10 (i.e., the operation when the learning unit 200 learns the likelihood ratio calculation unit 100) will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of the learning operation in the first information processing device.

[0034] 4, when the learning operation in the first information processing device 10 starts, the acquiring unit 50 first acquires sequence data (step S151). Note that the sequence data acquired here may be data prepared in advance for learning (e.g., learning data to which a correct answer label has been assigned). In other words, the sequence data used in the classification operation may be different from the sequence data used in the learning operation.

[0035] Next, the sampling cost loss function calculation unit 210 in the learning unit 200 calculates the sampling cost loss function based on the sequence data (step S152).The learning unit 200 then uses the calculated sampling cost loss function to train the likelihood ratio calculation unit 100 (step S153).

[0036] For example, the learning unit 200 calculates a gradient from the sampling cost loss function, and then changes the parameters based on the calculated gradient in a direction that reduces the sampling cost loss function, thereby progressing the learning of the likelihood ratio calculation unit 100.

[0037] (Technical Effects) Next, technical effects obtained by the first information processing device 10 will be described.

[0038] As described with reference to Figures 1 to 4, the first information processing device 10 performs learning related to likelihood ratio calculation using a sampling cost loss function. This allows for more appropriate learning because the cost of acquiring elements (in other words, the penalty for the time required for classification) is taken into consideration. Specifically, it becomes possible to accurately classify sequence data more quickly (i.e., with as few elements as possible).

[0039] Second Embodiment A second embodiment will be described with reference to Figures 5 to 7. The second embodiment differs from the first embodiment described above only in some configurations and operations, and other parts may be the same as the first embodiment. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit a description of other overlapping parts as appropriate.

[0040] (Functional Configuration) First, the functional configuration of the second information processing device 10 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the functional configuration of the second information processing device. Note that in Fig. 5, elements similar to those shown in Fig. 2 are denoted by the same reference numerals.

[0041] 5, the second information processing device 10 is configured to include, as components for realizing its functions, an acquisition unit 50, a likelihood ratio calculation unit 100, a classification unit 150, and a learning unit 200. In particular, the classification unit 150 in the second information processing device 10 includes a decision-making time output unit 155.

[0042] The decision-making time output unit 155 is configured to be able to output the decision-making time τ of the classification unit 150 to the learning unit 200. The decision-making time τ is the time when the classification unit 150 classified the sequence data (i.e., the time when the likelihood ratio exceeded the threshold). Note that in this embodiment, "time" advances each time an element is acquired. Therefore, the time corresponds to the number of elements acquired so far.

[0043] The learning unit 200 performs learning using the decision-making time τ output from the decision-making time output unit 155. Specifically, the sampling cost loss function calculation unit 210 calculates the sampling cost function L as a function of the decision-making time τ, as shown in the following formula (1): ＳＣ Calculate.

[0044] Here, θ is a cost parameter matrix, and f(θ, τ) may be, for example, θ·τ.

[0045] (Learning Operation) Next, the learning operation in the second information processing device 10 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of the learning operation in the second information processing device.

[0046] 6 , when the learning operation in the second information processing device 10 starts, the acquisition unit 50 first acquires sequence data (step S251). Note that all elements included in the sequence data may be acquired together. Then, the likelihood ratio calculation unit 100 calculates likelihood ratios up to the final time T at which all elements have been acquired (step S252).

[0047] Next, the classification unit 150 determines the decision-making time τ, which is the time when the likelihood ratio calculated by the likelihood ratio calculation unit 100 exceeds the threshold (step S253). The decision-making time τ will be specifically described below with reference to Fig. 7. Fig. 7 is a graph showing an example of the decision-making time in the second information processing device.

[0048] As shown in FIG. 7, the decision-making time τ is the time when the likelihood ratio intersects with a preset threshold (i.e., the upper or lower threshold in the figure). For example, the decision-making time τ corresponding to the sequence data A is Ａ is the time when the likelihood ratio of sequence data A intersects with the upper threshold. Ｂ is the time when the likelihood ratio of sequence data B intersects with the lower threshold. Ｃ is the time when the likelihood ratio of the sequence data C intersects with the upper threshold.

[0049] As in the above example, multiple decision-making times τ are obtained depending on the number of sequential data. That is, if there are M pieces of sequential data, M decision-making times τ are obtained. The decision-making time output unit 155 outputs all of the decision-making times τ obtained from the likelihood ratios of the sequential data to the learning unit 200.

[0050] Returning to FIG. 6, the sampling cost loss function calculation unit 210 calculates the sampling cost loss function L as a function including the decision-making time τ. ＳＣ = f(θ, τ) (step S254). Then, the learning unit 200 calculates the calculated sampling cost loss function L ＳＣ is added to the loss function and learning is performed (step S255). Here, the sampling cost loss function L ＳＣ The loss function may be learned to minimize a loss function including the sampling cost loss function L ＳＣ The terms other than are not particularly limited, but may be classification penalties. A cross-entropy loss function may be used as the classification penalty. That is, the learning unit 200 uses the sampling cost loss function L ＳＣ and a cross-entropy loss function.

[0051] (Technical Effects) Next, technical effects obtained by the second information processing device 10 will be described.

[0052] 5 to 7, in the second information processing device 10, the sampling cost loss function is calculated as a function of the decision-making time τ. In this way, learning can be performed taking into account the time required to classify sequential data, thereby enabling early classification of sequential data.

[0053] Furthermore, by training to minimize a loss function that adds a sampling cost loss function to the cross-entropy loss function, it is possible to improve not only classification accuracy but also classification speed compared to when only the cross-entropy loss function is used.

[0054] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0055] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0056] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0057] (Supplementary Note 1) The information processing device described in Supplementary Note 1 includes an acquisition means that sequentially acquires a plurality of elements included in sequence data; a likelihood ratio calculation means that calculates, each time an element is acquired, a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on two or more adjacent elements; a classification means that classifies the sequence data into one of a plurality of classes based on the likelihood ratio; and a learning means that performs learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

[0058] (Supplementary Note 2) The information processing device described in Supplementary Note 2 is the information processing device described in Supplementary Note 1, wherein the classification means classifies the sequence data when the likelihood ratio exceeds any of thresholds corresponding to each of the plurality of classes, and the sampling cost loss function is a function of the time when the likelihood ratio exceeds the threshold.

[0059] (Supplementary Note 3) The information processing device according to Supplementary Note 3 is the information processing device according to Supplementary Note 2, wherein the learning means performs learning so as to minimize a loss function including the sampling cost loss function and a cross-entropy loss function.

[0060] (Supplementary Note 4) The information processing method described in Supplementary Note 4 sequentially acquires multiple elements included in sequence data, and for each acquired element, calculates a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on two or more adjacent elements, classifies the sequence data into one of multiple classes based on the likelihood ratio, and performs learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the element.

[0061] (Supplementary Note 5) The recording medium described in Supplementary Note 5 is a recording medium having recorded thereon a computer program that causes a computer to execute an information processing method, which includes sequentially acquiring multiple elements included in sequence data, calculating, for each acquired element, a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on two or more adjacent elements, classifying the sequence data into one of multiple classes based on the likelihood ratio, and performing learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

[0062] (Supplementary Note 6) The computer program described in Supplementary Note 6 causes a computer to execute an information processing method, which sequentially acquires multiple elements included in sequence data, and for each acquired element, calculates a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on two or more adjacent elements, classifies the sequence data into one of multiple classes based on the likelihood ratio, and performs learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

[0063] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0064] REFERENCE SIGNS LIST 10 Information processing device 11 Processor 50 Acquisition unit 100 Likelihood ratio calculation unit 150 Classification unit 155 Decision-making time output unit 200 Learning unit 210 Sampling cost loss function calculation unit

Claims

1. An information processing device comprising: an acquisition means for sequentially acquiring multiple elements included in sequence data; a likelihood ratio calculation means for calculating, each time an element is acquired, a likelihood ratio indicating the likelihood of a class to which the sequence data belongs based on two or more adjacent elements; a classification means for classifying the sequence data into one of a plurality of classes based on the likelihood ratio; and a learning means for performing learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

2. The information processing device according to claim 1, wherein the classification means classifies the sequence data when the likelihood ratio exceeds any of the thresholds corresponding to each of the multiple classes, and the sampling cost loss function is a function of the time when the likelihood ratio exceeds the threshold.

3. The information processing device according to claim 2, wherein the learning means performs learning so as to minimize a loss function including the sampling cost loss function and a cross-entropy loss function.

4. An information processing method comprising: sequentially acquiring multiple elements contained in sequence data; calculating, for each element acquired, a likelihood ratio indicating the likelihood of the class to which the sequence data belongs based on two or more adjacent elements; classifying the sequence data into one of multiple classes based on the likelihood ratio; and performing learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

5. A recording medium having a computer program recorded thereon that causes a computer to execute an information processing method comprising the steps of: sequentially acquiring multiple elements contained in sequence data; calculating, for each acquired element, a likelihood ratio indicating the likelihood of the class to which the sequence data belongs based on two or more adjacent elements; classifying the sequence data into one of multiple classes based on the likelihood ratio; and performing learning related to the calculation of the likelihood ratio using a sampling cost loss function related to the cost of acquiring the elements.

Images