Information processing device and method

Abstract

To acquire a prediction basis that can be easily used about a prediction related to a moving image.SOLUTION: An information processing device includes: a detection amount calculation part for calculating a prescribed detection amount for each time point on the basis of data of a moving image; a totaling section calculation part for calculating totaling sections into which the moving image and / or sound are partitioned by time, on the basis of data of sound corresponding to the moving image; a feature amount calculation part for calculating a feature amount for each totaling section on the basis of the detection amount for each time point; a prediction part for executing prediction on a prescribed item on the basis of the feature amount for each totaling section; and outputting a prediction result and contribution degrees of the feature amounts with respect to the prediction result; and an each-section-contribution-degree calculation part for calculating the contribution degrees of the totaling sections with respect to the prediction result on the basis of the contribution degrees of the feature amounts, the detection amount for each time point, and the totaling sections.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technology that makes it possible to obtain the basis for predictions made by a prediction model. [Background technology]

[0002] The internal workings of a predictive model obtained through machine learning are unknown, so even if a prediction result is obtained, it is difficult to identify the basis for that prediction result.

[0003] Patent Document 1 discloses a method for extracting a first feature that positively contributes to a prediction result by a prediction model and a second feature that negatively contributes to a prediction result by the prediction model from the features of input data input to a prediction model composed of a nonlinear model.

[0004] Non-Patent Document 1 discloses a method for calculating the importance of each feature value together with the prediction result when a feature value calculated from prediction target data is input to a model to obtain a predetermined prediction result. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication WO2019 / 130974 [Non-patent literature]

[0006] [Non-Patent Document 1] “A Unified Approach to Interpreting Model Predictions”, Scott M. Lundberg, Su-In Lee, Internet: Retrieved April 20, 2021, URL: https: / / arxiv.org / abs / 1705.07874 Summary of the Invention [Problem to be solved by the invention]

[0007] For example, when making some judgment about an interviewee based on a video of an interviewer interviewing the interviewee, a number of feature amounts are calculated from the video of the interview. The feature amounts are extracted, for example, from the facial expressions and movements of the interviewee. A predetermined judgment (prediction) is made based on the information of the calculated feature amounts. The judgment is, for example, an evaluation of the interviewee during the interview. Such a video of an interview may be of a certain length of time.

[0008] When making predictions from long videos, even if the importance of each feature is known, it is not easy for users such as interviewers and judges to use it. For example, to check whether a prediction based on the importance of each feature is valid, a user needs to watch a long video.

[0009] One object of the present disclosure is to provide a technology that makes it easy to know the basis for predictions regarding videos. [Means for solving the problem]

[0010] An information processing device according to one aspect included in the present disclosure has a detection amount calculation unit that calculates a predetermined detection amount for each time period based on video data; an aggregation time window calculation unit that calculates an aggregation time window that divides the video and / or audio by time based on audio data corresponding to the video; a feature calculation unit that calculates a feature for each aggregation time window based on the detection amount for each time period; a prediction unit that performs prediction of a predetermined item based on the feature for each aggregation time window, and outputs a prediction result and a contribution of the feature to the prediction result; and a per-interval contribution calculation unit that calculates the contribution of the feature, the detection amount for each time period, and the aggregation time window. Effect of the Invention

[0011] According to one aspect of the present disclosure, it is possible to obtain prediction basis that is easy to use for predictions regarding videos. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a functional configuration of an information processing device according to a first embodiment. [Diagram 2] 2 is a block diagram showing a hardware configuration of the information processing device shown in FIG. 1. [Diagram 3] FIG. 2 is a diagram illustrating the contribution of each feature value for each time window output from the prediction unit shown in FIG. 1. [Figure 4] FIG. 2 is a diagram illustrating the contribution degree for each time window calculated by the time window contribution degree calculation unit shown in FIG. 1. [Diagram 5] 2 is a flowchart for explaining a process in a time contribution degree calculation unit shown in FIG. [Figure 6] 2 is a diagram showing a time-based contribution degree table outputted from a time-based contribution degree calculation unit shown in FIG. 1. FIG. [Figure 7] 2 is a diagram showing an example of a screen showing a contribution degree displayed on the prediction basis display device shown in FIG. 1. FIG. [Figure 8] FIG. 11 is a block diagram showing a functional configuration of an information processing device according to a second embodiment. [Figure 9] FIG. 9 is a diagram illustrating feature amounts for each prediction time window calculated by a first feature amount calculation unit shown in FIG. 8. [Figure 10] 9 is a diagram illustrating feature amounts calculated by a second feature amount calculator shown in FIG. 8 for each time window for contribution degree calculation. FIG. [Figure 11] 9 is a diagram showing the contribution degree output from the prediction unit shown in FIG. 8. FIG. [Figure 12] 9 is a diagram showing similarities calculated by a similarity calculation unit shown in FIG. 8. FIG. [Figure 13] FIG. 9 is a diagram illustrating the contribution of a time window for contribution calculation calculated by the sub-time window contribution calculation unit shown in FIG. 8. [Figure 14] 9 is a diagram showing the contribution degree for each time calculated by a time contribution degree calculation unit shown in FIG. 8. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] (First embodiment)

[0015] FIG. 1 is a block diagram showing a functional configuration of an information processing device according to a first embodiment.

[0016] As shown in FIG. 1, the information processing device of this embodiment is a prediction basis display device 1 having a detection amount calculation unit 10, an aggregation time window calculation unit 20, a feature amount calculation unit 30, a prediction unit 40, and an interval contribution calculation unit 50.

[0017] FIG. 2 is a block diagram showing a hardware configuration of the information processing device shown in FIG.

[0018] The hardware configuration of the information processing device shown in FIG. 1 is, as shown in FIG. 2, a processor 61, a main memory 62, a storage device 63, a communication device 64, an input device 65, and a display device 66 connected via a bus 67.

[0019] The processor 61 executes software programs read from the storage device 63 to the main memory 42 to realize the functions shown in FIG.

[0020] The communication device 64 is for connecting to other devices.

[0021] The input device 65 is for inputting information into various tables set up in the storage device 63 .

[0022] The display device 66 displays and outputs the results of execution by the processor 61 .

[0023] The function of each block of the prediction basis display device 1 configured as shown in FIG. 1 will be described below together with the overall processing.

[0024] The detection amount calculation unit 10 calculates a predetermined detection amount for each time based on video data input via the input device 65. For example, when the prediction basis display device 1 is used to judge an interviewee in an interview, the number of blinks made by the interviewee is calculated as the detection amount. The detection amount may also be vital data such as heart rate. It may also be data that quantifies gestures such as nodding and hand movements. It may also be data that quantifies emotions such as happiness or sadness obtained from facial expressions. The video data input via the input device 65 is data that includes frame images at a predetermined frame rate.

[0025] The time window calculation unit 20 calculates time windows that divide the video and / or audio based on the audio data corresponding to the video input via the input device 65. For example, a 30-second period including the time when the word of interest is uttered may be extracted as the time window. The time window may be 30 seconds from 15 seconds before to 15 seconds after the time when the word of interest is uttered. The time windows may overlap with each other. There may be a gap between the two time windows. Alternatively, if there is a gap between the extracted time windows and the other time windows, the gap may be treated as one time window. The audio data input via the input device 65 may be, for example, data obtained by converting a person's speech into text. Alternatively, the audio data may be information extracted from the text data that associates spoken words with the times when the words were uttered. The words referred to here may be, for example, notable words that are registered in a dictionary in advance.

[0026] The feature amount calculation unit 30 calculates the feature amount for each time interval calculated by the time window calculation unit 20, based on the detection amount for each time period calculated by the detection amount calculation unit 10. For example, the feature amount is calculated as the value of a statistical amount of the detection amount for the time interval. More specifically, the feature amount is each of the average value, variance value, maximum value, minimum value, etc. of the number of blinks per unit time in the time interval. Alternatively, the feature amount may include the total number of blinks in the time interval. Of course, the feature amount may be one or more of these, or all of them.

[0027] The prediction unit 40 predicts a predetermined item based on the feature for each time window calculated by the feature calculation unit 30, and outputs the prediction result and the contribution of the feature to the prediction result. For example, the prediction result may be the possibility of promotion or an evaluation value in an interview to determine promotion in a company. The contribution of a feature indicates the extent to which the feature has influenced the prediction result.

[0028] FIG. 3 is a diagram showing the contribution of each feature value for each time window outputted from the prediction unit 40 shown in FIG.

[0029] As shown in FIG. 3, the prediction unit 40 shown in FIG. 1 outputs the contribution of the feature calculated by the feature calculation unit 30 to the detection amount calculated by the detection amount calculation unit 10 for each of the counting time intervals calculated by the counting time interval calculation unit 20.

[0030] As shown in FIG. 1, the interval contribution calculation unit 50 includes an interval contribution calculation unit 51 and a time-of-day contribution calculation unit 52.

[0031] The time window contribution calculation unit 51 calculates the contribution of the time window to the prediction result output from the prediction unit 40 based on the contribution of the feature output from the prediction unit 40, the detection amount for each time calculated by the detection amount calculation unit 10, and the time window calculated by the time window calculation unit 20.

[0032] FIG. 4 is a diagram showing the contribution degree for each time window calculated by the time window contribution degree calculation unit 51 shown in FIG.

[0033] As shown in Fig. 4, the time window contribution calculation unit 51 shown in Fig. 1 calculates the contribution for each time window number output from the prediction unit 40. In this way, the contribution of a time window indicates the degree to which the time window (the multiple feature quantities in the time window as a whole) has influenced the prediction result. For example, the contribution of a time window may be the sum of the contributions of the feature quantities related to the detection quantities detected in that time window.

[0034] The time-of-day contribution calculation unit 52 calculates the contribution of each time period based on the contribution of each time window calculated by the time window contribution calculation unit 51 .

[0035] FIG. 5 is a flowchart for explaining the process in the time-based contribution degree calculation unit 52 shown in FIG.

[0036] The time contribution calculation unit 52 first obtains the start time of a time window and the end time of the time window plus one second from the feature value contribution table showing the contribution for each time window calculated by the time window contribution calculation unit 51, and lists these times as contribution calculation target times in the time contribution table (step S1). Note that at this point in time, only the time is recorded in the time contribution table.

[0037] Next, the time-based contribution degree calculation unit 52 acquires one target time for contribution degree calculation from the time-based contribution degree table (step S2).

[0038] The time-based contribution calculation unit 52 then extracts rows of the time window that includes the acquired contribution calculation time from the time-based contribution table, and records the sum of the contributions of the extracted rows as the contribution of that time window in the time-based contribution table (step S3).

[0039] Then, the processing in steps S2 and S3 is repeated until there are no unprocessed times for contribution calculation in the time-by-time contribution table (step S4), and when there are no unprocessed times for contribution calculation in the time-by-time contribution table, the time-by-time contribution calculation unit 52 outputs the time-by-time contribution table (step S5).

[0040] FIG. 6 is a diagram showing a time-based contribution degree table outputted from the time-based contribution degree calculation unit 52 shown in FIG.

[0041] As shown in Fig. 6, a time-based contribution table showing the contribution degree for each time is output from the time-based contribution degree calculation unit 52 shown in Fig. 1. This makes it possible to draw a step-like contribution degree graph.

[0042] FIG. 7 is a diagram showing an example of a screen showing the contribution degree displayed on the prediction basis display device 1 shown in FIG.

[0043] As shown in Figure 7, the screen displayed on the display unit 66 of the prediction basis display device 1 shown in Figure 1 displays a playback image 71 of the video input to the detection amount calculation unit 10 via the input device 65 of the prediction basis display device 1, a slide bar 72 for controlling the forwarding of the playback image 71, and a contribution degree according to the playback time as a contribution degree graph 73.

[0044] In this way, a video is divided into time windows and the contribution of each time window to the prediction result is calculated, so that easily usable information on the contribution of each time window can be obtained as the basis for prediction results for the video.

[0045] (Second embodiment)

[0046] FIG. 8 is a block diagram showing a functional configuration of an information processing device according to the second embodiment.

[0047] As shown in FIG. 8, the information processing device of this embodiment is a prediction basis display device 101 in which the configurations of the time window calculation unit 120, the feature calculation unit 130, and the interval contribution calculation unit 150 are different from those shown in FIG. 1.

[0048] The functions of each block of the prediction basis display device 101 configured as shown in FIG. 8 will be described below together with the processing thereof, focusing only on the parts that are different from those shown in FIG.

[0049] The aggregation interval calculation unit 120 is composed of a aggregation interval for prediction calculation unit 121 and a aggregation interval for contribution calculation calculation unit 122. The aggregation interval for prediction calculation unit 121 calculates an aggregation interval for prediction for tallying the learning data learned by the prediction unit 140 and input to the prediction model constructed by learning the learning data. The aggregation interval for contribution calculation calculation unit 122 calculates an aggregation interval for contribution calculation, which is an aggregation interval for calculating each contribution, by dividing the aggregation interval for prediction calculated by the aggregation interval for prediction calculation unit 121 into multiple intervals. Note that hereinafter, the aggregation interval for prediction will simply be referred to as an aggregation interval, and the aggregation interval for contribution calculation will sometimes be referred to as a sub-aggregation interval.

[0050] For example, the prediction interval calculation unit 121 calculates a time interval of a predetermined duration including a time point at which a predetermined word of interest is uttered as the prediction interval, and the contribution degree calculation calculation unit 122 calculates the contribution degree calculation interval by dividing the prediction interval calculated by the prediction interval calculation unit 121 into multiple intervals.

[0051] Specifically, the prediction interval calculation unit 121 may set a 30-second interval including the time when the word of interest is uttered as the prediction interval, and the contribution calculation interval calculation unit 122 may divide the prediction interval into intervals of 10 seconds to set them as contribution calculation intervals.

[0052] This allows the time period around the time when words related to the item to be predicted from the video are spoken to be used as the prediction interval, and then further divided to create a contribution calculation interval that represents the contribution in a shorter interval.

[0053] 8, the feature amount calculation unit 130 is composed of a first feature amount calculation unit 131 and a second feature amount calculation unit 132. The first feature amount calculation unit 131 calculates a feature amount for each aggregation window for prediction calculated by the prediction window calculation unit 121, based on the detection amount for each time point calculated by the detection amount calculation unit 110. The second feature amount calculation unit 132 calculates a feature amount for each aggregation window for contribution degree calculation calculated by the contribution degree calculation window calculation unit 122, based on the detection amount for each time point calculated by the detection amount calculation unit 110.

[0054] FIG. 9 is a diagram showing feature amounts for each of the prediction time windows calculated by the first feature amount calculation unit 131 shown in FIG.

[0055] As shown in FIG. 9 , the first feature amount calculation unit 131 shown in FIG. 8 outputs a value calculated by the first feature amount calculation unit 131 of a predetermined statistical amount for the detection amount calculated by the detection amount calculation unit 110 in association with each of the counting intervals calculated by the counting interval calculation unit 121 for prediction.

[0056] FIG. 10 is a diagram showing feature amounts for each time window for contribution degree calculation calculated by the second feature amount calculation unit 131 shown in FIG.

[0057] In the second feature calculation unit 132 shown in FIG. 8, as shown in FIG. 10, for each sub-counting time interval (counting time interval for contribution calculation) calculated by the counting time interval calculation unit 122 for contribution calculation, a value (i.e., a feature) calculated by the second feature calculation unit 132 of a predetermined statistical amount for the detection amount calculated by the detection amount calculation unit 110 is associated and output.

[0058] The total number of times may be calculated as a value of the statistic calculated by the first feature amount calculation section 131 and the second feature amount calculation section 132.

[0059] The prediction unit 140 uses a prediction model constructed by learning the learning data to output a prediction result and the contribution of the feature amount to the prediction result.

[0060] FIG. 11 is a diagram showing the contribution degree output from the prediction section 140 shown in FIG.

[0061] As shown in FIG. 11 , the prediction unit 140 shown in FIG. 8 outputs, for each time window (time window for prediction) calculated by the time window calculation unit 121 for prediction, the contribution of the feature calculated by the first feature calculation unit 131 to the detection amount calculated by the detection amount calculation unit 110.

[0062] As shown in FIG. 8, the interval contribution calculation unit 150 includes a time window contribution calculation unit 151, a time-of-day contribution calculation unit 152, a similarity calculation unit 153, and a sub-window contribution calculation unit 154.

[0063] The time window contribution calculation unit 151 calculates the contribution of the prediction time window based on the detection amount for each time instant calculated by the detection amount calculation unit 110, the prediction time window calculated by the prediction time window calculation unit 121, and the contribution of the feature amount calculated by the prediction unit 140.

[0064] The similarity calculation unit 153 calculates the similarity between the feature amount (the value of the statistical amount) of the calculation time window for prediction calculated by the first feature amount calculation unit 131 and the feature amount (the value of the statistical amount) of the calculation time window for contribution degree calculation calculated by the first feature amount calculation unit 132. Here, the similarity is expressed by distance. The distance is the difference between absolute values. The shorter the distance, the higher the similarity.

[0065] FIG. 12 is a diagram showing the similarity calculated by the similarity calculation unit 153 shown in FIG.

[0066] As shown in Figure 12, in the similarity calculation unit 153 shown in Figure 8, for each sub-counting interval (counting interval for contribution calculation) calculated by the counting interval calculation unit 122 for contribution calculation, the detection amount calculated by the detection amount calculation unit 110, the feature amount calculated by the second feature calculation unit 132, and the distance which is the difference in absolute value between the value output from the first feature calculation unit 131 and the value output from the second feature calculation unit 132 are output in correspondence with each other.

[0067] The sub-aggregation window contribution calculation unit 154 calculates the contribution of the aggregation window for contribution calculation from the contribution of the aggregation window for prediction output from the prediction unit 140, based on the similarity calculated in the similarity calculation unit 153 between the features of the aggregation window for prediction calculated by the first feature calculation unit 131 and the features of the aggregation window for contribution calculation calculated by the second feature calculation unit 132.

[0068] For example, the sub-window contribution calculation unit 154 calculates the contribution of the window for contribution calculation by correcting the contribution of the window for prediction so that the contribution is larger as the similarity calculated by the similarity calculation unit 153 is higher.

[0069] In this way, the contribution of the aggregation time interval for contribution calculation calculated by the aggregation time interval calculation unit for contribution calculation 122 can be calculated based on the similarity to the contribution of the aggregation time interval for prediction calculated by the aggregation time interval calculation unit for prediction 121.

[0070] Specifically, the subtime window contribution calculation unit 154 calculates the contribution i = Contribution rate for the aggregation interval + Contribution rate correction value i In this case, the contribution correction value is obtained by solving the following constraints.

[0071] (constraints) The longer the distance, the smaller the correction value. For sub-interval numbers i and j, Maximum correction value > Contribution correction value i >Contribution correction value j ,ij distance i <distance j Maximum correction value > Contribution correction value i = Contribution correction value j ,ij distance i = distance j The sum of the correction values ​​weighted by time is 0. Σ i time i × Contribution correction value i =0

[0072] In this case, the sub-window contribution calculation unit 154 may calculate the contribution of the counting window for contribution calculation so that the average value of the contribution of the counting window for contribution calculation included in the counting window for prediction, weighted by the time width, is equal to the contribution of the counting window for prediction. As a result, the contribution of the counting window for prediction as a whole is the contribution calculated for the counting window for prediction, and the contribution of the counting window for contribution calculation included in the counting window for prediction can be presented for shorter periods of time.

[0073] FIG. 13 illustrates the contributions of time windows calculated by the sub-time window contribution calculation unit 154 shown in FIG.

[0074] As shown in FIG. 13, the sub-window contribution calculation unit 154 shown in FIG. 8 calculates the contribution for each window for contribution calculation calculated by the window calculation unit for contribution calculation 122.

[0075] The time-of-day contribution calculation unit 152 calculates the contribution for each time period based on the contribution for each aggregation window for prediction calculated by the aggregation window contribution calculation unit 51 and the contribution for each aggregation window for contribution calculation calculated by the sub-aggregation window contribution calculation unit 154.

[0076] FIG. 14 is a diagram showing the contribution degree for each time calculated by the time contribution degree calculation unit 152 shown in FIG.

[0077] As shown in FIG. 14, in the time-based contribution degree calculation section 152 shown in FIG. 8, the contribution degree for each time of the moving image input to the detection amount calculation section 110 is calculated.

[0078] In this way, the contribution degree for each aggregation time window calculated by the aggregation time window contribution calculation unit 151, the contribution degree for each aggregation time window for contribution degree calculation calculated by the sub-aggregation time window contribution calculation unit 154, and the contribution degree for each time period calculated by the time period contribution calculation unit 152 are displayed on the display device 66 of the prediction basis display device 101.

[0079] In this manner, in this embodiment, by using an aggregation interval for prediction, which is a coarse unit of time, it is possible to reduce the amount of learning data used to train the prediction model, and by using an aggregation interval for contribution calculation, which is a fine unit of time, when a user looks back on a prediction result, a short time span of a video that is the basis for the prediction result is shown, enabling the user to easily check the video in a pinpoint manner, for example.

[0080] If the contribution degree calculated by the interval contribution degree calculation unit 150 can be either positive or negative, the contribution degrees in the prediction time window and / or the contribution degree calculation time window may be recorded in separate tables for time windows with positive contribution degrees and for time windows with negative contribution degrees. In this case, it becomes possible to present to the user a distinction between time periods that positively influenced the prediction result and time periods that negatively influenced the prediction result.

[0081] Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and these embodiments may be used in combination or some of the configurations may be changed within the scope of the technical concept of the present invention. [Explanation of symbols]

[0082] 1,101... prediction basis display device, 10,110... detection amount calculation unit, 20,120... calculation unit for time interval, 30,130... feature amount calculation unit, 40,140... prediction unit, 50,150... interval contribution calculation unit, 51,151... calculation unit for time interval contribution unit, 52,152... time-based contribution calculation unit, 61... processor, 62... main memory, 63... storage device, 64... communication device, 65... input device, 66... ​​display device, 67... bus, 71... played image, 72... slide bar, 73... contribution amount graph, 121... calculation unit for time interval for prediction, 122... calculation unit for time interval for contribution amount calculation, 131... first feature amount calculation unit, 132... second feature amount calculation unit, 153... similarity calculation unit, 154... calculation unit for contribution amount per sub-time interval

Claims

1. a detection amount calculation unit that calculates a predetermined detection amount for each time based on video data; a time window calculation unit that calculates a time window into which the video and / or audio is divided based on data of the audio corresponding to the video; a feature amount calculation unit that calculates a feature amount for each time window based on the detected amount for each time point; a prediction unit that performs prediction of a predetermined item based on the feature values ​​for each time window, and outputs a prediction result and a contribution of the feature values ​​to the prediction result; an interval contribution calculation unit that calculates a contribution of the time window to the prediction result based on the contribution of the feature amount, the detected amount for each time point, and the time window; An information processing device having the above configuration.

2. the prediction unit outputs the prediction result and a contribution degree of the feature amount by using a prediction model constructed by learning learning data; the aggregation interval calculation unit calculates an aggregation interval for prediction used to aggregate the learning data and the input to the prediction model, and an aggregation interval for contribution degree calculation, the aggregation interval for prediction being divided into a plurality of intervals for calculating the contribution degree of each of the learning data and the input to the prediction model; the feature amount calculation unit calculates a feature amount for each of the prediction time windows and a feature amount for each of the contribution degree calculation time windows; the interval contribution calculation unit calculates a contribution of the aggregation interval for prediction based on the contribution of the feature amount, the detected amount for each time, and the aggregation interval for prediction, and calculates a contribution of the aggregation interval for contribution calculation from the contribution of the aggregation interval for prediction based on a similarity between the feature amount of the aggregation interval for prediction and the feature amount of the aggregation interval for contribution calculation. The information processing device according to claim 1 .

3. the time interval calculation unit calculates a time interval of a predetermined duration including a time when a predetermined word of interest is spoken as a prediction time interval, and calculates the contribution degree calculation time interval by dividing the prediction time interval into a plurality of intervals. The information processing device according to claim 2 .

4. the interval contribution calculation unit calculates the contribution of the aggregation time window for contribution calculation by correcting the contribution of the aggregation time window for prediction so that the contribution becomes larger as the similarity is higher. The information processing device according to claim 2 .

5. the interval contribution calculation unit calculates the contribution of the aggregation window for contribution calculation such that an average value of the contribution of the aggregation window for contribution calculation, which is included in the aggregation window for prediction, weighted by a time width of the contribution is equal to the contribution of the aggregation window for prediction. The information processing device according to claim 4.

6. when the contribution degree can be a positive value or a negative value, the interval-by-interval contribution calculation unit records the contribution degrees in the prediction time interval and / or the contribution degree calculation time interval into separate tables, one for which the contribution degree is a positive value and one for which the contribution degree is a negative value. The information processing device according to claim 2 .

7. Calculate a predetermined detection amount for each time based on the video data, calculating an aggregation time interval that divides the video and / or audio by time based on audio data corresponding to the video; calculating a feature value for each time window based on the detected amount for each time instant; executing a prediction of a predetermined item based on the feature values ​​for each time window, and outputting a prediction result and a contribution of the feature values ​​to the prediction result; calculating a contribution of the time window to the prediction result based on the contribution of the feature amount, the detected amount for each time point, and the time window; An information processing method implemented by a computer.

Images