Information processing apparatus, information processing method, and program

Abstract

In order to provide a deduction technology that can be suitably applied even in a case where data of various lengths are inputted, an information processing device (1) comprises: a feature map generation unit (11) that generates, from input data, a plurality of feature maps for respective scales; a feature series generation unit (12) that generates, from the plurality of feature maps for respective scales, a feature series; and a feature information generation unit (13) that generates feature information by inputting the feature series into a recursive model.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program for generating feature information.

Background Art

[0002] A model that extracts features of variable-length data using the variable-length data as input data is known.

[0003] For example, Non-Patent Document 1 discloses a technique for extracting features by using variable-length speech as input, switching the input to convolutional layers with different kernel sizes between short-term and long-term contexts, and inputting the feature maps of a plurality of convolutional blocks into Multiscale Statistics Pooling.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the technique described in Non-Patent Document 1, all the feature maps input to Multiscale Statistics Pooling need to have a length of 1 or more. Therefore, in the technique described in Non-Patent Document 1, there is a lower limit on the applicable data length, and there is room for improvement in terms of handling data with various lengths.

[0006] One aspect of the present invention has been made in view of the above problems, and an example of its object is to provide an inference technique that can be suitably applied even when data having various lengths is input.

Means for Solving the Problems

[0007] An information processing apparatus according to one aspect of the present invention includes a feature map generation means for generating a plurality of scale-specific feature maps from input data, a feature sequence generation means for generating a feature sequence from the plurality of scale-specific feature maps, and a feature information generation means for generating feature information by inputting the feature sequence into a recursive model.

[0008] An information processing method according to one aspect of the present invention includes an information processing apparatus generating a plurality of scale-specific feature maps from input data, generating a feature sequence from the plurality of scale-specific feature maps, and generating feature information by inputting the feature sequence into a recursive model.

[0009] A program according to one aspect of the present invention causes a computer to function as a feature map generation means for generating a plurality of scale-specific feature maps from input data, a feature sequence generation means for generating a feature sequence from the plurality of scale-specific feature maps, and a feature information generation means for generating feature information by inputting the feature sequence into a recursive model.

Advantages of the Invention

[0010] According to one aspect of the present invention, it is possible to provide an inference technique that can be suitably applied even when data having various lengths is input.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0012] 〔Exemplary Embodiment 1〕 The first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for the exemplary embodiments described later.

[0013] (Configuration of Information Processing Apparatus 1) The configuration of the information processing apparatus 1 according to this exemplary embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of the information processing apparatus 1 according to this exemplary embodiment.

[0014] The information processing apparatus 1 is an apparatus that generates feature information regarding input data from the input data.

[0015] As an example of the input data, a video can be cited. Further, as an example of the feature information, the feature information including the result of predicting an object included in the input video can be cited, but it is not limited to this.

[0016] As shown in FIG. 1, the information processing apparatus 1 includes a feature map generation unit 11, a feature sequence generation unit 12, and a feature information generation unit 13. The feature map generation unit 11, the feature sequence generation unit 12, and the feature information generation unit 13 are configured to realize a feature map generation means, a feature sequence generation means, and a feature information generation means, respectively, in this exemplary embodiment.

[0017] The feature map generation unit 11 generates a plurality of scale-specific feature maps from the input data. The feature map generation unit 11 supplies the generated scale-specific feature maps to the feature sequence generation unit 12. The feature map generation unit 11 generates a plurality of scale-specific feature maps by using a plurality of convolutional layers that act serially on the input data.

[0018] By using a plurality of convolutional layers that act serially on the input data, a feature map (a feature map with a small scale) in which local information of the input data is reflected can be obtained from the upstream convolutional layer, and a feature map (a feature map with a large scale) in which global information of the input data is reflected can be obtained from the downstream convolutional layer. Note that the scale size has a positive correlation with the so-called RF (Receptive Field).

[0019] Here, the feature map generation unit 11 is not limited to using a plurality of convolutional layers that act serially on the input data, and may be configured to use a plurality of convolutional layers that act in parallel on the input data and obtain feature maps with different RFs.

[0020] The feature sequence generation unit 12 generates a feature sequence from a plurality of feature maps for each scale. The feature sequence generation unit 12 supplies the generated feature sequence to the feature information generation unit 13. The feature sequence is a sequence composed of the feature maps for each scale output from the feature map generation unit 11. As an example, these feature maps for each scale are arranged in the order of scale.

[0021] The feature information generation unit 13 generates feature information by inputting the feature sequence into a recurrent model. The recurrent model is a model that is input one unit at a time in order from the beginning of the sequence. Examples of the recurrent model include, but are not limited to, RNN (Recurrent Neural Network), LSTM (Long Short Term Memory), and GRU (Gated Recurrent Unit).

[0022] As described above, in the information processing apparatus 1 according to this exemplary embodiment, a configuration including a feature map generation unit 11 that generates a plurality of feature maps for each scale from input data, a feature sequence generation unit 12 that generates a feature sequence from the plurality of feature maps for each scale, and a feature information generation unit 13 that generates feature information by inputting the feature sequence into a recurrent model is adopted.

[0023] Therefore, according to the information processing apparatus 1 according to this exemplary embodiment, since the feature sequence composed of the plurality of feature maps for each scale generated from the input data is input into the recurrent model, an inference technique that can be suitably applied even when data having various lengths is input can be provided.

[0024] (Flow of the information processing method S1) The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to FIG. 2. FIG. 2 is a flowchart showing the flow of the information processing method S1 according to this exemplary embodiment.

[0025] (Step S11) In step S11, the feature map generation unit 11 generates a plurality of scale-specific feature maps from the input data.

[0026] (Step S12) In step S12, the feature sequence generation unit 12 generates a feature sequence from the plurality of scale-specific feature maps.

[0027] (Step S13) In step S13, the feature information generation unit 13 generates feature information by inputting the feature sequence into a recursive model.

[0028] As described above, in the information processing method S1 according to this exemplary embodiment, in step S11, the feature map generation unit 11 generates a plurality of scale-specific feature maps from the input data, in step S12, the feature sequence generation unit 12 generates a feature sequence from the plurality of scale-specific feature maps, and in step S13, the feature information generation unit 13 generates feature information by inputting the feature sequence into a recursive model. Therefore, according to the information processing method S1 according to this exemplary embodiment, the same effect as that of the information processing apparatus 1 can be obtained.

[0029] 〔Exemplary Embodiment 2〕 The second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in Exemplary Embodiment 1 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0030] (Configuration of Information Processing Apparatus 2) The configuration of the information processing apparatus 2 according to this exemplary embodiment will be described with reference to FIG. 3. FIG. 3 is a block diagram showing the configuration of the information processing apparatus 2 according to this exemplary embodiment.

[0031] The information processing apparatus 2 is an apparatus that acquires input data IN and generates feature information FI regarding the input data IN.

[0032] Similar to the above-described exemplary embodiments, as an example of the input data IN, a video can be mentioned. Further, as an example of the feature information FI, feature information including the result of predicting an object included in the input video can be mentioned, but it is not limited thereto.

[0033] As shown in FIG. 3, the information processing apparatus 2 includes a control unit 20, a storage unit 21, a communication unit 22, an input unit 23, and an output unit 24.

[0034] In the storage unit 21, data referred to by the control unit 20 described later is stored. As an example of the data stored in the storage unit 21, input data IN, scale-specific feature maps FM, maximum scale MS, feature series SF, and feature information FI can be mentioned, but they are not limited thereto. The input data IN, scale-specific feature maps FM, feature series SF, and feature information FI are as described above. The maximum scale MS will be described later.

[0035] The communication unit 22 is a communication module that communicates with other devices via a network (not shown). As an example, the communication unit 22 outputs data supplied from the control unit 20 described later to other devices via the network, or acquires data output from other devices via the network and supplies it to the control unit 20.

[0036] The input unit 23 is an interface that acquires data from other connected devices. The input unit 23 supplies the data acquired from other devices to the control unit 20 described later.

[0037] The output unit 24 is an interface that outputs data to other connected devices. The output unit 24 outputs the data supplied from the control unit 20 described later to other devices.

[0038] (Control Unit 20) The control unit 20 controls each unit included in the information processing apparatus 2. As an example, the control unit 20 stores the data acquired from the communication unit 22 and the input unit 23 in the storage unit 21, or supplies the data stored in the storage unit 21 to the communication unit 22 and the output unit 24.

[0039] As shown in FIG. 3, the control unit 20 also functions as a feature map generation unit 11, a feature sequence generation unit 12, a feature information generation unit 13, and a maximum scale calculation unit 14. The feature map generation unit 11, the feature sequence generation unit 12, the feature information generation unit 13, and the maximum scale calculation unit 14 are configured to realize a feature map generation means, a feature sequence generation means, a feature information generation means, and a maximum scale calculation means, respectively, in this exemplary embodiment.

[0040] The feature map generation unit 11 acquires the input data IN from the storage unit 21 and generates a plurality of scale-specific feature maps FM from the acquired input data IN. The feature map generation unit 11 generates a plurality of scale-specific feature maps FM by using a plurality of convolutional layers that act serially on the input data IN. The feature map generation unit 11 stores the generated plurality of scale-specific feature maps FM in the storage unit 21.

[0041] The feature sequence generation unit 12 acquires a plurality of scale-specific feature maps FM from the storage unit 21 and the maximum scale MS calculated by the maximum scale calculation unit 14 described later. Then, the feature sequence generation unit 12 generates a feature sequence SF having a length corresponding to the acquired maximum scale MS from the acquired plurality of scale-specific feature maps FM. The feature sequence generation unit 12 stores the generated feature sequence SF in the storage unit 21.

[0042] Further, as shown in FIG. 3, the feature sequence generation unit 12 also functions as a scale-specific shaping unit 121 and a multi-scale combining unit 122.

[0043] The scale-by-scale shaping unit 121 generates feature data that absorbs the differences in the lengths and dimensions of a plurality of scale-by-scale feature maps FM. As an example, the scale-by-scale shaping unit 121 is composed of a combination of a global pooling layer (GP: Grobal Pooling), a linear transformation layer that changes the number of channels, and an activation function. The linear transformation layer that changes the number of channels is, for example, composed of a fully connected layer (FC: Fully Connected) or a convolutional layer with a kernel size of 1 according to the input scale-by-scale feature map FM. The scale-by-scale shaping unit 121 supplies the generated feature data to the multi-scale combination unit 122.

[0044] The multi-scale combination unit 122 acquires the feature data output by the scale-by-scale shaping unit 121 and generates a feature sequence SF in which the feature data is arranged in the order of the scales corresponding to the feature data. Arranging in the order of the scales corresponding to the feature data means, in other words, arranging in the order of the convolutional layers corresponding to each fully connected layer.

[0045] The feature information generation unit 13 acquires the feature sequence SF from the storage unit 21 and generates feature information FI by inputting the acquired feature sequence SF into a recursive model. The recursive model is as described above.

[0046] The maximum scale calculation unit 14 acquires the input data IN from the storage unit 21 and calculates the maximum scale MS. The maximum scale is the largest scale that is less than or equal to the length of the input data IN. As an example, the maximum scale calculation unit 14 refers to the input data IN or related information associated with the input data IN (such as information indicating the length of the input data IN) to calculate the maximum scale MS. The maximum scale calculation unit 14 stores the calculated maximum scale MS in the storage unit 21.

[0047] (Example of processing executed by the control unit 20) An example of the processing executed by the control unit 20 will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of the processing executed by the control unit 20 according to this exemplary embodiment.

[0048] (Example of the process executed by the feature map generation unit 11) As shown in FIG. 4, the feature map generation unit 11 of the control unit 20 acquires the input data IN with a length of L and a dimensionality of C from the storage unit 21. Here, data with a length of L and a dimensionality of C may also be expressed as "data with a Shape of L*C". "*" represents the product operation symbol. Also, as shown in FIG. 4, the feature map generation unit 11 is a convolutional block CB composed of M (M≧2) convolutional layers that generate a plurality of scale-specific feature maps by acting serially on the input data IN 1 ~Convolutional block CB M (Convolutional block CB M is not shown in FIG. 4) and is provided with it.

[0049] When the feature map generation unit 11 acquires the input data IN, it serially acts on M convolutional blocks CB 1 ~Convolutional block CB M and inputs the acquired input data IN to the first convolutional block CB 1 to act on. From the convolutional block CB 1 a scale-specific feature map FM 1 *C 1 with a Shape of L 1 is output.

[0050] The feature map generation unit 11 may be configured such that when the convolutional block CB 1 is used as an identity mapping, a scale-specific feature map FM 0 identical to the input data IN is output.

[0051] Subsequently, the feature map generation unit 11 supplies the scale-specific feature map FM 1 output from the convolutional block CB 1 to a convolutional block CB 1 with a larger scale (larger RF) than the scale-specific feature map FM 2 . Also, the feature map generation unit 11 stores the scale-specific feature map FM 1 in the storage unit 21.

[0052] The feature map generation unit 11 supplies the scale-specific feature map FM 2 with the Shape of L 2 *C 2 output from the convolutional block CB 2 to a convolutional block CB (not shown in FIG. 4). Also, the feature map generation unit 11 stores the scale-specific feature map FM 3 in the storage unit 21. 2

[0053] In this way, the feature map generation unit 11 supplies the scale-specific feature map FM n with the Shape of L n *C n output from the convolutional block CB (1 ≤ n ≤ M - 2) to a convolutional block CB n . Also, the feature map generation unit 11 stores the scale-specific feature map FM n+1 in the storage unit 21 until the scale-specific feature map FM M-1 output from the convolutional block CB M-1 (the Shape of the scale-specific feature map FM M-1 is L M-1 *C M-1 ) is obtained. n

[0054] Here, the magnitude relationship between L j (1 ≤ j ≤ M - 1) and L j+1 is not particularly limited. As an example, the feature map generation unit 11 can be configured such that L j is larger than L j+1 . Also, the magnitude relationship between C j and C j+1 is not particularly limited. As an example, the feature map generation unit 11 can be configured such that C j is larger than C j+1 .

[0055] Finally, the feature map generation unit 11 supplies the scale-specific feature map FM M-1 output from the convolutional block CB to the convolutional block CB M-1 that is to be finally applied. M ​​Input it. The feature map generation unit 11 stores the scale-specific feature map FM M output from the convolutional block CB M *C M in the storage unit 21. M

[0056] (Example of the process executed by the feature sequence generation unit 12) As shown in FIG. 4, the feature sequence generation unit 12 includes, for each of the M convolutional blocks CB 1 ~ convolutional block CB M , a GP (global pooling layer) 12 k (1 ≤ k ≤ M) that acts on the scale-specific feature map FM k output from the convolutional block CB ak and an FC (fully connected layer) 12 ak that acts on the output of GP12 bk .

[0057] The feature sequence generation unit 12 inputs each of the scale-specific feature maps FM 1 ~ scale-specific feature map FM M stored in the storage unit 21 to the GP12 a1 ~ GP12 aM that acts on the scale-specific feature map.

[0058] Then, each of GP12 a1 ~ GP12 aM outputs a feature map FM 1 ~ feature map FM M that absorbs the difference in the mutual length (L) of the scale-specific feature maps FM 1 ~ scale-specific feature map FM M when the scale-specific feature maps FM 1_1 ~ FM M_1 are input.

[0059] As an example, when the scale-specific feature map FM 1 is input to GP12 a1 , the difference in length (L) is absorbed, and from GP12 a1 the Shape is (1*)C1 The feature map FM 1_1 is output. As another example, the feature map FM M for each scale aM is input to GP12, and the difference in length (L) is absorbed. From GP12 aM a feature map FM with Shape (1*)C M is output. M_1

[0060] Subsequently, the feature sequence generation unit 12 inputs the feature maps FM a1 ~FM aM output from each of GP12 1_1 ~FM M_1 to FC12 b1 ~FC12 bM respectively. When the feature maps FM b1 ~FM bM are input to each of FC12 1_1 ~FM M_1 FC12 outputs feature data FD 1_1 ~FD M_1 that absorbs the difference in the number of dimensions (C) between each other of the feature maps FM 1 ~FD M

[0061] As an example, when the feature map FM 1_1 is input to FC12 b1 the difference in the number of dimensions (C) is absorbed, and from FC12 b1 feature data FD with Shape Cf 1 is output. As another example, when the feature map FM M_1 is input to FC12 bM the difference in the number of dimensions (C) is absorbed, and from FC12 bM feature data FD with Shape C f M is output.

[0062] Next, the feature sequence generation unit 12 obtains the maximum scale MS from the storage unit 21, refers to the maximum scale MS, and the feature data FD b1 ~FD bM output from FC12 1 ​​~Characteristic data FD M is arranged in the order of the scales corresponding to the characteristic data FD 1 ~Characteristic data FD M to generate a feature sequence SF with Shape being Cf*m. Here, m is the number of blocks having RF with a length of L or less, and m ≦ M (that is, m is the value of the maximum scale MS). The feature sequence generation unit 12 supplies the generated feature sequence SF to the feature information generation unit 13.

[0063] (Example of the process executed by the feature information generation unit 13) The feature information generation unit 13 inputs the acquired feature sequence SF into a recursive block and generates feature information FI with Shape being Cf.

[0064] The feature information FI generated by the feature information generation unit 13 is supplied to FC18, or supplied to the communication unit 22 or the output unit 24 according to the output format.

[0065] (Example of application of the information processing apparatus 2) An example of the application of the information processing apparatus 2 will be described with reference to FIGS. 5 and 6. FIG. 5 is a diagram showing an example of the application of the information processing apparatus 2 according to this exemplary embodiment, and FIG. 6 is another diagram showing an example of the application of the information processing apparatus 2 according to this exemplary embodiment.

[0066] In the examples shown in FIGS. 5 and 6, the information processing apparatus 2 acquires a moving image obtained by photographing the state of a container CN filled with liquid as input data IN, and outputs feature information FI indicating whether the object contained in the container CN is a bubble.

[0067] As an example, the container CN shown on the left side of FIG. 5 contains an object DM1 and an object DM2 in the liquid. As shown on the left side of FIG. 5, when the container CN is swung, as shown on the right side of FIG. 5, the object DM1 draws a trajectory DL1, and the object DM2 moves by drawing trajectories DL2 and DL3. The control unit 20 of the information processing apparatus 2 acquires a moving image obtained by photographing the state of the container CN being swung as input data IN, and identifies the trajectory DL1 drawn by the object DM1 and the trajectories DL2 and DL3 drawn by the object DM2.

[0068] Next, the control unit 20 identifies the object contained in the container CN from the identified trajectory. In the figure shown in FIG. 5, the control unit 20 determines that the trajectory DL1 is a trajectory drawn by a bubble, and outputs feature information FI indicating that the object DM1 that drew the trajectory DL1 is a bubble. Also, in the figure shown in FIG. 5, the control unit 20 determines that the trajectory DL2 is a trajectory drawn by a bubble and the trajectory DL3 is a trajectory drawn by an object other than a bubble, and outputs feature information FI indicating that the object DM2 is an object other than a bubble.

[0069] As another example, in the container CN shown on the left side of FIG. 6, an object DM4 and an object DM5 are contained in the liquid. As shown on the left side of FIG. 6, when the container CN is swung, as shown on the right side of FIG. 6, the object DM4 draws a trajectory DL4 and the object DM5 draws a trajectory DL5 and moves. The control unit 20 acquires, as input data IN, a moving image obtained by photographing the state of swinging the container CN, and identifies the trajectories DL4 and DL5. Then, the control unit 20 determines that both the trajectories DL4 and DL5 are trajectories drawn by bubbles, and outputs feature information FI indicating that both the object DM4 that drew the trajectory DL4 and the object DM5 that drew the trajectory DL5 are bubbles.

[0070] Here, the process of analyzing a moving image obtained by photographing the state of swinging the container CN and identifying the trajectory of the object contained in the container CN may be performed by the control unit 20 or may be performed by a device other than the information processing device 2. When a device other than the information processing device 2 identifies the trajectory of the object contained in the container CN, the information processing device 2 may be configured to acquire, as input data IN, the trajectory identified by a device other than the information processing device 2.

[0071] For example, in the examples shown in FIGS. 5 and 6, the information processing device 2 acquires the trajectories DL1 to DL5 as input data IN. Then, the information processing device 2 refers to the trajectories DL1 to DL5 and outputs feature information FI indicating whether the objects DM1 to DM5 are bubbles or not.

[0072] In this way, the information processing apparatus 2 outputs feature information FI indicating whether an object contained in the container CN filled with liquid is a bubble. Therefore, the information processing apparatus 2 can be used for inspecting whether foreign matter is contained in the liquid. Further, even when the information processing apparatus 2 receives trajectory DL1 to trajectory DL5 having various lengths as input data IN, it can preferably estimate whether foreign matter is contained in the liquid.

[0073] (Process S2 executed by information processing apparatus 2) The flow of process S2 executed by the information processing apparatus 2 will be described with reference to FIG. 7. FIG. 7 is a flowchart showing the flow of process S2 executed by the information processing apparatus 2 according to this exemplary embodiment. Before the process shown in FIG. 7, the information processing apparatus 2 acquires input data IN via the communication unit 22 or the input unit 23, and stores the acquired input data IN in the storage unit 21. Further, before the process shown in FIG. 7 or before the process of step S24 described later, the maximum scale calculation unit 14 calculates the maximum scale MS, and stores the calculated maximum scale MS in the storage unit 21.

[0074] (Step S21) In step S21, the feature map generation unit 11 acquires the input data IN from the storage unit 21. The feature map generation unit 11 inputs the acquired input data IN to a plurality of convolutional layers that generate a plurality of scale-specific feature maps by acting serially on the input data, and performs calculations by the convolutional layers.

[0075] (Step S22) In step S22, the feature map generation unit 11 generates a plurality of scale-specific feature maps FM by calculation by the convolutional layers. The details of the process in which the feature map generation unit 11 generates the scale-specific feature maps FM are as described above. The feature map generation unit 11 stores the generated plurality of scale-specific feature maps FM in the storage unit 21.

[0076] (Step S23) In step S23, the feature sequence generation unit 12 acquires a plurality of scale-specific feature maps FM from the storage unit 21. The feature sequence generation unit 12 inputs the acquired plurality of scale-specific feature maps FM to the scale-specific shaping unit 121, shapes them by absorbing the differences in the length and dimensionality of each of the acquired plurality of scale-specific feature maps FM, and generates feature data. The details of the process by which the feature sequence generation unit 12 generates feature data are as described above. The feature sequence generation unit 12 supplies the generated feature data to the multi-scale combination unit 122.

[0077] (Step S24) In step S24, the multi-scale combination unit 122 acquires the feature data generated by the scale-specific shaping unit 121 and the maximum scale MS stored in the storage unit 21. Subsequently, the multi-scale combination unit 122 generates a feature sequence SF corresponding to the value indicated by the maximum scale MS. The details of the process by which the multi-scale combination unit 122 generates the feature sequence SF are as described above. The multi-scale combination unit 122 stores the generated feature sequence SF in the storage unit 21.

[0078] (Step S25) In step S25, the feature information generation unit 13 acquires the feature sequence SF stored in the storage unit 21. Then, the feature information generation unit 13 inputs the acquired feature sequence SF to a recursive block that is a recursive model, and generates feature information FI.

[0079] As described above, the feature information FI generated by the feature information generation unit 13 is supplied to the FC18, or supplied to the communication unit 22 or the output unit 24, depending on the output format.

[0080] As described above, the information processing apparatus 2 according to this exemplary embodiment includes a feature map generation unit 11 that generates a plurality of scale-specific feature maps FM from input data IN, a feature sequence generation unit 12 that generates a feature sequence SF from the plurality of scale-specific feature maps FM, and a feature information generation unit 13 that generates feature information FI by inputting the feature sequence SF into a recursive model. For this reason, according to the information processing apparatus 2 according to this exemplary embodiment, the same effects as those of the information processing apparatus 1 can be obtained.

[0081] [Exemplary Embodiment 3] A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the above exemplary embodiments are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0082] (Configuration of Information Processing Apparatus 2A) The configuration of the information processing apparatus 2A according to this exemplary embodiment will be described with reference to FIG. 8. FIG. 8 is a block diagram showing the configuration of the information processing apparatus 2A according to this exemplary embodiment.

[0083] The information processing apparatus 2A has a configuration in which a control unit 20A is provided instead of the control unit 20 included in the above-described information processing apparatus 2. The storage unit 21, the communication unit 22, the input unit 23, and the output unit 24 are as described above.

[0084] (Control Unit 20A) The control unit 20A controls each unit included in the information processing apparatus 2A. As an example, the control unit 20A stores data acquired from the communication unit 22 and the input unit 23 in the storage unit 21, or supplies the data stored in the storage unit 21 to the communication unit 22 and the output unit 24.

[0085] As shown in FIG. 8, the control unit 20A also functions as a feature map generation unit 11, a feature sequence generation unit 12, a feature information generation unit 13, a maximum scale calculation unit 14, and a learning unit 15. The feature map generation unit 11, the feature sequence generation unit 12, the feature information generation unit 13, and the maximum scale calculation unit 14 are configured to realize a feature map generation means, a feature sequence generation means, a feature information generation means, and a maximum scale calculation means, respectively, in this exemplary embodiment.

[0086] The feature map generation unit 11, the feature sequence generation unit 12, the feature information generation unit 13, and the maximum scale calculation unit 14 are as described above.

[0087] The learning unit 15 refers to the input data IN and the correct label associated with the input data IN, and updates at least one of the parameters of the convolutional block included in the feature map generation unit 11 and the GP and FC included in the feature sequence generation unit 12. For example, when the learning unit 15 inputs the input data IN to the feature map generation unit 11 and the feature information FI output from the feature information generation unit 13 does not match the correct label, the learning unit 15 updates at least one of the parameters of the convolutional block included in the feature map generation unit 11 and the GP and FC included in the feature sequence generation unit 12 so that the difference between the feature information FI and the correct label becomes smaller.

[0088] As described above, in the information processing apparatus 2A according to this exemplary embodiment, a configuration is adopted in which at least one of the parameters of the convolutional block, GP, and FC is updated by referring to the input data IN and the correct label associated with the input data IN. Thus, in the information processing apparatus 2A according to this exemplary embodiment, since the convolutional block, GP, and FC can be learned, an inference technique that can be suitably applied can be provided.

[0089] [Exemplary Embodiment 4] The fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the above exemplary embodiments are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0090] (Configuration of Information Processing Apparatus 2B) The configuration of the information processing apparatus 2B according to this exemplary embodiment will be described with reference to FIG. 9. FIG. 9 is a block diagram showing the configuration of the information processing apparatus 2B according to this exemplary embodiment.

[0091] The information processing apparatus 2B acquires target data TD and generates a plurality of input data IN by cutting the target data TD into a plurality of lengths. Then, the information processing apparatus 2B determines a recommended value RV indicating a recommended length among the plurality of lengths by referring to the feature information FI corresponding to each of the plurality of input data IN.

[0092] The information processing apparatus 2B has a configuration including a storage unit 21B and a control unit 20B instead of the storage unit 21 and the control unit 20 provided in the above-described information processing apparatus 2. The communication unit 22, the input unit 23, and the output unit 24 are as described above.

[0093] The storage unit 21B stores data referred to by the control unit 20B described later. As an example, in addition to the data stored in the above-described storage unit 21, the storage unit 21B stores the target data TD and the recommended value RV.

[0094] (Control Unit 20B) The control unit 20B controls each unit included in the information processing apparatus 2B. As an example, the control unit 20B stores the data acquired from the communication unit 22 and the input unit 23 in the storage unit 21B, or supplies the data stored in the storage unit 21B to the communication unit 22 and the output unit 24.

[0095] As shown in FIG. 9, the control unit 20B also functions as a feature map generation unit 11, a feature sequence generation unit 12, a feature information generation unit 13, a maximum scale calculation unit 14, an input data generation unit 16, and a recommendation unit 17. The feature map generation unit 11, the feature sequence generation unit 12, the feature information generation unit 13, the maximum scale calculation unit 14, the input data generation unit 16, and the recommendation unit 17 are configured to realize a feature map generation means, a feature sequence generation means, a feature information generation means, a maximum scale calculation means, an input data generation means, and a recommendation means, respectively, in this exemplary embodiment.

[0096] The feature map generation unit 11, the feature sequence generation unit 12, the feature information generation unit 13, and the maximum scale calculation unit 14 are as described above.

[0097] The input data generation unit 16 generates a plurality of input data IN by cutting the target data TD into a plurality of lengths. As an example, the input data generation unit 16 generates input data IN obtained by cutting the target data TD at a predetermined time (such as 3 seconds, 5 seconds, and 10 seconds). The input data generation unit 16 stores the generated input data IN in the storage unit 21B.

[0098] The recommendation unit 17 determines a recommendation value RV indicating a recommended length among the plurality of lengths cut by the input data generation unit 16 by referring to the feature information FI corresponding to each of the plurality of input data IN. As an example, the recommendation unit 17 determines the length that has the highest accuracy information included in the feature information FI and the shortest input data IN as the recommendation value RV. The recommendation unit 17 stores the determined recommendation value RV in the storage unit 21B.

[0099] (Processing S2A executed by the information processing apparatus 2B) The flow of processing executed by the information processing apparatus 2B will be described with reference to FIG. 10. FIG. 10 is a flowchart showing the flow of processing S2A executed by the information processing apparatus 2B according to this exemplary embodiment. Before the processing shown in FIG. 10, the information processing apparatus 2 acquires target data TD via the communication unit 22 or the input unit 23, and stores the acquired target data TD in the storage unit 21B. Also, similar to the processing shown in FIG. 7, before the processing shown in FIG. 10 or before the processing in step S24, the maximum scale calculation unit 14 calculates the maximum scale MS, and stores the calculated maximum scale MS in the storage unit 21B.

[0100] (Step S31) In step S31, the input data generation unit 16 acquires the target data TD from the storage unit 21B. Then, the input data generation unit 16 generates a plurality of input data IN by cutting the acquired target data TD into a plurality of lengths. The input data generation unit 16 stores the generated plurality of input data IN in the storage unit 21B.

[0101] (Steps S21 to S25) Steps S21 to S25, in which the feature map generation unit 11 acquires the input data IN from the storage unit 21B and the feature information generation unit 13 generates the feature information FI, are as described above.

[0102] (Step S32) In step S32, the recommendation unit 17 determines a recommended value RV indicating the recommended length among the plurality of lengths cut by the input data generation unit 16 in step S31, by referring to the feature information FI corresponding to each of the plurality of input data IN. The recommendation unit 17 stores the determined recommended value RV in the storage unit 21B.

[0103] The recommended value RV may be configured such that the input data generation unit 16 refers to it and generates input data IN having the length of the recommended value RV. Also, the recommended value RV may be referred to in a device other than the information processing apparatus 2, and input data IN having the length of the recommended value RV is generated in that device, and the generated input data IN is input to the information processing apparatus 2B.

[0104] As described above, in the information processing apparatus 2B according to the present exemplary embodiment, the input data generation unit 16 that generates a plurality of input data IN by cutting the target data TD into a plurality of lengths, and among the plurality of lengths, the recommendation unit 17 that determines a recommended value RV indicating the recommended length by referring to the feature information FI corresponding to each of the plurality of input data IN are provided. Thus, in the information processing apparatus 2B according to the present exemplary embodiment, it is possible to generate input data IN with high accuracy and short processing time.

[0105] 〔Example of Realization by Software〕 Some or all of the functions of the information processing apparatuses 1, 2, 2A, and 2B may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0106] In the latter case, the information processing apparatuses 1, 2, 2A, and 2B are realized by, for example, a computer that executes instructions of a program that is software for realizing each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 11. The computer C includes at least one processor C1 and at least one memory C2. A program P for operating the computer C as the information processing apparatuses 1, 2, 2A, and 2B is recorded in the memory C2. In the computer C, the processor C1 reads and executes the program P from the memory C2, whereby each function of the information processing apparatuses 1, 2, 2A, and 2B is realized.

[0107] As the processor C1, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating point number Processing Unit), PPU (Physics Processing Unit), a microcontroller, or a combination thereof can be used. As the memory C2, for example, a flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0108] Note that the computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and temporarily storing various data. Also, the computer C may further include a communication interface for transmitting and receiving data to and from other devices. Also, the computer C may further include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0109] Also, the program P can be recorded on a non-transitory tangible recording medium M readable by the computer C. As such a recording medium M, for example, a tape, disk, card, semiconductor memory, or programmable logic circuit can be used. The computer C can obtain the program P via such a recording medium M. Also, the program P can be transmitted via a transmission medium. As such a transmission medium, for example, a communication network or broadcast wave can be used. The computer C can also obtain the program P via such a transmission medium.

[0110] 〔Supplementary Note 1〕 The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0111] [Supplementary Note 2] Some or all of the above-described embodiments may also be described as follows. However, the present invention is not limited to the aspects described below.

[0112] (Supplementary Note 1) An information processing apparatus comprising: feature map generation means for generating a plurality of scale-specific feature maps from input data; feature sequence generation means for generating a feature sequence from the plurality of scale-specific feature maps; and feature information generation means for generating feature information by inputting the feature sequence into a recursive model.

[0113] According to the above configuration, it is possible to provide an inference technique that can be suitably applied even when data having various lengths is input.

[0114] (Supplementary Note 2) The information processing apparatus according to Supplementary Note 1, further comprising maximum scale calculation means for calculating a maximum scale, wherein the feature sequence generation means generates a feature sequence having a length corresponding to the maximum scale.

[0115] According to the above configuration, it is possible to omit the processing of unnecessary data.

[0116] (Supplementary Note 3) The information processing apparatus according to Supplementary Note 2, wherein the maximum scale calculation means calculates the maximum scale by referring to the input data or related information associated with the input data.

[0117] According to the above configuration, it is possible to suitably process the data.

[0118] (Supplementary Note 4) The feature map generation means includes a plurality of convolutional layers that generate the feature maps for different scales by acting on the input data in series, the information processing apparatus according to any one of Appendices 1 to 3.

[0119] According to the above configuration, it is possible to provide an inference technique that can be suitably applied even when data having various lengths is input.

[0120] (Appendix 5) The feature sequence generation means includes, for each of the plurality of convolutional layers, a global pooling layer that acts on the feature map for different scales output by the convolutional layer, and a fully connected layer that acts on the output of the global pooling layer, the information processing apparatus according to Appendix 4.

[0121] According to the above configuration, it is possible to provide an inference technique that can be suitably applied even when data having various lengths is input.

[0122] (Appendix 6) The feature sequence generation means generates the feature sequence by arranging the feature data output by each of the plurality of fully connected layers in the order of the scales corresponding to the feature data, the information processing apparatus according to Appendix 5.

[0123] According to the above configuration, data can be suitably processed.

[0124] (Appendix 7) Input data generation means for generating the plurality of input data by cutting the target data into a plurality of lengths, and recommendation means for determining a recommended length among the plurality of lengths by referring to the feature information corresponding to each of the plurality of input data, the information processing apparatus according to any one of Appendices 1 to 6.

[0125] According to the above configuration, data can be suitably processed.

[0126] (Appendix 8) An information processing method, comprising: an information processing apparatus generating a plurality of scale-specific feature maps from input data; generating a feature sequence from the plurality of scale-specific feature maps; and generating feature information by inputting the feature sequence into a recursive model.

[0127] According to the above configuration, it is possible to provide an inference technique that can be suitably applied even when data having various lengths is input.

[0128] (Appendix 9) The information processing method according to Appendix 8, wherein the information processing apparatus includes calculating a maximum scale, and generating a feature sequence having a length corresponding to the maximum scale in generating the feature sequence.

[0129] According to the above configuration, it is possible to omit the processing of unnecessary data.

[0130] (Appendix 10) The information processing method according to Appendix 9, wherein in calculating the maximum scale, the maximum scale is calculated by referring to the input data or related information associated with the input data.

[0131] According to the above configuration, it is possible to suitably process data.

[0132] (Appendix 11) The information processing method according to any one of Appendices 8 to 10, wherein in generating the feature map, a plurality of convolutional layers act on the input data in series to generate the plurality of scale-specific feature maps.

[0133] According to the above configuration, it is possible to provide an inference technique that can be suitably applied even when data having various lengths is input.

[0134] (Appendix 12) In generating the feature series, for each of the plurality of convolutional layers, the global pooling layer acts on the scale-specific feature map output by the convolutional layer, and the fully connected layer acts on the output of the global pooling layer. The information processing method according to Supplementary Note 11.

[0135] According to the above configuration, an inference technique that can be suitably applied even when data having various lengths is input can be provided.

[0136] (Supplementary Note 13) In generating the feature series, the feature series is generated by arranging the feature data output by each of the plurality of fully connected layers in the order of the scales corresponding to the feature data. The information processing method according to Supplementary Note 12.

[0137] According to the above configuration, data can be suitably processed.

[0138] (Supplementary Note 14) The information processing apparatus generates the plurality of input data by cutting the target data into a plurality of lengths, and determines a recommended length among the plurality of lengths by referring to the feature information corresponding to each of the plurality of input data. The information processing method according to any one of Supplementary Notes 8 to 13.

[0139] According to the above configuration, data can be suitably processed.

[0140] (Supplementary Note 15) A program for operating a computer as the information processing apparatus according to any one of Supplementary Notes 1 to 7, the program causing the computer to function as each of the above means.

[0141] According to the above configuration, an inference technique that can be suitably applied even when data having various lengths is input can be provided.

[0142] [Supplementary Item 3] Some or all of the above-described embodiments can also be expressed as follows.

[0143] An information processing apparatus including at least one processor, the processor executing a feature map generation process for generating a plurality of scale-specific feature maps from input data, a feature sequence generation process for generating a feature sequence from the plurality of scale-specific feature maps, and a feature information generation process for generating feature information by inputting the feature sequence into a recursive model.

[0144] Note that this information processing apparatus may further include a memory, and a program for causing the processor to execute the feature map generation process, the feature sequence generation process, and the feature information generation process may be stored in this memory. Further, this program may be recorded on a non-transitory tangible computer-readable recording medium.

Explanation of Reference Numerals

[0145] 1, 2, 2A, 2B Information processing apparatus 11 Feature map generation unit 12 Feature sequence generation unit 13 Feature information generation unit 14 Maximum scale calculation unit 15 Learning unit 16 Input data generation unit 17 Recommendation unit 20, 20A, 20B Control unit 21, 21B Storage unit 22 Communication unit 23 Input unit 24 Output unit 121 Scale-specific shaping unit 122 Multiple scale combination unit

Claims

1. Feature map generation means for generating a plurality of feature maps by scale from input data, Feature sequence generation means for generating a feature sequence from the plurality of feature maps by scale, Feature information generation means for generating feature information by inputting the feature sequence into a recursive model, Maximum scale calculation means for calculating a maximum scale and comprising, wherein the feature sequence generation means generates a feature sequence having a length corresponding to the maximum scale, and the maximum scale calculation means calculates the maximum scale by referring to the input data or related information associated with the input data An information processing apparatus.

2. The feature map generation means comprises a plurality of convolutional layers that generate the plurality of feature maps by scale by acting serially on the input data The information processing apparatus according to claim 1.

3. The feature sequence generation means for each of the plurality of convolutional layers, a global pooling layer that acts on the feature map by scale output by the convolutional layer, and a fully connected layer that acts on the output of the global pooling layer The information processing apparatus according to claim 2.

4. The feature sequence generation means generates the feature sequence by arranging the feature data output by each of the plurality of fully connected layers in the order of the scale corresponding to the feature data The information processing apparatus according to claim 3.

5. Feature map generation means for generating a plurality of feature maps by scale from input data, Feature sequence generation means for generating a feature sequence from the plurality of feature maps by scale, Feature information generation means for generating feature information by inputting the feature sequence into a recursive model, Input data generation means for generating a plurality of the input data by cutting target data into a plurality of lengths, Recommendation means for determining a recommended length among the plurality of lengths by referring to the feature information corresponding to each of the plurality of input data An information processing apparatus comprising.

6. An information processing apparatus generates a plurality of feature maps by scale from input data, generates a feature sequence from the plurality of feature maps by scale, generates feature information by inputting the feature sequence into a recursive model, calculates a maximum scale, and includes, in generating the feature sequence, generates a feature sequence having a length corresponding to the maximum scale, In calculating the maximum scale, calculate the maximum scale by referring to the input data or related information associated with the input data Information processing method.

7. An information processing apparatus, generating a plurality of scale-specific feature maps from input data, generating a feature sequence from the plurality of scale-specific feature maps, generating feature information by inputting the feature sequence into a recursive model, generating a plurality of the input data by cutting the target data into a plurality of lengths, determining a recommended length among the plurality of lengths by referring to the feature information corresponding to each of the plurality of the input data An information processing method including.

8. A computer, a feature map generation means for generating a plurality of scale-specific feature maps from input data, a feature sequence generation means for generating a feature sequence from the plurality of scale-specific feature maps, a feature information generation means for generating feature information by inputting the feature sequence into a recursive model, a maximum scale calculation means for calculating a maximum scale function as, The feature sequence generation means, generates a feature sequence having a length corresponding to the maximum scale, The maximum scale calculation means, calculates the maximum scale by referring to the input data or related information associated with the input data Program.

9. A computer, a feature map generation means for generating a plurality of scale-specific feature maps from input data, a feature sequence generation means for generating a feature sequence from the plurality of scale-specific feature maps, a feature information generation means for generating feature information by inputting the feature sequence into a recursive model, an input data generation means for generating a plurality of the input data by cutting the target data into a plurality of lengths, a recommendation means for determining a recommended length among the plurality of lengths by referring to the feature information corresponding to each of the plurality of the input data A program for functioning as.

Images