Machine learning apparatus, machine learning method and computer-readable storage medium

Abstract

A machine learning apparatus according to one aspect of the present discloser includes: a data splitting unit 11 that splits input data into a first group and a second group; a two-dimensional convolution processing unit 12 that performs a two-dimensional convolution process on the first group; a one-dimensional convolution processing unit 13 that performs a one-dimensional convolution process on the second group; and a linking unit 14 that links an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

Description

MACHINE LEARNING APPARATUS, MACHINE LEARNING METHOD AND COMPUTER-READABLE STORAGE MEDIUM

[0001] The present invention relates to machine learning of convolutional neural network (CNN).

[0002] In modern times, the popularity of convolutional neural network (CNN) models has surged significantly. The primary reason for trend is remarkable capability of CNN models to achieve high accuracy across a variety of computer vision tasks. These tasks encompass image classification, object detection, video recognition, semantic segmentation, and more. The versatility and efficacy of CNNs have made them indispensable tools in the field of artificial intelligence and machine learning.

[0003] The advent of CNNs brought a paradigm shift in how computer vision problems are approached. Traditional manual feature extraction methods, which were labor-intensive and less accurate, have been largely replaced by data-driven approaches enabled by CNNs. In CNN architectures, the convolutional layers automatically learn hierarchical feature representations directly from the data, which has led to unprecedented advancements in the performance of computer vision systems.

[0004] One of the critical challenges in designing a CNN model is the requirement for human expertise. Creating an optimized CNN for a specific task or dataset involves several stages, including selecting appropriate architectures, tuning hyperparameters, and ensuring that the model generalizes well to new, unseen data. The process is often iterative and resource-intensive, requiring deep knowledge of machine learning and domain-specific insights.

[0005] Furthermore, real-time applications impose additional constraints on CNN models. The real-time applications require that the CNN not only achieve high accuracy but also perform at high speeds to meet practical time-sensitive demands. Examples include autonomous driving, real-time surveillance, and interactive applications, where latency and computational efficiency are crucial.

[0006] Video recognition, in particular, presents unique challenges. Unlike still images, video consists of sequential frames over time, providing an added temporal dimension to the data. Standard three-dimensional convolutional neural networks (3D CNNs) are commonly employed to handle such spatiotemporal data. The 3D CNNs apply standard 3D convolutions that process both spatial and temporal dimensions simultaneously, effectively capturing motion and appearance information.

[0007] However, while 3D CNNs are powerful, they are also computationally demanding. The operations involved in 3D convolution require significant computational resources and memory, making these models less practical for real-time applications. High computational complexity and a large number of parameters contribute to extended inference times and increased energy consumption. Consequently, the deployment of 3D CNNs in real-world scenarios, especially on devices with limited computational capabilities, becomes challenging.

[0008] The computational expense and parameter overhead associated with 3D CNNs have prompted researchers and practitioners to explore more efficient alternatives (ex. see Non-Patent Literatures 1 to 3). The techniques disclosed in non-patent literatures 1 to 3 are expected to solve the problems of computational expense and parameter overhead associated with 3D CNNs.

[0009] [NPL1] Zhikang Qiu, Xu Zhao, and Zhilan Hu, “EFFICIENT TEMPORAL-SPATIAL FEATURE GROUPING FOR VIDEO ACTION RECOGNITION”, Department of Automation, Shanghai Jiao Tong University, The Central Media Technology Institute of Huawei Co., Ltd.ICIP2020, p.2176 to p.2180 [NPL2] Zhaofan Qiu, Ting Yao, and Tao Mei, “Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks”, University of Science and Technology of China, Hefei, China, Microsoft Research, Beijing, ChinaarXiv:1711.10305v1 [cs.CV] 28 Nov 2017 [NPL3] Chenxu Luo and Alan Yuille, “Grouped Spatial-Temporal Aggregation for Efficient Action Recognition”, Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA

[0010] However, the technique disclosed in non-patent literature 1 has the advantage of being able to reduce parameters and multiply accumulate operations (MACs) to improve calculation efficiency and reduce memory usage, but has the problem of low interaction and low accuracy. On the other hand, the techniques disclosed in non-patent literatures 2 and 3 have the advantage of having many interactions and high accuracy, but have the problem of being unable to reduce parameters and MACs.

[0011] Reducing the computational burden without compromising accuracy is a critical objective to enhance the feasibility of video recognition models for real-time and resource-constrained environments.

[0012] One example of an object of the present discloser is to enhance computational efficiency and reduce memory usage while maintaining high accuracy in video recognition tasks.

[0013] In order to achieve the foregoing object, a machine learning apparatus according to one aspect of the present discloser includes: a data splitting unit that splits input data into a first group and a second group; a two-dimensional convolution processing unit that performs a two -dimensional convolution process on the first group; a one-dimensional convolution processing unit that performs a one-dimensional convolution process on the second group; and a linking unit that links an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

[0014] In order to achieve the foregoing object, a machine learning method according to another aspect of the present discloser includes: a data splitting step of splitting input data into a first group and a second group; a two-dimensional convolution processing step of performing a two-dimensional convolution process on the first group; a one-dimensional convolution processing step of performing a one-dimensional convolution process on the second group; and a linking step of linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

[0015] In order to achieve the foregoing object, a computer-readable recording medium according to still another aspect of the present discloser has recorded therein a program, and the program includes an instruction to cause the computer to execute: a data splitting step of splitting input data into a first group and a second group; a two-dimensional convolution processing step of performing a two-dimensional convolution process on the first group; a one-dimensional convolution processing step of performing a one-dimensional convolution process on the second group; and a linking step of linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

[0016] As described above, according to the present discloser, it is possible to enhance computational efficiency and reduce memory usage while maintaining high accuracy in video recognition tasks.

[0017] FIG. 1 is a diagram illustrating a schematic configuration of an example of the machine learning apparatus.FIG. 2 is a block diagram more specifically illustrating a configuration of the machine learning apparatus.FIG. 3 is a diagram illustrating a configuration of the convolutional neural network in the example embodiment.FIG. 4 is a diagram illustrating the processing of the convolutional neural network in the example embodiment.FIG. 5 is a flowchart illustrating an example of the operation of the machine learning apparatus.FIG. 6 is a diagram illustrating the number of parameters calculation in the convolution processing.FIG. 7 is a table comparing the number of parameters calculation in the convolution processing.FIG. 8 is a table comparing the number of spatial and temporal interactions between convolutional layers.FIG. 9 is a block diagram illustrating an example of a computer that realizes the machine learning apparatus.

[0018] (Example Embodiment) Example embodiment of the present discloser is described in detail below referring to the accompanying drawings.

[0019] Configuration of Apparatus First, the configuration of an example of a machine learning apparatus will be described with reference to FIG. 1. FIG. 1 is a diagram illustrating a schematic configuration of an example of the machine learning apparatus.

[0020] The machine learning apparatus 10 illustrated in FIG. 1 is an apparatus that functions as a machine learning model. As illustrated in FIG. 1, the machine learning apparatus 10 includes a data splitting unit 11, a two-dimensional convolution processing unit 12, a one-dimensional convolution processing unit 13, and a linking unit 13.

[0021] The data splitting unit 11 splits input data into a first group and a second group. The two-dimensional convolution processing unit 12 performs a two-dimensional convolution process on the first group. The one-dimensional convolution processing unit performs a one-dimensional convolution process on the second group. The linking unit links an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

[0022] In this way, the machine learning apparatus 10 performs three-dimensional convolution processing on a portion of the input data, and performs two-dimensional convolution processing on the remaining portion of the input data. The two-dimensional convolution processing and the one-dimensional convolution processing use less memory and have higher calculation efficiency than the three-dimensional convolution processing. Therefore, the machine learning apparatus 10 can improve calculation efficiency and reduce memory usage while maintaining high accuracy in a video recognition task.

[0023] Next, the configuration and function of the machine learning apparatus 10 will be described in detail using FIGS. 2 to 4 in addition to FIG. 1. FIG. 2 is a block diagram more specifically illustrating a configuration of the machine learning apparatus.

[0024] As illustrated in FIG. 2, the machine learning apparatus 10 includes a data acquisition unit 15 and an output unit 16 in addition to the data splitting unit 11, the two-dimensional convolution processing unit 12, the one-dimensional convolution processing unit 13, and the linking unit 14 described above.

[0025] The data acquisition unit 15 acquires the input data and inputs the acquired input data to the splitting unit 11. In the example embodiment, the input data includes a channel in image data. More specifically, the input data includes each channel of a plurality of image data 21 that constitutes the video data 30, as illustrated in FIG. 2.

[0026] The data splitting unit 11 splits the input channels into the first group and the second group. Specifically, the data splitting unit 11 splits the input channels into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group. As a method of channel splitting by the data splitting unit 11, for example, the method disclosed in the above-mentioned non-patent literature 1 can be mentioned.

[0027] In this example embodiment, the linking unit 14 links the output from the two-dimensional convolution process and the output from the one-dimensional convolution process, and then performs depth wise convolution process on the output obtained by the linking.

[0028] As illustrated in FIG. 2, the machine learning apparatus 10 includes a convolutional neural network (CNN) 20. The convolutional neural network 20 functions as a two-dimensional convolution processing unit 12, a one-dimensional convolution processing unit 13, and a connection unit 14.

[0029] The convolutional neural network 20 will be described with reference to FIG. 3 and Fig. 4. FIG. 3 is a diagram illustrating a configuration of the convolutional neural network in the example embodiment. FIG. 4 is a diagram illustrating the processing of the convolutional neural network in the example embodiment.

[0030] As illustrated FIG. 3, in the convolutional neural network 20, a two-dimensional convolutional layer (1×3×3 SD (Sigma Delta)) functions as the two-dimensional convolutional processing unit 12. Also, a one-dimensional convolutional layer (3×1×1 SD) functions as the one-dimensional convolutional processing unit 13. Furthermore, a connector and a depth wise convolutional layer (3×3×3 DW (Depth wise Convolution)) function as the linking unit 14. As illustrated in FIG. 4, the convolutional neural network 20 can efficiently execute three-dimensional convolution processing by the configuration illustrated in FIG. 3.

[0031] The output unit 16 outputs the output obtained by performing the depth wise convolution process as a final output. The machine learning apparatus 10 can also update parameters of the two-dimensional convolution layer, the one-dimensional convolution layer, and the depth wise convolution layer described above using the output result of the output unit 16.

[0032] Operation of Apparatus Next, operation of the machine learning apparatus 10 will be described using FIG. 5. FIG. 5 is a flowchart illustrating an example of the operation of the machine learning apparatus. In the following description, FIGS. 1 to 4 are referred to as appropriate. In the example embodiment, the machine learning method is implemented by operating the machine learning apparatus 10. Accordingly, the description of the machine learning method according to the example embodiment is replaced with the following description of the operations of the machine learning apparatus 10.

[0033] As illustrated in FIG. 5, first, the data acquisition unit 15 acquires input data and inputs the acquired input data to the splitting unit 11 (step A1). The input data is each channel of a plurality of image data 21 that constitutes the video data 30.

[0034] Next, the data splitting unit 11 splits the input data into the first group and the second group (step A2). Specifically, the data splitting unit 11 splits the input channels into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group.

[0035] Next, the two-dimensional convolution processing unit 12 performs two-dimensional convolution processing on the first group (step A3). At the same time as step A3, the one-dimensional convolution processing unit 13 performs one-dimensional convolution processing on the second group (step A4).

[0036] Next, the linking unit 14 links the output from the two-dimensional convolution process and the output from the one-dimensional convolution process (step A5).

[0037] Next, the linking unit 14 performs depth wise convolution process on the output obtained by the linking of step A5 (step A6).

[0038] The output unit 16 outputs the output obtained by performing the depth wise convolution process of step A6 as a final output (step A7).

[0039] In this way, the machine learning apparatus 10 performs three-dimensional convolution processing on a portion of the input data, and performs two-dimensional convolution processing on the remaining portion of the input data. As a result, the machine learning apparatus 10 can improve calculation efficiency and reduce memory usage while maintaining high accuracy in a video recognition task.

[0040] Effects of the example embodiment The effects of the example embodiment will be described in detail below. At first, the bifurcation of channels into spatial and temporal convolution paths allows the model to specialize in different aspects of the data. Spatial convolutions are adept at capturing appearance information within individual frames, while temporal convolutions focus on the motion and interactions across frames. By combining the outputs of these two types of convolutions, the model integrates spatial feature and temporal features effectively. Subsequent three-dimensional depth wise convolution refines information, maintaining coherence across the spatiotemporal dimensions without the full parameter overhead of standard three-dimensional convolutions.

[0041] Several advantages are anticipated from the machine learning apparatus 10. Primarily, the compute efficiency improves due to the lighter weight of standard two-dimensional and standard one-dimensional convolutions compared to standard three-dimensional convolutions. Better computer efficiency leads to faster inference time, making the 3D CNN models more suitable for real-time applications. Additionally, the reduction in the number of parameters can significantly decrease memory usage, which is especially beneficial for deployment on edge devices and environments with limited resources.

[0042] The ability to separately process spatial and temporal features offers another key benefit. The standard three-dimensional convolutions treat spatiotemporal data homogeneously, potentially leading to redundant or less meaningful representations. In contrast, the convolution layer of the machine learning apparatus 10 leverages the intrinsic characteristics of video data by explicitly separating spatial and temporal processing. Thereby resulting in richer, more specialized feature extraction, enhancing the network's ability to discern subtle patterns and dynamics in the input data.

[0043] Moreover, the concatenation of outputs from the standard two-dimensional and standard one-dimensional convolutions followed by the depth wise convolution allows for an efficient fusion of spatial and temporal information. Depth wise convolutions, which perform separate convolutions over each input channel without combining them, significantly cut down on the parameter count and computational load compared to conventional convolutions. The convolution layer of the machine learning apparatus 10 can produce high-quality feature maps with a lower computational footprint, making it possible to achieve similar or even superior accuracy to standard three-dimensional convolutions but with greater efficiency.

[0044] The number of parameters calculation in the convolution processing will be described with reference to FIG. 6. FIG. 6 is a diagram illustrating the number of parameters calculation in the convolution processing. The number of parameters calculation for the standard three-dimensional convolution is illustrated by equation (1) in FIG. 6. Additionally, the number of Multiplication and Accumulation operations (MACs) calculation for the standard three-dimensional convolution is shown by equation (2) in FIG. 6. On the other hand, the number of parameter calculation for the convolution processing of the machine learning apparatus 10 is illustrated by equation (3) in FIG.6. Additionally, the number of Multiplication and Accumulation operations (MACs) calculation for the convolution processing of the machine learning apparatus 10 is illustrated by equation (4) in FIG. 6.

[0045] FIG. 7 is a table comparing the number of parameters calculation in the convolution processing. To highlight the efficiency of the convolution layer of the machine learning apparatus 10 in terms of parameter count, table in FIG. 7 illustrates the comparison of number of parameters required by standard three-dimensional convolutions layer versus the convolution layer of the machine learning apparatus 10. For the simplicity of comparison, the table in FIG. 7 assumes the kernel size equal to 3x3x3. Also, the table in FIG.7 illustrates the number of parameters comparison for different scenarios, including cases where the number of input channels is less than, greater than, or equal to the number of output channels.

[0046] In standard three-dimensional convolutional layers, spatial, temporal, and channel interactions occur simultaneously within a single layer, leading to comprehensive but computationally expensive operations. The standard three-dimensional convolution layer captures spatiotemporal dependencies but at the cost of high computational complexity and memory usage.

[0047] Conversely, the convolution layer of the machine learning apparatus 10 introduces a more efficient interaction scheme. FIG. 8 is a table comparing the number of spatial and temporal interactions between convolutional layers. The convolution layer of the machine learning apparatus 10 achieves 1.5 times spatial interaction compared to standard three-dimension convolution layer. Half of interactions is by standard two-dimensional convolution and one full interaction by three-dimensional depth wise convolution. Similarly, the convolution layer of the machine learning apparatus 10 realizes 1.5 times temporal interaction compared to standard three-dimensional convolution layer. Half of the interaction by one-dimensional standard convolution and one full interaction by three-dimensional depth wise convolution.

[0048] In summary, the convolution layer of the machine learning apparatus 10 offers a promising alternative to standard three-dimensional convolutions, with substantial benefits in terms of computational efficiency and specialized feature extraction. The convolution layer of the machine learning apparatus 10 addresses the pressing need for high-performance, real-time video recognition models that can be deployed in resource-constrained environments. The novel arrangement of splitting the input channels for separate spatial and temporal convolutions, followed by their fusion through three-dimensional depth wise convolution, presents a balanced approach to capturing the intricate details in spatiotemporal data. By reducing computational and memory overheads while maintaining high accuracy, the convolution layer of the machine learning apparatus 10 can significantly enhance the viability of CNN-based video recognition in practical applications.

[0049] Program It suffices for a program in the example embodiment that causes a computer to carry out steps A1 to A7 illustrated in FIG. 8. Also, by this program being installed and executed in the computer, the machine learning apparatus 10 and the machine learning method can be realized. In this case, a processor of the computer functions and performs processing as the data splitting unit 11, the two-dimensional convolution processing unit 12, the one-dimensional convolution processing unit 13, the linking unit 14, the data acquisition unit 15 and the output unit 16. The computer includes a general-purpose PC, a server computer, a smartphone and a tablet-type terminal device.

[0050] Furthermore, the program may be executed by a computer system constructed with a plurality of computers. In this case, for example, each computer may function as one of the data splitting unit 11, the two-dimensional convolution processing unit 12, the one-dimensional convolution processing unit 13, the linking unit 14, the data acquisition unit 15 and the output unit 16.

[0051] Physical configuration Physical configuration Using FIG. 9, the following describes a computer that realizes the machine learning apparatus 10 by executing the program according to the example embodiment. FIG. 9 is a block diagram illustrating an example of a computer that realizes the machine learning apparatus.

[0052] As shown in FIG. 9 a computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected in such a manner that they can perform data communication with one another via a bus 121.

[0053] The computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to the CPU 111, or in place of the CPU 111. In this case, the GPU or the FPGA can execute the programs according to the example embodiment.

[0054] The CPU 111 deploys the program according to the example embodiment, which is composed of a code group stored in the storage device 113 to the main memory 112 and carries out various types of calculation by executing the codes in a predetermined order. The main memory 112 is typically a volatile storage device, such as a DRAM (dynamic random-access memory).

[0055] Also, the program according to the example embodiment is provided in a state where it is stored in a computer-readable recording medium 120. Note that the program according to the present example embodiment may be distributed over the Internet connected via the communication interface 117.

[0056] Also, specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device, such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and an input device 118, such as a keyboard and a mouse. The display controller 115 is connected to a display device 119, and controls display on the display device 119.

[0057] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads out the program from the recording medium 120, and writes the result of processing in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.

[0058] Specific examples of the recording medium 120 include: a general-purpose semiconductor storage device, such as CF (CompactFlash (registered trademark)) and SD (Secure Digital); a magnetic recording medium, such as a flexible disk; and an optical recording medium, such as a CD-ROM (Compact Disk Read Only Memory).

[0059] Note that the machine learning apparatus 10 can also be realized by using items of hardware that respectively correspond to the components, such as an electronic circuit, rather than the computer in which the program is installed. Furthermore, a part of the machine learning apparatus 10 may be realized by the program, and the remaining part of the machine learning apparatus 10 may be realized by hardware. The computer is not limited to the computer illustrated in FIG. 12.

[0060] A part or an entirety of the above-described example embodiment can be represented by (Supplementary Note 1) to (Supplementary Note 12) described below but is not limited to the description below.

[0061] (Supplementary Note 1) A machine learning apparatus comprising: a data splitting means for splitting input data into a first group and a second group; a two-dimensional convolution processing means for performing a two-dimensional convolution process on the first group; a one-dimensional convolution processing means for performing a one-dimensional convolution process on the second group; and a linking means for linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

[0062] (Supplementary Note 2) The machine learning apparatus according to supplementary note 1, wherein the input data is a channel in image data, and the data splitting means splits the input channel into the first group and the second group.

[0063] (Supplementary Note 3) The machine learning apparatus according to supplementary note 2, wherein the input data is a channel in each of a plurality of image data constituting video data, the data splitting means splits the input channel into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group.

[0064] (Supplementary Note 4) The machine learning apparatus according to supplementary note 2, wherein the linking means performs a depth wise convolution process on the output obtained by the linking.

[0065] (Supplementary Note 5) A machine learning method comprising: splitting input data into a first group and a second group; performing a two-dimensional convolution process on the first group; performing a one-dimensional convolution process on the second group; and linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

[0066] (Supplementary Note 6) The machine learning method according to supplementary note 5, wherein the input data is a channel in image data, and in the data splitting, splits the input channel into the first group and the second group.

[0067] (Supplementary Note 7 The machine learning method according to supplementary note 6, wherein the input data is a channel in each of a plurality of image data constituting video data, in the data splitting, splits the input channel into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group.

[0068] (Supplementary Note 8) The machine learning method according to supplementary note 6, wherein, in the linking, performs a depth wise convolution process on the output obtained by the linking.

[0069] (Supplementary Note 9) A transitory computer-readable recording medium that includes a program including instructions recorded thereon, the instructions causing a computer to carry out: splitting input data into a first group and a second group; performing a two-dimensional convolution process on the first group; performing a one-dimensional convolution process on the second group; and linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

[0070] (Supplementary Note 10) The computer-readable medium according to supplementary note 9, wherein the input data is a channel in image data, and in the data splitting, splits the input channel into the first group and the second group.

[0071] (Supplementary Note 11) The computer-readable medium according to supplementary note 10, wherein the input data is a channel in each of a plurality of image data constituting video data, in the data splitting, splits the input channel into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group.

[0072] (Supplementary Note 12) The computer-readable medium according to supplementary note 10, wherein, in the linking, performs a depth wise convolution process on the output obtained by the linking.

[0073] Although the invention of the present application has been described above with reference to the example embodiment, the invention of the present application is not limited to the above-described example embodiment. Various changes that can be understood by a person skilled in the art within the scope of the invention of the present application can be made to the configuration and the details of the invention of the present application. INDUSTRIAL APPLICABILITY

[0074] As described above, according to the discloser, it is possible to enhance computational efficiency and reduce memory usage while maintaining high accuracy in video recognition tasks. The present disclosure is useful for image monitoring system and like. REFERENCE SIGNS LIST

[0075] 10 Machine learning apparatus 11 Data splitting unit 12 Two-dimensional convolution processing unit 13 One-dimensional convolution processing unit 14 Linking unit 15 Data acquisition unit 16 Output unit 20 CNN 30 Video data 31 Image data 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. A machine learning apparatus comprising: a data splitting means for splitting input data into a first group and a second group; a two-dimensional convolution processing means for performing a two-dimensional convolution process on the first group; a one-dimensional convolution processing means for performing a one-dimensional convolution process on the second group; and a linking means for linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

2. The machine learning apparatus according to claim 1, wherein the input data is a channel in image data, and the data splitting means splits the input channel into the first group and the second group.

3. The machine learning apparatus according to claim 2, wherein the input data is a channel in each of a plurality of image data constituting video data, the data splitting means splits the input channel into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group.

4. The machine learning apparatus according to claim 2, wherein the linking means performs a depth wise convolution process on the output obtained by the linking.

5. A machine learning method comprising: splitting input data into a first group and a second group; performing a two-dimensional convolution process on the first group; performing a one-dimensional convolution process on the second group; and linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

6. The machine learning method according to claim 5, wherein the input data is a channel in image data, and in the data splitting, splits the input channel into the first group and the second group.

7. The machine learning method according to claim 6, wherein the input data is a channel in each of a plurality of image data constituting video data, in the data splitting, splits the input channel into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group.

8. The machine learning method according to claim 6, wherein, in the linking, performs a depth wise convolution process on the output obtained by the linking.

9. A transitory computer-readable recording medium that includes a program including instructions recorded thereon, the instructions causing a computer to carry out: splitting input data into a first group and a second group; performing a two-dimensional convolution process on the first group; performing a one-dimensional convolution process on the second group; and linking an output from the two-dimensional convolution process and an output from the one-dimensional convolution process.

10. The computer-readable medium according to claim 9, wherein the input data is a channel in image data, and in the data splitting, splits the input channel into the first group and the second group.

11. The computer-readable medium according to claim 10, wherein the input data is a channel in each of a plurality of image data constituting video data, in the data splitting, splits the input channel into a spatial convolution path that constitutes the first group and a temporal convolution path that constitutes the second group.

12. The computer-readable medium according to claim 10, wherein, in the linking, performs a depth wise convolution process on the output obtained by the linking.

Images