Inference device, model generation device, inference method, and inference program
By expressing model parameters as a matrix of submatrices and adjusting their values to match a diagonal matrix product, the computational burden and resource demands are reduced, effectively addressing the limitations of existing methods in machine learning model calculations.
Patent Information
- Application Number
- JP2021036254
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-08
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2041-03-08
AI Technical Summary
Conventional methods for reducing the computational burden and calculation amount in machine learning models are inadequate, as distributed processing and fast Fourier transform methods fail to effectively decrease the computational resources required and are limited in applicability.
The proposed solution involves expressing at least some parameters of the machine learning model as a matrix comprising a first and second submatrix, where the number of elements in each row and column are the same, and adjusting the value of each element of the second submatrix to match the product of the first submatrix and a diagonal matrix, allowing for the reuse of calculation results and reducing the number of parameters.
This approach significantly reduces the amount of calculation required and alleviates the burden on computational resources by minimizing the number of parameters in the machine learning model, particularly when applied to neural networks for product inspection tasks.
Smart Images

Figure 0007581979000046 
Figure 0007581979000047 
Figure 0007581979000048
Abstract
Description
[Technical field]
[0001] The present invention relates to an inference device, a model generation device, an inference method, and an inference program. [Background technology]
[0002] Conventionally, in a scene where a product is manufactured, such as on a production line, a technology has been developed to inspect the quality of the product by photographing the product to be manufactured with an imaging device and analyzing the obtained image data. For example, Patent Document 1 proposes an inspection device that uses a trained first neural network to determine whether an object shown in an image is normal or abnormal, and if it is determined to be abnormal, uses a trained second neural network to classify the type of abnormality.
[0003] According to the method using a trained neural network, it is possible to carry out visual inspection of a product based on the calculation results of the trained neural network without manually specifying the contents of image processing in detail. Therefore, it is possible to simplify the information processing of the visual inspection and reduce the effort required to create an inspection program. This effect is not limited to the case where a neural network is used. Even in the method using a trained machine learning model other than a neural network (e.g., a principal component vector obtained by principal component analysis, a support vector machine, etc.), it is possible to simplify the information processing of the visual inspection and reduce the effort required to create an inspection program.
[0004] In addition, the inference tasks that can be performed by the trained machine learning model are not limited to visual inspection and inference on images. By using learning data corresponding to the desired information processing for machine learning, a trained machine learning model that has acquired the ability to perform the desired inference task on a predetermined type of data can be generated. Therefore, according to the method of using the trained machine learning model, the information processing for performing the desired inference task on the predetermined type of data can be simplified, and the effort required to create an inference program can be reduced.
[0005] In general, a machine learning model has parameters used for computing an inference task. The values of the parameters are adjusted by machine learning so that a desired output can be obtained for a desired input (i.e., so that the ability to perform a desired inference task can be acquired). When a neural network is adopted as a machine learning model, the parameters are, for example, weights of connections between neurons. The more one tries to improve the accuracy of performing an inference task, the more the number of parameters constituting the machine learning model tends to increase. In particular, when a neural network is adopted as a machine learning model, there is a tendency to deepen the hierarchy of the neural network in order to improve the accuracy of performing an inference task, and the deeper the hierarchy of the neural network, the more the number of parameters constituting the neural network increases.
[0006] Naturally, the more parameters that make up a machine learning model, the more the accuracy of the inference task can be expected to improve, but the amount of calculation required for the computation of the machine learning model increases. Therefore, in both machine learning and inference task scenarios, problems associated with the increase in computational complexity arise, such as delayed computation time, memory pressure, high power consumption due to the long computation time, and the need for expensive computers to complete the computation (i.e., a burden on computational resources).
[0007] Therefore, Patent Document 2 and Patent Document 3 propose a method for speeding up the computational processing of a machine learning model by distributed processing. Specifically, Patent Document 2 proposes a method for distributing deep learning by exchanging quantized gradients between a plurality of learning devices and a distributed deep learning device. Patent Document 3 also proposes a method for connecting a plurality of learning nodes via a ring-type communication network capable of one-way communication, and distributing deep learning by cooperative processing between the learning nodes. In addition, Non-Patent Document 1 proposes a method for reducing the number of calculations in a convolution operation by converting the calculation of the product of input data and parameters into a calculation formula of a different expression by fast Fourier transform. [Prior art documents] [Patent documents]
[0008] [Patent Document 1] JP 2012-026982 A [Patent Document 2] JP 2018-120441 A [Patent Document 3] JP 2020-003848 A [Non-patent literature]
[0009] [Non-Patent Document 1] Tyler Highlander, Andres Rodriguez, "Very Efficient Training of Convolutional Neural Networks using Fast Fourier Transform and Overlap-and-Add", arXiv:1601.06815 [cs.NE], 25 Jan 2016. Summary of the Invention [Problem to be solved by the invention]
[0010] The inventor of the present invention found that the above conventional method has the following problems. That is, in a method that relies only on distributed processing, by connecting multiple computers, the load on each computer can be reduced and the calculation processing can be accelerated, but it is difficult to reduce the amount of calculation required for the calculation processing of the machine learning model. Rather, the amount of calculation required for the calculation processing of the machine learning model increases by the amount of information on the calculation process communicated between each computer, and the overall burden on the calculation resources increases. In addition, in a method using a fast Fourier transform such as that in Non-Patent Document 1, it is possible to speed up the convolution operation, but it is difficult to apply it to other operations. In addition, the burden on the calculation resources increases by the amount of information on the parameters of the machine learning model and information on another expression obtained by the fast Fourier transform (especially, memory is compressed). Therefore, in the conventional method, it is difficult to reduce the amount of calculation required for the calculation processing of the machine learning model and suppress the burden on the calculation resources.
[0011] In one aspect, the present invention has been made in consideration of the above circumstances, and its purpose is to provide a technology for reducing the amount of calculation required for the computational processing of a machine learning model and reducing the burden on computational resources. [Means for solving the problem]
[0012] In order to solve the above-mentioned problems, the present invention employs the following configuration.
[0013] That is, an inference device according to one aspect of the present invention includes a data acquisition unit that acquires target data, an inference unit that performs an inference task on the acquired target data using an inference model trained by machine learning, and an output unit that outputs information regarding the result of performing the inference task. At least a portion of a plurality of parameters of the inference model is expressed in a matrix. The matrix includes a first submatrix and a second submatrix. The first submatrix and the second submatrix have the same number of elements in each row and column, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and a diagonal matrix.
[0014] In this configuration, at least a part of the parameters of the inference model can be expressed by a matrix including a first submatrix and a second submatrix, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and the diagonal matrix. Therefore, even if the information of the second submatrix is not held, the calculation result of the first submatrix can be reused to obtain the calculation result of the second submatrix. Therefore, according to this configuration, the number of parameters constituting the inference model can be substantially reduced. Specifically, the number of parameters can be reduced by the difference between the number of elements of the second submatrix and the number of elements other than 0 in the diagonal matrix. This reduces the amount of calculation required for the calculation processing of the inference model and suppresses the burden on calculation resources.
[0015] In the inference device according to the above aspect, at least a part of the matrix may be divided into M×N submatrices such that M submatrices having the same number of elements in each row and column are arranged in each column, and a scaling relationship may be established in each column such that a submatrix arranged in one row constitutes the first submatrix for each submatrix arranged in the other rows except for the one row, and each submatrix arranged in the other rows constitutes the second submatrix. According to this configuration, the number of parameters of the inference model can be effectively reduced by establishing the scaling relationship. This effectively reduces the amount of calculation required for the arithmetic processing of the inference model and suppresses the burden on computational resources.
[0016] In the inference device according to the above aspect, the M and the N may be the same prime number S. With this configuration, it is possible to easily adjust the range in which the scaling relationship holds.
[0017] In the inference device according to the above aspect, the scaling relationship may be repeatedly established in submatrices constituting the first submatrix, so that the scaling relationship is recursively established in at least a part of the matrix. According to this configuration, since the scaling relationship is recursively established, it is possible to reduce the number of parameters even in the first subsequence in the scaling relationship that is finally established (i.e., on the outermost side). Therefore, it is possible to further effectively reduce the amount of calculation required for the arithmetic processing of the inference model, and further reduce the burden on computational resources. Note that the establishment of the scaling relationship may be repeated until each submatrix becomes a matrix of an arbitrary size (for example, a 1×1 matrix).
[0018] In the inference device according to the above aspect, the M and the N may be the same prime number S, and at least a part of the matrix may be formed of a square matrix whose number of elements is a power of the prime number S. With this configuration, it is possible to easily adjust the range in which the scaling relationship holds.
[0019] In the inference device according to the above aspect, the M and the N may each be 2. With this configuration, it is possible to easily adjust the range in which the scaling relationship holds.
[0020] In the inference device according to the above aspect, the inference model may be configured by a neural network, and each element of the matrix may be configured to correspond to a weight of a connection between each neuron in the neural network. With this configuration, when a neural network is adopted as an inference model (machine learning model), it is possible to reduce the amount of calculation required for the arithmetic processing of the inference model and suppress the burden on computational resources.
[0021] In the inference device according to the above aspect, the target data may be composed of image data showing a product, and the inference task may be to determine whether or not the product shown in the image data has a defect. With this configuration, when performing an appearance inspection of a product using a trained inference model, it is possible to reduce the amount of calculation required for the arithmetic processing of the inference model and suppress the burden on computational resources.
[0022] Furthermore, the aspects of the present invention are not limited to the inference device described above. One aspect of the present invention may be a model generation device for generating a trained inference model for use in the inference device according to any of the above aspects.
[0023] For example, a model generating device according to one aspect of the present invention includes a data acquiring unit that acquires a plurality of learning data sets each composed of a combination of training data and a correct answer label indicating a correct answer to an inference task for the training data, and a learning processing unit that performs machine learning of an inference model using the plurality of learning data sets. At least a portion of a plurality of parameters of the inference model is expressed in a matrix. The matrix includes a first submatrix and a second submatrix. The number of elements in each row and column of the first submatrix and the second submatrix is the same. The machine learning is configured by training the inference model for each of the learning data sets such that a result of performing the inference task on the training data using the inference model matches the correct answer indicated by the correct answer label, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and a diagonal matrix. With this configuration, it is possible to reduce the amount of calculation required for the arithmetic processing of the inference model and to suppress the burden on computational resources when generating a trained inference model.
[0024] In the model generating device according to the above aspect, training the inference model may include dividing at least a part of the matrix into M×N submatrices such that M submatrices having the same number of elements in each row and column are arranged in each column, and adjusting values of each element of the matrix so that a scaling relationship is established in which a submatrix arranged in one row constitutes the first submatrix for each submatrix arranged in other rows except for the one row, and each submatrix arranged in the other rows constitutes the second submatrix. According to this configuration, the amount of calculation required for the calculation processing of the inference model can be reduced by the amount of the scaling relationship established, and the burden on calculation resources can be suppressed. Furthermore, the amount of reduction in the amount of calculation required for the calculation processing of the inference model can be managed based on the establishment of this scaling relationship. The establishment of the scaling relationship may be repeated in the submatrices constituting the first submatrix until each submatrix becomes a matrix of an arbitrary size (for example, a 1×1 matrix).
[0025] In the model generating device according to the above aspect, the inference model may be configured by a neural network. Each element of the matrix may be configured to correspond to a weight of a connection between each neuron in the neural network. Training the inference model may include adjusting values of each element of at least a part of the matrix so that the scaling relationship is recursively established in at least a part of the matrix by repeating the establishment of the scaling relationship in the submatrices constituting the first submatrix until each submatrix becomes a 1×1 matrix. Adjusting values of each element of at least a part of the matrix may include providing initial values of each element of at least a part of the matrix so that the scaling relationship is recursively established, deriving a result of performing an inference task on training data of each of the learning data sets in a forward propagation phase, and adjusting values of each element of at least a part of the matrix and each element of the diagonal matrix by backpropagating a gradient of an error between the derived result of performing the inference task and a correct answer indicated by a corresponding correct answer label in a backpropagation phase. The operation of the forward propagation phase includes a first step of calculating a product of the 1×1 matrix constituting an initial first submatrix and an input vector; a second step of deriving a product of the 1×1 matrix constituting an initial second submatrix and an input vector by calculating a product of the initial result of the multiplication of the first submatrix and a corresponding element of the diagonal matrix; a third step of deriving a product of a submatrix constituting a first submatrix of a next stage and an input vector from a result of the multiplication of the first submatrix and a result of the multiplication of the second submatrix obtained in a target stage; a fourth step of deriving a product of a submatrix constituting the second submatrix of a next stage and an input vector by calculating a result of the multiplication of the first submatrix of the next stage and a corresponding element of the diagonal matrix; and a fourth step of substituting the initial result of the multiplication of the first submatrix and the initial result of the multiplication of the second submatrix obtained in the first step and the second step, respectively, as initial values of the result of the multiplication of the first submatrix and the result of the multiplication of the second submatrix obtained in the target stage, respectively, and recursively repeating the third step and the fourth step.and a fifth step of deriving a product of at least a portion of the matrix and an input vector. The operations of the back propagation phase may include a first step of acquiring a gradient of the error with respect to a product of at least a portion of the matrix and an input vector, a second step of deriving a gradient of the error for each element of a diagonal matrix used in a final stage of recursive iteration of the forward propagation phase based on the acquired gradient of the error and a value of a product of a submatrix constituting the first submatrix and an input vector in the final stage, a third step of deriving a gradient of the error for each element of a diagonal matrix used in the final stage based on the acquired gradient of the error and a value of a product of a submatrix constituting the first submatrix and an input vector in a target stage of the recursive iteration of the forward propagation phase, and a fourth step of deriving a gradient of the error for each element of a diagonal matrix used in the target stage based on the acquired gradient of the error, a value of a product of a submatrix constituting the first submatrix and an input vector in a target stage of the recursive iteration of the forward propagation phase, and a fourth step of deriving a gradient of the error for each element of a diagonal matrix used in the target stage. a fourth step of deriving a gradient; a fifth step of deriving an error gradient for a product of a submatrix constituting the first submatrix and an input vector in the target phase based on the obtained error gradient, the value of each element of the diagonal matrix used in the target phase, and the error gradient for a product of the submatrix constituting the first submatrix and an input vector in the next phase; a sixth step of deriving an error gradient for each element of a diagonal matrix used in each phase by substituting the error gradient for a product of a submatrix constituting the first submatrix and an input vector in the final phase derived in the third step of the backpropagation phase as an initial value of the error gradient for a product of a submatrix constituting the first submatrix and an input vector in the next phase, and recursively repeating the fourth step and the fifth step of the backpropagation phase until an error gradient for each element of a diagonal matrix used in an initial stage of recursive iteration of the forward propagation phase is derived;The method may further include a seventh step of deriving an error gradient for elements of the 1×1 matrix constituting the initial first submatrix based on an error gradient for a product of a submatrix constituting the first submatrix and an input vector in the initial stage, and an eighth step of adjusting values of each element of the diagonal matrix used in each stage and each element of the 1×1 matrix constituting the initial first submatrix based on the error gradient derived for each element of the diagonal matrix used in each stage and each element of the 1×1 matrix constituting the initial first submatrix. According to this configuration, in a situation where a neural network is adopted as an inference model (machine learning model), the values of each parameter of the inference model can be adjusted so that a scaling relationship is recursively established during the machine learning process. As a result, the number of parameters can be reduced even in the first submatrix in the scaling relationship that is finally established. Therefore, according to this configuration, the amount of calculation required for the arithmetic processing of the inference model can be further reduced, and the burden on computational resources can be further suppressed. In this embodiment, the 1×1 matrix may be replaced with a unit matrix of any size.
[0026] As another aspect of each of the inference device and the model generation device according to the above-mentioned embodiments, one aspect of the present invention may be an information processing method or a program that realizes all or part of each of the above-mentioned configurations, or may be a storage medium that stores such a program and is readable by a computer or other device, machine, etc. Here, a storage medium that is readable by a computer, etc. is a medium that accumulates information such as a program by electrical, magnetic, optical, mechanical, or chemical action. Another aspect of the present invention may be an inference system constituted by the inference device and the model generation device according to any of the above-mentioned embodiments.
[0027] For example, an inference method according to one aspect of the present invention is an information processing method in which a computer executes the steps of acquiring target data, performing an inference task on the acquired target data using an inference model trained by machine learning, and outputting information related to the results of performing the inference task, wherein at least a portion of a plurality of parameters of the inference model are expressed in a matrix, the matrix includes a first submatrix and a second submatrix, the number of elements in each row and column of the first submatrix and the second submatrix are identical, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and a diagonal matrix.
[0028] For example, an inference program according to one aspect of the present invention is a program for causing a computer to execute the steps of acquiring target data, performing an inference task on the acquired target data using an inference model trained by machine learning, and outputting information regarding the results of performing the inference task, wherein at least a portion of a plurality of parameters of the inference model are represented in a matrix, the matrix includes a first submatrix and a second submatrix, the number of elements in each row and column of the first submatrix and the second submatrix are identical, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and a diagonal matrix. Effect of the Invention
[0029] According to the present invention, it is possible to reduce the amount of calculation required for the computational processing of a machine learning model and to suppress the burden on computational resources. [Brief description of the drawings]
[0030] [Figure 1] FIG. 1 illustrates a schematic example of a situation in which the present invention is applied. [Diagram 2] FIG. 2 illustrates an example of a hardware configuration of the model generating device according to the embodiment. [Diagram 3]FIG. 3 illustrates an example of a hardware configuration of the inference device according to the embodiment. [Figure 4] FIG. 4 illustrates an example of a software configuration of the model generating device according to the embodiment. [Figure 5A] FIG. 5A shows an example of a method for setting the first submatrix and the second submatrix. [Figure 5B] FIG. 5B shows an example where scaling relationships hold recursively within a 4×4 parameter matrix. [Figure 5C] FIG. 5C shows an example of a generalization of element decomposition where the scaling relationship holds recursively and at each stage the submatrix in the first row constitutes the first submatrix. [Figure 6] FIG. 6 illustrates an example of a software configuration of the inference device according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of a processing procedure of the model generating device according to the embodiment. [Figure 8A] FIG. 8A shows an example of the process of calculating a parameter matrix (2×2 matrix) in the forward propagation phase. [Figure 8B] FIG. 8B shows an example of the calculation process of the parameter matrix (2×2 matrix) in the backpropagation phase. [Figure 9A] FIG. 9A shows an example of the calculation process of a parameter matrix (4×4 matrix) in the forward propagation phase. [Figure 9B] FIG. 9B shows an example of the calculation process of the parameter matrix (4×4 matrix) in the backpropagation phase. [Figure 10A] FIG. 10A shows an example of the process of calculating a parameter matrix (a matrix of any size) in the forward propagation phase. [Figure 10B] FIG. 10B shows an example of the process of calculating a parameter matrix (a matrix of any size) in the backpropagation phase. [Figure 11] FIG. 11 is a flowchart illustrating an example of a processing procedure of the inference device according to the embodiment. [Figure 12] FIG. 12 illustrates a schematic example of another scene to which the present invention is applied. [Figure 13] FIG. 13 illustrates a schematic example of another scene to which the present invention is applied. [Figure 14] FIG. 14 illustrates a schematic example of another scene to which the present invention is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0031] An embodiment according to one aspect of the present invention (hereinafter, also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, in implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.
[0032] §1 Examples of application 1 is a schematic diagram illustrating an example of a situation in which the present invention is applied. As shown in FIG. 1, an inference system 100 according to this embodiment includes a model generating device 1 and an inference device 2.
[0033] The model generation device 1 according to this embodiment is a computer configured to generate a trained inference model 5 by machine learning. In this embodiment, the model generation device 1 acquires a plurality of learning data sets 3. Each learning data set 3 is composed of a combination of training data 31 and a correct answer label 32. The training data 31 may be appropriately configured according to an inference task, etc. The correct answer label 32 is configured to indicate a correct answer (true value) of the inference task for the training data 31.
[0034] Any type of task for inferring features contained in data may be selected as the inference task of the inference model 5. As an example, when generating a trained inference model 5 for visual inspection using images, the inference task may be to determine whether or not a product shown in image data has a defect (e.g., may include tasks such as extraction of the presence or absence of a defect, the type of defect, and the range in which the defect exists). In this case, the training data 31 may be configured with image data showing the product, and the correct answer label 32 may be configured to indicate the correct answer of the inference task for the associated training data 31 (e.g., the presence or absence of a defect, the type of defect, the range in which the defect exists, etc.). The model generation device 1 performs machine learning of the inference model 5 using multiple learning datasets 3.
[0035] The inference model 5 is configured by a machine learning model having a plurality of parameters. The type of the machine learning model is not particularly limited and may be appropriately selected according to the embodiment. For example, a neural network, a principal component vector obtained by principal component analysis, a support vector machine, etc. may be adopted as the inference model 5. For convenience of explanation, an example in which a neural network is adopted as the machine learning model constituting the inference model 5 will be shown below (FIG. 4 described later). When the inference model 5 is configured by a neural network, the weight of the connection between each neuron (node) is an example of a parameter.
[0036] In this embodiment, at least a portion of the multiple parameters of the inference model 5 are configured to be expressible by a matrix. The matrix is configured to include a first submatrix and a second submatrix. The first submatrix and the second submatrix are configured to have the same number of elements in each row and column. The number of rows and columns of each submatrix may be appropriately selected depending on the embodiment. The number of rows and the number of columns may be the same or different from each other. The first submatrix and the second submatrix may be scalars (1×1 matrices).
[0037] The machine learning is configured by training the inference model 5 for each learning data set 3 so that the result of performing an inference task on the training data 31 using the inference model 5 matches the correct answer indicated by the correct answer label 32, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and the diagonal matrix. This makes it possible to generate a trained inference model 5 that has acquired the ability to perform an inference task and in which the value of each element of the second submatrix is adjusted to match the product of the first submatrix and the diagonal matrix.
[0038] In the example of FIG. 1, w1-w4 are an example of parameters expressed by a matrix. Among these, w1 and w2 are an example of a first submatrix. w3 and w4 are an example of a second submatrix. (y1, y2) indicates the calculation result (output) of the component part expressed by this matrix in the inference model 5. (x1, x2) indicates the input to this component part. As an example, when each layer constituting a neural network is expressed by the above matrix, the input to each layer corresponds to (x1, x2), and the output from each layer corresponds to (y1, y2). (a1, a2) are an example of elements of the diagonal components of a diagonal matrix. When the values of each element of the diagonal components of a diagonal matrix are the same, the diagonal matrix may be expressed by one value (scalar).
[0039] On the other hand, the inference device 2 according to this embodiment is a computer configured to perform an inference task using a trained inference model 5. In this embodiment, the inference device 2 acquires target data 221. Next, the inference device 2 performs an inference task on the acquired target data 221 using the inference model 5 trained by machine learning. As described above, at least a part of the multiple parameters of the inference model 5 is expressed in a matrix. The matrix includes a first submatrix and a second submatrix. The first submatrix and the second submatrix have the same number of elements in each row and column. The value of each element of the second submatrix is adjusted to match the product of the first submatrix and the diagonal matrix. The inference device 2 outputs information related to the result of performing the inference task.
[0040] As described above, in this embodiment, at least a part of the parameters of the inference model 5 can be expressed by a matrix including the first submatrix and the second submatrix. According to the model generating device 1 of this embodiment, in the trained inference model 5, the value of each element of the second submatrix is adjusted so as to match the product of the first submatrix and the diagonal matrix. Therefore, even if the information of the second submatrix is not held, the calculation result of the first submatrix can be reused to obtain the calculation result of the second submatrix. In the example of FIG. 1, the calculation result of the second submatrix (a1w1x1 and a2w2x2) can be obtained from the calculation result of the first submatrix (w1x1 and w2x2) without performing the calculation of w3x1 and w4x2. Therefore, according to this embodiment, the number of parameters constituting the inference model 5 can be substantially reduced. Specifically, the number of parameters can be reduced by the difference between the number of elements of the second submatrix and the number of elements other than 0 in the diagonal matrix. This reduces the amount of calculation required for the calculation processing of the inference model 5 and suppresses the burden on the calculation resources.
[0041] In the example of FIG. 1, the model generation device 1 and the inference device 2 are connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like. However, the method of exchanging data between the model generation device 1 and the inference device 2 is not limited to this example, and may be appropriately selected depending on the embodiment. For example, data may be exchanged between the model generation device 1 and the inference device 2 using a storage medium.
[0042] 1, the model generation device 1 and the inference device 2 are each configured by separate computers. However, the configuration of the inference system 100 according to this embodiment is not limited to this example and may be determined appropriately depending on the embodiment. For example, the model generation device 1 and the inference device 2 may be a single computer. Also, for example, at least one of the model generation device 1 and the inference device 2 may be configured by multiple computers.
[0043] §2 Configuration example [Hardware configuration] <Model generation device> Fig. 2 illustrates a schematic example of a hardware configuration of the model generating device 1 according to this embodiment. As shown in Fig. 2, the model generating device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17 are electrically connected. In Fig. 2, the communication interface and the external interface are described as a "communication I / F" and an "external I / F".
[0044] The control unit 11 includes a hardware processor such as a central processing unit (CPU), a random access memory (RAM), and a read only memory (ROM), and is configured to execute information processing based on programs and various data. The storage unit 12 is an example of a memory, and is configured, for example, by a hard disk drive or a solid state drive. In this embodiment, the storage unit 12 stores various information such as a model generation program 81, a plurality of learning data sets 3, learning result data 125, and the like.
[0045] The model generation program 81 is a program for causing the model generation device 1 to execute machine learning information processing ( FIG. 7 ) described below for generating a trained inference model 5. The model generation program 81 includes a series of instructions for the information processing. A plurality of learning datasets 3 are used to generate the trained inference model 5. The learning result data 125 indicates information about the generated trained inference model 5. In this embodiment, the learning result data 125 is generated as a result of executing the model generation program 81. Details will be described later.
[0046] The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. The model generating device 1 can use the communication interface 13 to execute data communication via a network with other information processing devices. The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting to an external device. The type and number of the external interfaces 14 may be selected arbitrarily. The model generating device 1 may be connected to a device for obtaining training data 31 (for example, a camera when the training data 31 is image data) via at least one of the communication interface 13 and the external interface 14.
[0047] The input device 15 is a device for performing input, such as a mouse, a keyboard, etc. The output device 16 is a device for performing output, such as a display, a speaker, etc. An operator such as a user can operate the model generation device 1 by using the input device 15 and the output device 16.
[0048] The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as a program stored in a storage medium 91. The storage medium 91 is a medium that accumulates information such as a program by electrical, magnetic, optical, mechanical, or chemical action so that a computer or other device, machine, or the like can read various information such as the stored program. At least one of the model generation program 81 and the multiple learning data sets 3 may be stored in the storage medium 91. The model generation device 1 may acquire at least one of the model generation program 81 and the multiple learning data sets 3 from the storage medium 91. Note that, in FIG. 2, a disk-type storage medium such as a CD or a DVD is illustrated as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to the disk type, and may be other than the disk type. As an example of a storage medium other than the disk type, a semiconductor memory such as a flash memory can be cited. The type of the drive 17 may be arbitrarily selected according to the type of the storage medium 91.
[0049] Regarding the specific hardware configuration of the model generating device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), or the like. The storage unit 12 may be configured with a RAM and a ROM included in the control unit 11. At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16, and the drive 17 may be omitted. The model generating device 1 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. In addition, the model generating device 1 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a PC (Personal Computer), an industrial PC (IPC), or the like.
[0050] <Inference device> Fig. 3 illustrates a schematic example of a hardware configuration of the inference device 2 according to this embodiment. As shown in Fig. 3, the inference device 2 according to this embodiment is a computer in which a control unit 21, a storage unit 22, a communication interface 23, an external interface 24, an input device 25, an output device 26, and a drive 27 are electrically connected.
[0051] The control unit 21 to the drive 27 and the storage medium 92 of the inference device 2 may be configured similarly to the control unit 11 to the drive 17 and the storage medium 91 of the model generation device 1. The control unit 21 includes a CPU, RAM, ROM, etc., which are hardware processors, and is configured to execute various information processing based on programs and data. The storage unit 22 is configured, for example, with a hard disk drive, a solid state drive, etc. In this embodiment, the storage unit 22 stores various information such as an inference program 82, learning result data 125, etc.
[0052] The inference program 82 is a program for causing the inference device 2 to execute information processing ( FIG. 11 ) described below that performs an inference task using a trained inference model 5. The inference program 82 includes a series of instructions for the information processing. At least one of the inference program 82 and the learning result data 125 may be stored in a storage medium 92. Furthermore, the inference device 2 may acquire at least one of the inference program 82 and the learning result data 125 from the storage medium 92.
[0053] The inference device 2 may be connected to a device (sensor, other computer, etc.) for acquiring target data 221 via at least one of a communication interface 23 and an external interface 24. The inference device 2 may receive operations and inputs from an operator such as a user by using an input device 25 and an output device 26.
[0054] Regarding the specific hardware configuration of the inference device 2, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 21 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, an FPGA, a DSP, or the like. The storage unit 22 may be configured with a RAM and a ROM included in the control unit 21. At least one of the communication interface 23, the external interface 24, the input device 25, the output device 26, and the drive 27 may be omitted. The inference device 2 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the inference device 2 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a general-purpose PC, an industrial PC, a PLC (programmable logic controller), or the like.
[0055] [Software configuration] <Model generation device> 4 illustrates a schematic example of a software configuration of the model generating device 1 according to the present embodiment. The control unit 11 of the model generating device 1 loads the model generating program 81 stored in the storage unit 12 in the RAM. The control unit 11 then uses the CPU to interpret and execute instructions included in the model generating program 81 loaded in the RAM to control each component. As a result, as shown in FIG. 4, the model generating device 1 according to the present embodiment operates as a computer including a data acquiring unit 111, a learning processing unit 112, and a storage processing unit 113 as software modules. That is, in the present embodiment, each software module of the model generating device 1 is realized by the control unit 11 (CPU).
[0056] The data acquiring unit 111 is configured to acquire a plurality of learning data sets 3. Each learning data set 3 is configured by a combination of training data 31 and a correct answer label 32. The training data 31 may be configured appropriately according to an inference task, etc. The correct answer label 32 is configured to indicate a correct answer (true value) of the inference task for the training data 31.
[0057] The learning processing unit 112 is configured to perform machine learning of the inference model 5 using a plurality of learning datasets 3. The inference model 5 is configured by a machine learning model having a plurality of parameters. As long as at least a portion of the plurality of parameters of the inference model 5 can be expressed by a matrix, the type of the inference model 5 is not particularly limited and may be appropriately selected according to the embodiment.
[0058] In this embodiment, at least a part of the multiple parameters of the inference model 5 is configured to be expressible by a matrix. The matrix (hereinafter also referred to as a parameter matrix) is configured to include a first submatrix and a second submatrix. The first submatrix and the second submatrix are configured to have the same number of elements in each row and column. The machine learning is configured by training the inference model 5 for each learning dataset 3 so that the result of performing an inference task on the training data 31 using the inference model 5 matches the correct answer indicated by the correct answer label 32, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and the diagonal matrix.
[0059] The storage processing unit 113 is configured to generate information about the trained inference model 5 generated by machine learning as learning result data 125, and to store the generated learning result data 125 in a predetermined storage area. The learning result data 125 may be appropriately configured to include information for reproducing the trained inference model 5.
[0060] (Inference model) As shown in FIG. 4, an example of the inference model 5 may be configured by a neural network. In the example of FIG. 4, the neural network configuring the inference model 5 includes an input layer 51, one or more intermediate (hidden) layers 52, and an output layer 53. The number of intermediate layers 52 may be appropriately determined depending on the embodiment. Each of the layers 51 to 53 includes one or more neurons (nodes). The number of neurons included in each of the layers 51 to 53 may be appropriately determined depending on the embodiment. In addition, the connection relationship between the neurons included in each of the layers 51 to 53 may also be appropriately determined depending on the embodiment. In one example, each neuron included in each of the layers 51 to 53 may be connected to all neurons in the adjacent layers. As a result, the inference model 5 may be configured by a fully connected neural network.
[0061] A weight (connection weight) is set for each connection in each layer 51 to 53. A threshold is set for each neuron, and basically, the output of each neuron is determined depending on whether the sum of the products of each input and each weight exceeds the threshold. The threshold may be expressed by an activation function. In this case, the output of each neuron is determined by inputting the sum of the products of each input and each weight into the activation function and executing the calculation of the activation function. The type of activation function may be selected arbitrarily. The weight of the connection between each neuron included in each layer 51 to 53 is an example of a parameter of the inference model 5. That is, when the inference model 5 is configured by a neural network, each element of the parameter matrix may be configured to correspond to the weight of the connection between each neuron in the neural network.
[0062] When the inference model 5 is configured by a neural network as illustrated in FIG. 4, the parameters of each of the layers 51 to 53 can be expressed by different matrices. In this case, at least a part of at least one of the layers 51 to 53 configuring the neural network may be configured by a parameter matrix including a first submatrix and a second submatrix as a result of machine learning. In one example, each of the layers 51 to 53 may be configured by a parameter matrix different from each other. In this case, the above (x1, x2) in each parameter matrix corresponds to the input data for each of the layers 51 to 53, and the above (y1, y2) corresponds to the calculation result (output) of each of the layers 51 to 53.
[0063] In machine learning, the learning processing unit 112 uses the training data 31 of each learning data set 3 as input data, and uses the correct answer label 32 as a teacher signal. That is, the learning processing unit 112 inputs the training data 31 of each learning data set 3 to the input layer 51, and executes the forward propagation arithmetic processing of the inference model 5 (forward propagation phase). As a result of this forward propagation arithmetic processing, the learning processing unit 112 acquires an output value corresponding to the result of performing an inference task on the training data 31 (i.e., the result of inferring the solution of the inference task) from the output layer 53. The format of the output value obtained from the output layer 53 is not particularly limited as long as the inference result can be specified, and may be appropriately determined according to the embodiment. In one example, the output value of the output layer 53 may be configured to directly indicate the inference result. In another example, the output value of the output layer 53 may be configured to indirectly indicate the inference result via any arithmetic processing such as threshold processing.
[0064] The learning processing unit 112 calculates the error between the obtained result of performing the inference task and the correct label 32 associated with the input training data 31, and further calculates the gradient of the calculated error. Then, the learning processing unit 112 backpropagates the calculated gradient of the error by the backpropagation method to adjust the value of each parameter (backpropagation phase). During this adjustment, the learning processing unit 112 adjusts the value of each element of the second submatrix so that it matches the product of the first submatrix and the diagonal matrix. The learning processing unit 112 repeats the above series of processes for adjusting the value of each parameter for each training data set 3 so that the sum of the calculated errors becomes smaller. As a result of this machine learning, it is possible to generate a trained inference model 5 that has acquired the ability to perform the inference task and has been adjusted so that the value of each element of the second submatrix matches the product of the first submatrix and the diagonal matrix.
[0065] The storage processing unit 113 generates learning result data 125 for reproducing the trained inference model 5 generated by the machine learning. As long as the trained inference model 5 can be reproduced, the configuration of the learning result data 125 is not particularly limited and may be appropriately determined according to the embodiment. As an example, the learning result data 125 may include information indicating the values of each parameter obtained by the adjustment of the machine learning. In this case, since the value of each element of the second submatrix coincides with the product of the first submatrix and the diagonal matrix, the information indicating the values of the parameters constituting the second submatrix can be omitted. That is, the learning result data 125 may be configured to include information indicating the values of the parameters constituting the first submatrix obtained by the adjustment of the machine learning and each value of the diagonal elements of the diagonal matrix. In some cases, the learning result data 125 may further include information indicating the structure of the inference model 5. The structure may be specified, for example, by the number of layers from the input layer to the output layer, the type of each layer, the number of neurons included in each layer, the connection relationship between neurons in adjacent layers, etc. The storage processing unit 113 stores the generated learning result data 125 in a predetermined storage area.
[0066] (Relationship between the first and second submatrices) Here, an example of the relationship between the first submatrix and the second submatrix will be described. In this embodiment, if some elements of the parameter matrix correspond to the first submatrix and at least some of the remaining elements correspond to the second submatrix (i.e., if a proportional relationship exists in at least a part of the parameter matrix of the trained inference model 5), the effect of reducing the number of parameters can be obtained. Therefore, it is sufficient that the proportional relationship exists in at least a part of the parameter matrix. However, in order to effectively reduce the number of parameters, it is preferable that the range that corresponds to the first submatrix and the second submatrix (particularly the second submatrix) is wide in the parameter matrix.
[0067] 5A shows an example of a method for setting the first and second submatrices to effectively reduce the number of parameters. In this setting method, the first and second submatrices are set so that the scaling relationship defined by the following conditions (1) to (3) holds in at least a part of the parameter matrix. Condition (1): At least a part of the parameter matrix is divided into M × N submatrices such that M rows and N columns of submatrices each have the same number of elements. Condition (2): In each column, a submatrix arranged in any one of the rows constitutes a first submatrix corresponding to each of the submatrices arranged in the other rows excluding the said one of the rows. Condition (3): Each submatrix placed in another row constitutes a second submatrix.
[0068] In the example of FIG. 5A, W corresponds to at least a part of the parameter matrix, X corresponds to the input data for that part, and Y corresponds to the operation result (output) of that part. MN corresponds to M × N submatrices. Each submatrix w1-w MN The number of elements in each row and column of each submatrix w1-w MN As long as the submatrix w1-w MN can be a 1×1 matrix (scalar). Each input (x1-xN ) The number of elements in each submatrix w1-w MN corresponds to the number of elements in the column.
[0069] In the example of FIG. 5A, each submatrix w1-w MN Of these, each submatrix w1-w N constitutes the first submatrix for the submatrices of other rows (second row and each row after that) in each column. Then, each of the remaining submatrices w N+1 -w MN constitutes the second submatrix for the first submatrix in the corresponding column. In other words, the first submatrix w t is the submatrix w for each row from the second row onwards. rN+t Construct the first submatrix for each submatrix w rN+t is the submatrix w t (where t is a natural number from 1 to N, and r is a natural number from 1 to M-1). 2,1 -a M,N is a diagonal matrix. 2,1 -a M,N The number of elements in the submatrix w1-w N corresponds to the number of elements in a row of
[0070] The values of M and N are not particularly limited as long as they are natural numbers equal to or greater than 2, and may be appropriately determined depending on the embodiment. In one example, M and N may be the same prime number S. As a specific example, M and N may each be 2 (example of FIG. 1). In this case, at least some of the rows and columns of the parameter matrix are each divided into two, and the submatrices corresponding to the first submatrix and the second submatrix can be easily identified. In another example, M and N may be different values from each other.
[0071] In the example of FIG. 5A, the submatrix arranged in the first row of each column constitutes a first submatrix for the submatrix arranged in the other rows (second row and subsequent rows) of each column, and the submatrix arranged in the second row and subsequent rows of each column constitutes a second submatrix. However, the arrangement of the first submatrix and the second submatrix may not be limited to such an example. In each column, the first submatrix may be arranged in any row from the second row onwards. Also, in the example of FIG. 5A, the submatrix in the same row of each column constitutes the first submatrix, but the arrangement of the first submatrix may not be limited to such an example. That is, the rows in which the submatrix constituting the first submatrix is arranged may differ between each column (for example, in the first column, the submatrix in the first row constitutes the first submatrix, while in the second column, the submatrix in any row from the second row onwards may constitute the first submatrix).
[0072] In order to further effectively reduce the number of parameters, the scaling relationship may be recursively established in at least a part of the parameter matrix (i.e., the scaling relationship may be further established in each submatrix). The number of times the scaling relationship is recursively established may not be particularly limited and may be appropriately determined according to the embodiment. The establishment of the scaling relationship may be repeated until each submatrix becomes a matrix of any size. In this case, the matrix of any size may be called a "unit matrix". However, in order to further effectively reduce the number of parameters, it is desirable that the scaling relationship is recursively established such that the establishment of the scaling relationship in the submatrices constituting the first submatrix is repeated until each submatrix becomes a 1×1 matrix (scalar). This effectively reduces the amount of calculation required for the arithmetic processing of the inference model 5 and suppresses the burden on the computational resources.
[0073] FIG. 5B shows an example of a scene where the scaling relationship is recursively established in a 4×4 parameter matrix until each submatrix becomes a 1×1 matrix. First, the 4×4 parameter matrix W is divided into 2×2 submatrices (size 2×2). Among them, W1 and W2 in the first row of each column constitute the first submatrix, and the submatrix in the second row of each column constitute the second submatrix. A1 and A2 (size 2×2) are diagonal matrices used in this stage. As a result, the scaling relationship is established in the 4×4 parameter matrix. Then, each first submatrix (W1, W2) is further divided into 2×2 submatrices (size 1×1). Among them, the submatrix (w1-w4) in the first row of each column in each submatrix constitutes the first submatrix, and the submatrix in the second row of each column constitutes the second submatrix. a1-a4 (size 1×1) are diagonal matrices used in this stage. Within each first submatrix (W1, W2), a scaling relationship is further established. Only the diagonal elements of the diagonal matrix need to be retained. Therefore, in the example of FIG. 5B, the 16 parameters in W can be reduced to 12 (w1-w4, a1-a4, A1, A2).
[0074] In the example of FIG. 5B, the scaling relationship is established recursively in two stages, and the number of divisions M×N in each stage is the same (2×2). However, the number of times that the relationship is established recursively is not limited to this example, and may be determined appropriately depending on the embodiment. Also, the number of divisions M×N in each stage may differ at least in part. For example, if there is a difference in M between the number of divisions M1×N1 in the first stage and the number of divisions M2×N2 in the second stage, i and N i At least one of them may be different.
[0075] FIG. 5C shows an example of a generalization of element decomposition where the scaling relationship holds recursively and at each stage the submatrix in the first row constitutes the first submatrix. j i But, p i-1 ×q i-1Assuming that the first row of the submatrix at each stage constitutes the first submatrix, the corresponding matrix W j i The submatrix in can be expressed by the formula in FIG. 5C. i corresponds to the recursion stage. A natural number from 1 to l+1 is substituted for i. The element division in FIG. 5C is valid when i is in the range from 2 to l+1. l indicates the number of times the scaling relationship is recursively valid. j corresponds to the number of the arrangement column of the corresponding matrix at each stage. j can range from 1 to J i A natural number up to is substituted. i is shown by the following equation 1.
[0076]
number
[0077] p at each stage i-1 and q i-1 corresponds to the number of divisions M and N at each stage. p i-1 and q i-1 The value of p is not particularly limited and may be appropriately determined depending on the embodiment. The number of divisions in each stage may be the same or different between the stages. i-1 and q i-1 The values of p may be the same or different. i-1 and q i-1 may be the same prime number S, and the number of divisions may be the same between each stage. For example, the prime number S may be 2. In this case, at least a part of the parameter matrix (W l+1 ) is a function whose number of elements is a power of a prime number S (S 2l) This makes it easy to grasp the submatrices corresponding to the first and second submatrices.
[0078] In the following, for convenience of explanation, unless otherwise specified, it is assumed that the submatrix constituting the first submatrix of each column in each stage is arranged in the first row, and the submatrices in the second row and onward constitute the second submatrix. However, the arrangement of the submatrix constituting the first submatrix of each column in each stage does not have to be limited to the first row. The first submatrix of each column in each stage may be arranged anywhere in the second row and onward. The arrangement of the first submatrix of each column may be the same or different between stages. In addition, the arrangement of W in the final division j 1 need not be limited to a 1×1 matrix. In another example, W in the final division j 1 can be a matrix of any size.
[0079] <Inference device> 6 illustrates a schematic example of a software configuration of the inference device 2 according to this embodiment. The control unit 21 of the inference device 2 loads the inference program 82 stored in the storage unit 22 onto the RAM. The control unit 21 then uses the CPU to interpret and execute instructions included in the inference program 82 loaded onto the RAM to control each component. As a result, as shown in FIG. 6, the inference device 2 according to this embodiment operates as a computer including a data acquisition unit 211, an inference unit 212, and an output unit 213 as software modules. That is, in this embodiment, each software module of the inference device 2 is also realized by the control unit 21 (CPU) like the model generation device 1.
[0080] The data acquisition unit 211 is configured to acquire target data 221. The inference unit 212 holds the learning result data 125 and is thus provided with an inference model 5 that has been trained by machine learning. The inference unit 212 is configured to perform an inference task on the acquired target data 221 using the trained inference model 5. The output unit 213 is configured to output information relating to a result of performing the inference task.
[0081] <Other> Each software module of the model generating device 1 and the inference device 2 will be described in detail in an operation example described later. In this embodiment, an example is described in which each software module of the model generating device 1 and the inference device 2 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors (e.g., graphics processing units). Each of the above modules may be realized as a hardware module. Furthermore, with regard to the software configurations of the model generating device 1 and the inference device 2, software modules may be omitted, replaced, or added as appropriate depending on the embodiment.
[0082] §3 Example of operation [Model generation device] 7 is a flowchart showing an example of a processing procedure related to machine learning by the model generation device 1 according to the present embodiment. The processing procedure of the model generation device 1 described below is an example of a model generation method. However, the processing procedure of the model generation device 1 described below is merely an example, and each step may be changed as much as possible. In addition, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.
[0083] (Step S101) In step S101, the control unit 11 operates as a data acquisition unit 111 and acquires a plurality of learning data sets 3, each of which is composed of a combination of training data 31 and a correct answer label 32 indicating a correct answer to an inference task for the training data 31.
[0084] Each learning data set 3 may be generated as appropriate. For example, an environment in which an inference task is performed in real space or virtual space is assumed, and training data 31 (training sample) can be acquired by observing any object in the environment. The observed object may be selected as appropriate according to the inference task. Any sensor may be used to acquire the training data 31. As an example, when the inference task is an appearance inspection of a product, the training data 31 can be acquired by photographing a product with or without a defect using a camera. Next, information indicating the result (correct answer / true value) of performing the inference task on the acquired training data 31 is acquired as a correct answer label 32. In this case, the inference task may be performed manually by an operator or the like. Then, the acquired correct answer label 32 is associated with the training data 31. In this way, each learning data set 3 can be generated.
[0085] Each learning data set 3 may be generated automatically by the operation of a computer, or may be generated manually by at least partially including an operation of an operator. Furthermore, each learning data set 3 may be generated by the model generating device 1, or may be generated by a computer other than the model generating device 1. When each learning data set 3 is generated by the model generating device 1, the control unit 11 acquires a plurality of learning data sets 3 by automatically executing the above series of generation processes or manually by an operation of an operator via the input device 15. On the other hand, when each learning data set 3 is generated by another computer, the control unit 11 acquires a plurality of learning data sets 3 generated by the other computer, for example, via a network, a storage medium 91, or the like. Some of the learning data sets 3 may be generated by the model generating device 1, and the other learning data sets 3 may be generated by one or more other computers.
[0086] There may be any number of acquired learning data sets 3. When a plurality of learning data sets 3 are acquired, the control unit 11 advances the process to the next step S102.
[0087] (Step S102) In step S102, the control unit 11 operates as the learning processing unit 112 and performs machine learning of the inference model 5 using a plurality of learning data sets 3. In this embodiment, the machine learning is configured by training the inference model 5 so that, for each learning data set 3, a result of performing an inference task on the training data 31 using the inference model 5 matches a correct answer indicated by the correct answer label 32, and the value of each element of the second submatrix is adjusted to match the product of the first submatrix and the diagonal matrix.
[0088] When generating a trained inference model 5 in which the above-mentioned scaling relationship holds, training the inference model 5 includes adjusting the value of each element of at least a part of the parameter matrix so that the scaling relationship holds. Furthermore, when generating a trained inference model 5 in which the above-mentioned scaling relationship holds recursively, training the inference model 5 includes adjusting the value of each element of at least a part of the parameter matrix so that the scaling relationship holds recursively within at least a part of the parameter matrix by repeating the establishment of the scaling relationship within the submatrices constituting the first submatrix. This recursive establishment of the scaling relationship may be repeated until each submatrix becomes a 1×1 matrix (scalar). In other words, the first submatrix (W j 1 ) may be a 1 × 1 matrix. The machine learning method may be appropriately selected depending on the type of machine learning model that constitutes the inference model 5.
[0089] In one example, the training of the inference model 5 (adjusting the values of at least some of the elements of the parameter matrix) is - Initialize the inference model 5 (give initial values to each parameter); In the forward propagation phase, deriving the results of trial execution of an inference task on the training data 31 of each learning data set 3; and In a backpropagation phase, adjusting the values of the parameters of the inference model 5 (at least some elements of the parameter matrix and each element of the diagonal matrix) by backpropagating the gradient of the error between the derived inference task execution result and the correct answer indicated by the corresponding correct answer label 32; It is composed of:
[0090] (Initial setting) First, the control unit 11 performs initial settings of the inference model 5 to be processed by machine learning. In this embodiment, the control unit 11 may provide the structure of the neural network (e.g., the number of layers, the type of each layer, the number of neurons included in each layer, the connection relationship between neurons in adjacent layers, etc.) and the initial values of the weights of the connections between each neuron by a template or by input from an operator. Furthermore, when re-learning is performed, the control unit 11 may perform initial settings of the neural network based on learning result data obtained by past machine learning.
[0091] In this embodiment, each element of the parameter matrix corresponds to a weight of a connection between each neuron. Therefore, providing an initial value of the weight of a connection between each neuron corresponds to providing an initial value of each element of the parameter matrix. Since the second submatrix is expressed by the product of the first submatrix and the diagonal matrix, providing an initial value of an element corresponding to the second submatrix may be omitted. In other words, providing an initial value of a weight of a connection between each neuron may be configured by providing an element corresponding to the first submatrix and an initial value of each element of the diagonal components of the diagonal matrix.
[0092] When generating a trained inference model 5 in which the scaling relationship holds recursively, the control unit 11 gives initial values to at least some of the elements of the parameter matrix so that the scaling relationship holds recursively. As a specific example, the control unit 11 sets the initial values of W j 1 and the initial values of the diagonal matrices used at each stage.
[0093] (Forward propagation phase) Next, in the forward propagation phase, the control unit 11 inputs the training data 31 of each learning data set 3 to the input layer 51, and executes the forward propagation calculation process of the inference model 5. In this forward propagation calculation process, the control unit 11 first calculates the product of the submatrix constituting the first submatrix and the input vector. When the parameter matrix corresponds to the input layer 51, the input vector is the training data 31 of each learning data set 3. When the parameter matrix corresponds to the intermediate layer 52 or the output layer 53, the input vector is the calculation result (output) of the layer preceding the corresponding layer. Next, the control unit 11 derives the product of the submatrix constituting the second submatrix and the input vector by calculating the product of the result of the product of the first submatrix and the corresponding elements of the diagonal matrix. That is, the control unit 11 multiplies the result of the product of the first submatrix and the diagonal matrix, and obtains the calculation result obtained by the multiplication as the result of the product of the second submatrix and the input vector.
[0094] When generating a trained inference model 5 in which a scaling relationship holds recursively, the operations in the forward propagation phase may be composed of the following operations in steps 1 to 5. First step: In at least a part of the parameter matrix for which the scaling relationship is recursively established, the product of the submatrix constituting the initial first submatrix and the input vector is calculated. Second step: derive the product of the submatrix and the input vector that constitutes the initial second submatrix by calculating the product of the initial first submatrix and the corresponding elements of the diagonal matrix. Third step: Derive the product of the submatrix and the input vector that constitutes the first submatrix of the next step from the result of the multiplication of the first submatrix and the result of the multiplication of the second submatrix obtained in the target step of the recursive iteration. Fourth step: derive the product of the submatrix and the input vector that constitutes the second submatrix of the next stage by calculating the product of the first submatrix of the next stage obtained and the corresponding elements of the diagonal matrix. Fifth step: Substitute the initial results of the multiplication of the first submatrix and the initial results of the multiplication of the second submatrix obtained in the first step and the second step, respectively, as the initial values of the results of the multiplication of the first submatrix and the results of the multiplication of the second submatrix obtained in the target step, and recursively repeat the third step and the fourth step (i.e., substitute the results of the multiplication of each submatrix in the next step as the results of the multiplication of each submatrix in the new target step, and repeat the processing of the third step and the fourth step), thereby deriving the product of at least a part of the parameter matrix and the input vector. In addition, the initial first submatrix and the initial second submatrix may be configured as 1×1 matrices.
[0095] (Backpropagation phase) In the backpropagation phase, the control unit 11 calculates an error between the result of the execution of the inference task obtained in the forward propagation phase and the correct answer indicated by the correct answer label 32 associated with the input training data 31, and further calculates the gradient of the calculated error. Then, the control unit 11 backpropagates the gradient of the calculated error by the error backpropagation method to adjust the value of each parameter. During this adjustment, the control unit 11 adjusts the value of each parameter so that the value of each element of the second submatrix coincides with the product of the first submatrix and the diagonal matrix. In other words, the control unit 11 adjusts the value of each element of the first submatrix and the diagonal matrix so as to correspond to each element of the second submatrix.
[0096] When generating a trained inference model 5 in which a scaling relationship holds recursively, the operations in the backpropagation phase may be composed of the following operations in steps 1 to 8. First step: Obtain the gradient of the error with respect to the product of at least a part of the parameter matrix and the input vector. Second step: Derive the error gradient for each element of the diagonal matrix used in the final step (i.e., multiplied by the first submatrix in the final step) based on the corresponding elements of the obtained error gradient and the value of the product of the submatrix and the input vector that constitutes the first submatrix in the final step of the recursive iteration of the fifth step of the forward propagation phase. Third step: Based on the obtained error gradient and the values of each element of the diagonal matrix used in the final step, derive the error gradient for each element of the product of the submatrix and the input vector that constitutes the first submatrix in the final step. Fourth step: deriving an error gradient for each element of the diagonal matrix used in the target stage based on the obtained error gradient, the value of the product of the submatrix constituting the first submatrix and the input vector in the target stage of the recursive iteration of the forward propagation phase, and the error gradient for the product of the submatrix constituting the first submatrix and the input vector in the next stage of the target stage in the recursive iteration of the forward propagation phase. Fifth step: Derive the error gradient for each element of the product of the submatrix and the input vector constituting the first submatrix in the target stage based on the obtained error gradient, the value of each element of the diagonal matrix used in the target stage, and the error gradient for the product of the submatrix and the input vector constituting the first submatrix in the next stage. Sixth step: Substitute the error gradient for each element of the product of the submatrix and the input vector constituting the first submatrix in the final step derived in the third step of the backpropagation phase as the initial value of the error gradient for each element of the product of the submatrix and the input vector constituting the first submatrix in the next step, and derive the error gradient for each element of the diagonal matrix used in each step by recursively repeating the fourth and fifth steps of the backpropagation phase until the error gradient for each element of the diagonal matrix used in the initial step of the recursive iteration of the forward propagation phase is derived. Seventh step: Derive an error gradient for elements of a submatrix constituting an initial first submatrix based on the error gradient for the product of the input vector and the submatrix constituting the initial first submatrix derived in the sixth step. Eighth step: Based on the error gradients derived for each element of the diagonal matrix used in each step and each element of the submatrix constituting the initial first submatrix, the values of each element of the diagonal matrix used in each step and each element of the submatrix constituting the initial first submatrix are adjusted. When the parameter matrix corresponds to the output layer 53, the gradient of the error in the first step is calculated from the error between the result of the execution of the inference task obtained by the forward propagation phase and the correct answer indicated by the correct answer label 32 associated with the input training data 31. When the parameter matrix corresponds to the intermediate layer 52 or the input layer 51, the gradient of the error in the first step is the gradient back-propagated from the layer subsequent to the corresponding layer. As described above, the initial first submatrix may be composed of a 1×1 matrix.
[0097] (A) First concrete example (2×2 matrix) First, an example of the calculation process of forward propagation and back propagation in a simple case (2×2 matrix) where a scaling relationship holds will be described with reference to Figures 8A and 8B. Figures 8A and 8B show an example of the calculation process in the phase of forward propagation and back propagation of a parameter matrix (2×2 matrix) where a scaling relationship holds. This case corresponds to the case where each of the submatrices w1-w4 in Figure 1 is a 1×1 matrix.
[0098] In the forward propagation phase, the control unit 11 calculates the product of the submatrix (w1, w2) constituting the first submatrix and the input vector (x1, x2). If each element is expressed as in the following formula 2 and formula 3, this calculation results in Y1 1 and Y2 1 can be obtained.
[0099]
number
number
[0100] Next, the control unit 11 calculates the product of the first submatrix (Y1 1 , Y2 1 ) and the corresponding elements (a1, a2) of the diagonal matrix, the product of the submatrix that constitutes the second submatrix and the input vector (Z1 1 , Z2 1) is derived. FIG. 8A illustrates the calculation process up to this point. As a result of this calculation process, it is possible to derive the product (y1, y2) of the parameter matrix and the input vector. When the parameter matrix corresponds to the input layer 51 or the intermediate layer 52, the control unit 11 forward propagates the obtained product result to the next layer. On the other hand, when the parameter matrix corresponds to the output layer 53, the control unit 11 obtains the product result as the execution result of the inference task.
[0101] In the backpropagation phase, the control unit 11 obtains the gradient (∂L / ∂y1, ∂L / ∂y2) of the error L with respect to the product of the parameter matrix and the input vector. As described above, when the parameter matrix corresponds to the output layer 53, the control unit 11 can obtain the gradient (∂L / ∂y1, ∂L / ∂y2) of the error by partially differentiating the error between the result of the execution of the inference task obtained by the forward propagation phase and the correct answer indicated by the corresponding correct answer label 32. On the other hand, when the parameter matrix is the intermediate layer 52 or the input layer 51, the control unit 11 can obtain the gradient backpropagated from the layer after the corresponding layer as the gradient (∂L / ∂y1, ∂L / ∂y2) of the error.
[0102]
number
number
[0103]
number
number
[0104] From the above formula 3, (∂y1 / ∂Y1 1 ) is 1. From the above formula 2, (∂Z1 1 / ∂Y1 1 ) is a1. Also, (∂Y1 1 / ∂w1) is x1. Therefore, (∂L / ∂Y1 1 ) is "(∂L / ∂y1)+(∂L / ∂Z1 1 )×a1", and (∂L / ∂w1) is "(∂L / ∂Y1 1 )×x1". Similarly, (∂L / ∂Y2 1 ) is "(∂L / ∂y1)+(∂L / ∂Z2 1 )×a2", and (∂L / ∂w2) is "(∂L / ∂Y2 1 )×x2". (∂L / ∂Z1 1 ) and (∂L / ∂Z2 1) has already been calculated by the previous calculation (for example, Equation 5). Therefore, the control unit 11 calculates the error gradient (∂L / ∂y1, ∂L / ∂y2) for each element of the product of the submatrix constituting the first submatrix and the input vector based on the acquired error gradient (∂L / ∂y1, ∂L / ∂y2) and the values of each element of the diagonal matrix (a1, a2). 1 , ∂L / ∂Y2 1 Then, the control unit 11 derives the gradient of the error (∂L / ∂Y1 1 , ∂L / ∂Y2 1 ), the error gradient (∂L / ∂w1, ∂L / ∂w2) for each element of the submatrix that makes up the first submatrix is derived.
[0105]
number
[0106] (B) Second concrete example (4×4 matrix) Next, an example of the calculation process of forward propagation and back propagation in a simple case (4×4 matrix) where the scaling relationship holds recursively will be described with reference to Figures 9A and 9B. Figures 9A and 9B show an example of the calculation process in the phases of forward propagation and back propagation of a parameter matrix (4×4 matrix) where the scaling relationship holds recursively. This case corresponds to the case where each submatrix (w1-w4, etc.) in the final division of Figure 5B is a 1×1 matrix.
[0107] In the forward propagation phase, the control unit 11 calculates the product of the submatrix (w1-w4) constituting the initial first submatrix and the input vector (x1-x4). This calculation corresponds to the operation of the first step of the forward propagation phase. If each element is expressed as in the following Equation 9 and Equation 10, the result of the product of the initial first submatrix (Y1 1 , Y2 1 , Y3 1 , Y4 1 ) can be obtained.
[0108]
number
number
[0109] Next, the control unit 11 calculates the product of the first submatrix (Y1 1 -Y4 1) and the corresponding elements (a1-a4) of the diagonal matrix, the products of the submatrices constituting the initial second submatrix and the input vector (Z1 1 -Z4 1 ) is derived. This calculation corresponds to the second step of the forward propagation phase. (w1, w2, a1w1, a2w2) and (w3, w4, a3w3, a4w4) correspond to the first submatrix (W1, W2) of the next stage. Each element of the first submatrix of the next stage can be expressed by the following Equations 11 and 12.
[0110]
number
number
[0111] Therefore, the control unit 11 calculates the initial result of the first submatrix product (Y1 1 -Y4 1 ) and the result of the multiplication of the second submatrix (Z1 1 -Z4 1 ) to the product of the submatrix constituting the first submatrix of the next stage and the input vector (Y1 2 -Y4 2 This calculation corresponds to the operation of the third step of the forward propagation phase, which is executed by substituting the initial results of the multiplication of the first submatrix and the initial results of the multiplication of the second submatrix obtained in the first step and the second step, respectively, as the initial values of the results of the multiplication of the first submatrix and the results of the multiplication of the second submatrix obtained in the target phase, respectively.
[0112] Then, the control unit 11 calculates the result of the first submatrix multiplication in the next stage (Y1 2 -Y4 2 ) and the corresponding elements (A1, A2) of the diagonal matrix, the product of the submatrix and the input vector that constitutes the second submatrix of the next stage (Z1 2 -Z4 2This calculation corresponds to the fourth step of the forward propagation phase. The product of the second submatrix can be expressed by the following Equation 13.
[0113]
number
[0114] In the backpropagation phase, the control unit 11 obtains the gradient (∂L / ∂y1-∂L / ∂y4) of the error L with respect to the product of the parameter matrix and the input vector. The method of obtaining the gradient is the same as in the first specific example. This process corresponds to the first step process of the backpropagation phase.
[0115] Next, the control unit 11 derives the error gradient for each element of the diagonal matrix (A1, A2) used in the final stage. The diagonal elements of the diagonal matrix A1 are denoted as diag(A 11 , A 12 ) and the diagonal elements of the diagonal matrix A2 are expressed as diag(A 21 , A 22 ), then, as in the first example above, the gradient of the error of the second submatrix in the final stage (∂L / ∂Z1 2 -∂L / ∂Z4 2 ) to obtain the gradient of each element (∂L / ∂A 11 , ∂L / ∂A 12 , ∂L / ∂A 21 , ∂L / ∂A 22 ) satisfies the following Equations 14 to 17.
[0116]
number
number
number
number
[0117] Similarly, by the chain rule, the gradient of the error for the product of the second submatrix in the final step (∂L / ∂Z1 2 -∂L / ∂Z4 2 ) and then the gradient of the error for each element of the product of the submatrix and the input vector that constitutes the first submatrix in the final stage (∂L / ∂Y1 2 -∂L / ∂Y4 2 ) satisfies the following Equations 18 to 21.
[0118]
number
number
number
number
[0119] Next, the control unit 11 derives the error gradient (∂L / ∂a1-∂L / ∂a4) for each element of the diagonal matrix used in the previous stage (target stage). By the chain rule, the error gradient (∂L / ∂Z1 1 -∂L / ∂Z4 1 ), the error gradient of each element (∂L / ∂a1-∂L / ∂a4) satisfies the following Equations 22-25.
[0120]
number
number
number
number
[0121] Next, the control unit 11 calculates the gradient of the error (∂L / ∂Y1 1 -∂L / ∂Y4 1 ) is derived. By the chain rule, the gradient of the error for the multiplication of the second submatrix at this stage (∂L / ∂Z1 1 -∂L / ∂Z4 1 ) via the error gradient (∂L / ∂Y1 1 -∂L / ∂Y4 1 ) satisfies the following Equations 26 to 29.
[0122]
number
number
number
number
[0123] Next, the control unit 11 derives the error gradient (∂L / ∂w1-∂L / ∂w4) for the elements of the submatrix that constitute the initial first submatrix. According to the chain rule, the error gradient (∂L / ∂w1-∂L / ∂w4) of the initial first submatrix satisfies the following Equations 30 to 33.
[0124]
number
number
number
number
[0125] Therefore, the control unit 11 calculates the gradient of the error (∂L / ∂Y1 1 -∂L / ∂Y41 ), the error gradient (∂L / ∂w1-∂L / ∂w4) for the elements of the submatrix that make up the initial first submatrix is derived. This calculation corresponds to the seventh step of the backpropagation phase. The calculations up to this point have yielded the error gradient used to adjust (update) the parameter matrix, that is, the error gradient for the elements of the submatrix that make up the initial first submatrix and each element of the diagonal matrix used at each stage (in this case, ∂L / ∂w1-∂L / ∂w4, ∂L / ∂A 11 , ∂L / ∂A 12 , ∂L / ∂A 21 , ∂L / ∂A 22 , ∂L / ∂a1-∂L / ∂a4) can be obtained. Based on the gradient of the error derived for each element of the diagonal matrix used at each stage and each element of the submatrix constituting the initial first submatrix, the control unit 11 adjusts the values of each element of the diagonal matrix used at each stage and each element of the submatrix constituting the initial first submatrix in a direction that reduces the error from the inference result. The amount of adjustment may be appropriately adjusted by the learning rate. This calculation corresponds to the eighth step of the backpropagation phase.
[0126] When the parameter matrix corresponds to the intermediate layer 52 or the output layer 53, the control unit 11 derives the gradient of the error (∂L / ∂x1-∂L / ∂x4) for the input vector in order to back-propagate the error to the previous layer. According to the chain rule, the gradient of the error (∂L / ∂x1-∂L / ∂x4) for the input vector satisfies the following Equations 34 to 37.
[0127]
number
number
number
number
[0128] Therefore, the control unit 11 calculates the values (w1-w4) of the elements of the submatrix constituting the first submatrix at the initial stage, and the gradient of the error (∂L / ∂Y1 1 -∂L / ∂Y4 1 ), the error gradient (∂L / ∂x1-∂L / ∂x4) for the input vector is derived. FIG. 9B illustrates the calculation process up to this point. The control unit 11 back-propagates the derived error gradient to the previous layer.
[0129] (C) Third concrete example (arbitrary size matrix) Next, an example of the calculation process of forward propagation and backward propagation in a generalized case (matrix of any size) where a scaling relationship holds recursively will be described with reference to Figures 10A and 10B. Figures 10A and 10B show an example of the calculation process in the phases of forward propagation and backward propagation of a parameter matrix where a scaling relationship holds recursively. This case corresponds to the case of the parameter matrix exemplified in Figure 5C. The calculation process in this case is the same as the second specific example, except that the number of elements is generalized.
[0130] In the forward propagation phase, the control unit 11 performs the first step of processing by dividing the submatrix (W j 1 ) and the input vector (x j ) (Equation 38). Through this calculation, the control unit 11 calculates the initial first submatrix product result (Y j 1 ) is obtained.
[0131]
number
[0132] Next, in the second step, the control unit 11 calculates the product of the first submatrix (Y j 1 ) and the diagonal matrix (A k,j 1) to obtain the product of the submatrix and the input vector that constitutes the initial second submatrix (Z k,j 1 ) is derived (for i=1 in Equation 39).
[0133]
number
[0134] Next, in the third step, the control unit 11 calculates the result of the multiplication of the first submatrix obtained in the target stage of the recursive iteration (Y j i ) and the result of the multiplication of the second submatrix (Z k,j i ) to the product of the submatrix that constitutes the first submatrix of the next stage and the input vector (Y j i+1 ) is derived, where Y j i and Z k,j i Each element of Y j,m i , Z k,j,m i In addition, the control unit 11 performs the fourth step of processing by dividing the result of the multiplication of the first submatrix in the next step (Y j i+1 ) and the diagonal matrix (A k,j i+1 ) is calculated, and the product of the submatrix and the input vector that constitute the second submatrix of the next stage (Z k,j i+1 ) is derived (Equation 39).
[0135]
number
[0136] The control unit 11 calculates the result of the first submatrix multiplication (Y j i ) and the result of the multiplication of the second submatrix (Z k,j i) are the initial values of the first submatrix products (Y j 1 ) and the result of the initial multiplication of the second submatrix (Z k,j 1 ) into the input vector 11. Then, the control unit 11 recursively repeats the third and fourth steps. This series of processing corresponds to the processing of the fifth step of the forward propagation phase. In this case, the control unit 11 repeats the processing of the third and fourth steps l times. From this calculation result, the control unit 11 can derive the result of the product of the parameter matrix and the input vector (the case of i=l in Equation 40). FIG. 10A illustrates the calculation process so far. When the parameter matrix corresponds to the input layer 51 or the intermediate layer 52, the control unit 11 forward propagates the obtained product result to the next layer. On the other hand, when the parameter matrix corresponds to the output layer 53, the control unit 11 obtains the product result as the result of performing the inference task.
[0137] In the backpropagation phase, the control unit 11 calculates, as a first step, the gradient (∂L / ∂Y j l+1 The method of obtaining the information is the same as that of the first specific example. Next, the control unit 11 performs the second step of the process as follows: The gradient of the obtained error (∂L / ∂Y j l+1 ), and the value of the product of the submatrix and the input vector that constitutes the first submatrix in the final stage of the recursive iteration of the fifth step of the forward propagation phase (Y j l ), the diagonal matrix (A k,j l ) each element (A k,j,m l ) (for the case i=l in Eqs. 41 and 42).
[0138]
number
number
[0139] Next, in the third step, the control unit 11 calculates the gradient of the obtained error (∂L / ∂Y j l+1 ) and the diagonal matrix (A k,j l ) each element (A k,j,m l ) for each element of the product of the submatrix and the input vector that constitutes the first submatrix in the final stage, the gradient of the error (∂L / ∂Y j l ) each element (∂L / ∂Y j,m l ) is derived (for the case of i = l in Eqs. 42 and 43).
[0140]
number
[0141] Next, in the fourth step, the control unit 11 calculates the gradient of the obtained error (∂L / ∂Y j l+1 ), the product of the submatrix constituting the first submatrix and the input vector at the target stage of the recursive iteration of the forward propagation phase (Y j i ), and the gradient of the error for the product of the submatrix and the input vector that constitutes the first submatrix in the next step of the target step in the recursive iteration of the forward propagation phase (∂L / ∂Y j i+1 ), the gradient of the error for each element of the diagonal matrix used in the target stage (∂L / ∂A k,j i ) is derived (Equation 41, Equation 42).
[0142] In the fifth step, the control unit 11 calculates the gradient of the obtained error (∂L / ∂Y j l+1 ), the diagonal matrix used in the target stage (A k,j i) and the gradient of the error (∂L / ∂Y j i+1 ), the gradient of the error (∂L / ∂Y j i ) is derived (Equation 42, Equation 43).
[0143] The control unit 11 calculates the gradient of the error (∂L / ∂Y j i+1 ) is the initial value of the gradient of the error (∂L / ∂Y j l ) for each element of the diagonal matrix used in the initial stage of the recursive iteration of the forward propagation phase. k,j 1 ) is derived by recursively repeating the fourth and fifth steps of the backpropagation phase described above, and deriving the gradient of the error (∂L / ∂A k,j i , i is a natural number from 1 to l). This series of processes corresponds to the sixth step of the backpropagation phase. The process so far derives the error gradient (∂L / ∂A k,j i ), and the gradient of the error for the product of the submatrix constituting the first submatrix at the initial stage and the input vector (∂L / ∂Y j 1 ) is derived.
[0144] Next, in the seventh step, the control unit 11 calculates the input vector (x j ), and the gradient of the error for the product of the submatrix constituting the first submatrix at the initial stage and the input vector, which is derived in the sixth step (∂L / ∂Y j 1), the gradient of the error for the elements of the submatrix that constitute the initial first submatrix (∂L / ∂W j 1 ) is derived.
[0145]
number
[0146] In the eighth step, the control unit 11 calculates the gradient of the error (∂L / ∂A k,j i , ∂L / ∂W j 1 ), the values of each element of the diagonal matrix used in each stage and each element of the submatrix constituting the initial first submatrix (A k,j i , W j 1 ) to adjust the volume.
[0147] In addition, when the parameter matrix corresponds to the intermediate layer 52 or the output layer 53, the control unit 11 back-propagates the error to the previous layer by calculating the values of each element of the submatrix constituting the first submatrix at an early stage (W j 1 ), and the gradient of the error for the product of the submatrix constituting the first submatrix at the initial stage and the input vector (∂L / ∂Y j 1 ), the gradient of the error with respect to the input vector (∂L / ∂x j ) is derived (Equation 45). FIG. 10B illustrates the calculation process up to this point. The control unit 11 back-propagates the derived error gradient to the previous layer.
[0148]
number
[0149] (Small summary) Through each of the above calculation processes, in each case, the control unit 11 can execute the processing of the forward propagation phase and the backward propagation phase to adjust the parameter values of the inference model 5. The control unit 11 may repeatedly execute each of the processing of the forward propagation phase and the backward propagation phase until a predetermined condition is met (e.g., executing a specified number of times, or the sum of the calculated errors being equal to or less than a threshold value).
[0150] As a result of the above machine learning processing, the control unit 11 can generate a trained inference model 5 that has acquired the ability to perform an inference task and in which the value of each element of the second submatrix has been adjusted to match the product of the first submatrix and the diagonal matrix. In particular, according to the above second and third specific examples, a trained inference model 5 can be generated in which the value of each parameter has been adjusted so that a scaling relationship is recursively established within at least a portion of the parameter matrix. When the machine learning processing is completed, the control unit 11 proceeds to the next step S103.
[0151] (Step S103) Returning to FIG. 7, in step S103, the control unit 11 operates as a storage processing unit 113 and generates information on the trained inference model 5 generated by machine learning as learning result data 125. The learning result data 125 holds information for reproducing the trained inference model 5. As an example, the learning result data 125 may be configured to include information indicating the values of the parameters constituting the first submatrix obtained by adjusting the machine learning and the values of the diagonal components of the diagonal matrix, and information on the parameters constituting the second submatrix may be omitted. As another example, when a scaling relationship holds recursively, the learning result data 125 includes the values of the parameters constituting the initial first submatrix obtained by adjusting the machine learning (W j 1 ) and the values of each element of the diagonal matrix used at each stage (A k,j i ) and information regarding parameters other than these may be omitted. The control unit 11 stores the generated learning result data 125 in a predetermined storage area.
[0152] The predetermined storage area may be, for example, a RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination of these. The storage medium may be, for example, a CD, a DVD, or the like, and the control unit 11 may store the learning result data 125 in the storage medium via the drive 17. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). In this case, the control unit 11 may use the communication interface 13 to store the learning result data 125 in the data server via a network. The external storage device may also be, for example, an external storage device connected to the model generation device 1 via the external interface 14.
[0153] When the storage of the learning result data 125 is completed, the control unit 11 ends the processing procedure of the model generating device 1 according to this operation example.
[0154] The generated learning result data 125 may be provided to the inference device 2 at any timing. For example, the control unit 11 may transfer the learning result data 125 to the inference device 2 as part of the process of step S103 or separately from the process of step S103. The inference device 2 may acquire the learning result data 125 by receiving this transfer. Also, for example, the inference device 2 may acquire the learning result data 125 by accessing the model generation device 1 or a data server via a network using the communication interface 23. Also, for example, the inference device 2 may acquire the learning result data 125 via the storage medium 92. Also, for example, the learning result data 125 may be incorporated in advance into the inference device 2.
[0155] Furthermore, the control unit 11 may update or newly generate the learning result data 125 by periodically or irregularly repeating the processes of steps S101 to S103. During this repetition, at least a part of the learning dataset 3 used for machine learning may be changed, modified, added, deleted, or the like as appropriate. Then, the control unit 11 may update the learning result data 125 held by the inference device 2 by providing the updated or newly generated learning result data 125 to the inference device 2 by any method.
[0156] [Inference device] 11 is a flowchart showing an example of a processing procedure for performing an inference task by the inference device 2 according to this embodiment. The processing procedure of the inference device 2 described below is an example of an inference method. However, the processing procedure of the inference device 2 described below is merely an example, and each step may be modified as much as possible. Furthermore, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.
[0157] (Step S201) In step S201, the control unit 21 operates as the data acquisition unit 211 and acquires the target data 221. The target data 221 is a sample of a predetermined type of data to be subjected to the estimation task. A method for acquiring the target data 221 may be appropriately determined according to the type of data. As an example, the target data 221 may be generated by observing the inference target by a sensor (for example, a camera). In addition, a route for acquiring the target data 221 may not be particularly limited and may be appropriately selected according to the embodiment. As an example, the control unit 21 may directly acquire the target data 221 from a device such as a sensor. In another example, the control unit 21 may indirectly acquire the target data 221 via another computer, a storage medium 92, or the like. When the target data 221 is acquired, the control unit 21 advances the process to the next step S202.
[0158] (Step S202) In step S202, the control unit 21 operates as the inference unit 212 and sets the trained inference model 5 by referring to the learning result data 125. Then, the control unit 21 uses the trained inference model 5 to perform an inference task on the acquired target data 221.
[0159] The inference process in step S202 may be the same as the calculation process of the forward propagation (i.e., inference trial) phase in the training process of the above machine learning. That is, the control unit 21 inputs the target data 221 to the input layer 51 and executes the calculation process of the forward propagation of the inference model 5. In this calculation process of the forward propagation, the control unit 21 calculates the product of the submatrix constituting the first submatrix and the input vector. When the parameter matrix corresponds to the input layer 51, the input vector is the target data 221. When the parameter matrix corresponds to the intermediate layer 52 or the output layer 53, the input vector is the calculation result (output) of the layer preceding the corresponding layer. Next, the control unit 21 derives the product of the submatrix constituting the second submatrix and the input vector by calculating the product of the result of the product of the first submatrix and the corresponding elements of the diagonal matrix. That is, the control unit 21 multiplies the result of the product of the first submatrix and the diagonal matrix, and acquires the calculation result obtained by the multiplication as the result of the product of the second submatrix and the input vector. When the scaling relationship holds recursively, the control unit 21 executes the calculations of the first to fifth steps of the forward propagation phase. As a result of these calculations, the control unit 21 obtains an output value corresponding to the result of performing the inference task on the target data 221 from the output layer 53. When the inference process is completed, the control unit 21 proceeds to the next step S203.
[0160] (Step S203) In step S203, the control unit 21 operates as the output unit 213 and outputs information about the result of performing the inference task (inference result).
[0161] The output destination and the contents of the information to be output may be appropriately determined according to the embodiment. For example, the control unit 21 may output the inference result obtained in step S202 directly to the output device 26. Furthermore, the control unit 21 may execute any information processing based on the inference result obtained. The control unit 21 may then output the result of the information processing as information related to the inference result. The output of the result of the information processing may include controlling the operation of the controlled device according to the inference result. The output destination may be, for example, the output device 26, an output device of another computer, the controlled device, etc.
[0162] When the output of information relating to the result of performing the inference task is completed, the control unit 21 ends the processing procedure of the inference device 2 according to this operation example. The control unit 21 may continuously and repeatedly execute the series of information processing from step S201 to step S203. The timing of repetition may be appropriately determined according to the embodiment. In this way, the inference device 2 may be configured to continuously and repeatedly perform the above inference task.
[0163] [Features] As described above, according to the model generating device 1 of this embodiment, the process of steps S101 to S103 can generate a trained inference model 5 that is expressed by a matrix including a first submatrix and a second submatrix, and in which the value of each element of the second submatrix is adjusted to match the product of the first submatrix and the diagonal matrix. As a result, even if the information of the second submatrix is not held in the process of step S202 of the inference device 2, the calculation result of the first submatrix can be reused to obtain the calculation result of the second submatrix. Similarly, by giving initial values to each parameter so as to satisfy such a relationship (i.e., setting the values of each element of the first submatrix and the diagonal matrix), the calculation result of the first submatrix can be reused to obtain the calculation result of the second submatrix in the process of step S102 without holding the information of the second submatrix. Therefore, according to this embodiment, the information on the second submatrix is omitted in the learning result data 125, and the number of parameters constituting the inference model 5 can be substantially reduced. Specifically, the number of parameters can be reduced by the difference between the number of elements of the second submatrix and the number of elements other than 0 in the diagonal matrix. When the scaling relationship holds recursively, the number of elements of at least a part of the parameter matrix can be reduced to the number of elements of the submatrix constituting the initial first submatrix and the number of elements of the diagonal matrix used at each stage. Therefore, according to this embodiment, the amount of calculation required for the calculation processing of the inference model 5 can be reduced, and the burden on the calculation resources can be suppressed. Note that, in this embodiment, other speed-up methods such as distributed processing exemplified in Patent Document 2 and Patent Document 3 and high-speed processing of convolution calculation exemplified in Non-Patent Document 1 may also be used in combination.
[0164] § 4 Variations Although the embodiment of the present invention has been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present invention. For example, the following modifications are possible. In the following, the same reference numerals are used for the same components as in the above embodiment, and the description of the same points as in the above embodiment is omitted as appropriate. The following modifications can be combined as appropriate.
[0165] <4.1> The inference system 100 according to the above embodiment may be applied to any situation where an arbitrary inference task is performed on a predetermined type of data. The inference task may be, for example, determining whether a product shown in image data has a defect, identifying the type of an object shown in image data, inferring the state of an object shown in sensing data, etc. The training data 31 and the object data 221 may be, for example, image data, sound data, numerical data, text data, and other measurement data obtained by various sensors. The training data 31 and the object data 221 may be sensing data generated by observing an arbitrary object with a sensor. The sensor may be, for example, a camera, a microphone, an encoder, an environmental sensor, a vital sensor, a medical testing device, an in-vehicle sensor, a home security sensor, etc. The environmental sensor may be, for example, a barometer, a thermometer, a hygrometer, a sound pressure meter, a sound sensor, an ultraviolet sensor, a light meter, a rain gauge, a gas sensor, etc. The vital sensor may be, for example, a blood pressure monitor, a pulse rate monitor, a heart rate monitor, an electrocardiograph, an electromyograph, a thermometer, a skin electrodermal response monitor, a microwave sensor, an electroencephalograph, a magnetoencephalograph, an activity monitor, a blood glucose level meter, an electrooculography sensor, an eye movement measuring instrument, or the like. The medical examination device may be, for example, a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, or the like. The in-vehicle sensor may be, for example, an image sensor, a light detection and ranging (Lidar) sensor, a millimeter wave radar, an ultrasonic sensor, an acceleration sensor, or the like. The home security sensor may be, for example, an image sensor, an infrared sensor, an activity (audio) sensor, a gas (CO2, etc.) sensor, a current sensor, a smart meter (a sensor that measures the amount of power used by home appliances, lighting, etc.), or the like. Below, examples of modified examples in which the application scene is limited will be given.
[0166] (A) Visual inspection scene 12 illustrates a schematic example of an application scene of the inspection system 100A according to the first modified example. This modified example is an example of the application of the above embodiment to a scene in which an appearance inspection of a product is carried out using image data showing the product. The inspection system 100A according to this modified example is an example of the above inference system 100, and includes a model generating device 1 and an inspection device 2A. As in the above embodiment, the model generating device 1 and the inspection device 2A may be connected to each other via a network.
[0167] The predetermined type of data (training data 31A and target data 221A) handled in this modification is composed of image data (image sample) showing a product RA. The image data may be obtained by photographing the product RA with a camera SA. The inference task is to determine whether or not the product RA shown in the image data has a defect. Determining whether or not there is a defect may consist of identifying the presence or absence of a defect, estimating the probability that the product RA contains a defect, identifying the type of defect contained in the product RA (which may include a type indicating "no defect"), extracting the range of defects contained in the product RA, or a combination of these. Except for these limitations, the inspection system 100A according to this modification may be configured in the same way as the inference system 100 according to the above embodiment.
[0168] The product RA may be, for example, a product conveyed on a manufacturing line of electronic devices, electronic components, automobile parts, medicines, food, etc. The electronic components may be, for example, a circuit board, a chip capacitor, a liquid crystal, a relay winding, etc. The automobile parts may be, for example, a connecting rod, a shaft, an engine block, a power window switch, a panel, etc. The medicines may be, for example, packaged tablets, unpackaged tablets, etc. The product RA may be a final product generated after the manufacturing process is completed, an intermediate product generated during the manufacturing process, or an initial product prepared before the manufacturing process. The defects may be, for example, scratches, stains, cracks, dents, burrs, color unevenness, foreign matter contamination, etc.
[0169] (Model Generator) In this modification, the model generating device 1 acquires a plurality of learning data sets 3A in the above step S101. Each learning data set 3A is composed of a combination of training data 31A and a correct answer label 32A. The training data 31A is composed of image data showing a product RA. Any type of camera may be used to acquire the training data 31A. The correct answer label 32A is configured to indicate the correct answer of an inference task related to a defect of a product shown in the training data 31A, such as the presence or absence of the defect, the type of the defect, and the range in which the defect exists.
[0170] The model generating device 1 performs machine learning of the inference model 5 using the multiple learning data sets 3A acquired by the process of step S102 above. This makes it possible to generate a trained inference model 5 that has acquired the ability to determine whether or not the product RA shown in the image data has a defect and in which the values of each element of the second submatrix are adjusted to match the product of the first submatrix and the diagonal matrix. The model generating device 1 appropriately stores information about the trained inference model 5 as learning result data 125 in a predetermined storage area by the process of step S103 above. The generated learning result data 125 may be provided to the inspection device 2A at any timing.
[0171] (Inspection equipment) The inspection device 2A is an example of the inference device 2. The hardware configuration and software configuration of the inspection device 2A may be the same as those of the inference device 2 according to the above embodiment. In this modification, the inspection device 2A may be connected to the camera SA via a communication interface or an external interface. Alternatively, the camera SA may be connected to another computer, and the inspection device 2A may be configured to be able to acquire image data from the camera SA by connecting to the other computer. The camera SA may be, for example, a general RGB camera, a depth camera, an infrared camera, or the like. The camera SA may be appropriately disposed in a location where it is possible to photograph the product RA. For example, the camera SA may be disposed near a conveyor device that transports the product RA.
[0172] The inspection device 2A executes a series of information processes related to the appearance inspection by the same processing procedure as the inference device 2. That is, in step S201, the inspection device 2A acquires the target data 221A from the camera SA. The target data 221A is composed of image data showing the product RA to be inspected. In step S202, the inspection device 2A uses the trained inference model 5 to determine whether or not the product RA shown in the target data 221A (image data) has a defect. Then, in step S203, the inspection device 2A outputs information related to the result of determining whether or not the product RA has a defect. In one example, the inspection device 2A may directly output the result of determining whether or not the product RA has a defect to the output device. In another example, when the inspection device 2A determines that the product RA contains a defect, it may output a warning to the output device to notify the user of the fact. The output destination of this information is not limited to the output device of the inspection device 2A, but may be an output device of another computer. In yet another example, when the inspection device 2A is connected to a conveyor device that transports the product RA, the inspection device 2A may control the conveyor device so that defective products and non-defective products are transported on separate lines based on the judgment results.
[0173] (Features) According to this modification, the number of parameters constituting the inference model 5 is substantially reduced, thereby reducing the amount of calculation required for the computational processing of the inference model 5 used in the visual inspection, and the burden on computational resources can be reduced. This can be expected to have the effect of speeding up the visual inspection and enabling the visual inspection to be performed using a low-cost computer.
[0174] (B) Image recognition scene FIG. 13 illustrates a schematic example of an application scene of a classification system 100B according to a second modified example. This modified example is an example in which the above embodiment is applied to a scene in which an object captured in image data is to be classified. The classification system 100B according to this modified example is an example of the above inference system 100, and includes a model generating device 1 and a classification device 2B. As in the above embodiment, the model generating device 1 and the classification device 2B may be connected to each other via a network.
[0175] The predetermined type of data (training data 31B and target data 221B) handled in this modification is composed of image data (image samples) in which the target RB appears. The image data may be obtained by photographing the target RB with the camera SB. The inference task is to identify the type of the target RB appearing in the image data. Identifying the type of the target RB may include segmentation (extracting the range in which the target appears). The target RB may be a person or any object. The range in the image data may be a specific part of the target RB (e.g., a face, etc.) or the entire target RB. When the target RB is a person, the identification target may be, for example, a part of the body such as a face. Identifying the type of a person may be, for example, estimating an individual or a body part (face, arm, leg, joint, etc.). The same applies to any object. Except for these limitations, the identification system 100B according to this modification may be configured in the same manner as the inference system 100 according to the above embodiment.
[0176] (Model Generator) In this modification, the model generating device 1 acquires a plurality of learning data sets 3B in the above step S101. Each learning data set 3B is composed of a combination of training data 31B and a correct answer label 32B. The training data 31B is composed of image data in which the object RB appears. Any type of camera may be used to acquire the training data 31B. The correct answer label 32B is configured to indicate the type (correct answer) of the object RB appearing in the training data 31B.
[0177] The model generating device 1 performs machine learning of the inference model 5 using the multiple learning data sets 3B acquired by the process of step S102. This makes it possible to generate a trained inference model 5 that has acquired the ability to identify the type of the object RB captured in the image data and in which the values of each element of the second submatrix are adjusted to match the product of the first submatrix and the diagonal matrix. The model generating device 1 appropriately stores information about the trained inference model 5 in a predetermined storage area as learning result data 125 by the process of step S103. The generated learning result data 125 may be provided to the classification device 2B at any timing.
[0178] (Identification device) The identification device 2B is an example of the inference device 2. The hardware configuration and software configuration of the identification device 2B may be the same as those of the inference device 2 according to the above embodiment. In this modification, the identification device 2B may be connected to the camera SB via a communication interface or an external interface. Alternatively, the camera SB may be connected to another computer, and the identification device 2B may be configured to be able to acquire image data from the camera SB by connecting to the other computer. The camera SB may be, for example, a general RGB camera, a depth camera, an infrared camera, or the like. The camera SB may be appropriately disposed in a location where it is possible to photograph the object RB.
[0179] The identification device 2B executes a series of information processes related to the identification of the object RB by the same processing procedure as the inference device 2. That is, in step S201, the identification device 2B acquires the object data 221B from the camera SB. The object data 221B is composed of image data in which the object RB to be identified appears. In step S202, the identification device 2B uses the trained inference model 5 to identify the type of the object RB appearing in the object data 221B (image data). Then, in step S203, the identification device 2B outputs information on the result of identifying the type of the object RB. In one example, the identification device 2B may output the result of identifying the object RB to the output device as it is. In another example, the identification device 2B may execute any information process according to the result of identifying the object RB. For example, the identification device 2B may execute information process such as unlocking when the identified target person (object RB) is a specific individual.
[0180] (Features) According to this modification, the number of parameters constituting the inference model 5 is substantially reduced, thereby reducing the amount of calculation required for the arithmetic processing of the inference model 5 used to identify the object RB, and suppressing the burden on the computational resources. As a result, it is possible to expect effects such as speeding up the identification processing of the object RB and performing the identification processing on a low-cost computer.
[0181] (C) A situation of state inference FIG. 14 illustrates a schematic example of an application scene of an inference system 100C according to the third modified example. This modified example is an example of application of the above embodiment to a scene in which the state of an object RC appearing in sensing data is inferred (identification / regression). The inference system 100C according to this modified example is an example of the above inference system 100, and includes a model generation device 1 and an inference device 2C. As in the above embodiment, the model generation device 1 and the inference device 2C may be connected to each other via a network.
[0182] The predetermined types of data (training data 31C and target data 221C) handled in this modification are composed of sensing data generated by observing an object RC with a sensor SC. The sensor SC may be, for example, a camera, a microphone, an encoder, an environmental sensor, a vital sensor, a medical testing device, an in-vehicle sensor, a home security sensor, or the like. The inference task is to infer (identify / regress) the state of the object RC appearing in the sensing data. Inferring may include predicting (i.e., inferring a future state from the time when the sensing data was obtained). Except for these limitations, the inference system 100C according to this modification may be configured similarly to the inference system 100 according to the above embodiment.
[0183] The type of the sensor SC may be appropriately selected according to the inference task. As an example, the target object RC may be a person, and inferring the state of the target object RC may be inferring the state of the target person. In this case, the sensor SC may be composed of at least one of a camera, a microphone, a vital sensor, and a medical examination device, for example. Inferring the state of the target person may be inferring a health state, such as the probability of developing a certain disease or the probability of a change in physical condition. Alternatively, the target person may be, for example, a driver of a vehicle, and inferring the state of the target person may be inferring the state of the driver (for example, a degree of drowsiness, a degree of fatigue, a degree of ease, etc.).
[0184] As another example, the object RC may be an industrial machine, and inferring the state of the object RC may be inferring (detecting or predicting) whether or not there is an abnormality in the industrial machine. In this case, the sensor SC may be composed of at least one of a microphone, an encoder, and an environmental sensor, for example. The sensing data may be composed of an encoder value, temperature, operating sound, etc. of a motor.
[0185] As another example, the object RC may be an object existing outside the vehicle, and inferring the state of the object RC may be inferring the situation outside the vehicle. In this case, the sensor SC may be, for example, composed of at least one of a camera and an on-board sensor. Inferring the situation outside the vehicle may be, for example, inferring the attributes of an object existing outside the vehicle, inferring a congestion situation, inferring the risk of an accident, etc. The object existing outside the vehicle may be, for example, a road, a traffic light, an obstacle (a person, an object), etc. Inferring the attributes of an object existing outside the vehicle may include, for example, inferring the occurrence of an event such as a person or a vehicle jumping out, a sudden start, a sudden stop, a lane change, etc.
[0186] As another example, the object RC may be an object present in a specific location, for example, outdoors or in a specific indoor location (for example, in a vinyl greenhouse), and inferring the state of the object RC may be inferring the situation of the specific location. In this case, the sensor SC may be composed of, for example, at least one of a camera, a microphone, and an environmental sensor. As a specific example, the object RC may be a plant, and inferring the situation of the specific location may be inferring the cultivation situation of the plant.
[0187] As another example, the object RC may be an object present in a house, and inferring the state of the object RC may be inferring the situation in the house. In this case, the sensor SC may be, for example, a camera, a microphone, an environmental sensor, or a home security sensor.
[0188] (Model Generator) In this modification, the model generating device 1 acquires a plurality of learning data sets 3C in the above step S101. Each learning data set 3C is composed of a combination of training data 31C and a correct answer label 32C. The training data 31C is composed of sensing data generated by observing an object RC by a sensor SC. The correct answer label 32C is configured to indicate the state (correct answer) of the object RC appearing in the training data 31C.
[0189] The model generation device 1 performs machine learning of the inference model 5 using the multiple learning data sets 3C acquired by the process of step S102 above. This makes it possible to generate a trained inference model 5 that has acquired the ability to infer the state of the object RC appearing in the sensing data and in which the values of each element of the second submatrix are adjusted to match the product of the first submatrix and the diagonal matrix. The model generation device 1 appropriately stores information about the trained inference model 5 in a predetermined storage area as learning result data 125 by the process of step S103 above. The generated learning result data 125 may be provided to the inference device 2C at any timing.
[0190] (Inference device) The inference device 2C is an example of the inference device 2. The hardware configuration and software configuration of the inference device 2C may be similar to those of the inference device 2 according to the above embodiment. In this modification, the inference device 2C may be connected to the sensor SC via a communication interface or an external interface. Alternatively, the sensor SC may be connected to another computer, and the inference device 2C may be configured to be able to acquire sensing data from the sensor SC by connecting to the other computer.
[0191] The inference device 2C executes a series of information processes related to state inference of the object RC by the same processing procedure as the inference device 2 described above. That is, in step S201, the inference device 2C acquires object data 221C from the sensor SC. The object data 221C is composed of sensing data generated by observing the object RC, which is the inference object, by the sensor SC. In step S202, the inference device 2C uses the trained inference model 5 to infer the state of the object RC that appears in the object data 221C (sensing data).
[0192] Then, in step S203, the inference device 2C outputs information on the result of inferring the state of the object RC. In one example, the inference device 2C may output the result of inferring the state of the object RC to the output device as it is. In another example, the inference device 2C may execute any information processing according to the result of inferring the state of the object RC. As a specific example, when inferring the state of the object RC is inferring the health state of a target person, the inference device 2C may output a warning to inform the target person of an abnormality in their health state when it is determined that there is an abnormality (for example, the probability of developing a predetermined disease exceeds a threshold). As another specific example, when inferring the state of the object RC is inferring the state of the driver, the inference device 2C may execute an output such as notifying a message encouraging the driver to take a break from driving or prohibiting switching from automatic driving to manual driving when the driver's drowsiness level or fatigue level exceeds a threshold. As another specific example, when inferring the state of the object RC is inferring whether or not there is an abnormality in an industrial machine, when it is determined that there is an abnormality or a sign of an abnormality in the industrial machine, the inference device 2C may output a warning to notify the fact. As another specific example, when inferring the state of the object RC is inferring a situation outside the vehicle, the inference device 2C may determine an operation command for the vehicle according to the inferred situation outside the vehicle, and control the vehicle according to the determined operation command (for example, executing a temporary stop of the vehicle when it detects a person jumping out).
[0193] (Features) According to this modification, the number of parameters constituting the inference model 5 is substantially reduced, thereby reducing the amount of calculation required for the arithmetic processing of the inference model 5 used to infer the state of the object RC, and reducing the burden on computational resources. As a result, it is possible to expect effects such as speeding up the processing for inferring the state of the object RC and implementing the inference processing on a low-cost computer.
[0194] <4.2> In the above embodiment, the inference model 5 is configured by a fully connected neural network. However, the type of the neural network configuring the inference model 5 may not be limited to this example. In another example, the inference model 5 may be configured by a convolutional neural network, a recurrent neural network, or the like. The neural network configuring the inference model 5 may include other types of layers, such as a convolutional layer, a pooling layer, a normalization layer, a dropout layer, or the like.
[0195] In the above embodiment, the type of the machine learning model constituting the inference model 5 may not be limited to a neural network. As long as the parameters can be expressed by a matrix, the type of the machine learning model constituting the inference model 5 may not be particularly limited and may be appropriately selected depending on the embodiment. As another example, the inference model 5 may be composed of, for example, a principal component vector obtained by principal component analysis, a support vector machine, etc. The machine learning method may be appropriately determined depending on the type of machine learning model adopted. [Explanation of symbols]
[0196] 1...Model generation device, 11: control unit, 12: storage unit, 13: communication interface, 14...External interface, 15...input device, 16...output device, 17...drive, 81...model generation program, 91...storage medium, 111: data acquisition unit; 112: learning processing unit; 113...storage processing unit, 125...learning result data, 2... Reasoning device, 21: control unit, 22: storage unit, 23: communication interface, 24…External interface, 25...input device, 26...output device, 27...drive, 82...inference program, 92...storage medium, 211: data acquisition unit, 212: inference unit, 213: output unit, 221...Target data, 3…Training data set, 31...training data, 32...correct label, 5…Inference model, 51...input layer, 52...intermediate (hidden) layer, 53...output layer
Claims
1. A data acquisition unit that acquires target data; An inference unit that performs an inference task on the acquired target data using an inference model trained by machine learning; an output unit that outputs information regarding a result of performing the inference task; Equipped with At least a portion of the parameters of the inference model are expressed in a matrix; the matrix includes a first submatrix and a second submatrix; the number of elements in each row and column of the first submatrix and the second submatrix are the same; The value of each element of the second submatrix is adjusted to match the product of the first submatrix and a diagonal matrix. Reasoning device.
2. In at least a portion of the matrix, At least a portion of the matrix is divided into M×N submatrices such that there are M submatrices in each row and N submatrices in each column, each submatrix having the same number of elements in each row and column; In each column, a submatrix arranged in any row constitutes the first submatrix for each submatrix arranged in the other rows except for the certain row; and Each of the sub-matrices arranged in the other rows constitutes the second sub-matrix. A scaling relationship exists. The inference device of claim 1 .
3. The M and the N are the same prime number S. The inference device according to claim 2.
4. the scaling relationship is repeatedly satisfied in the submatrices constituting the first submatrix, so that the scaling relationship is recursively satisfied in at least a part of the matrix.
4. An inference device according to claim 2 or 3.
5. The M and the N are the same prime number S, At least a part of the matrix is composed of a square matrix whose number of elements is a power of a prime number S.
5. The inference device according to claim 4.
6. each of M and N is 2; An inference device according to any one of claims 2 to 5.
7. The inference model is configured by a neural network, Each element of the matrix is configured to correspond to a weight of a connection between each neuron in the neural network. An inference device according to any one of claims 1 to 6.
8. The target data is composed of image data showing a product, the inference task being to determine whether a product depicted in the image data is defective; An inference device according to any one of claims 1 to 7.
9. A data acquisition unit that acquires a plurality of learning data sets each composed of a combination of training data and a correct answer label indicating a correct answer of an inference task for the training data; A learning processing unit that performs machine learning of an inference model using the plurality of learning data sets, At least a portion of the parameters of the inference model are expressed in a matrix; the matrix includes a first submatrix and a second submatrix; the number of elements in each row and column of the first submatrix and the second submatrix are the same; The machine learning is configured by training the inference model for each of the learning data sets such that a result of performing the inference task on the training data using the inference model matches a correct answer indicated by the correct answer label, and a value of each element of the second submatrix is adjusted to match a product of the first submatrix and a diagonal matrix. A learning processing unit; Equipped with Model generation device.
10. Training the inference model includes dividing at least a portion of the matrix into M×N submatrices such that, in at least a portion of the matrix, M submatrices having the same number of elements in each row and column are arranged in rows and N submatrices in each column, and adjusting values of each element of the matrix so that a scaling relationship is established in which a submatrix arranged in any row constitutes the first submatrix for each submatrix arranged in other rows excluding the one row, and each submatrix arranged in the other rows constitutes the second submatrix.
10. The model generating apparatus of claim 9.
11. The inference model is configured by a neural network, Each element of the matrix corresponds to a weight of a connection between each neuron in the neural network; Training the inference model includes adjusting values of each element of at least a portion of the matrix so that the scaling relationship recursively holds within at least a portion of the matrix by repeating the establishment of the scaling relationship within the submatrices constituting the first submatrix until each submatrix becomes a 1×1 matrix; Adjusting the values of each element of at least a portion of the matrices includes: providing initial values for at least some of the elements of the matrices such that the scaling relationship holds recursively; deriving a result of performing an inference task on the training data of each of the learning data sets in a forward propagation phase; and In a backpropagation phase, adjusting values of at least some of the elements of the matrix and the diagonal matrix by backpropagating a gradient of an error between the derived result of the inference task and a correct answer indicated by a corresponding correct answer label; It is composed of The operations in the forward propagation phase are: a first step of computing the product of said 1×1 matrix and an input vector constituting an initial first submatrix; a second step of deriving a product of the 1×1 matrix and an input vector constituting an initial second submatrix by calculating a product of the initial first submatrix product and corresponding elements of the diagonal matrix; a third step of deriving a product of a submatrix constituting a first submatrix of a next step and an input vector from the result of the multiplication of the first submatrix and the result of the multiplication of the second submatrix obtained in the current step; A fourth step of deriving a product of a submatrix constituting the second submatrix of the next stage and an input vector by calculating a product of the result of the product of the first submatrix of the next stage and a corresponding element of the diagonal matrix; and a fifth step of substituting the initial results of the multiplication of the first submatrix and the initial results of the multiplication of the second submatrix obtained in the first step and the second step, respectively, as initial values of the results of the multiplication of the first submatrix and the results of the multiplication of the second submatrix obtained in the target step, and recursively repeating the third step and the fourth step, thereby deriving a product of at least a part of the matrix and an input vector; It is composed of The backpropagation phase of the calculation is A first step of obtaining a gradient of said error with respect to a product of at least a portion of said matrix and an input vector; a second step of deriving, based on the obtained error gradient and the value of the product of the submatrix constituting the first submatrix and the input vector in the final stage of the recursive iteration of the forward propagation phase, an error gradient for each element of the diagonal matrix used in the final stage; a third step of deriving, based on the obtained error gradient and the values of each element of the diagonal matrix used in the final stage, an error gradient for a product of a submatrix constituting the first submatrix and an input vector in the final stage; a fourth step of deriving an error gradient for each element of a diagonal matrix used in a target stage based on the obtained error gradient, a value of a product of a submatrix constituting the first submatrix and an input vector in a target stage of the recursive iteration of the forward propagation phase, and an error gradient for a product of a submatrix constituting the first submatrix and an input vector in a next stage of the target stage in the recursive iteration of the forward propagation phase; a fifth step of deriving an error gradient for a product of a submatrix constituting the first submatrix and an input vector in the target stage based on the obtained error gradient, the values of each element of the diagonal matrix used in the target stage, and an error gradient for a product of a submatrix constituting the first submatrix and an input vector in the next stage; a sixth step of substituting the error gradient for a product of the submatrix constituting the first submatrix in the final step and the input vector derived in the third step of the backpropagation phase as an initial value of the error gradient for a product of the submatrix constituting the first submatrix in the next step and the input vector, and recursively repeating the fourth step and the fifth step of the backpropagation phase until an error gradient for each element of the diagonal matrix used in an initial step of the recursive iteration of the forward propagation phase is derived; a seventh step of deriving an error gradient for elements of the 1×1 matrix constituting the initial first submatrix based on the input vector and an error gradient for a product of a submatrix constituting the first submatrix at the initial stage and the input vector derived in the sixth step; and an eighth step of adjusting values of each element of the diagonal matrix used in each step and each element of the 1×1 matrix constituting the initial first submatrix based on the gradient of the error derived for each element of the diagonal matrix used in each step and each element of the 1×1 matrix constituting the initial first submatrix; Consists of: The model generating device according to claim 10.
12. The computer acquiring target data; performing an inference task on the acquired target data using a trained inference model through machine learning; outputting information regarding a result of performing the inference task; An inference method for performing At least a portion of the parameters of the inference model are expressed in a matrix; the matrix includes a first submatrix and a second submatrix; the number of elements in each row and column of the first submatrix and the second submatrix are the same; The value of each element of the second submatrix is adjusted to match the product of the first submatrix and a diagonal matrix. Reasoning method.
13. On the computer, acquiring target data; performing an inference task on the acquired target data using a trained inference model through machine learning; outputting information regarding a result of performing the inference task; An inference program for executing At least a portion of the parameters of the inference model are expressed in a matrix; the matrix includes a first submatrix and a second submatrix; the number of elements in each row and column of the first submatrix and the second submatrix are the same; The value of each element of the second submatrix is adjusted to match the product of the first submatrix and a diagonal matrix. Inference program.
Citation Information
Patent Citations
Inspection device
JP2012026982A
Device and method for increasing processing speed of neural network, and application of the same
JP2018109947A
Distributed deep layer learning apparatus and distributed deep layer learning system
JP2018120441A
Distributed deep learning system, distributed deep learning method and computing interconnect apparatus
JP2020003848A
Neural network model compaction device
JP2020155010A
