Information processing method, information processing system, and program

Abstract

This information processing method is executed by an information processing system comprising: an input signal DAC that outputs a first analog signal to an analog computer; a weighting signal DAC that outputs a second analog signal to the analog computer; and an output signal ADC that performs AD conversion on an analog signal output by the analog computer. The input signal DAC executes, on an input matrix A, matrix normalization processing for normalizing a matrix to be processed, and then executes, on the input matrix A normalized by the matrix normalization processing, quantization matrix acquisition processing for quantizing the matrix to be processed. The weighting signal DAC executes matrix normalization processing on a weighting matrix X and then executes quantization matrix acquisition processing on the weighting matrix X normalized by the matrix normalization processing. The output signal ADC corrects the result of AD conversion on the basis of a parameter used in the matrix normalization processing by the input signal DAC and a parameter used in the matrix normalization processing by the weighting signal DAC.

Description

Information processing methods, information processing systems, and programs

[0001] This invention relates to an information processing method, an information processing system, and a program.

[0002] Artificial intelligence, exemplified by deep learning, plays a central role in current information processing technology. In recent years, as society has become increasingly reliant on ICT (Information and Communication Technology), the increasing power consumption associated with the execution of artificial intelligence has become a problem. The majority of this power consumption is accounted for by matrix multiplication, which is commonly used in calculations involving input matrices and weight matrices in artificial intelligence, including deep learning.

[0003] Just to clarify, matrix multiplication is matrix multiplication. More specifically, the matrix product of the first matrix and the second matrix is ​​the multiplication of the first matrix and the transpose of the second matrix, where the first matrix acts on the transpose of the second matrix from the left. Hereafter, the first matrix will be called the input matrix and the second matrix will be called the weight matrix. Specifically, when A is an n × I input matrix, X is an m × I weight matrix, and Y is an n × m output matrix, the matrix product of the input matrix A and the weight matrix X means the calculation of equation (1) below.

[0004]

[0005] Here, the symbol in equation (2) below represents the transpose of matrix X.

[0006]

[0007] One method for achieving low power consumption in artificial intelligence is to perform matrix multiplication calculations using an analog computer based on physical phenomena (see Non-Patent Literature 1). In this method, the input matrix A and the weight matrix X are first converted from digital to analog and quantized using a DAC (Digital-to-Analog Converter) for the input signal and a DAC (Digital-to-Analog Converter) for the weight signal, respectively, connected to an external computer. Here, quantization is the process of reducing the precision of a digital signal, which is processed by a digital computer, from a high-precision format to a low-precision format for processing by an analog computer.

[0008] Then, an analog computer is used to calculate the product of the analog-converted matrices, and the result is converted from analog to digital using an ADC (Analog-to-Digital Converter) for the output signal. Finally, the output matrix Y can be obtained as a digital value. This method achieves low power consumption because it performs the computationally expensive matrix multiplication calculation based on physical phenomena.

[0009] Nakajima, Mitsumasa, Tsurutani, Takuma, Tanaka, Akishi, and Hashimoto, Toshikazu (2022) Physical implementation of reservoir computing using optical devices. NTT Technical Journal / Nippon Telegraph and Telephone Corporation (ed.), 34(6), 39-42.

[0010] However, in the case of matrix multiplication calculations using an analog computer as described in Non-Patent Document 1, the calculation accuracy sometimes decreased. Note that an analog computer can be any device that returns an output matrix by performing analog calculations when given an input matrix and a weight matrix. Thus, with the technologies proposed so far, the accuracy of matrix multiplication calculations using analog computers was sometimes poor. In other words, the calculation accuracy was sometimes low when digital signals were converted to analog, the matrix multiplication was calculated, and then the signal was digitized again.

[0011] In view of the above circumstances, the present invention aims to provide a technique for improving the calculation accuracy when a digital signal is converted to analog, a matrix product is calculated, and then the signal is converted back to digital.

[0012] One aspect of the present invention comprises an input signal DAC that outputs a first analog signal to an analog computer, a weight signal DAC that outputs a second analog signal to the analog computer, and an output signal ADC that performs AD conversion on the analog signal output by the analog computer, wherein the input signal DAC performs matrix normalization processing on an input matrix A to normalize the matrix to be processed, and then performs quantization matrix acquisition processing on the input matrix A normalized by the matrix normalization processing to quantize the matrix to be processed, the weight signal DAC performs matrix normalization processing on a weight matrix X, and then performs quantization matrix acquisition processing on the weight matrix X normalized by the matrix normalization processing, and the output signal ADC performs AD conversion on the result of the AD conversion using the matrix normalization processing performed by the input signal DAC An information processing method executed by an information processing system, which applies a correction based on parameters and parameters used in the matrix normalization process by the weight signal DAC, comprising: a first matrix normalization step in which the input signal DAC performs the matrix normalization process on the input matrix A; a first quantization step in which the input signal DAC performs the quantization matrix acquisition process on the input matrix A normalized by the matrix normalization process; a second matrix normalization step in which the weight signal DAC performs the matrix normalization process on the weight matrix X; a second quantization step in which the weight signal DAC performs the quantization matrix acquisition process on the input matrix X normalized by the matrix normalization process; and a correction step in which the output signal ADC applies the correction.

[0013] One aspect of the present invention is an information processing system comprising: an input signal DAC that outputs a first analog signal to an analog computer; a weight signal DAC that outputs a second analog signal to the analog computer; and an output signal ADC that performs AD conversion on the analog signal output by the analog computer. The input signal DAC performs a matrix normalization process on an input matrix A to normalize the matrix to be processed, and then performs a quantization matrix acquisition process on the input matrix A normalized by the matrix normalization process to quantize the matrix to be processed. The weight signal DAC performs the matrix normalization process on a weight matrix X, and then performs the quantization matrix acquisition process on the weight matrix X normalized by the matrix normalization process. The output signal ADC applies a correction to the result of the AD conversion based on the parameters used in the matrix normalization process by the input signal DAC and the parameters used in the matrix normalization process by the weight signal DAC.

[0014] One aspect of the present invention is a program for causing a computer to function as the above-mentioned information processing system.

[0015] This invention makes it possible to improve the calculation accuracy when converting digital signals to analog, calculating matrix products, and then digitizing them again.

[0016] An explanatory diagram illustrating the information processing system of the embodiment. An explanatory diagram illustrating the input signal DAC in the embodiment. An explanatory diagram illustrating the weight signal DAC in the embodiment. cos(θ) in the embodiment ａｉ～,ΔｘｊAn explanatory diagram for explaining the expected value and theoretical error of ). An explanatory diagram for explaining the output signal DAC in the embodiment. An explanatory diagram for explaining a second example of the technique for calculating the lower limit range in the embodiment. An explanatory diagram for explaining a second example of the technique for calculating the upper limit range in the embodiment. A first explanatory diagram for explaining the output correction process in the embodiment. A second explanatory diagram for explaining the output correction process in the embodiment. A third explanatory diagram for explaining the output correction process in the embodiment. A fourth explanatory diagram for explaining the output correction process in the embodiment. A fifth explanatory diagram for explaining the output correction process in the embodiment. A diagram showing an example of the hardware configuration of the input signal DAC in the embodiment. A flowchart showing an example of the processing flow executed by the input signal DAC in the embodiment. A diagram showing an example of the hardware configuration of the weight signal DAC in the embodiment. A flowchart showing an example of the processing flow executed by the weight signal DAC in the embodiment. A diagram showing an example of the hardware configuration of the output signal ADC in the embodiment. A flowchart showing an example of the processing flow executed by the output signal ADC in the embodiment. A flowchart showing an example of the processing flow executed by the analog computer 1 in the embodiment.

[0017] (Embodiment) Figure 1 is an explanatory diagram illustrating an information processing system 100 of an embodiment. The information processing system 100 is equipped with an analog computer and is communicatively connected to an external computer 9. In the example of Figure 1, the device denoted by reference numeral 1 is the analog computer equipped with the information processing system 100. Hereafter, this analog computer will be referred to as analog computer 1. An analog computer is a device that returns an output matrix by performing analog calculations when given an input matrix and a weight matrix. Therefore, analog computer 1 is, for example, the analog computer described in Non-Patent Document 1. The external computer 9 is a computing device that outputs at least two matrices to the information processing system 100, such as the external computer described in Non-Patent Document 1.

[0018] Here, one of the two matrices is called the input matrix, and the other is called the weight matrix. As mentioned above, the matrix product of the input matrix and the weight matrix is ​​the input matrix acting on the transpose of the weight matrix from the left, and is the multiplication of the input matrix and the transpose of the weight matrix. The result of this multiplication is the output matrix. The information processing system 100 obtains the value of the matrix product of the input matrix and the weight matrix. This matrix product will now be explained.

[0019] <Matrix Product> Hereafter, the symbol A will represent the input matrix and the symbol X will represent the weight matrix. The input matrix A is an n × l matrix, and the weight matrix X is an m × l matrix. The matrix product of the input matrix A and the weight matrix X is represented by the symbol Y. Therefore, the matrix product of the input matrix A and the weight matrix X is the output matrix Y. Since the output matrix Y is the matrix product of the input matrix A and the weight matrix X, the relationship between the output matrix Y, the input matrix A, and the weight matrix X is expressed by the following equation (3).

[0020]

[0021] As can be seen from the definition of matrix multiplication, if the input matrix A is an n × l matrix and the weight matrix X is an m × l matrix, then the output matrix Y is an n × m matrix.

[0022] Here, the i-th row vector from the top of matrix A is a ｉ The j-th row vector from the top of matrix X is x ｊ Then, the (i, j) element y[i, j] of matrix Y is expressed by equation (4).

[0023]

[0024] In other words, the (i, j) element y[i, j] of matrix Y is a vector a ｉ and vector x ｊ This is the dot product of vector a. ｉ and vector x ｊ Since both lengths are l, the computational cost of y[i, j] is O(l) using Landau notation. Therefore, a matrix Y of size n × m can be obtained with a computational cost of O(nml). This computational cost increases as the size of the matrix increases.

[0025] Analog computer 1 performs calculations using analog signals, thereby reducing the power consumption required for these calculations.

[0026] <About the DAC> The information processing system 100 includes not only the analog computer 1, but also an input signal DAC 2, a weight signal DAC 3, and an output signal ADC 4. The input signal DAC 2 obtains the input matrix A from the external computer 9 and outputs an analog signal to the analog computer 1. The weight signal DAC 3 obtains the weight matrix X from the external computer 9 and outputs an analog signal to the analog computer 1. The output signal ADC 4 performs an AD (Analog-to-Digital) conversion on the analog signal output by the analog computer 1. The output signal ADC 4 then outputs the output matrix Y to the external computer 9.

[0027] <<Input Signal DAC and Weight Signal DAC>> Figure 2 is an explanatory diagram illustrating the input signal DAC2 in the embodiment. Figure 3 is an explanatory diagram illustrating the weight signal DAC3 in the embodiment.

[0028] The input signal DAC2 comprises a first converter 20 and a first control unit 21 which is a control unit equipped with a processor 91 such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Network Processing Unit) connected by a bus, and a memory 92, and executes a program.

[0029] The first converter 20 performs a Digital-to-Analog (DA) conversion on the matrix to be converted. A DA conversion is a process that converts discrete values ​​to analog values. In other words, a DA conversion means converting the signal representing the value to be represented from a digital signal to an analog signal. Furthermore, a DA conversion on a matrix means converting the elements of the matrix from discrete values ​​to analog values. In other words, a DA conversion on a matrix means converting the signal representing the matrix to be represented from a digital signal to an analog signal.

[0030] The target of the DA conversion performed by the first converter 20 is the result of the quantization matrix acquisition process, which will be described later, performed by the first control unit 21.

[0031] The first control unit 21 performs matrix normalization processing, parameter transmission processing, quantization value acquisition processing, quantization matrix acquisition processing, and quantization error transmission processing on the matrix to be processed. The object to be processed here, that is, the object to be processed by the matrix normalization processing performed by the first control unit 21, is the input matrix A. The matrix normalization processing, parameter transmission processing, quantization value acquisition processing, quantization matrix acquisition processing, and quantization error transmission processing will be described later.

[0032] The weight signal DAC3 comprises a second converter 30 and a second control unit 31 which is a control unit equipped with a processor 93 such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Network Processing Unit) connected by a bus, and a memory 94, and executes a program.

[0033] The second converter 30 performs a Digital-to-Analog (DA) transformation on the matrix to be transformed. The target of the DA transformation performed by the second converter 30 is the result of the quantization matrix acquisition process, which is performed by the second control unit 31 and will be described later.

[0034] The second control unit 31 performs matrix normalization processing, parameter transmission processing, quantization value acquisition processing, quantization matrix acquisition processing, and quantization error transmission processing on the matrix to be processed. The object to be processed here, that is, the object to be processed by the matrix normalization processing performed by the second control unit 31, is the weight matrix X.

[0035] <<<Matrix Normalization Process>>> Matrix normalization is the process of normalizing the matrix being processed. Therefore, by performing matrix normalization, a normalized matrix is ​​obtained. Hereafter, the normalized matrix will be referred to as the normalized matrix. Normalization is the process of transforming a matrix so that each element falls within a predetermined range.

[0036] The matrix normalization process may be any process as long as the matrix to be processed is normalized. For example, it may be a process of shifting and scaling elements of each row of the matrix to be processed to normalize the elements into values ranging from 0 to 1. Note that "normalizing into values ranging from 0 to 1" means a process of converting each element of the matrix such that the element takes a value ranging from 0 to 1.

[0037] <<Description of an Example of Matrix Normalization Process Using Mathematical Formulas>> As described above, the matrix to be subjected to the matrix normalization process executed by the first control unit 21 is an input matrix A. Accordingly, an example of the matrix normalization process when the input matrix A is the processing target will be described using mathematical formulas.

[0038] Note that, as described above, the second control unit 31 also executes the matrix normalization process. However, the processing target is not the input matrix A, but a weight matrix X. Therefore, the example of the matrix normalization process described below using mathematical formulas is also an example of the matrix normalization process executed by the second control unit 31 if the input matrix A is replaced with the weight matrix X.

[0039] Let the i-th row vector a from the top of the input matrix A ｉ have the j-th element a ｉ [j]. The normalized vector is represented by the symbols in the following formula (5).

[0040]

[0041] The shift and scaling executed in the matrix normalization process at this time is represented by the following formula (6). Formula (6) indicates that shifting and scaling a ｉ gives the value of the element represented by the symbol in formula (7). Note that the symbol in formula (7) means the j-th element of the vector represented by formula (5).

[0042]

[0043]

[0044] Here, the parameter represented by the symbol in formula (8) (hereinafter referred to as "μ parameter") and the parameter represented by the symbol in formula (9) (hereinafter referred to as "ν parameter") are values defined by formula (10) and formula (11), and are parameters for shifting and scaling a vector.

[0045]

[0046]

[0047]

[0048]

[0049] As shown in equation (10), the μ parameter represents the smallest element in each row vector, and the ν parameter is the reciprocal of the range of elements that each row vector can take.

[0050] Let's look at a concrete example of normalization. Let the input matrix A be given by the following equation (12).

[0051]

[0052] Here, the minimum values ​​for each row are 1, 2, and 1, respectively, as shown in equation (13).

[0053]

[0054] Furthermore, the maximum values ​​for each row are 5, 6, and 3, respectively, as shown in equation (14).

[0055]

[0056] To make the minimum value of the normalized vector zero, the matrix normalization process first subtracts the value of the μ parameter from each element of the matrix. As a result, matrix A becomes the matrix given by equation (15) below.

[0057]

[0058] Next, in order to make the maximum value of the normalized vector 1, each element of the matrix is ​​multiplied by the value of the ν parameter. As a result, the normalized matrix A (hereinafter referred to as "normalized matrix A") is obtained. － This yields the result of equation (16) below, where the symbols represent the normalized matrix A. － This represents the normalized matrix A when matrix A is given by equation (12). － The following equation (17) is given.

[0059]

[0060]

[0061] As the example in equation (17) shows, each element of the normalized matrix is ​​between 0 and 1.

[0062] The μ-parameter and ν-parameter for each row are obtained by referencing each element of the n×l matrix A. Therefore, the computational cost of obtaining the μ-parameter and ν-parameter for each row is O(nl). Also, the normalized matrix A － Since each element is calculated from each element of matrix A, the computational cost is O(nl).

[0063] As described above, changing matrix A to matrix X is an example of the matrix normalization process performed by the DAC3 for weight signals. Therefore, in the matrix normalization process performed by the DAC3 for weight signals, the normalization of the m × l matrix X is performed at a computational cost of O(ml). That is, when matrix X is an m × l matrix, the normalized matrix X (hereinafter referred to as "normalized matrix X") is obtained. － This can be obtained with a computational cost of O(ml).

[0064] <<

[0065] row vector a ｉ The sum of the elements σ ａｉ Defines σ. ａｉ The definition of can be expressed as shown in equation (18). The symbol on the left side of equation (18) is σ. ａｉ It represents.

[0066]

[0067] Here, in matrix A, σ ａｉ Since this is obtained for each of the n rows, the total computational cost is O(nl). Row vector x in matrix X ｊ The sum of the elements σ ｘｊ Since this is obtained for each of the m rows, the total computational cost is O(ml).

[0068] From equation (6), the following equation (19) holds true.

[0069]

[0070] The symbol on the left side of equation (19) represents the row vector a of the normalized input matrix A. ｉ and the row vector x of the normalized weight matrix X ｊ This represents the inner product of the two. The symbol to the left of the symbol on the left side of equation (19) (i.e., the symbol on the left side of equation (6)) is the row vector a of the normalized input matrix A. ｉ This represents the row vector x of the normalized weight matrix X. The symbol to the right of the symbol on the left side of equation (19) (that is, the remaining symbol on the left side of equation (19) after removing the symbol on the left side of equation (6)) represents the row vector x of the normalized weight matrix X. ｊ This represents the transpose of the vector.

[0071] Now, since equation (19) holds true, we can derive equation (20) from equation (4).

[0072]

[0073] Equation (20) shows that the elements of matrix Y can be calculated from the dot product of normalized vectors. Therefore, the sum of the elements of the row vectors σ ａｉ The μ parameter and ν parameter are transmitted to the output signal ADC via the external computer 9, and the dot product value (i.e., the value represented by the symbol on the left side of equation (19)) is calculated using the normalized matrix. Based on the result, the dot product value of the matrix before normalization can be obtained using equation (20). The dot product value of the matrix before normalization is the value represented by the symbol on the right side of equation (4).

[0074] Furthermore, the analog computer 1 will perform calculations on the inner product value of the normalized matrix, that is, the value represented by the symbol on the left side of equation (19). This is because the analog computer 1 does not directly calculate the normalized vector on the left side of equation (19), but rather calculates the vector obtained by quantizing the normalized vector.

[0075] <<<Parameter Transmission Process>>> The parameter transmission process is the process of transmitting the parameters used in the matrix normalization process to the external computer 9 by controlling the operation of the interface unit 22 or interface unit 32, which will be described later. In the parameter transmission process performed by the first control unit 21, the parameters used in the matrix normalization process performed by the first control unit 21 are transmitted to the external computer 9 via the interface unit 22. In the parameter transmission process performed by the second control unit 31, the parameters used in the matrix normalization process performed by the second control unit 31 are transmitted to the external computer 9 via the interface unit 32.

[0076] The parameters used in matrix normalization are, for example, the parameters used for shift scaling. The parameters used for shift scaling are, for example, the μ parameter and the ν parameter.

[0077] <<<Quantized Value Acquisition Process and Quantized Matrix Acquisition Process>>> The quantized value acquisition process is the process of obtaining the number of quantized values ​​indicated by the quantization number that will be used to quantize the normalized matrix obtained in the matrix normalization process. Specifically, matrix quantization is the process of quantizing each element of the matrix. More specifically, matrix quantization is the process of replacing the value of each element of the matrix with the closest value from a given set of discrete values.

[0078] The discrete value itself, or the discrete value that gives this discrete value (i.e., the discrete value to be replaced) under a predetermined rule, is generally called the quantized value. The latter, "the discrete value that gives the discrete value to be replaced under a predetermined rule," is, for example, the quantized value when the average of the i-th quantized value from the bottom and the (i+1)-th quantized value from the top is taken as "this discrete value." An example of the quantized value in this latter example is, for example, y in the output correction process described later. ｌ － [i, j] and y ｕ － [i, j]. In this case, the “discrete value to be replaced” is, for example, y ～ [i, j]

[0079] The quantization number is the number of quantized values. The quantization number is a predetermined number and is pre-stored in a predetermined storage device, such as the first storage unit 23 or the second storage unit 33. The quantization number may be entered by the user each time the quantization value acquisition process is executed.

[0080] <<<<Quantization Matrix Acquisition Process>>>> The quantization matrix acquisition process is a process that performs quantization on the target of processing. The target of processing is a normalized matrix. Furthermore, quantization is performed using quantized values. Therefore, quantized values ​​are also used in the quantization matrix acquisition process. The quantized values ​​used in the quantization matrix acquisition process are, for example, the quantized values ​​obtained in the quantized value acquisition process. Note that it is not necessarily required that quantized values ​​be obtained in the quantized value acquisition process; in the quantization matrix acquisition process, quantization may be performed using, for example, predetermined quantized values.

[0081] The quantization matrix acquisition process involves replacing the values ​​of each element of the normalized matrix being processed with the closest quantized value. Hereafter, the quantized matrix will be referred to as the quantized matrix. Using this terminology, the quantization matrix acquisition process can be described as obtaining a quantized matrix by quantizing the normalized matrix being processed.

[0082] The normalized matrix to be processed in the quantization matrix acquisition process is normalized matrix A when the quantization matrix acquisition process is performed by the input signal DAC2. － Therefore, when the quantization matrix acquisition process is performed by the DAC3 for weight signals, the normalized matrix X － That is the case.

[0083] Furthermore, the quantized values ​​obtained in the quantized value acquisition process are, for example, the quantized values ​​used in the quantization matrix acquisition process performed by the input signal DAC2 when the quantized value acquisition process is performed by the input signal DAC2. Also, the quantized values ​​obtained in the quantized value acquisition process are, for example, the quantized values ​​used in the quantization matrix acquisition process performed by the weight signal DAC3 when the quantized value acquisition process is performed by the weight signal DAC3.

[0084] By the way, the quantization value acquisition process can be any process that yields quantized values. Here, we will show some examples of quantization value acquisition processes and explain an example of a quantization matrix acquisition process that uses the quantized values ​​obtained in the quantization value acquisition process.

[0085] <<<<<<First example of quantization value acquisition process>>>>> The quantization value acquisition process may be a process that acquires quantization values ​​that satisfy the condition that the intervals between quantization values ​​are equal or approximately equal (hereinafter referred to as the "first quantization value condition"). For example, consider the case where the number of bits for quantization is 2. In this case, the quantization values ​​that satisfy the first quantization value condition are, for example, 0.00, 0.33, 0.66, and 1.00. Therefore, the quantization value acquisition process is a process that acquires 0.00, 0.33, 0.66, and 1.00 as quantization values.

[0086] Note that the four values ​​0.00, 0.33, 0.66, and 1.00 have four normalized elements between 0 and 1 (= 2 ２ These are the values ​​obtained by dividing the number. The interval between 0.33 and 0.00 is 0.33. The interval between 0.33 and 0.66 is 0.33. The interval between 0.66 and 1.00 is 0.34, which is not 0.33, but 0.33 and 0.34 are almost the same. Therefore, the four values ​​0.00, 0.33, 0.66, and 1.00 satisfy the first quantization value condition.

[0087] As described above, the quantization matrix acquisition process is a process that replaces the value of each element of the normalization matrix with the closest quantization value. Therefore, if the quantization values ​​obtained in the quantization value acquisition process are 0.00, 0.33, 0.66, and 1.00, then the normalization matrix A － If it is expressed by equation (21), then the normalized matrix A after quantization － (hereinafter referred to as “quantization matrix A ～ The expression is given by the following equation (22).

[0088]

[0089]

[0090] Note that the symbol on the left side of equation (22) represents the quantization matrix A.～ It represents.

[0091] <<

[0092] For example, when dividing the elements from 0 to 1 into four parts, the quantization values ​​that satisfy the second quantization value condition are, for example, 0.00, 0.77, 0.81, and 1.00. Note that 0.00 is the square root of 0.00, 0.77 is the square root of 0.33, 0.81 is the square root of 0.66, and 1.00 is the square root of 1.00.

[0093] The interval between 0.00 and 0.77 is 0.77, the interval between 0.77 and 0.81 is 0.04, and the interval between 0.81 and 1.00 is 0.19. Therefore, with the exception of the interval between 0.81 and 1.00, from 0.00 to 0.81, the precision becomes finer (i.e., the interval between quantized values ​​becomes smaller) as the quantization value increases. The second quantization condition is satisfied. For this exception, the maximum quantization value is 1, so the second quantization condition is actually satisfied.

[0094] In the case where the quantized values ​​obtained in the quantum value acquisition process are 0.00, 0.77, 0.81, and 1.00, and the normalization matrix A － When expressed by equation (23), the quantization matrix A ～ This is given by the following equation (24).

[0095]

[0096]

[0097] This example of obtaining the quantization matrix in equation (24) is an example of a quantization value acquisition process that obtains quantization values ​​that satisfy the second quantization value condition, and a quantization matrix acquisition process that uses the quantization values ​​obtained in that quantization value acquisition process.

[0098] When the number of quantization values ​​used in quantization (i.e., the number of quantization values) is denoted as b, obtaining the quantization values ​​can be done in a computational cost of O(b). Also, the n × l normalization matrix A － The quantization of each element can be performed with a computational cost of O(nl). Furthermore, the normalized matrix X is m×l. － The quantization of each element can be performed in O(ml). Therefore, the total computational cost required for these two quantizations is O((n+m)l)).

[0099] <<<<<Third example of quantization value acquisition process>>>> The quantization value acquisition process may be a process that acquires a quantization value that satisfies the condition (hereinafter referred to as the "third quantization value condition") that it is the average of each cluster obtained as a result of performing clustering on each element of the matrix.

[0100] For example, normalized matrix A － Each element is treated as a one-dimensional data point, and using the k-means method, each element is divided into b clusters corresponding to the quantization number b. The mean of each resulting cluster is used as the quantization value to create the normalization matrix A. － Each element is quantized. This process is an example of obtaining a quantized value that satisfies the third quantization value condition and then quantizing using the obtained quantized value. In other words, this process is an example of obtaining a quantized value that satisfies the third quantization value condition and obtaining a quantized matrix using the quantized value obtained in that quantization value acquisition process.

[0101] In the quantization value acquisition process that obtains quantized values ​​that satisfy the third quantization value condition, clustering is used. Therefore, efficient quantization according to the data distribution is possible.

[0102] Let's look at a specific example: A 3x4 normalized matrix A. － Assume that is given by the following equation (25).

[0103]

[0104] This normalized matrix A － When we obtain 12 (=3 × 4) elements, we get the result shown in equation (26).

[0105]

[0106] When the quantization number b = 2, if we use clustering to divide the 12 elements shown in equation (26) into two, we obtain two clusters: cluster 1 shown in equation (27) and cluster 2 shown in equation (28).

[0107]

[0108]

[0109] As can be seen from equation (27), the mean of cluster 1 is 0.79, and as can be seen from equation (28), the mean of cluster 2 is 0.10.

[0110] Using the results of this clustering, the normalized matrix A of equation (25) － The result of quantization (i.e., quantization matrix A) ～ The answer is given by equation (29) below.

[0111]

[0112] In this quantization value acquisition process and quantization matrix acquisition process, the n × l normalized matrix A － The computational cost required to extract elements from is O(nl). Also, t Ａ－ Normalized matrix A － If the number of iterations of the k-means method used to obtain the quantized values ​​used for quantization is O(nlbt), then the computational cost of clustering is O(nlbt). Ａ－ ) That is, the normalized matrix A － The computational cost of obtaining the quantized value from is O(nlbt). Ａ－ )

[0113] t Ｘ－ The normalized matrix X － If the number of iterations of the k-means method used to obtain the quantized values ​​used for quantization is O(mlbt), then the computational cost of clustering is O(mlbt). Ｘ－ Therefore, the computational cost of obtaining the quantized value in DAC is O(lb(nt)). Ａ－ +mt Ｘ－ ))

[0114] Up to this point, we have explained the matrix normalization process, and then explained the quantization value acquisition process and the quantization matrix acquisition process with three examples. As explained above, the quantization matrix is ​​obtained by performing matrix normalization, followed by quantization value acquisition, and then quantization matrix acquisition.

[0115] <<<Input destination of the quantization matrix>>> The quantization matrix obtained in this way is then input to the first converter 20 if it was obtained by the first control unit 21. In other words, the input destination of the quantization matrix obtained by the quantization matrix acquisition process performed by the first control unit 21 is the first converter 20.

[0116] On the other hand, if the obtained quantization matrix was obtained by the second control unit 31, it is input to the second converter 30. In other words, the input destination for the quantization matrix obtained by the quantization matrix acquisition process performed by the second control unit 31 is the second converter 30.

[0117] The first converter 20 performs a D / A conversion on the input quantization matrix. As a result, an analog signal representing the quantization matrix input to the first converter 20 is obtained. This obtained analog signal is output to the analog computer 1.

[0118] The second converter 30 performs a D / A conversion on the input quantization matrix. As a result, an analog signal representing the quantization matrix input to the second converter 30 is obtained. This obtained analog signal is output to the analog computer 1.

[0119] <<<Quantization Error Transmission Process>>> The quantization error transmission process is a process that transmits quantization errors that occur in the process of obtaining a quantized matrix from a normalized matrix to an external computer 9 by controlling the operation of the interface unit 22 or interface unit 32, which will be described later. In the quantization error transmission process performed by the first control unit 21, the quantization errors that occur in the quantized matrix acquisition process performed by the first control unit 21 are transmitted to the external computer 9 via the interface unit 22. In the quantization error transmission process performed by the second control unit 31, the quantization errors that occur in the quantized matrix acquisition process performed by the second control unit 31 are transmitted to the external computer 9 via the interface unit 32. The quantization errors transmitted to the external computer 9 are used to correct the dot product value in the output correction process, which will be described later.

[0120] <<

[0121] << ｉ The quantization error vector in Δa ｉ Let's assume that Δa ｉ Specifically, it can be expressed by the following equation (30).

[0122]

[0123] row vector x ｊ The quantization error vector in Δx ｊ Let's assume Δx ｊ Specifically, this can be expressed by the following equation (31).

[0124]

[0125] Hereafter, the symbol in equation (32) will be a ｉ － It is written as such, and the symbol in formula (33) is a ｉ ～ Write this and change the symbol in equation (34) to x ｊ － Write this and change the symbol in equation (35) to x ｊ ～ It should be written as follows.

[0126]

[0127]

[0128]

[0129]

[0130] Symbol a ｉ － represents the i-th row vector counted from the top of normalization matrix A － , and symbol a ｉ ～ represents the i-th row vector counted from the top of quantization matrix A ～ . Symbol x ｊ － represents the j-th row vector counted from the top of normalization matrix X － , and symbol x ｊ ～ represents the j-th row vector counted from the top of quantization matrix X ～ .

[0131] Quantization error vector Δa ｉ and quantization error vector Δx ｊ satisfy the relationship of the following formula (36).

[0132]

[0133] Vector a ｉ ～ and vector Δx ｊ form an angle θ represented by the following formula (37) ａｉ～,Δｘｊ . Vector Δa ｉ and vector x ｊ ～ form an angle θ represented by the following formula (38) Δａｉ,ｘｊ～ . Furthermore, vector Δa ｉ and vector Δx ｊ form an angle θ represented by the following formula (39) Δａｉ,Δｘｊ . In this case, the relationship of the following formula (40) holds.

[0134]

[0135]

[0136]

[0137]

[0138] In this equation, ||・|| represents the square norm of the vector. The dot product value a is expressed by the following equation (41). ｉ ～ (x ｊ ～ ) Ｔ This is obtained by analog computer 1. The inner product value a ｉ ～ (x ｊ ～ ) Ｔ is vector a ｉ ～ and vector x ｊ ～ This is the value of the dot product (i.e., the dot product value) between the two.

[0139]

[0140] Here, the computational cost of the square norm of each vector in the last three terms on the right-hand side of equation (40) is as small as O((n+m)l). This is because the process of obtaining the square norm of each vector is based on the normalization matrix and the quantization matrix corresponding to the n×l matrix A and the m×l matrix X.

[0141] On the other hand, calculating cosine is computationally expensive because the calculation cost of finding the angle between n vectors of length l and m vectors is O(nml). However, if we approximate the values ​​of the cosine function in the last three terms on the right-hand side of equation (40), we can obtain the dot product a ｉ ～ (x ｊ ～ ) Ｔ The correction is performed at a lower computational cost.

[0142] Note that one of the cosine function values ​​in the last three terms on the right-hand side of equation (40) is the value cos(θ) expressed by equation (42) below. ａｉ～,Δｘｊ ) is the value of the cosine function in the last three terms on the right-hand side of equation (40), which is the value cos(θ) expressed in equation (43) below. Δａｉ,ｘｊ～). The last one of the values of the cosine function in the last three terms on the right-hand side of formula (40) is the value cos(θ represented by the following formula (44) Δａｉ,Δｘｊ ).

[0143]

[0144]

[0145]

[0146] Here, the angle between vectors is approximated. For this approximation, the reference vector b represented by the following formula (45) is used Ａ . The reference vector b Ａ is obtained from matrix A.

[0147]

[0148] As shown by formula (45), the reference vector b Ａ is the quantization matrix A ～ is the average of the row vectors in . The computational cost of the process of obtaining the reference vector b ～ from the quantization matrix A Ａ is O(nl).

[0149] A specific example of the reference vector b Ａ is shown. Consider the quantization matrix A ～ of the following formula (46).

[0150]

[0151] In this case, the reference vector b Ａ is the quantization matrix A ～ as the average of the row vectors in , which is represented by the following formula (47).

[0152]

[0153] The angle θ between the reference vector b Ａ and the vector a ｉ ～ is represented by the following formula (48). The angle θ between the reference vector b ａｉ～,ｂＡ is represented by the following formula (48). The angle θ between the reference vector b Ａ and the vector x ｊ ～ is represented by the following formula (49). The angle θ between the reference vector b ｘｊ～,ｂＡ is represented by the following formula (49). The reference vector bＡ and vector Δa ｉ angle θ Δａｉ,ｂＡ This is expressed by the following equation (50): Reference vector b Ａ and vector Δx ｊ angle θ Δｘｊ,ｂＡ This is expressed by the following equation (51).

[0154]

[0155]

[0156]

[0157]

[0158] At this time, the value cos(θ) ａｉ～,Δｘｊ ) and the value cos(θ) Δａｉ,ｘｊ～ ) and the value cos(θ) Δａｉ,Δｘｊ The expected value with respect to is given by the following equation (52).

[0159]

[0160] Using this expected value, we can find the value cos(θ) ａｉ～,Δｘｊ ) and the value cos(θ) Δａｉ,ｘｊ～ ) and the value cos(θ) Δａｉ,Δｘｊ When approximating ) with the following equation (53), the theoretical error is given by the following equation.

[0161]

[0162] Here, the angle θ used to calculate the matrix product is used. ａｉ～,ｂＡ And, angle θ ｘｊ～,ｂＡ And, angle θ Δａｉ,ｂＡ And, angle θ Δｘｊ,ｂＡ This can be obtained in O((n+m)l) computational cost based on each quantization matrix and each quantization error matrix of an n×l matrix A and an m×l matrix X.

[0163] <<<<<<<<Proof of expected value and theoretical error>>>>>>> Here, the value cos(θ) ａｉ～,Δｘｊ We will show that the expected value of ) is expressed by the first equation of equation (52), and that its theoretical error is expressed by the first equation of equation (53). Note that the value cos(θ) Δａｉ,ｘｊ～ ) and the value cos(θ Δａｉ,ΔｘｊThe expected values ​​and theoretical errors for each of these can be explained using a similar logical progression, so for simplicity, we will use the value cos(θ) here. ａｉ～,Δｘｊ This section explains the expected value and theoretical error of ).

[0164] Figure 4 shows cos(θ) in the embodiment. ａｉ～,Δｘｊ This is an explanatory diagram to explain the expected value and theoretical error of ). Figure 4 shows vector b Ａ and vector a ｉ ～ And, vector Δx ｊ This shows a three-dimensional space given by and . In this three-dimensional space, vector b Ａ and vector a ｉ ～ The plane and vector Δx given by ｊ Let φ be the angle between the two points.

[0165] Then, vector b Ａ and vector a ｉ ～ And, vector Δx ｊ The coordinates in three-dimensional space are expressed by the following equation (54).

[0166]

[0167] Therefore, vector a ｉ ～ and vector Δx ｊ The square of the distance between the coordinates is given by the following equation (55).

[0168]

[0169] Vector a ｉ ～ and vector Δx ｊ We assume that the norms of each are 1. Therefore, from the Law of Cosines, the following equation (56) holds.

[0170]

[0171] Vector b Ａ and vector a ｉ ～ The plane and vector Δx given by ｊ Since the angle φ formed by the two is between 0 and π, cos(θ) ａｉ～,ΔｘｊThe expected value of ) is as shown in formula (57).

[0172]

[0173] Further, from formula (56), cos(θ ａｉ～,Δｘｊ ) takes the maximum value represented by formula (58) when φ=0, and takes the minimum value represented by formula (59) when φ=π. Therefore, when cos(θ ａｉ～,Δｘｊ ) is approximated using the expected value, the theoretical error is as shown in formula (60).

[0174]

[0175]

[0176]

[0177] So far, the expected value and theoretical error of the value cos(θ ａｉ～,Δｘｊ ) have been described. As mentioned above, the expected values and theoretical errors of the value cos(θ Δａｉ,ｘｊ～ ) and the value cos(θ Δａｉ,Δｘｊ ) can also be similarly shown.

[0178] Since the expected value can be obtained as shown in formula (52), the inner product value a ｉ ～ (x ｊ ～ ) Ｔ is approximated by the following formula (61).

[0179]

[0180] In formula (61), the reference vector b Ａ is calculated from the normalized matrix A ～ of matrix A as shown in formula (45). From the normalized matrix X ～ of matrix X, the reference vector b Ｘ represented by the following formula (62) can be obtained.

[0181]

[0182] When the reference vector b Ｘ is used, the value cos(θ ａｉ～,Δｘｊ ), the value cos(θ Δａｉ,ｘｊ～ ), and the value cos(θ Δａｉ,ΔｘｊThe expected values ​​and theoretical errors for each of the following are given by equation (63):

[0183]

[0184] Reference vector b Ａ When using the dot product a ｉ ～ (x ｊ ～ ) Ｔ This could be approximated as shown in equation (61). Reference vector b Ｘ Even when using the dot product a ｉ ～ (x ｊ ～ ) Ｔ An approximation is possible. When approximating the inner product value, using a reference vector that reduces the theoretical error improves calculation accuracy. Therefore, in analog computer 1, the reference vector b Ａ and reference vector b Ｘ Of these, the dot product value obtained is approximated using the one that results in a smaller theoretical error.

[0185] Specifically, the analog computer 1 obtains an approximate dot product value represented by the following equation (64).

[0186]

[0187] Here, the reference vector is given by equation (65) below.

[0188]

[0189] In equation (65), the computational cost required to obtain the reference vector is O(1). Also, in equation (64), the dot product value a ｉ ～ (x ｊ ～ ) Ｔ Since this is calculated by analog computer 1, the dot product value a ｉ ～ (x ｊ ～ ) Ｔ The computational cost required to obtain this is O(1).

[0190] The size of the matrix Y to be calculated in the matrix multiplication is n × m, and the reference vector can be obtained in a computational cost of O(n+m)l). The angle between the reference vector and each vector can be obtained in a computational cost of O(n+m)l, and the norm of each vector can be obtained in a computational cost of O(n+m)l. Therefore, the computational cost required to obtain the matrix multiplication using equation (64) is O(n+m)l.

[0191] Each of the control units, the first control unit 21 and the second control unit 31, transmits a reference vector, the angle between the reference vector and each vector, and the norm of each vector to an external computer in order to correct the error in the dot product value caused by the quantization error.

[0192] <<DAC for Output Signal>> Figure 5 is an explanatory diagram illustrating the ADC4 for output signals in the embodiment. The ADC4 for output signals performs analog-to-digital conversion on the analog signal representing the dot product value obtained by the analog computer 1. The ADC4 for output signals further outputs the digital signal after the AD conversion, which is a digital signal representing the dot product value obtained by the analog computer 1, to the external computer 9.

[0193] The output signal ADC 4 comprises a third converter 40 and a third control unit 41 which is a control unit equipped with a processor 95 such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Network Processing Unit) connected by a bus, and a memory 96, and executes a program.

[0194] The third converter 40 performs an analog-to-digital (AD) conversion on the value to be converted. AD conversion is the process of converting an analog value to a discrete value. In other words, AD conversion means converting the signal representing the value to be represented from an analog signal to a digital signal. AD conversion on a matrix means converting the elements of the matrix from analog values ​​to discrete values. In other words, AD conversion on a matrix means converting the signal representing the matrix to be represented from an analog signal to a digital signal. Furthermore, the target of the AD conversion performed by the third converter 40 is the dot product value obtained by the analog computer 1.

[0195] More specifically, the third converter 40 is an AD converter that uses quantized values ​​between the lower limit range and the upper limit range (quantized value distribution range described later) obtained in the range adjustment process described later to convert the dot product value obtained by the analog computer 1 from an analog value to a discrete value.

[0196] The result of the AD conversion by the third converter 40 is output to the third control unit 41, and the value is corrected by the output correction process described later.

[0197] The third control unit 41 performs range adjustment processing and output correction processing.

[0198] <<<Range Adjustment Process>>> The range adjustment process determines the range in which the quantization values ​​are distributed when the analog signal output by the analog computer 1 is converted into a digital signal by the third converter 40. Hereinafter, the range in which the quantization values ​​are distributed will be called the quantization value distribution range.

[0199] Quantization values ​​are used when converting analog signals to digital signals. These quantization values ​​are set to divide a given quantization value distribution range into equal intervals. From the viewpoint of measurement accuracy of analog signals, it is desirable that the quantization value distribution range satisfies at least the first distribution condition. The first distribution condition is that the quantization value distribution range includes the minimum and maximum values ​​of the analog signal being quantized.

[0200] The measurement accuracy of an analog signal is such that a higher accuracy yields more information from the analog signal, while a lower accuracy yields less information. For example, if the minimum and maximum values ​​of an analog signal do not fall within the quantization distribution range, the information of the analog signal outside the quantization distribution range is lost, and the analog signal is converted to a digital signal.

[0201] Therefore, compared to the case where the minimum and maximum values ​​of the analog signal fall within the quantization distribution range, the amount of information contained in the converted digital signal is less. Thus, the measurement accuracy of an analog signal refers to the amount of information contained in the converted digital signal.

[0202] Furthermore, the quantized value distribution range may satisfy either or both of the second or third distribution conditions in addition to the first distribution condition. The second distribution condition is that the difference between the maximum value of the quantized value distribution range and the maximum value of the analog signal is small. The third distribution condition is that the difference between the minimum value of the quantized value distribution range and the minimum value of the analog signal is also small.

[0203] For these reasons, the third control unit 41 determines the quantization value distribution range by range adjustment processing in order to increase the amount of information contained in the converted digital signal.

[0204] As shown in equation (40), the analog computer 1 calculates the inner product value a corresponding to each element of matrix Y. ｉ ～ (x ｊ ～ ) Ｔ Calculate the following: i is an integer between 1 and n (inclusive), and j is an integer between 1 and m (inclusive).

[0205] The range adjustment process is performed using the element y, which is represented by the following equation (66). ～ The value of [i, j] is the dot product a as shown in equation (67). ｉ ～ (x ｊ ～ ) Ｔ In this case, the analog signal y ～ This process obtains the range [i, j] and quantizes the interval between the lower and upper limits of the obtained range to be equal. Therefore, the obtained analog signal y ～ The range [i, j] can be said to be determined as the quantized value distribution range.

[0206] Note that the analog computer 1 has a dot product value a ｉ ～ (x ｊ ～ ) Ｔ Therefore, the analog signal has an inner product value a ｉ ～ (x ｊ ～ ) Ｔ This is a signal indicating that. Therefore, if the relationship in equation (67) is satisfied, a ｉ ～ (x ｊ ～) Ｔ Analog signal a is an analog signal that indicates ｉ ～ (x ｊ ～ ) Ｔ is an analog signal y ～ It can be said that [i, j]

[0207] Note that element y ～ The definition of [i, j] is a ｉ ～ (x ｊ ～ ) Ｔ This is because, while the element y[i,j] in the i row and j column of the matrix Y mentioned above is calculated from the input matrix and weight matrix which are not normalized and quantized, y ～ [i, j] are different in that they are calculated from normalized and quantized input matrices and weight matrices, and therefore different notations are used.

[0208]

[0209]

[0210] The analog signal y obtained during range adjustment processing ～ The range [i, j] (hereinafter referred to as the "acquired range") can be expressed mathematically as shown in equation (68). Here, R ｌ This is the lower limit of the range to be acquired, R ｕ This is the upper limit of the acquisition range. Hereafter, the lower limit of the acquisition range will be called the lower limit range, and the upper limit of the acquisition range will be called the upper limit range.

[0211]

[0212] <<<<<Calculation of Lower Limit Range>>>> Lower Limit Range R ｌ The lower limit range R ｌ This can be calculated using any computational technique. Here, the lower limit range R ｌ Two techniques for calculating this are illustrated below.

[0213] <<<<<<<First example of the technique for calculating the lower bound range>>>>>>> The first example of the technique for calculating the lower bound range is the quantization matrix A ～ The lower limit range R is based on the minimum value of each column in the table. ｌThis is a technique for calculating the quantization matrix A. Specifically, the quantization matrix A ～ The minimum vector v obtained from the minimum value of each column in [the specified location] ｍｉｎ,Ａ～ Using the lower limit range R ｌ The minimum vector v is calculated. ｍｉｎ,Ａ～ The minimum vector v is defined by the following equation (69). The symbol on the left side of equation (69) is the minimum vector v. ｍｉｎ,Ａ～ It represents.

[0214]

[0215] Minimum vector v ｍｉｎ,Ａ～ The following equation (70) holds true for this.

[0216]

[0217] Therefore, the lower limit range R ｌ The lower limit range R is given by the following equation (71). That is, by performing the calculation in equation (71), the lower limit range R is obtained. ｌ This can be obtained.

[0218]

[0219] This technology allows for a lower range R ｌ A concrete example of obtaining this is shown. Quantization matrix A ～ and the quantization matrix X ～ Let and be as shown in equation (72) below.

[0220]

[0221] From equation (69), the minimum vector v ｍｉｎ,Ａ～ is the quantization matrix A ～ The smallest element in each column is given by the following equation (73).

[0222]

[0223] Furthermore, from equation (71), the lower limit range R ｌ The answer is given by the following equation (74).

[0224]

[0225] In this technique, the minimum vector v ｍｉｎ,Ａ～ is the quantization matrix A ～It is obtained with a computational cost of O(nl) based on this. Also, the lower bound range R ｌ The computational cost of obtaining from equation (71) is O(ml). Therefore, the computational cost of this technique is O((n+m)l).

[0226] << Ａ This is what is used.

[0227] Figure 6 is an explanatory diagram illustrating a second example of the technique for calculating the lower limit range in the embodiment. In the second example of the technique for calculating the lower limit range, as shown in Figure 6, the reference vector b Ａ and vector a ｉ ～ and vector x ｊ ～ The following equation (75) is used for the angle between and .

[0228]

[0229] Also, vector a ｉ ～ and vector x ｊ ～ Since each element can take a value between 0 and 1, the following equation (76) holds true.

[0230]

[0231] Therefore, the following equation (77) holds true.

[0232]

[0233] Therefore, the following equation (78) holds true.

[0234]

[0235] From this, the following equation (79) holds true.

[0236]

[0237] Therefore, in this technology, the lower limit range R ｌ The result is as shown in equation (80) below.

[0238]

[0239] From equation (79), element y ～ The lower limit of [i, j] can be obtained with a computational cost of O(1). Therefore, the lower limit range R of equation (80) can be obtained. ｌ The computational cost required to obtain this is O(nm).

[0240] The first example of the technique for calculating the lower bound range (hereinafter referred to as the "first lower bound technique") and the second example of the technique for calculating the lower bound range (hereinafter referred to as the "second lower bound technique") described so far are both techniques that can obtain the lower bound values ​​of the elements of matrix Y.

[0241] In the range adjustment process, the lower limit range R ｌ To effectively obtain this, the lower limit range R is set to the larger of the lower limit values ​​of the elements of matrix Y obtained using the first lower limit technique and the second lower limit technique. ｌ This decision is made. This can be expressed mathematically as equation (81) below.

[0242]

[0243] In this context, "effective" means that the lower bound range is close to the values ​​of the elements of matrix Y.

[0244] Let's explain the definition of "effective" in more detail. The lower bound of the elements of matrix Y obtained in the first lower bound technique is R ｌ，１ The lower bound of the elements of matrix Y obtained using the second lower bound technique is R ｌ，２ The lower bound of the elements of matrix Y is defined by the following equation (82).

[0245]

[0246] In this case, the lower bound obtained by the first lower bound technique is more effective because the difference between the lower bound of the elements of matrix Y and the lower bound obtained by the second lower bound technique is smaller, which means that the following equation (83) holds.

[0247]

[0248] Furthermore, since the lower bound of the elements of matrix Y is less than or equal to the value shown in equation (82), the lower bound R ｌ，１ and the lower limit is R ｌ，２The relationship between equations (84) and (85) below holds true for this.

[0249]

[0250]

[0251] Conversely, when the following equation (86) holds, the lower bound obtained by the second lower bound technique is more effective than the lower bound obtained by the first lower bound technique because the difference with respect to the lower bound of the elements of matrix Y is smaller.

[0252]

[0253] Furthermore, when the following equation (87) holds, either the first lower bound technique or the second lower bound technique may be used.

[0254]

[0255] <<<<Quantization matrix X ～ Lower limit range R using ｌ Determination >>>> In the first and second lower bound techniques, the quantization matrix A ～ The minimum vector or reference vector was obtained from this. By the way, the quantization matrix X ～ The lower limit range R can also be calculated by using the minimum vector or reference vector. ｌ The following is obtained: Quantization matrix X ～ The minimum vector v obtained from this ｍｉｎ,Ｘ～ This is as shown in equation (88) below. Note that the symbol on the left side of equation (88) is the minimum vector v ｍｉｎ,Ｘ～ It represents.

[0256]

[0257] Minimum vector v ｍｉｎ,Ｘ～ The following equation (89) holds true for this.

[0258]

[0259] Therefore, the lower limit range R ｌ This is as shown in equation (90) below.

[0260]

[0261] Also, the quantization matrix X ～Reference vector b obtained from Ａ When using this, the following equation (91) holds true.

[0262]

[0263] As a result, the lower limit range R ｌ The answer is given by the following equation (92).

[0264]

[0265] Therefore, in the range adjustment process, the lower limit range R is adjusted using the following equation (93) instead of equation (81). ｌ This is calculated.

[0266]

[0267] By using equation (93), the lower limit range R ｌ This can be obtained with a computational cost of O(mn).

[0268] <<<<<Calculation of Upper Range>>>> Upper Range R ｕ The upper limit range R ｕ This can be calculated using any computationally feasible technique. Here, the upper limit range R ｕ Two techniques for calculating this are illustrated below.

[0269] << ～ Upper limit range R based on the maximum value of each column ｕ This is a technique for obtaining the quantization matrix A. Specifically, the quantization matrix A ～ The maximum vector v obtained from the maximum values ​​of each column. ｍａｘ,Ａ～ Using the upper limit range R ｕ This is a technique for calculating the maximum vector v. ｍａｘ,Ａ～ The maximum vector v is defined by the following equation (94). The symbol on the left side of equation (94) is the maximum vector v. ｍａｘ,Ａ～ It represents.

[0270]

[0271] Maximum vector v ｍａｘ,Ａ～ For this, the following equation (95) holds true.

[0272]

[0273] Therefore, the upper range R ｕ The answer is given by the following equation (96).

[0274]

[0275] In this technique, the maximum vector v ｍａｘ,Ａ～ is the quantization matrix A ～ It can be obtained with a computational cost of O(nl) based on this. Also, the upper limit range R ｕ The computational cost of obtaining from equation (96) is O(ml). Therefore, the computational cost of this technique is O((n+m)l).

[0276] << Ａ This is what is used.

[0277] Figure 7 is an explanatory diagram illustrating a second example of the technique for calculating the upper limit range in the embodiment. In the second example of the technique for calculating the upper limit range, as shown in Figure 7, the reference vector b Ａ and vector a ｉ ～ and vector x ｊ ～ The following equation (97) is used for the angle between and .

[0278]

[0279] From equation (97), the following equation (98) holds true.

[0280]

[0281] Therefore, in this technology, the upper limit range R ｕ The answer is given by the following equation (99).

[0282]

[0283] From equation (98), element y ～ The upper limit of [i, j] can be obtained with a computational cost of O(1). Therefore, the upper limit range R in equation (99) ｕ The computational cost required to obtain this is O(nm).

[0284] The first example of the technique for calculating the upper limit range (hereinafter referred to as "first upper limit technique") and the second example of the technique for calculating the upper limit range (hereinafter referred to as "second upper limit technique") described so far are both techniques that can obtain the upper limit values ​​of the elements of matrix Y.

[0285] In the range adjustment process, the upper limit range R ｕ To effectively obtain this, the smaller of the upper limit values ​​of the elements of matrix Y obtained by the first upper limit technique and the second upper limit technique is set to the upper limit range R. ｕ This decision is then made. This can be expressed mathematically as the following equation (100). Note that the definition of "effective" here is the same as the definition above.

[0286]

[0287] <<<<Quantization matrix X ～ Upper limit range R using ｕ Determination >>>> In the first and second lower bound techniques, the quantization matrix A ～ The maximum vector or reference vector was obtained from this. By the way, the quantization matrix X ～ The upper limit range R can also be calculated by using the maximum vector or reference vector. ｕ The following is obtained: Quantization matrix X ～ The maximum vector v obtained from this ｍａｘ,Ｘ～ This is as shown in equation (101) below. Note that the symbol on the left side of equation (101) is the maximum vector v ｍａｘ,Ｘ～ It represents.

[0288]

[0289] Maximum vector v ｍａｘ,Ｘ～ The following equation (102) holds true for this.

[0290]

[0291] Therefore, the upper range R ｕ This is as shown in equation (103) below.

[0292]

[0293] Also, the quantization matrix X ～ Reference vector b obtained fromＡ When using this, the following equation (104) holds true.

[0294]

[0295] As a result, the upper limit range R ｕ The answer is given by the following equation (105).

[0296]

[0297] Therefore, in the range adjustment process, the upper limit range R is adjusted using the following equation (106) instead of equation (99). ｕ This is calculated.

[0298]

[0299] By using equation (106), the upper limit range R ｕ This can be obtained with a computational cost of O(nm).

[0300] In the range adjustment process, the quantization matrix is ​​obtained from the external computer 9, and the lower limit range R is set. ｌ This is obtained, for example, by the first lower limit technique or the second lower limit technique, and the upper limit range R ｕ This is obtained, for example, by the first upper limit technique or the second upper limit technique. Then, in the range adjustment process, the obtained lower limit range R ｌ and upper limit range R ｕ The intervals between them are quantized to be equal.

[0301] <<<Output Correction Processing>>> Output correction processing is a process that corrects the discrete values ​​of the dot product obtained by the third converter 40 based on the parameters transmitted to the external computer 9 by the parameter transmission processing. Therefore, output correction processing is a process that corrects the discrete values ​​of the dot product obtained by the third converter 40 based on the parameters used in the matrix normalization processing. The parameters here include the parameters used in the matrix normalization processing by the input signal DAC2 and the parameters used in the matrix normalization processing by the output signal DAC3.

[0302] The output matrix Y is the result of the correction performed by the output correction process. Therefore, the output correction process can be said to be a process for obtaining the output matrix Y. Furthermore, since the output correction process is a correction process, this process improves the accuracy of the dot product value. Now, we will explain the theory behind the correction performed by the output correction process.

[0303] <<<<<Theory regarding the correction performed in output correction processing>>>>> In analog computer 1, the input vector a is used to calculate the matrix product. ｉ ～ and vector x ｊ ～ The dot product of the two is calculated. Unlike digital computers that perform calculations with a CPU, analog computer 1's output contains noise. Therefore, N(σ) ２ Let ) be the symbol representing Gaussian noise with a standard deviation of σ, and the dot product value calculated by analog computer 1 is y ～ Let '[i, j]' be the inner product value y ～ '[i, j] is as shown in equation (107). Note that the symbol on the left side of equation (101) is y ～ ' represents [i, j].

[0304]

[0305] Note y ～ [i, j] is the theoretical value of the inner product (i.e., calculated digitally or analogically in a noise-free environment), and y ～ '[i, j] is the value of the dot product with noise calculated using an analog computer.

[0306] Here, we assume that the standard deviation σ of the Gaussian noise generated by analog computer 1 is known in advance. The inner product value y ～ Since [i, j] is calculated by analog computer 1, its value needs to be measured by a measuring instrument. However, due to the limitations of the measuring instrument, the dot product value y ～ The value of [i, j] cannot be directly observed. Similarly, the inner product value y ～ The values ​​of [i, j] cannot be directly observed.

[0307] To explain the meaning of "directly" here, it means that because the measuring instrument itself affects the calculation results of analog computer 1, it is not possible to observe the calculation results of analog computer 1 themselves. In other words, the inability to directly observe refers to an event known as Heisenberg's uncertainty principle in the realm of quantum mechanics.

[0308] Figure 8 is the first explanatory diagram illustrating the output correction process in the embodiment. Note that "unobservable" in Figure 8 means "cannot be directly observed" as described above. As shown in Figure 8, the value y represented by the symbol in equation (108) ｌ － The value y, represented by the symbols [i, j] and equation (109). ｕ － [i, j] and the inner product value y ～ Let [i, j] be the quantized value. Then, the value y － [i, j] is valued y ～ Let this be the quantized value of [i, j]. In this case, the quantization shown by equation (110) is the inner product value y. ～ Performing this operation on [i, j] reduces the quantization error.

[0309]

[0310]

[0311]

[0312] This quantization results in a quantized value y ｌ － [i, j] and quantization value y ｕ － The inner product value y between [i, j] ～ The inner product value y based on [i, j] ～ [i, j] quantized to y ｌ － [i, j] and quantization value y ｕ － This quantization method uses the average value of [i, j], making it an intuitively easy-to-understand quantization method.

[0313] Note that here the quantization value y ｌ － [i, j] and quantization value y ｕ －[i, j] refers to the lower limit range R ｌ and upper limit range R ｕ It is assumed that it lies between these two values. However, the value of the dot product y calculated by an analog computer as in equation (107) ～ Since noise is superimposed on [i, j], the quantized value y ｌ － [i, j] is the lower limit range R ｌ The fact that it becomes smaller and the quantization value y ｕ － [i, j] is the upper limit range R ｕ It is possible that it could become larger.

[0314] To calculate the dot product of vectors of length l whose elements are between 0 and 1, an analog computer uses the quantization value y. ｌ － [i, j] is the lower limit range R ｌ If it becomes smaller, then y ｌ － The setting [i, j] = 0 is performed. Also, in analog computers, the quantization value y ｕ － [i, j] is the upper limit range R ｕ If it becomes larger, then y ｕ － The setting [i, j] = l is performed.

[0315] Next, we will further explain this quantization technique using Figures 9 to 11.

[0316] Figure 9 is a second explanatory diagram illustrating the output correction process in the embodiment. Figure 10 is a third explanatory diagram illustrating the output correction process in the embodiment. Figure 11 is a fourth explanatory diagram illustrating the output correction process in the embodiment.

[0317] As shown in Figure 9, the dot product value y ～ [i, j] is the quantized value y ｌ － [i, j] and quantization value y ｕ － Assume that it exists uniformly between [i, j]. In this case, the inner product value y ～ The expected value of the probability distribution of [i, j] is as shown in Figure 10, y － [i, j]. In this case, y－ [i, j] is y ～ This is the point that halves the area under the integral of the probability distribution of [i, j]. Therefore, the inner product value y ～ The point that halves the area under the integral of the probability distribution [i, j] is the dot product y. ～ Using the quantization values ​​[i, j] reduces the quantization error.

[0318] The output correction process is based on this theory and uses the inner product value y ～ This is the process of quantizing [i, j]. Specifically, as shown in Figure 11, first the inner product value y ～ Lower limit y [i, j] ｌ [i, j] and the upper limit y ｕ [i, j] are calculated. Next, the lower limit y ｌ － [i, j] to upper limit y ｕ － The inner product value y in the interval [i, j] ～ The point that halves the integral of the probability distribution of [i, j] is the inner product value y. ～ It is determined as the quantized value of [i, j].

[0319] Now, let's find the lower limit of the dot product y. ｌ ～ [i, j] and the upper limit y ｕ ～ This section describes the technique for calculating [i, j].

[0320] y ～ For [i, j], equations (70), (79), (89), and (91) hold.

[0321] In the output correction process, the y shown in equation (70) ～ The lower limit of [i, j] and the y shown in equation (79) ～ The lower limit of [i, j] and the y shown in equation (89) ～ The lower limit of [i, j] and the y shown in equation (91) ～ Among the lower bounds of [i, j], the largest is y. ～ This is obtained as the lower limit of [i, j]. This can be expressed mathematically as shown in equation (111).

[0322]

[0323] On the other hand, y～ For [i, j], equations (95), (98), (102), and (104) hold.

[0324] In the output correction process, the y shown in equation (95) ～ The upper limit of [i, j] and the y shown in equation (98) ～ The upper limit of [i, j] and the y shown in equation (102) ～ The upper limit of [i, j] and the y shown in equation (104) ～ The smallest of the upper and lower limits of [i, j] is y. ～ This is obtained as the upper limit of [i, j]. This can be expressed mathematically as shown in equation (112).

[0325]

[0326] These lower and upper bounds are obtained for each element of the n × m output matrix with a computational cost of O(nm).

[0327] Next, the dot product value y in the output correction process. ～ Quantized value y of [i, j] ｌ － An example of a technique for calculating [i, j] will be explained using Figure 12.

[0328] Figure 12 is a fifth explanatory diagram illustrating the output correction process in the embodiment. As shown in Figure 12, according to equation (107), the dot product value y ～ The probability distribution of [i, j] is the inner product y ～ It follows a Gaussian distribution centered at [i, j] with a standard deviation of σ. Therefore, the inner product y ～ The probability distribution of [i, j] can be expressed mathematically as shown in equation (113).

[0329]

[0330] Inner product value y ～ [i, j] is the lower bound y ｌ － [i, j] to upper limit y ｕ － Since it is in the interval up to [i, j], the dot product y ～ The expected value E(y) of the probability distribution of [i, j] ～[i, j]) is expressed by equation (114). Note that the symbol on the left side of equation (114) is the expected value E(y ～ Represents [i, j])

[0331]

[0332] erf is the error function. The symbol in equation (115) below is y. ～ [i, j] represents the symbol y in equation (116) below. ～ ' represents [i, j], and the symbol in the following equation (117) is y ｌ － [i, j] represents [i, j], and the symbol in the following equation (118) is y ｕ － Represents [i, j]

[0333]

[0334]

[0335]

[0336]

[0337] In the output correction process, the dot product value y ～ The expected value E(y) of the probability distribution of [i, j] ～ [i, j]) lower limit y ｌ － [i, j] to upper limit y ｕ － The point y that halves the area when integrated over the interval [i, j] － [i, j] dot product value y ～ Let this be the quantized value of [i, j]. Expected value E(y ～ If S(t) is the area obtained by integrating [i, j] over the interval from 0 to t, then S(t) can be expressed by the following equation (119).

[0338]

[0339] The definitions of functions f and g are given in equation (120).

[0340]

[0341] Therefore, the inner product y ～ The expected value E(y) of the probability distribution of [i, j] ～[i, j]) lower limit y ｌ － [i, j] to upper limit y ｕ － The area obtained by integrating over the interval [i, j] is S(y ｕ － [i,j])-S(y ｌ － [i, j]) is the point that halves this area y － The area under [i, j] is S(y － [i,j])-S(y ｌ － Since [i, j], the following equation (121) holds. Note that “point y － The area defined by [i, j] is the dot product value y ～ Lower bound y of the probability distribution of [i, j] ｌ － [i, j] to y － This is the value obtained by integrating up to [i, j].

[0342]

[0343] Therefore, the following equation (122) holds true.

[0344]

[0345] Note that the symbol in the following equation (123) is y － Represents [i, j]

[0346]

[0347] As a result, equation (124) is obtained from equation (119).

[0348]

[0349] Here, with respect to the function f(u,t) in equation (120), the following equation (125) holds through Taylor expansion.

[0350]

[0351] Therefore, the following equation (126) holds true.

[0352]

[0353] Substituting equation (126) into equation (124), we get the quantized value y. － An approximate formula for [i, j] is obtained. Specifically, the following formula (127) is obtained.

[0354]

[0355] The function f in this equation is obtained by equation (120) with a computational cost of O(1). Therefore, for each element of the n × m output matrix, its quantized value is obtained with a computational cost of O(nm).

[0356] The output correction process uses equations (111) and (112) to determine the lower limit y of the inner product value, based on the quantized value obtained by the range adjustment process and the quantization matrix obtained from the external computer 9. ｌ ～ [i, j] and the upper limit y ｕ ～ We obtain [i, j]. Then, the lower limit y ｌ ～ [i, j] and the upper limit y ｕ ～ Based on [i, j], use equations (120) and (127) to find y in equation (107). ～ Quantized value y corresponding to [i, j] － We obtain [i, j]. Note that a in equation (107) ｉ ～ (x ｊ ～ ) Ｔ The corresponding quantization value y － [i, j] are quantized values ​​y that satisfy the relationship in equation (110). － It is [i, j].

[0357] y in equation (107) ～ Quantized value y corresponding to [i, j] － As a result of obtaining [i, j], the calculation result in analog computer 1 is obtained in the approximate form represented by equation (128).

[0358]

[0359] Subsequently, the obtained a ｉ ～ (x ｊ ～ ) ＴFrom equation (64), the matrix product a in the normalized matrix ｉ － (x ｊ － ) Ｔ This can be obtained. ｉ ～ (x ｊ ～ ) Ｔ From equation (64), matrix product a ｉ － (x ｊ － ) Ｔ Since a is obtained, ｉ ～ (x ｊ ～ ) Ｔ Matrix product a ｉ － (x ｊ － ) Ｔ In the process of obtaining this, quantization error is used. Finally, a ｉ － (x ｊ － ) Ｔ Using this, each element y[i,j] of the output matrix Y is obtained according to the following equation (20).

[0360] <Regarding computational costs> The computational costs are summarized below. Matrix normalization is the process of normalizing a given n x l input matrix A and an m x l weight matrix X with a computational cost of O((n+m)l). Quantization value acquisition is the process of obtaining quantized values ​​from the normalized matrix with a computational cost of O((n+m)l).

[0361] The first converter 20 performs a DA transformation of the input matrix A in O((n+m)l) computation time. At the same time, the first converter 20 calculates the reference vector, the angle of each vector, and the norm of each vector, which are necessary for correcting the error in the dot product, in O((n+m)l) computation time.

[0362] The range adjustment process calculates the range of values ​​that the analog signal can take for an n × m output matrix Y with a computation cost of O(nm). The third converter 40 performs AD conversion of the n × m output matrix Y with a computation cost of O(nm).

[0363] The output correction process corrects the discrete values ​​of the dot product obtained from the AD conversion with a computational cost of O(nm). As a result, the computational cost to obtain the output matrix Y from the input matrix A and the weight matrix X is O((n+m)l+nm). This is the same computational cost required to access the n×l input matrix A, the m×l weight matrix X, and the n×m output matrix Y. Furthermore, this computational cost is smaller than the computational cost O(nml) required to calculate the matrix product.

[0364] <Example of Hardware Configuration of Input Signal DAC2> Figure 13 shows an example of the hardware configuration of the input signal DAC2 in the embodiment. The input signal DAC2 includes a first control unit 21 and executes a program. The input signal DAC2 functions as a device comprising a first converter 20, a first control unit 21, an interface unit 22, and a first storage unit 23 through the execution of the program.

[0365] More specifically, the processor 91 reads the program stored in the first storage unit 23 and stores the read program in the memory 92. By executing the program stored in the memory 92, the input signal DAC 2 functions as a device comprising the first converter 20, the first control unit 21, the interface unit 22, and the first storage unit 23.

[0366] The first control unit 21 controls the operation of each functional unit of the input signal DAC 2. The first control unit 21 performs, for example, the matrix normalization process, parameter transmission process, quantization value acquisition process, quantization matrix acquisition process, and quantization error transmission process described above. The first control unit 21 controls, for example, the operation of the interface unit 22. The first control unit 21 acquires, for example, the information stored in the first storage unit 23. Specifically, the process of acquiring the information stored in the first storage unit 23 is a read operation.

[0367] The interface unit 22 is configured to include a communication interface for connecting the input signal DAC 2 to an external device. The interface unit 22 communicates with the external device via wired or wireless connection. The external device is, for example, an analog computer 1. The external device is, for example, an external computer 9.

[0368] The first converter 20 is connected to the analog computer 1 by a wire, and the analog signal obtained by the first converter 20 is output to the analog computer 1 via this wire, rather than through the interface unit 22.

[0369] The interface unit 22 may include input devices such as a mouse, keyboard, touch panel, and microphone. The interface unit 22 may also be configured as an interface connecting these input devices to the input signal DAC 2. In this way, the input devices of the interface unit 22 receive various information or signals to the input signal DAC 2 via wired or wireless connections. Note that the information or signals do not necessarily have to be input to the communication interface of the interface unit 22, but may also be input to the input devices of the interface unit 22.

[0370] The interface unit 22 outputs various types of information, for example. The interface unit 22 includes, for example, a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display, as well as a speaker. The interface unit 22 may be configured as an interface for connecting these display devices or speakers to the input signal DAC 2. Therefore, the interface unit 22 may output information indicated by information or signals input to the input device of the interface unit 22 as an image or sound.

[0371] The first storage unit 23 is configured using a computer-readable recording medium such as a magnetic hard disk drive or a semiconductor memory device. The first storage unit 23 stores various information related to the input signal DAC 2. The first storage unit 23 stores various information generated by the operation of the first control unit 21, for example. The first storage unit 23 may reside, for example, on the cloud.

[0372] <Example of the processing flow executed by the input signal DAC2> Figure 14 is a flowchart showing an example of the processing flow executed by the input signal DAC2 in the embodiment. The first control unit 21 acquires the input matrix A (step S101). Next, the first control unit 21 performs matrix normalization processing on the input matrix A (step S102). Next, the first control unit 21 performs parameter transmission processing (step S103). Next, the first control unit 21 performs quantization value acquisition processing (step S104).

[0373] Next, the first control unit 21 performs a quantization matrix acquisition process on the normalized input matrix A obtained by the matrix normalization process in step S102 (step S105). Next, the first control unit 21 performs a quantization error transmission process (step S106). Next, the first converter 20 performs a DA conversion on the quantization matrix obtained in the quantization matrix acquisition process in step S105 (step S107). The quantization matrix after the DA conversion in step S107 is output to the analog computer 1.

[0374] Thus, the input signal DAC2 performs matrix normalization on the input matrix A, and then performs quantization matrix acquisition on the input matrix A that has been normalized by the matrix normalization process.

[0375] Step S103 may be executed at any time after step S102 and before the output correction process is performed. Similarly, step S106 may be executed at any time after step S105 and before the output correction process is performed.

[0376] <Example of Hardware Configuration of Weight Signal DAC3> Figure 15 shows an example of the hardware configuration of the weight signal DAC3 in the embodiment. The weight signal DAC3 includes a second control unit 31 and executes a program. The weight signal DAC3 functions as a device comprising a second converter 30, a second control unit 31, an interface unit 32, and a second storage unit 33 through the execution of the program.

[0377] More specifically, the processor 93 reads the program stored in the second storage unit 33 and stores the read program in the memory 94. By executing the program stored in the memory 94, the weight signal DAC 3 functions as a device comprising the second converter 30, the second control unit 31, the interface unit 32, and the second storage unit 33.

[0378] The second control unit 31 controls the operation of each functional unit of the weight signal DAC 3. The second control unit 31 performs, for example, the matrix normalization process, parameter transmission process, quantization value acquisition process, quantization matrix acquisition process, and quantization error transmission process described above. The second control unit 31 controls, for example, the operation of the interface unit 32. The second control unit 31 acquires, for example, the information stored in the second storage unit 33. Specifically, the process of acquiring the information stored in the second storage unit 33 is read.

[0379] The interface unit 32 is configured to include a communication interface for connecting the weight signal DAC 3 to an external device. The interface unit 32 communicates with the external device via wired or wireless connection. The external device is, for example, an analog computer 1. The external device is, for example, an external computer 9.

[0380] The second converter 30 is connected to the analog computer 1 by a wire, and the analog signal obtained by the second converter 30 is output to the analog computer 1 via this wire, rather than through the interface unit 32.

[0381] The interface unit 32 may include input devices such as a mouse, keyboard, touch panel, or microphone. The interface unit 32 may also be configured as an interface connecting these input devices to the weight signal DAC 3. In this way, the input devices of the interface unit 32 receive various information or signals to the weight signal DAC 3 via wired or wireless connections. Note that the information or signals do not necessarily have to be input to the communication interface of the interface unit 32, but may also be input to the input devices of the interface unit 32.

[0382] The interface unit 32 outputs various types of information, for example. The interface unit 32 is comprised of a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display, as well as a speaker. The interface unit 32 may be configured as an interface for connecting these display devices or speakers to the weight signal DAC 3. Therefore, the interface unit 32 may output information indicated by information or signals input to the input device of the interface unit 32 as an image or sound.

[0383] The second storage unit 33 is configured using a computer-readable recording medium such as a magnetic hard disk drive or a semiconductor memory device. The second storage unit 33 stores various information related to the weight signal DAC 3. The second storage unit 33 stores various information generated by, for example, the operation of the second control unit 31. The second storage unit 33 may reside, for example, on the cloud.

[0384] <Example of the processing flow executed by the DAC3 for weight signals> Figure 16 is a flowchart showing an example of the processing flow executed by the DAC3 for weight signals in the embodiment. The second control unit 31 acquires the weight matrix X (step S201). Next, the second control unit 31 performs matrix normalization processing on the weight matrix X (step S202). Next, the second control unit 31 performs parameter transmission processing (step S203). Next, the second control unit 31 performs quantization value acquisition processing (step S204).

[0385] Next, the second control unit 31 performs a quantization matrix acquisition process on the normalized weight matrix X obtained by the matrix normalization process in step S202 (step S205). Next, the second control unit 31 performs a quantization error transmission process (step S206). Next, the second converter 30 performs a DA conversion on the quantization matrix obtained in the quantization matrix acquisition process in step S205 (step S207). The quantization matrix after the DA conversion in step S207 is output to the analog computer 1.

[0386] Thus, the DAC3 for weight signals performs matrix normalization on the input matrix X, and then performs quantization matrix acquisition on the input matrix X that has been normalized by the matrix normalization process.

[0387] Step S203 may be executed at any time after step S202 but before the output correction process is performed. Similarly, step S206 may be executed at any time after step S205 but before the output correction process is performed.

[0388] <Example of Hardware Configuration of Output Signal ADC4> Figure 17 shows an example of the hardware configuration of the output signal ADC4 in the embodiment. The output signal ADC4 includes a third control unit 41 and executes a program. By executing the program, the output signal ADC4 functions as a device comprising a third converter 40, a third control unit 41, an interface unit 42, and a third storage unit 43.

[0389] More specifically, the processor 95 reads the program stored in the third storage unit 43 and stores the read program in the memory 96. By executing the program stored in the memory 96, the ADC 4 for output signals functions as a device comprising the third converter 40, the third control unit 41, the interface unit 42, and the third storage unit 43.

[0390] The third control unit 41 controls the operation of each functional unit of the output signal ADC 4. The third control unit 41 performs, for example, the range adjustment process and the output correction process described above. The third control unit 41 controls, for example, the operation of the interface unit 42. The third control unit 41 acquires, for example, the information stored in the third storage unit 43. Specifically, the process of acquiring the information stored in the third storage unit 43 is a read operation.

[0391] The interface unit 42 is configured to include a communication interface for connecting the output signal ADC 4 to an external device. The interface unit 42 communicates with the external device via wired or wireless means. The external device is, for example, an analog computer 1. The external device is, for example, an external computer 9. In such a case, the interface unit 42 obtains, for example, a quantization error from the external computer 9.

[0392] The third converter 40 is connected to the analog computer 1 by a wire, and the analog signal obtained by the third converter 40 is output to the analog computer 1 via this wire, rather than through the interface unit 42.

[0393] The interface unit 42 may include input devices such as a mouse, keyboard, touch panel, or microphone. The interface unit 42 may also be configured as an interface connecting these input devices to the output signal ADC 4. In this way, the input devices of the interface unit 42 receive various information or signals to the output signal ADC 4 via wired or wireless connections. Note that the information or signals do not necessarily have to be input to the communication interface of the interface unit 42, but may also be input to the input devices of the interface unit 42.

[0394] The interface unit 42 outputs various types of information, for example. The interface unit 42 is comprised of a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display, as well as a speaker. The interface unit 42 may be configured as an interface for connecting these display devices or speakers to the ADC 4 for output signals. Therefore, the interface unit 42 may output information indicated by information or signals input to the input device of the interface unit 42 as an image or sound.

[0395] The third storage unit 43 is configured using a computer-readable recording medium such as a magnetic hard disk drive or a semiconductor memory device. The third storage unit 43 stores various information related to the output signal ADC 4. The third storage unit 43 stores various information generated by the operation of the third control unit 41, for example. The third storage unit 43 may reside, for example, on the cloud.

[0396] <Example of the processing flow performed by the output signal ADC4> Figure 18 is a flowchart showing an example of the processing flow performed by the output signal ADC4 in the embodiment. The third control unit 41 obtains the quantization error transmitted in the quantum error transmission process of step S106 and the quantization error transmitted in the quantization error transmission process of step S206 from the external computer 9 via the interface unit 42 (step S301). Next, the third control unit 41 obtains the dot product value output by the analog computer 1 (step S302). Next, the third control unit 41 performs range adjustment processing (step S303). Next, the third converter 40 performs AD conversion on the dot product value obtained in step S301 (step S304).

[0397] Next, the third control unit 41 performs an output correction process on the result of the AD conversion in step S304 (step S305). In this output correction process, as described above, corrections are made based on the parameters used in the matrix normalization process by the input signal DAC2 and the parameters used in the matrix normalization process by the output signal DAC3. The output matrix Y is then obtained through the output correction process. The obtained output matrix Y is output to, for example, an external computer 9.

[0398] Step S301 may be executed at any timing as long as it is performed before the output correction process. Therefore, step S301 may be executed in parallel with any or more of steps S302 to S204. Step S301 may be executed after step S302, after step S303, or after step S304.

[0399] Thus, the ADC for the output signal applies a correction to the AD conversion result based on the parameters used in the matrix normalization process by the DAC2 for the input signal and the parameters used in the matrix normalization process by the DAC3 for the weight signal.

[0400] <Example of the processing flow of analog computer 1> Figure 19 is a flowchart showing an example of the processing flow executed by analog computer 1 in the embodiment. Analog computer 1 processes the quantization matrix (i.e., quantization matrix A) obtained in step S107. ～ ) and the quantization matrix obtained in step S207 (i.e., the quantization matrix X ～ ) obtain (step S401).

[0401] Next, analog computer 1 processes the quantization matrix A ～ and the quantization matrix X ～ The dot product value is obtained based on this (step S402). Next, the analog computer 1 outputs the dot product value obtained in step S402 to the third converter 40 of the output signal ADC 4 (step S403). The analog computer 1 and the third converter 40 are connected by a wire, and the output of the analog computer 1 propagates through this wire to the third converter 40. This propagation is step S301.

[0402] <An example of the processing flow executed by the information processing system 100> In the information processing system, for example, after the processing of steps S101 to S107 is executed in the order described in Figure 14, the processing of steps S201 to S207 is executed in the order described in Figure 16. Then, after the processing of steps S402 to S403 is executed by the analog computer 1 in the order shown in Figure 19, the processing of steps S301 to 304 is executed in the order shown in Figure 18.

[0403] Note that the processes in steps S101 to S107 do not necessarily have to be executed before the processes in steps S201 to S207. For example, the processes in steps S101 to S107 may be executed in parallel with some or all of the processes in steps S201 to S207.

[0404] The information processing system 100 configured in this way performs matrix normalization and quantization matrix acquisition, then performs DA conversion and causes the analog computer 1 to perform the calculation. After performing AD conversion on the results of the calculation by the analog computer 1, it performs correction based on the parameters used in the matrix normalization process.

[0405] Therefore, as explained in detail using mathematical formulas, the information processing system 100 can obtain matrix products with higher precision. Specifically, in order to obtain the matrix product by the analog computer 1, it is necessary to perform a digital-to-analog conversion of the matrix in the input signal DAC 2. In this conversion, the input signal DAC 2 quantizes each element of the matrix to 0 to 1 by shifting and scaling, as shown in equation (20).

[0406] In the analog computer 1, errors occur because the dot product calculation is performed on quantized numerical values. Therefore, the third control unit 41 corrects the dot product calculation using the error that occurs when the matrix is ​​quantized, as shown in equation (64). The output signal ADC 4 needs to set the range of the analog output of the analog computer 1 and perform analog-to-digital conversion. At this time, the third control unit 41 sets the range in the output signal ADC 4 from the theoretical upper and lower limits of the dot product value, as shown in equations (93) and (106). Also, the dot product value calculated by the analog computer 1 contains noise. Therefore, the third control unit 41 estimates (corrects) the dot product value using the measured value and the standard deviation of noise in the output signal ADC 4, as shown in equation (127).

[0407] Therefore, the information processing system 100 can obtain matrix products with higher precision. Consequently, the information processing system 100 configured in this way can improve the accuracy of matrix product calculations using analog computers. In other words, the information processing system 100 can improve the calculation accuracy when converting digital signals to analog, calculating matrix products, and then converting them back to digital.

[0408] (Variation) The input signal DAC2 may be implemented using multiple information processing devices connected to each other via a network. In this case, each process performed by the first control unit 21 may be performed in a distributed manner by the multiple information processing devices.

[0409] The DAC 3 for weight signals may be implemented using multiple information processing devices connected to each other via a network. In this case, each process performed by the second control unit 31 may be performed in a distributed manner by multiple information processing devices.

[0410] The ADC 4 for output signals may be implemented using multiple information processing devices connected to each other via a network. In this case, each process performed by the third control unit 41 may be performed in a distributed manner by multiple information processing devices.

[0411] The analog signal output by the input signal DAC2 to the analog computer 1 is an example of the first analog signal. That is, the quantization matrix A ～ The analog signal representing this is an example of the first analog signal.

[0412] The analog signal output by the weight signal DAC3 to the analog computer 1 is an example of a second analog signal. That is, the quantization matrix X ～ The analog signal representing this is an example of a second analog signal.

[0413] The process in step S102 is an example of the first matrix normalization step. The process in step S105 is an example of the first quantization step. The process in step S202 is an example of the second matrix normalization step. The process in step S205 is an example of the second quantization step. The process in step S304 is an example of the correction step.

[0414] Furthermore, all or part of the functions of the input signal DAC2, weight signal DAC3, and output signal ADC4 may be implemented using hardware such as ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, magnetic disks, magneto-optical disks, optical disks (CD-ROM, DVD-ROM, etc.), portable media such as semiconductor memory (volatile memory, non-volatile memory, etc.) (ROM, RAM, etc.), and storage devices such as hard disks built into computer systems. The program may also be transmitted via telecommunications lines.

[0415] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0416] <Analysis of Non-Patent Document 1> In the case of the technology described in Non-Patent Document 1, it is analyzed that the following four causes reduced the accuracy of the matrix multiplication calculation.

[0417] One of the reasons is the low bit count of the input signal DAC and weight signal DAC in the technology described in Non-Patent Document 1. External computers generally allow for 32-bit numerical representation using float type and 64-bit numerical representation using double type. However, because analog computers perform processing based on physical phenomena (i.e., processing using analog signals), due to device limitations, it is necessary to quantize the input numerical value to a low-bit value before converting it to an analog value.

[0418] Another cause is calculation errors within the analog computer. The result of matrix multiplication performed within the analog computer deviates significantly from the actual matrix multiplication value. This is because the matrix multiplication is calculated based on analog values ​​corresponding to low-bit numbers.

[0419] Another contributing factor is the low bit depth of the ADC used for output signals in the technology described in Non-Patent Document 1. The ADC for output signals measures and digitizes the analog output of an analog computer, but due to limitations of the measuring device, it can only quantize the analog output to a low-bit value.

[0420] The last cause is noise present in the output of analog computers. Because analog computers perform calculations based on physical phenomena, their output contains noise.

[0421] 100... Information processing system, 1... Analog computer, 2... Input signal DAC, 3... Weight signal DAC, 4... Output signal ADC, 20... First converter, 30... Second converter, 40... Third converter, 21... First control unit, 31... Second control unit, 41... Third control unit, 22... Interface unit, 32... Interface unit, 42... Interface unit, 23... First storage unit, 33... Second storage unit, 43... Third storage unit, 91... Processor, 92... Memory, 93... Processor, 94... Memory, 95... Processor, 96... Memory

Claims

1. An information processing method executed by an information processing system, comprising: an input signal DAC that outputs a first analog signal to an analog computer; a weight signal DAC that outputs a second analog signal to the analog computer; and an output signal ADC that performs AD conversion on the analog signal output by the analog computer, wherein the input signal DAC performs a matrix normalization process on an input matrix A to normalize the matrix to be processed, and then performs a quantization matrix acquisition process on the input matrix A normalized by the matrix normalization process to quantize the matrix to be processed; the weight signal DAC performs the matrix normalization process on a weight matrix X, and then performs the quantization matrix acquisition process on the weight matrix X normalized by the matrix normalization process; and the output signal ADC applies a correction to the result of the AD conversion based on the parameters used in the matrix normalization process by the input signal DAC and the parameters used in the matrix normalization process by the weight signal DAC, wherein the input signal DAC performs the matrix normalization process on the input matrix A, An information processing method comprising: a first quantization step in which the input signal DAC performs the quantization matrix acquisition process on the input matrix A normalized by the matrix normalization process; a second matrix normalization step in which the weight signal DAC performs the matrix normalization process on the weight matrix X; a second quantization step in which the weight signal DAC performs the quantization matrix acquisition process on the input matrix X normalized by the matrix normalization process; and a correction step in which the output signal ADC applies the correction.

2. The information processing method according to claim 1, wherein the quantization matrix acquisition process performs quantization, wherein the average of each cluster obtained as a result of performing clustering on each element of the normalized matrix obtained in the matrix normalization process is used as the quantization value.

3. An information processing system comprising: an input signal DAC that outputs a first analog signal to an analog computer; a weight signal DAC that outputs a second analog signal to the analog computer; and an output signal ADC that performs AD conversion on the analog signal output by the analog computer, wherein the input signal DAC performs a matrix normalization process on an input matrix A to normalize the matrix to be processed, and then performs a quantization matrix acquisition process on the input matrix A normalized by the matrix normalization process to quantize the matrix to be processed; the weight signal DAC performs the matrix normalization process on a weight matrix X, and then performs the quantization matrix acquisition process on the weight matrix X normalized by the matrix normalization process; and the output signal ADC applies a correction to the result of the AD conversion based on the parameters used in the matrix normalization process by the input signal DAC and the parameters used in the matrix normalization process by the weight signal DAC.

4. A program for causing a computer to function as the information processing system described in claim 3.

Images