Information processing method, information processing apparatus, and non-transitory computer-readable medium

Abstract

To reduce an expected square error in Monte Carlo Approximate Matrix Multiplication (AMM) without significantly increasing computation time.SOLUTION: Aspects of the present disclosure can involve systems and methods for accelerating a process to search for optimal solutions for interaction models, the process involving iterative matrix multiplication between a first matrix and a second matrix, which include: replacing the first matrix with a first approximate matrix and the second matrix with a second approximate matrix by adding a first set of constant values to the first matrix and a second set of constant values to the second matrix; executing a Monte Carlo Approximate Matrix Multiplication between the first approximate matrix and the second approximate matrix to generate an approximate matrix product; executing the process to search for the optimal solutions based on the approximate matrix product according to the interaction models; and providing the optimal solutions found from the process.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates generally to parallel computing, and more specifically to graphics processing units (GPUs) and operations for matrix multiplication. [Background technology]

[0002] Complementary metal-oxide-semiconductor (CMOS) annealing is a technique that implements the principles of Ising computing in digital circuits. Ising computing is a method focused on finding the ground state of the Ising model using non-planar graphs, an NP-hard problem. This computational principle can be applied to efficiently solve other combinatorial optimization problems, and is suitable for improving productivity, resource conservation, and efficient system operation in thermal power plants and urban infrastructure.

[0003] In this specification, CMOS annealing machine and momentum annealing (MA) are described. Although both are CMOS annealing techniques, they have different characteristics and are suitable for different problems. CMOS annealing machine is an annealing machine that uses semiconductor circuits and is excellent in terms of power performance and scalability, but due to implementation reasons, there are restrictions on the optimization problems that can be handled. On the other hand, MA is an algorithm that is suitable for computers with many cores such as GPUs, and can deal with the ground state search problem of the Ising model using arbitrary coupling, so it is applicable to various optimization problems. The differences in the characteristics of each technique provided are as follows.

[0004] The CMOS annealing machine is a computing device that efficiently and energy-efficiently finds a solution to the ground state search problem of the non-planar Ising model with an external magnetic field, which is known to be NP-hard. The Hamiltonian of the Ising model is expressed by the following equation.

[0005]

number

[0006] The outline of the CMOS annealing machine can be explained as follows. The spin state is simulated using a memory cell of a spin array, and the spin state (-1 / +1) corresponds to the two states (0 / 1) of the memory cell. Furthermore, the interaction coefficient J ij and the external magnetic field factor h i is stored in the memory cells of the spin array. This structure represents the Ising model using a non-planar graph. ij , and the external magnetic field factor h i All the components of the Ising model, such as the above, are realized in memory cells, and their values ​​are read and written through an SRAM-compatible interface (address bus, data bus, read / write control section, and input / output clock). When searching for the ground state of the Ising model, the host computer controls the SRAM-compatible interface to write the initial values ​​of the spin, external magnetic field coefficient, and interaction coefficient. The solution is obtained by reading the value of the spin after completing the ground state search.

[0007] FIG. 1 shows an example of the computation flow of a CMOS annealing machine. First, a desired combinatorial optimization problem is formulated as a ground-state search problem of an Ising model (Ising format). For example, the traveling salesman problem (TSP), a typical combinatorial optimization problem, can be converted into a ground-state search problem of an Ising model. Next, the coefficients of the Ising model obtained from the above process are written into the CMOS annealing machine (graph embedding), and a ground-state search is performed (solution search). The obtained ground state of the Ising model corresponds to the solution of the original problem such as the TSP. Therefore, the solution of the desired combinatorial optimization problem can be obtained by decoding the received search result.

[0008] Figures 2A and 2B show the conversion from the Ising model to a bipartite graph. The MA algorithm is a method for quickly solving the ground state search problem of the Ising model by utilizing massively parallel computing. In this algorithm, to find the ground state of the fully connected Ising model (Figure 2A), the ground state of the Ising model is searched on the bipartite graph (Figure 2B).

[0009] Specifically, Fig. 2A shows a fully connected Ising model with N=7, and Fig. 2B shows an Ising model on a complete bipartite graph G. When the value of the thick line coupling is large enough, the spin configuration on the left side matches the spin configuration on the right side in the ground state.

[0010] Spin σ i L and spin σ i R The interaction between i (shown by the bold line in Figure 2B) is determined to satisfy the following equation, it is guaranteed that the ground state of the Ising model on the complete bipartite graph coincides with the ground state of the fully connected Ising model, which is the original problem.

[0011]

number

[0012] where λ is the largest eigenvalue of -J and C is any subset of the vertex set V. Equation (2.2) allows to simultaneously update all spins of the fully connected Ising model. The computational flow is outlined in Figure 3, and the Ising problem represented as a mathematical model of optimization is shown with respect to Figure 4.

[0013] In the field of annealing, annealing machines that utilize digital circuits and D-waves have been commercially available, and quantum annealing has been realized using analog devices that utilize quantum fluctuations. Summary of the Invention [Problem to be solved by the invention]

[0014] Figure 5 shows an example process of performing CMOS annealing by Ising form and annealing technique. First, the combinatorial optimization problem is converted into an Ising problem, and then its solution can be obtained by CMOS annealing. CMOS annealing repeatedly performs strict matrix multiplication. Acceleration of matrix multiplication can be achieved by running on GPU, but it has reached its limit. To reduce the overall calculation time of CMOS annealing, a faster matrix multiplication method is required. As shown in the performance comparison with the conventional method in Figure 6, the momentum annealing technique can operate up to 250 times faster than the conventional computation technique.

[0015] The exemplary implementation described herein aims to reduce the expected squared error in Monte Carlo Approximation Matrix Multiplication (AMM) without significantly increasing the computation time. This is achieved by leveraging additional information used in traditional AMM. The essential steps of the method are outlined below.

[0016] The mean and perturbation vectors are used to define matrices X and Y by adding or subtracting constant values ​​from column A and row B, respectively.

[0017]

number

[0018] We exploit the fact that certain terms of the matrix product AB can be computed using matrix-vector products and dot products instead of matrix-matrix products, thereby reducing the number of operations required for those terms and resulting in more efficient computations.

[0019] Instead of using A and B directly, we find the AMM for matrices X and Y. This approximation depends only on the error in XY, which is shown to be less than using the AMM for AB directly.

[0020] We analyze the expected squared error of the Monte Carlo AMM for computing XY. The error is denoted by E and is proportional to the difference between the sum of the squares of the absolute values ​​and the squared norm of XY.

[0021] By proposing several theorems, the exemplary implementation reduces the expected squared error without significantly increasing computation time, resulting in more accurate approximations of matrix products, which are important in a variety of applications involving eigenvalue operations, combinatorial optimization algorithms, and other information processing tasks.

[0022] An exemplary implementation introduces a computationally-low cost pre-processing technique that is incorporated into the existing matrix multiplication algorithm as an approximation method. By employing this technique, the optimization time of CMOS annealing can be accelerated.

[0023] Many algorithms for approximate matrix multiplication have been proposed, but are specifically designed for such applications (e.g., specifically designed to run efficiently on a central processing unit (CPU)). However, since CMOS annealing leverages the extensive parallel computing capabilities of GPUs, it is essential to have an approximation algorithm that runs fast on a GPU platform. An exemplary implementation of such an algorithm is presented. [Means for solving the problem]

[0024] Aspects of the present disclosure may include a method of accelerating a process involving iterative matrix multiplication between a first matrix and a second matrix to search for an optimal solution to an interaction model, the method including: replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix; performing Monte Carlo approximation matrix multiplication between the first and second approximation matrices to generate an approximation matrix product; running a process to search for an optimal solution based on the approximation matrix product according to the interaction model; and providing the optimal solution found from the process.

[0025] Aspects of the present disclosure may include a system for accelerating a process involving iterative matrix multiplication between a first matrix and a second matrix to search for an optimal solution to an interaction model, the system including: means for replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix; means for performing Monte Carlo approximation matrix multiplication between the first and second approximation matrices to generate an approximation matrix product; means for executing a process to search for an optimal solution based on the approximation matrix product according to the interaction model; and means for providing the optimal solution found from the process.

[0026] Aspects of the disclosure may include a non-transitory computer-readable medium storing instructions for accelerating a process involving iterative matrix multiplication between a first matrix and a second matrix for searching for an optimal solution to an interaction model, the instructions including: replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix; performing a Monte Carlo approximation matrix multiplication between the first and second approximation matrices to generate an approximation matrix product; executing a process for searching for an optimal solution based on the approximation matrix product according to the interaction model; and providing the optimal solution found from the process.

[0027] An aspect of the present disclosure may include an information processing apparatus configured to accelerate a process involving iterative matrix multiplication between a first matrix and a second matrix for searching for an optimal solution to an interaction model, the information processing apparatus including a processor configured to control one or more computing devices to perform the following operations using parallel processing: replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix; performing Monte Carlo approximation matrix multiplication between the first and second approximation matrices to generate an approximation matrix product; executing a process for searching for an optimal solution based on the approximation matrix product according to the interaction model; and providing the optimal solution found from the process. Effect of the Invention

[0028] The expected square error in Monte Carlo Approximation Matrix Multiplication (AMM) can be reduced without significantly increasing the calculation time. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiment of the invention. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 is a diagram illustrating an example of a computation flow of a CMOS annealing machine. [Figure 2A] FIG. 1 illustrates a conversion from an Ising model to a bipartite graph. [Figure 2B] FIG. 1 illustrates a conversion from an Ising model to a bipartite graph. [Diagram 3] FIG. 13 is a diagram illustrating an example of a calculation flow of momentum annealing. [Figure 4] FIG. 1 illustrates an example Ising problem as a mathematical model for optimization. [Diagram 5] FIG. 1 is a diagram showing an example of an optimization flow using CMOS annealing. [Figure 6] FIG. 1 illustrates a comparative performance example of momentum annealing versus conventional techniques. [Figure 7] FIG. 1 illustrates an example pseudo-code (Algorithm 1) for Monte Carlo AMM according to one exemplary implementation. [Figure 8] FIG. 1 shows an example of pseudocode (Algorithm 2) for Monte Carlo AMM including the proposed preprocessing according to one illustrative implementation. [Figure 9] FIG. 1 illustrates an example of an overall process according to one exemplary implementation. [Figure 10] FIG. 2 illustrates an example of an optimal solution search process according to one illustrative implementation. [Figure 11A] FIG. 1 illustrates an example of the improvement in computation time using an exemplary implementation compared to related art. [Figure 11B] FIG. 1 illustrates an example of the improvement in computation time using an exemplary implementation compared to related art. [Figure 12] 1 is a block diagram showing a schematic configuration of an information processing device capable of implementing an exemplary implementation example. [Figure 13] FIG. 1 is a block diagram illustrating a computing system in which an example implementation may be implemented. [Figure 14] FIG. 1 is a detailed block diagram illustrating a computing system in which example implementations can be applied. [Figure 15] FIG. 1 is a block diagram showing units constituting a computing system in which an exemplary implementation can be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] The following detailed description provides details of the drawings and exemplary implementations of the present application. Reference numbers and descriptions of elements that overlap between drawings are omitted for clarity. The terms used throughout the description are provided by way of example and are not intended to be limiting. For example, the use of the term "automatic" may include a fully automatic implementation or a semi-automatic implementation with user or administrator control over certain aspects of the implementation, depending on the desired implementation in the art of practicing the implementation of the present application. The selection may be performed by a user through a user interface or other input means, or may be realized through a desired algorithm. The exemplary implementations as described herein may be utilized alone or in combination, and the functionality of the exemplary implementations may be realized through any means according to the desired implementation. EXAMPLES

[0031] Monte Carlo AMM Theory An example implementation described herein involves an apparatus that performs approximate matrix calculations as shown in the following pseudocode: m×k and B∈R k×n We describe the theory of the Monte Carlo method for computing the AMM of M. For any real matrix M, M (l) and M. (l) Let denote the l-th column and l-th row of the matrix M, respectively. Let |M| denote the Frobenius norm of M.

[0032] Figure 7 shows an example pseudocode (Algorithm 1) for Monte Carlo AMM according to one illustrative implementation. This is a randomized algorithm designed to return an approximate solution Z close to an exact solution AB with high probability. It has been demonstrated that the expected squared Frobenius error can be evaluated between AB and Z as follows:

[0033]

number

[0034] The error can be minimized if:

[0035]

number

[0036] In that case, the error is:

[0037]

number

[0038] AB=Σ l=1 k A (l) B (l) The statement in (3.1) and equation (3.3) suggest that Algorithm 1, which uses optimal Monte Carlo sampling, is designed to assign higher probabilities to columns / rows with larger Frobenius norms during sampling and compute their product as an approximate solution.

[0039] Extended Monte Carlo AMM A simple way to reduce the expected squared error is to increase the number of sampling iterations, c. The error decreases inversely proportional to the number of iterations. However, increasing c increases the computation time required for Monte Carlo AMM. Therefore, as described herein, example implementations can involve a method to reduce the expected squared error without significantly increasing computation time by further leveraging information used in conventional AMMs.

[0040]

number

[0041] In the formula, α=[α1,…,α k ] T ,β=[β1,…,β k ] T ,x=[x1,…,x k ] T,y=[y1,…,y k ] T and e l is an l-dimensional vector with all elements equal to 1. They are a function of the constant value α i +x i From the i-th column of A, β i +y i This means that X and Y are obtained by subtracting from the i-th row of B.

[0042]

number

[0043] The second, third, and fourth terms in equation (3.5) can be calculated using matrix-vector and dot products rather than matrix-matrix products. By exploiting this fact, these terms can be computed exactly given A, B, x, and y with a reduced number of operations. Thus, the exemplary implementation can be used to compute the AMM for matrices X and Y. The expected error of this approximation depends only on the error in XY. For simplicity of notation, A' = Ae m α T and B' = B - βe n T The matrices A' and B' are constant, and the important fact is that the column sums of A' and the row sums of B' are all zero.

[0044] Taking into account the considerations of equation (3.3), the expected squared error of the Monte Carlo AMM when computing XY is given by

[0045]

number

[0046] Here, the following theorem is provided: A function E is globally optimal at x=y=0.

[0047] This theorem allows maximizing the approximation performance of the exemplary implementation. The proposed method is incorporated into the Monte Carlo AMM to obtain Algorithm 2 for computing AB. Figure 8 shows an example of pseudocode (Algorithm 2) for the Monte Carlo AMM including the proposed preprocessing according to one exemplary implementation.

[0048] Algorithms 1 and 2 are designed to efficiently compute approximate solutions to the matrix multiplication AM while minimizing computational overhead. Furthermore, these algorithms have unique properties that enhance their compatibility with GPU implementations and facilitate efficient execution without compromising memory transfer efficiency or computational performance.

[0049] An example implementation focuses on replacing the matrix multiplication steps performed on the GPU with approximate matrix products.

[0050] Exemplary implementations involve target devices capable of massively parallel computing, such as GPUs and FPGAs. Exemplary implementations may include:

[0051] (1) In applications that perform repeated matrix multiplications, replace exact matrix products with approximate matrix products.

[0052] (2) In applications that perform repeated matrix multiplications, replace the exact matrix multiplications with Monte Carlo approximate matrix multiplication (AMM) as described in the embodiments.

[0053] (3) When searching for optimal solutions for interaction models including the Ising model, exact matrix products are replaced with approximate matrix products.

[0054] (4) When searching for an optimal solution for an interaction model including an Ising model, the strict matrix multiplication is replaced with the Monte Carlo AMM described in the embodiment.

[0055] The overall flow of the optimization calculation incorporating the proposed method is shown in Figs. 9 and 10.

[0056] 9 shows an example of the overall process according to one exemplary implementation. At S911, problem data is converted to quadratic programming problem data. At S912, model coefficients are set for each term in memory. At S913, weights for each term are set in memory. At S914, variable values ​​in variable memory are initialized. At S915, temperature parameters are initialized. At S916, an optimal solution search process is then performed, and at S917, output variable values ​​are read and interpreted.

[0057] 10 shows an example of an optimal solution search process S916 according to one example implementation. At S916a, the flow generates data for a mixed binary quadratic programming problem. At S916b, an optimal solution search using AMM is performed using Algorithm 1 or 2 as described above. At S916c, the solution can be rounded to a feasible solution. At S916d, a determination is made whether the iterations are complete, and if not (No), the flow proceeds to S916a.

[0058] The proposed method, which improves Monte Carlo Approximate Matrix Multiplication (AMM) on the GPU, offers several notable benefits compared to related fields.

[0059] Improved Matrix Multiplication Approximation: The proposed exemplary implementation improves the approximation of matrix products without increasing computation time compared to conventional AMMs. This means that users can achieve more accurate results without sacrificing efficiency. By reducing the error in the matrix product approximation, scientific and practical calculations that rely on matrix multiplication, such as eigenvalue and eigenvector operations, can be performed with greater precision.

[0060] GPU compatibility: The proposed exemplary implementation is specifically designed to work well with parallel operations on GPUs. GPUs have enormous computing power and are widely used in high performance computing. By leveraging the potential of GPUs for fast and accurate matrix multiplication calculations, the proposed exemplary implementation can significantly speed up scientific and practical calculations, resulting in faster processing times and improved overall performance.

[0061] Versatility and integration: The proposed exemplary implementation can be easily incorporated into various algorithms that rely on matrix multiplication. This flexibility allows researchers and developers to improve the efficiency and accuracy of existing algorithms without significant modifications. By seamlessly integrating the method into various applications, users can benefit from improved matrix multiplication operations across a wide range of information processing tasks.

[0062] Optimal hyperparameter solutions: The example implementations also provide analytical solutions for the optimal values ​​of the hyperparameters. This means that users can fine-tune the method to achieve the best possible results for their particular use case. Analytical solutions to hyperparameter optimization simplify the implementation process and allow users to achieve optimal performance without extensive trial and error.

[0063] Overall, customer value derived from the proposed exemplary implementations includes faster and more accurate matrix multiplication operations, improved precision in scientific and practical calculations, compatibility with GPUs for improved performance, easy integration into existing algorithms, and analytical solutions for optimizing hyperparameters. These benefits enable researchers and practitioners to accelerate their operations, achieve better results, and advance work in areas such as data analysis, machine learning, scientific simulation, and optimization algorithms.

[0064] 11A and 11B are diagrams illustrating an example of the improvement of computation time by using the exemplary implementation compared to the related art. Specifically, FIG. 11A shows an example of the variation of convergence over time when running a power method CUDA program on a GPU. Both FIG. 11A and FIG. 11B share a common horizontal axis representing computation time. The vertical axis of FIG. 11A represents the relative error between the maximum eigenvalue during an iteration and the estimated eigenvalue. In contrast, the vertical axis of FIG. 11B represents the deviation between the eigenvector corresponding to the maximum eigenvalue during an iteration and the estimated eigenvector. As shown in the exemplary results of FIG. 11A and FIG. 11B, the method using Monte Carlo AMM demonstrates favorable convergence in both indicators.

[0065] FIG. 12 is a block diagram showing a schematic configuration of an information processing device capable of implementing an exemplary implementation example. Specifically, FIG. 12 is an example of an information processing device that searches for an optimal solution to a mixed binary quadratic programming problem. As shown in the drawing, the information processing device 10 includes a processor 11, a primary storage device 12, an auxiliary storage device 13, an input device 14, an output device 15, a communication device 16, one or more computing devices 20, and a system bus 5 that communicatively connects the devices. For example, the information processing device 10 may be realized by using virtual information processing resources, such as a cloud server provided in part or in whole by a cloud system. Furthermore, the information processing device 10 may be realized by, for example, a plurality of information processing devices that cooperate with each other to operate and are communicatively connected to each other.

[0066] The processor 11 is configured by using, for example, a central processing unit (CPU) or a microprocessor unit (MPU). The primary storage device 12 is a device for storing programs or data, such as a read-only memory (ROM), a static random access memory (SRAM), a non-volatile RAM (NVRAM), a masked read-only memory (mask ROM), a programmable ROM (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), etc. The secondary storage device 13 is a hard disk drive, a flash memory, a solid-state drive (SSD), and an optical storage device (such as a compact disk (CD), a digital versatile disk (DVD)). The programs and data stored in the secondary storage device 13 are read into the primary storage device 12 at any time.

[0067] The input device 14 is a user interface that receives information input from a user, such as a keyboard, a mouse, a card reader, or a touch panel. The output device 15 is a user interface that provides information to a user, such as a display device (liquid crystal display (LCD) and a graphic card) that visualizes various types of information, an audio output device (speaker), and a printing device. The communication device 16 is a communication interface that communicates with other devices, such as a network interface card (NIC), a wireless communication module, a universal serial interface (USB) module, and a serial communication module.

[0068] The computing device 20 is a device that executes a process related to an optimal solution search for a mixed binary quadratic programming problem. The computing device 20 may take the form of an expansion card mounted on the information processing device 10, such as a graphics processing unit (GPU). The computing device 20 is composed of hardware such as a complementary metal oxide semiconductor (CMOS) circuit, a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC). The computing device 20 includes a control device, a storage device, and an interface that connects to a system bus 5, and transmits and receives commands and information to and from the processor 11 via the system bus 5. The computing device 20 may be communicatively connected to another computing device 20 via a communication line, for example, and may operate in cooperation with the other computing device 20. The functions realized by the computing device 20 may be realized, for example, by having a processor (such as a CPU and a GPU) execute a program.

[0069] The computing device 20 shown in Figure 12 is described below in Figure 13. One or more of the computing devices 20 may be installed.

[0070] In an exemplary implementation, an information processing device as shown in FIG. 12 can be configured to accelerate a process of searching for an optimal solution to an interaction model, the process including repeated matrix multiplication between a first matrix and a second matrix. The processor 11 can be configured to control one or more computing devices 20 to perform the following using parallel processing: replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix, as shown with respect to FIG. 7 and FIG. 8; performing Monte Carlo approximation matrix multiplication between the first approximation matrix and the second approximation matrix to generate an approximation matrix product; performing a process of searching for an optimal solution based on the approximation matrix product according to the interaction model; and providing the optimal solution found from the process. The one or more computing devices 20 can include a graphics processing unit (GPU) or a field programmable gate array (FPGA).

[0071] As described with respect to Figures 4 and 5, the process of searching for an optimal solution can be a complementary metal oxide semiconductor (CMOS) annealing process and the interaction model can be an Ising model, but example implementations are not limited thereto. For example, the process can also be a neural network operation or an eigenvalue operation, depending on the desired implementation.

[0072] As illustrated with respect to FIG. 8, processor 11 may be configured to control one or more computing devices to perform Monte Carlo approximation matrix multiplication using parallel processing by determining a first vector based on a first set of constant values, determining a second vector based on a second set of constant values, and generating an approximation matrix product by Monte Carlo approximation matrix multiplication from a sum of a result of the matrix product of the Monte Carlo approximation matrix multiplication, a vector product of the second vector and the first approximation matrix, a vector product of the second approximation matrix and the first vector, and an inner product involving the first set of constant values ​​and the second set of constant values.

[0073] As shown with respect to the equations in Figures 7 and 8, the first set of constant values ​​and the second set of constant values ​​can be selected such that a matrix multiplication result of the first matrix and the second matrix is ​​equivalent to a sum of a matrix multiplication result of the first approximation matrix and the second approximation matrix, one or more vector multiplication operations involving the first approximation matrix and the second approximation matrix, and one or more dot product operations involving the first set of constant values ​​and the second set of constant values.

[0074] 13 is a diagram showing the operation principle of the computing device 20, and is a block diagram of a system (hereinafter referred to as a computing system 500) constituting the computing device 20. The computing system 500 calculates a variable array x1, ..., x N or an array of variables y1,…,y N Hereinafter, the operation principle of the computing device 20 will be described with reference to FIG.

[0075] As shown in the figure, the computation system 500 includes a variable memory 511 , a non-linear coefficient memory 512 , a linear coefficient memory 513 , a difference calculation block 514 , a sampling block 515 , and a next state determination block 516 .

[0076] The variables x1,…,x N and y1,…,y N The information indicating is stored in the variable memory 511 of each computing system 500.

[0077] Information indicative of the matrix J is stored in the nonlinear coefficient memory 512. The matrix J is generally a symmetric matrix, and by utilizing this symmetry, it is possible to reduce the amount of usage of the nonlinear coefficient memory 512. Information indicative of the vector h is stored in the linear coefficient memory 513.

[0078] As shown in the figure, a control signal EN, a weight signal SW, and a temperature signal TE are input to a computing system 500.

[0079] The signal EN indicates which of the variable arrays x and y are updated, and the signal cycles between high (H) and low (L) values. For example, when EN is H, the variable array x i It is determined that s is updated, and when EN is L, it is determined that the variable array y is updated. N are updated simultaneously, and the variables y1,…,y N are updated simultaneously. In Figure 13, signal EN is input only to sampling block 515 for simplicity, but it is equally applicable to other places, such as variable memories, that require this signal.

[0080] The signal SW is a signal indicating a vector of N elements representing the diagonal elements of the diagonal matrix W.

[0081] The values ​​of the matrix J stored in the nonlinear coefficient memory 512, the vector h stored in the linear coefficient memory 513, the signal SW, and the variable s (x or y) stored in the variable memory 511 are input to a difference calculation block 514. When the signal EN is H, the difference calculation block 514 calculates (J+diag(w1, ..., w N ))y+h, and when the signal EN is L, (J+diag(w1,…,w N ))x+h. This output value is the A i is equivalent to.

[0082] A sampling block 515 receives the output of the difference calculation block 514, the signal SW, the signal TW that stores the value of the temperature parameter, the signal EN, and the values ​​of other variables. The i-th element is randomly sampled and is −(2−|y i |) or more and (2-|y i |) or less is the domain, and when the signal EN is L, -(2-|x i |) or greater and (2-|x i The output is from a truncated normal distribution, represented by Equation 12, whose domain is |

[0083] The next state determination block 516 determines the next state of the variable based on one or more values ​​output from the sampling block 515. If the MCMC update rule is defined as a simple heat bath algorithm, the next state determination block 516 may receive only one output value of the sampling block 515 and write the received output value directly to the variable memory 511. If the well-known over-relaxation method is used as the MCMC update rule, the next state determination block 516 receives multiple values ​​from the sampling block 515 and the current value of the variable to be updated from the variable memory 511, selects one according to the over-relaxation method, and writes the value to the variable memory 511. As is well known, in the over-relaxation method, the next state is determined so that it has a negative correlation with the previous state.

[0084] Fig. 14 is a detailed block diagram of a computing system to which an exemplary implementation example can be applied. Specifically, Fig. 14 is a block diagram showing a circuit configuration example when SRAM technology is applied to a computing system 500. A plurality of units 801 constitute an array unit 802. Such a configuration can be manufactured by applying semiconductor manufacturing technology.

[0085] One unit 801 is the variable x1,…,x N and y1,…,y N and a configuration for updating the value of the multi-value memory 901. That is, 2N units 801 are prepared.

[0086] The configuration example of FIG. 14 will be described with reference to the generalized configuration of FIG. 13. The data stored in the nonlinear coefficient memory 512 and the linear coefficient memory 513 are set by the model coefficient setting unit 612. The nonlinear coefficient memory 512 stores an N×N matrix J commonly used by all units 801. In addition, an N-dimensional vector h is stored in the linear coefficient memory 513 and commonly used by all units 801. In order to reduce the circuit scale, these memories are common to each unit 801. Thus, the nonlinear coefficient memory 512 and the linear coefficient memory 513 supply the coefficients J and h to all units 801. However, the signal lines for this purpose are omitted in FIG. 8. In principle, each unit 801 may include the nonlinear coefficient memory 512 and the linear coefficient memory 513 individually.

[0087] The N elements (w1, ..., w N ) is stored in the weight memory 803. This data is set by the weight setting unit 613. The i-th unit that stores x and y is the i-th component w i is used, it is necessary to switch the value of the signal SW for each unit 801. In Fig. 8, the signal line that supplies the signal SW to the unit 801 is omitted.

[0088] A temperature signal TE provided by the temperature setting unit 615 is supplied to all units 801. The function and configuration of the temperature signal is according to the prior art. The signal line supplying the signal TE to the units 801 is omitted.

[0089] The interaction driver 804 alternately inputs a signal that enables the update of the variable x and a signal that enables the update of the variable y to each unit 801. As a result, the variables x1 to x N are updated simultaneously, and variables y1 to y N are updated simultaneously.

[0090] The SRAM interface 805 writes and reads the memory that stores the variables of the unit 801 by applying the circuit configuration of the SRAM. The variables read after the process is completed in the computing system 500 are sent to the variable value reading unit 617. The variable value reading unit 617 outputs the read variables as continuous or binary values ​​based on the domain data 603 to obtain a solution to the mixed binary quadratic programming problem.

[0091] The controller 806 initializes the computing system 500 and reports the completion of processes according to the instructions of the interaction computation execution unit 616 .

[0092] 15 is a block diagram of a unit that constitutes a computing system to which an exemplary implementation example can be applied. Specifically, FIG. 15 is a diagram showing an example of a circuit configuration of one unit 801. Continuous variables x1, ..., x N and y1,…,y N A multi-value memory 901 that stores any one of the above is included in one unit.

[0093] The difference calculation circuit 902 realizes the function of the difference calculation block 514. If the variables stored in the multi-value memory 901 are x1, ..., x N When either (y1,…,y N ) is input to the difference calculation circuit 902. The variables stored in the multi-value memory 901 are y1, ..., y N When either (x1,…,x N ) vectors are input. These variable vectors are read and generated from a multi-value memory 901 of another unit 801 by the SRAM interface 805. In addition, an N×N matrix J and an N-dimensional vector h, which are coefficients, are input. In addition, weights w i The difference calculation circuit 902 receives these inputs and calculates (J+diag(w1,...,w N ))s+h (where s is a vector of x or y variables) i Output.

[0094] The sampling circuit 903 realizes the function of the sampling block 515. The output Ai, the signal EN, the signal SW, the signal TE, and the variables stored in the multi-value memory 901 are x i If y i , or the variable stored in the multi-value memory 901 is y i When x i is input to the sampling circuit 903. Next, the next state candidate of the variable s i The probability of existence of p(s i )

[0095] The state determination circuit 904 determines the next state of the variable based on one or more candidates output from the sampling circuit 903. In the state determination circuit 904, when following, for example, the over-relaxation method, if multiple candidates are obtained from the sampling circuit 903, the candidate having a negative correlation with the immediately preceding state in the multi-value memory 901 is selected, and the next state is determined. The determined next state is stored in the multi-value memory 901.

[0096] In the above, the difference calculation block 514, the sampling block 515, and the next state determination block 516 are assumed to be hardware such as an FPGA. However, software can be implemented that utilizes multiple computing devices such as those found in GPUs, vector computers, etc. By providing multiple units 801 in this way, variables can be updated in parallel.

[0097] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the substance of their innovations to others skilled in the art. An algorithm is a sequence of predefined steps leading to a desired end state or result. In exemplary implementations, the steps performed require physical manipulations of specific quantities to achieve specific results.

[0098] Unless otherwise specifically indicated, and as will be apparent from the discussion, discussion throughout the description utilizing terms such as "processing," "computing," "calculating," "determining," "displaying," and the like will be recognized to include operations and processes of a computer system or other information processing device that manipulates and converts data represented as physical (electronic) quantities in the computer system's registers and memory into other data similarly represented as physical quantities in the computer system's memory or registers or other information storage, transmission, or display device.

[0099] Exemplary implementations may also relate to an apparatus for performing the operations of the present specification. The apparatus may be specially constructed for the required purposes or may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored on a computer-readable medium, such as a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium may include tangible media, such as optical disks, magnetic disks, read-only memory, random access memory, solid-state devices and drives, or any other type of tangible or non-transitory medium suitable for storing electronic information. The computer-readable signal medium may include media such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. The computer program may include a pure software implementation, including instructions for performing the operations of a desired implementation.

[0100] Various general-purpose systems may be used with the programs and modules according to the examples herein, or it may prove useful to construct more specialized apparatus to perform the desired method steps. In addition, the example implementations are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the techniques of the example implementations as described herein. Instructions in the programming language may be executed by one or more processing devices, such as a central processing unit (CPU), processor, or controller.

[0101] As known in the art, the operations described above may be implemented by hardware, software, or some combination of software and hardware. Various aspects of the exemplary implementations may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (software) that, when executed by a processor, cause the processor to perform a method for implementing the examples of the present application. Furthermore, some exemplary implementations of the present application may be implemented exclusively by hardware, while other exemplary implementations may be implemented exclusively by software. Furthermore, the various functions described may be implemented in a single unit or may be spread across multiple components in various manners. When implemented by software, the method may be executed by a processor, such as a general-purpose computer, based on instructions stored on a machine-readable medium. If necessary, the instructions may be stored on the medium in compressed and / or encrypted form.

[0102] Moreover, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the techniques of the present application. Various aspects and / or components of the described example implementations may be used alone or in any combination. It is intended that the specification and example implementations be considered as examples only, with a true scope and spirit of the present application being indicated by the following claims. [Explanation of symbols]

[0103] 5: System bus 10: Information processing device 11: Processor 12: Primary memory device 13: Auxiliary storage device 14: Input devices 15: Output device 16: Communication devices 20: Computing devices

Claims

1. 1. An information processing method for accelerating a process involving repeated matrix multiplications between a first matrix and a second matrix for searching an optimal solution to an interaction model, comprising: replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix; performing a Monte Carlo approximation matrix multiplication between the first approximation matrix and the second approximation matrix to generate an approximation matrix product; executing the process of searching for the optimal solution based on the approximate matrix product according to the interaction model; providing the optimal solution found from the process; and An information processing method comprising:

2. 2. The information processing method according to claim 1, An information processing method performed using parallel processing on a graphics processing unit (GPU) or a field programmable gate array (FPGA).

3. 2. The information processing method according to claim 1, 11. The information processing method, wherein the process of searching for the optimal solution is a complementary metal-oxide semiconductor (CMOS) annealing process, and the interaction model includes an Ising model.

4. 2. The information processing method according to claim 1, An information processing method, wherein the process is a neural network operation.

5. 2. The information processing method according to claim 1, An information processing method, wherein the process is an eigenvalue operation.

6. 2. The information processing method according to claim 1, performing the Monte Carlo approximation matrix multiplication, determining a first vector based on the first set of constant values ​​and a second vector based on the second set of constant values; generating the approximation matrix product by the Monte Carlo approximation matrix multiplication from a sum of a matrix product result of the Monte Carlo approximation matrix multiplication, a vector product of the second vector and the first approximation matrix, a vector product of the second approximation matrix and the first vector, and an inner product including the first set of constant values ​​and the second set of constant values; An information processing method comprising:

7. 2. The information processing method according to claim 1, 13. An information processing method, comprising: selecting a first set of constant values ​​and a second set of constant values ​​such that a result of a matrix multiplication of the first matrix and the second matrix is ​​equivalent to a sum of a result of a matrix multiplication of the first approximate matrix and the second approximate matrix, one or more vector multiplication operations involving the first approximate matrix and the second approximate matrix, and one or more dot product operations involving the first set of constant values ​​and the second set of constant values.

8. 1. An information processing device configured to accelerate a process of searching for an optimal solution to an interaction model, the process including repeated matrix multiplications between a first matrix and a second matrix, the information processing device comprising: a processor configured to control one or more computing devices to perform the above-described operations using parallel processing; replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix; performing a Monte Carlo approximation matrix multiplication between the first approximation matrix and the second approximation matrix to generate an approximation matrix product; executing the process of searching for the optimal solution based on the approximate matrix product according to the interaction model; providing the optimal solution found from the process; and An information processing device configured to execute the above.

9. The information processing apparatus of claim 8 , wherein the one or more computing devices comprise a graphics processing unit (GPU) or a field programmable gate array (FPGA).

10. The information processing device according to claim 8 , wherein the process for searching for the optimal solution is a complementary metal-oxide semiconductor (CMOS) annealing process, and the interaction model includes an Ising model.

11. The information processing apparatus according to claim 8 , wherein the process is a neural network operation.

12. The information processing apparatus according to claim 8 , wherein the process is an eigenvalue calculation.

13. The processor controls the one or more computing devices to use parallel processing to: determining a first vector based on the first set of constant values ​​and a second vector based on the second set of constant values; generating the approximation matrix product by the Monte Carlo approximation matrix multiplication from a sum of a matrix product result of the Monte Carlo approximation matrix multiplication, a vector product of the second vector and the first approximation matrix, a vector product of the second approximation matrix and the first vector, and an inner product including the first set of constant values ​​and the second set of constant values; The information processing apparatus of claim 8 , configured to perform the Monte Carlo approximation matrix multiplication by:

14. 9. The information processing device of claim 8, wherein the first set of constant values ​​and the second set of constant values ​​are selected such that a result of matrix multiplication of the first matrix and the second matrix is ​​equivalent to a sum of a result of matrix multiplication of the first approximate matrix and the second approximate matrix, one or more vector multiplication operations involving the first approximate matrix and the second approximate matrix, and one or more inner product operations involving the first set of constant values ​​and the second set of constant values.

15. 1. A non-transitory computer readable medium storing instructions for accelerating a process of searching for an optimal solution to an interaction model, the process including repeated matrix multiplications between a first matrix and a second matrix, the instructions comprising: replacing the first matrix with a first approximation matrix and replacing the second matrix with a second approximation matrix by adding a first set of constant values ​​to the first matrix and adding a second set of constant values ​​to the second matrix; performing a Monte Carlo approximation matrix multiplication between the first approximation matrix and the second approximation matrix to generate an approximation matrix product; executing the process of searching for the optimal solution based on the approximate matrix product according to the interaction model; providing the optimal solution found from the process; and A non-transitory computer readable medium comprising:

Images