Data processing method, data processing apparatus, and program

Abstract

To acquire an appropriate value of a parameter.SOLUTION: The processing unit 12 specifies each axial direction of a class coordinate system having an origin corresponding to a class center based on a feature vector of each instance in the same class, and acquires a set of second universal values of a plurality of parameters for each second instance group corresponding to each axial direction. The processing unit 12 generates, based on the first feature vector corresponding to the second instance group, the class center coordinates, and a set of first and second universal values of each parameter corresponding to the class, a unit vector indicating a correction direction of the set of first universal values. The processing unit 12 generates a learning model M1 that outputs class-coordinate-system coordinates in response to an input of an instance in the class. The processing unit 12 calculates, for a new instance classified into the class, a value of each solution-seeking parameter of the new instance based on a set of first coordinates and first generic values and a unit vector of the new instance in the class coordinates system obtained by the learning model M1.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a data processing method, a data processing device, and a program. [Background technology]

[0002] Information processing devices are sometimes used to solve combinatorial optimization problems. A combinatorial optimization problem is converted into an evaluation function that represents the energy of an Ising model, which is a model that represents the spin behavior of magnetic materials. The information processing device searches for a combination of state variable values ​​included in the evaluation function that minimizes or maximizes the evaluation function. The combination of state variable values ​​that minimizes or maximizes the evaluation function corresponds to the ground state or optimal solution represented by the set of state variables. Solution methods that obtain an approximate solution to a combinatorial optimization problem in a practical amount of time include, for example, the tabu search (TS) method, the simulated annealing (SA) method, and the parallel tempering (PT) method.

[0003] These solution methods use various parameters to control the search for a solution. For example, the TS method uses tabu tenure as one of the parameters. Tabu tenure is the length of time for which the value of a state variable is fixed once it has been changed during the search. In addition, the SA method and PT method, for example, use parameters related to temperature conditions such as maximum and minimum temperature values.

[0004] Statistical analysis and machine learning techniques are sometimes used to obtain predicted output results for input data. For example, a case study learning device has been proposed that takes as input information about component placement examples on an existing circuit board, classifies the components placed on the board based on their similarity from the placement example information, and saves and outputs design rules determined for the components included in each classification for each classification. The proposed case study learning device includes a neural network that takes as input information about component placement examples within a classification and extracts design rules for elements for which design rules should be determined from the placement example information.

[0005] There have also been proposals for systems that build client profiles to detect incidents of abnormal behavior according to a training corpus of events occurring at the client, systems that generate machine learning models customized for healthcare organizations using synthetic datasets that extend training and validation sets, and systems that use neural networks to generate masks of object instances in images.

[0006] There has been a proposal for a device that performs parameter optimization for a neural network during model training using a function called a cross-entropy loss function and a triple loss function. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 11-306222 [Patent Document 2] Special Publication No. 2022-512195 [Patent Document 3] Special Publication No. 2023-544335 [Patent Document 4] US Patent Application Publication No. 2022 / 0092869 [Patent Document 5] US Patent Application Publication No. 2020 / 0364406 Summary of the Invention [Problem to be solved by the invention]

[0008] As described above, multiple parameters are used in solving combinatorial optimization problems. The value of each parameter affects the solution performance. For example, if the value of each parameter is inappropriate, the possibility of arriving at a better solution may decrease. Therefore, it is possible to classify multiple problem data representing multiple combinatorial optimization problems into multiple classes and prepare generic values ​​for each parameter for each class in advance.

[0009] The generalized values ​​of each parameter can be obtained in advance by parameter search. Parameter search is a process in which solutions are previously performed for multiple problem data belonging to the class with various values ​​for each parameter, and the generalized values ​​of each parameter corresponding to the class are obtained by selecting the values ​​that best satisfy, for example, a predetermined standard regarding solution performance. When new problem data is generated, the generalized values ​​of each parameter corresponding to the class into which the new problem data is classified can be used to perform a solution for the problem data.

[0010] However, simply using the generalized values ​​of the parameters corresponding to the class into which the problem data is classified may not be enough to improve the solution performance. In one aspect, the present invention aims to obtain appropriate values ​​for parameters. [Means for solving the problem]

[0011] In one aspect, a data processing method is provided. In this data processing method, a data processing system acquires a plurality of instances, each of which is information indicating a problem to be solved and which are classified into the same class, a plurality of feature vectors corresponding to the plurality of instances, class center coordinates corresponding to the class calculated from the plurality of feature vectors, and a set of first generic values ​​of a plurality of parameters obtained by a parameter search using a first group of the plurality of instances. The data processing system identifies a plurality of axis directions defining a class coordinate system having an origin corresponding to the class center coordinates based on the plurality of feature vectors, extracts a plurality of second group of instances from the plurality of instances corresponding to the plurality of axis directions based on the plurality of feature vectors and the plurality of axis directions, and acquires a set of second generic values ​​of the plurality of parameters for each second group of instances by a parameter search using each of the second group of instances. The data processing system generates a unit vector indicating a correction direction for the set of first generic values ​​for each second group of instances based on the first feature vector, the class center coordinates, the set of first generic values, and the set of second generic values ​​corresponding to the second group of instances. A data processing system generates a learning model that outputs coordinates in the class coordinate system corresponding to an instance in response to an input of an instance belonging to a class, using multiple instances and multiple coordinates in a class coordinate system corresponding to the instance. When a first instance classified into the class is input, the data processing system obtains first coordinates in the class coordinate system corresponding to the first instance using the first instance and the learning model. The data processing system calculates values ​​of multiple parameters used to solve for the first instance, based on a first set of universal values, a unit vector, and the first coordinates.

[0012] In one aspect, a data processing device having a storage unit and a processing unit is provided. In another aspect, a program executed by a computer is provided. [Effects of the Invention]

[0013] In one aspect, the appropriate values ​​of the parameters can be obtained. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating a data processing system according to a first embodiment. [Figure 2] FIG. 10 illustrates an example of hardware of a data processing device according to a second embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of the distribution of instances in a feature space. [Figure 4] FIG. 2 is a diagram illustrating an example of functions of a data processing device. [Figure 5] FIG. 10 is a diagram illustrating an example of data held by a data processing device. [Figure 6] FIG. 2 is a diagram illustrating an example of functions of a data processing device. [Figure 7] FIG. 10 is a diagram illustrating a shift direction in a feature space. [Figure 8] FIG. 10 is a diagram illustrating a corrected unit vector. [Figure 9] FIG. 10 is a diagram illustrating the definition of variables. [Figure 10] FIG. 10 is a diagram illustrating the definition of variables. [Figure 11] 10 is a flowchart illustrating an example of a learning process. [Figure 12] 10 is a continuation of the flowchart illustrating an example of the learning process. [Figure 13] 10 is a flowchart showing an example of sub-processing for class cm. [Figure 14] 10 is a flowchart illustrating an example of a prediction process. [Figure 15] 10 is a flowchart illustrating an example of a classification process. [Figure 16] 10 is a flowchart illustrating an example of feature regression processing. [Figure 17] 10 is a flowchart illustrating an example of a generalized parameter search. [Figure 18] FIG. 10 is a diagram illustrating an example of parameters to be tuned. [Figure 19]FIG. 10 is a diagram illustrating tuning target parameters. DETAILED DESCRIPTION OF THE INVENTION

[0015] The present embodiment will be described below with reference to the drawings. [First embodiment] A first embodiment will be described.

[0016] FIG. 1 is a diagram illustrating a data processing system according to a first embodiment. The data processing system 10 controls the values ​​of parameters used to solve a combinatorial optimization problem. The data processing system 10 includes a storage unit 11 and a processing unit 12.

[0017] The storage unit 11 may be a volatile semiconductor memory such as a random access memory (RAM), or a non-volatile storage such as a hard disk drive (HDD) or flash memory. The storage unit 11 stores data used in processing by the processing unit 12. The processing unit 12 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). However, the processing unit 12 may also include an application-specific electronic circuit such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processor executes a program stored in a memory such as a RAM (which may be the storage unit 11). A set of multiple processors is sometimes called a "multiprocessor" or simply a "processor."

[0018] A combinatorial optimization problem is formulated using a predetermined evaluation function and is converted into a problem of minimizing the value of the evaluation function, for example. The value of the evaluation function, i.e., the evaluation value, represents the energy of an Ising model, for example. The evaluation function is sometimes called an energy function or an objective function. The evaluation function includes multiple state variables. The state variables are binary variables that take on values ​​of 0 or 1. The state variables may also be called bits. The solution to the combinatorial optimization problem is represented by the values ​​of the multiple state variables. The solution that minimizes the value of the evaluation function represents the ground state of the Ising model and corresponds to the optimal solution to the combinatorial optimization problem.

[0019] The Ising-type evaluation function is expressed by equation (1).

[0020]

number

[0021] The state vector x has multiple state variables as elements and represents the state of the Ising model. Equation (1) is an evaluation function formulated in the QUBO (Quadratic Unconstrained Binary Optimization) format. If the problem is to maximize the value of the evaluation function, simply reverse the sign of the evaluation function.

[0022] The first term on the right side of equation (1) is the sum of the values ​​of the two state variables and the weighting coefficients for all combinations of two state variables that can be selected from all state variables, without omissions or duplications. The subscripts i and j are the indexes of the state variables. i is the i-th state variable. x j is the jth state variable. W ij is the weighting coefficient that indicates the strength of the connection between the i-th state variable and the j-th state variable. ij =W ji and W ii = 0. N is the number of all state variables.

[0023] The second term on the right side of equation (1) is the sum of the products of the biases of all state variables and the values ​​of the state variables. i denotes the bias for the i-th state variable. The solution of the combinatorial optimization problem is performed by a search unit (not shown). The search unit is realized by a processor such as a CPU or a GPU. However, the solution may be performed by the processing unit 12. Methods for solving the combinatorial optimization problem include, for example, the TS method, the SA method, and the PT method.

[0024] The TS method is a search method based on the greedy search. The greedy search finds the energy change ΔE when the values ​​of all state variables are changed for the current state. i Calculate ΔE i This is a method of repeating the procedure of selecting the state variable for which ΔE is the smallest, inverting the value of that state variable, and setting it as the next state. On the other hand, in the TS method, the value of a state variable once changed is fixed for a predetermined period. The value of a state variable is updated by i This allows the TS method to find a local solution, i.e., all ΔE i > 0, it suppresses the repetition of the same state transition, making it possible to search a wider solution space. In the TS method, the length of time for which the value of a state variable is fixed once it has been changed is called tabu tenure. Tabu tenure is an example of a parameter used in solving problems with the TS method.

[0025] The SA method uses the Metropolis method or the Gibbs method to determine the transition probability from one state to the next state by changing the state variables. In the SA method, even changes that increase the value of the evaluation function are probabilistically allowed based on a comparison between the change in the evaluation function value and the thermal noise value. This allows for escape from local solutions. The thermal noise value is calculated based on the temperature value and random numbers. The larger the temperature value, the larger the amplitude of the thermal noise value. The larger the amplitude of the thermal noise value, the more likely it is that a state transition with a large increase in the evaluation function value will be allowed. In the SA method, for example, the transition probability p from the current state to the candidate state to which the transition is to occur in the Metropolis method is expressed by equation (2).

[0026]

number

[0027] E prop is the energy of the candidate state. E prev is the energy of the current state. T is the temperature value. Furthermore, in the PT method, the Metropolis method searches for a solution at multiple temperature values. In the PT method, the states between adjacent temperatures are probabilistically exchanged at a predetermined timing to escape from a local solution. The execution unit of the search at each temperature value is sometimes called a replica. The PT method is also sometimes called the replica exchange method. In the PT method, the exchange probability p of states between temperatures is expressed by equation (3).

[0028]

number

[0029] Δ in equation (3) is expressed by equation (4).

[0030]

number

[0031] In equation (4), i is the identification number of the temperature value.i+1 and the temperature value T i About T i+1 >T i E i is the temperature value T i is the energy of the state corresponding to E i+1 is the temperature value T i+1 is the energy of the state corresponding to

[0032] The SA method reduces the amplitude of the thermal noise by gradually changing the temperature from the highest to the lowest temperature, converging the state of the Ising model to the ground state or a state close to the ground state, thereby obtaining a solution. The PT method performs independent searches for each temperature value within the range from the lowest to the highest temperature, swaps the states obtained at each temperature value at a predetermined timing, and continues the search using the swapped state as the initial solution, repeating this process to extract a good solution obtained in the process. The lowest temperature value, highest temperature value, and temperature change range are examples of parameters used in the SA and PT methods to find a solution.

[0033] In addition to the above examples, there are various other parameters used in finding a solution, such as a coefficient for determining the maximum number of repetitions of state transitions in one search, and a threshold for the number of state transitions to determine whether the solution has converged. The values ​​of multiple parameters are used in finding a solution.

[0034] The parameter values ​​affect the solution performance, so the data processing system 10 controls the parameter values ​​used to solve each problem as follows. The processing unit 12 acquires a plurality of instances, each of which is information indicating a problem to be solved and which are classified into the same class. ij This is the data.

[0035] Here, the processing unit 12 can pre-cluster each instance according to the feature vector of each instance included in the existing instance set, and classify each instance into one of a plurality of classes. For example, HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) can be used as the clustering method. The processing unit 12 can obtain a learning model for classifying each instance into one of the groups using the clustering results.

[0036] A feature vector is a set of two or more types of feature quantities. One feature vector is obtained for one instance. The types of feature quantities include, for example, the size of the instance, density, and solution performance when parameters are set to experimental values. The size of the instance is determined by W in Equation (1). ij The density of instances is W ij is an index showing the degree to which non-zero values ​​are included in the solution. Solution performance is the time required to converge on a solution in a search using trial values ​​(default values) of parameters, and the result of judging whether the energy of the final solution is good or bad. In addition, feature quantities can include other indicators such as the difficulty of the problem, which is evaluated based on the type and number of constraints.

[0037] FIG. 1 illustrates an example of an instance distribution 30 of a certain class in a feature space 20. The number of dimensions of the feature space 20, i.e., the number of feature types, is 2, for example. However, the number of dimensions of the feature space 20 may be greater than 2. The position of an instance in the feature space 20 is represented by the feature vector of the instance.

[0038] The processing unit 12 acquires multiple feature vectors corresponding to multiple instances classified into the same class. For example, the multiple instances are classified into the same class based on the similarity of the multiple feature vectors corresponding to the multiple instances. The processing unit 12 acquires class center coordinates corresponding to the class, calculated from the multiple feature vectors. The class center coordinates are the center of gravity or geometric center of the multiple feature vectors corresponding to the class. For example, the class center coordinate o in FIG. 1 is obtained by calculating the arithmetic mean for each component of the feature vector of each instance in the instance distribution 30.

[0039] The processing unit 12 acquires a set of first universal values ​​of a plurality of parameters obtained by a parameter search using a first group of instances among the plurality of instances, the set corresponding to the class. For example, the processing unit 12 may extract a first group of instances whose feature vectors are located near the class center coordinate o from the plurality of instances belonging to the class. The processing unit 12 may acquire the set of first universal values ​​of the plurality of parameters by performing a parameter search using the first group of instances. However, the set of first universal values ​​may be stored in advance in the storage unit 11. In this case, the processing unit 12 can acquire the set of first universal values ​​from the storage unit 11.

[0040] In parameter search, the value of each parameter is changed and a solution is found using that value repeatedly, and the set of values ​​(best parameter values) that best meets evaluation criteria such as short solution time and low energy of the solution reached is obtained as the general-purpose value. Methods for determining the value of each parameter used in parameter search include grid search, random search, and TPE (Tree-structured Parzen Estimator).

[0041] The first instance group may be all or a part of the multiple instances belonging to the class. For example, the first instance group may be instances randomly extracted from the multiple instances belonging to the class. FIG. 1 illustrates a parameter space 40. In the example of FIG. 1, the number of dimensions of the parameter space 40, i.e., the number of types of parameters, is two or more. The coordinate p of the parameter space 40 O corresponds to a set of first generalized values ​​of the plurality of parameters.

[0042] The processing unit 12 identifies a plurality of axis directions that define a class coordinate system having an origin corresponding to the class center coordinate, based on a plurality of feature vectors belonging to the class. For example, the class coordinate system 31 is a class coordinate system that corresponds to the instance distribution 30. The origin of the class coordinate system 31 corresponds to the class center coordinate o in the feature space 20.

[0043] In one example, the processing unit 12 performs principal component analysis (PCA) on the instance distribution 30 to obtain two principal component directions of the instance distribution 30 as axis directions d1 and d2 of the class coordinate system 31. The unit vectors indicating the principal component directions of the instance distribution 30 are calculated as eigenvectors of a covariance matrix based on the feature vectors of each instance in the instance distribution 30 and the class center coordinate o. The eigenvectors of the covariance matrix are called the first principal component, the second principal component, ... in descending order of their eigenvalues. In the example of FIG. 1, since the feature space 20 is two-dimensional, the processing unit 12 obtains two principal component directions (the first principal component direction and the second principal component direction). The axis directions of the class coordinate system 31 may also be obtained using other techniques, such as singular value decomposition (SVD).

[0044] The processing unit 12 extracts a second group of instances corresponding to the multiple axis directions from the multiple instances based on the multiple feature vectors and the multiple axis directions. For example, the processing unit 12 obtains, in the feature space 20, coordinates of the ends of the instance distribution 30 corresponding to each of the axis directions d1 and d2 in the class coordinate system 31. The coordinates a1 and a2 are the coordinates of the ends of the instance distribution 30 corresponding to the two axis directions d1 and d2. The processing unit 12 may calculate the coordinate a1 based on, for example, the eigenvector (h1) and eigenvalue (λ1) of the covariance matrix indicating the principal component corresponding to the axis direction d1. For example, the calculation may be performed as a1 = o + α * λ1 * h1, where α is a positive constant commonly used for each axis direction. The coordinate a2 may be calculated in a similar manner.

[0045] The processing unit 12 acquires one or more instances that belong within a predetermined distance from the coordinate a1 as a first second instance group. The processing unit 12 also acquires one or more instances that belong within a predetermined distance from the coordinate a2 as a second second instance group. The vector indicating the coordinate a1 can be said to be a feature vector corresponding to the first second instance group. The vector indicating the coordinate a2 can be said to be a feature vector corresponding to the second second instance group.

[0046] The processing unit 12 performs a parameter search using each of the second instance groups to acquire a set of second universal values ​​of the parameters for each second instance group. For example, coordinate p1 in the parameter space 40 corresponds to the set of second universal values ​​of the parameters corresponding to the first second instance group. Coordinate p2 in the parameter space 40 corresponds to the set of second universal values ​​of the parameters corresponding to the second second instance group.

[0047] Furthermore, the processing unit 12 acquires a first feature vector corresponding to the second instance group. As described above, the vector corresponding to the coordinate a1 in the feature space is the first feature vector corresponding to the first second instance group. Furthermore, the vector corresponding to the coordinate a2 in the feature space is the first feature vector corresponding to the second second instance group.

[0048] The processing unit 12 generates a unit vector indicating a correction direction for the set of first generic values ​​for each second instance group based on the first feature vector, the class center coordinates, the set of first generic values, and the set of second generic values. Here, each component of the coordinates of the instance in the class coordinate system 31 corresponds to the set of first generic values ​​p in the parameter space 40. o The unit vector e indicates the direction of deviation of the second generalized value pair p1, p2 relative to p_1 ,e p_2 The processing unit 12 corresponds to, for example, the unit vector e p_1 ,e p_2 is calculated using equations (5) and (6). Note that the subscript "p_i" stands for "p i " indicates.

[0049]

number

[0050]

number

[0051] unit vector e p_1 corresponds to the first component of the coordinate in the class coordinate system 31 of the instance, p o The unit vector e indicates the direction of correction. p_2 corresponds to the second component of the coordinate in the class coordinate system 31 of the instance, p o The number of unit vectors is the same as the number of dimensions of the feature space 20. The processing unit 12 generates the unit vector e p_1 ,e p_2 is stored in the storage unit 11 in association with the corresponding class.

[0052] Furthermore, the processing unit 12 generates a learning model that outputs coordinates corresponding to an instance in the class coordinate system in response to input of an instance belonging to a class, using the multiple instances and multiple coordinates in the class coordinate system corresponding to the multiple feature vectors. For input to the learning model, the processing unit 12 converts each feature vector of an instance belonging to the class into coordinates in the class coordinate system of the class.

[0053] For example, the learning model M1 is a model that outputs coordinates corresponding to an instance in the class coordinate system 31 in response to input of an instance belonging to a class corresponding to the class center coordinate o. The processing unit 12 generates the learning model M1 using machine learning technology. The learning model M1 may be a neural network (NN). For example, when the learning model M1 is a graph neural network (GNN), a W corresponding to the instance is generated. ij The graphed data is input to the learning model M1. The instance may be data that has already been converted into an input format for the learning model. The processing unit 12 stores the generated learning model M1 in the storage unit 11 in association with the corresponding class.

[0054] In this way, the processing unit 12 calculates the learning model M1 and the unit vector e p_1 ,e p_2 When a new instance belonging to a certain class is received, the processing unit 12 prepares a learning model M1 and a unit vector e corresponding to the class. p_1 ,e p_2 Based on this, the values ​​of the parameters to be used in solving the instance are identified as follows.

[0055] The processing unit 12 acquires an instance A1 classified into a class corresponding to the class center coordinate o. Then, the processing unit 12 inputs the instance A1 into a learning model M1 corresponding to the class, thereby acquiring coordinates (b1, b2) of the instance A1 in the class coordinate system 31.

[0056] The processing unit 12 generates a set p of first general-purpose values ​​corresponding to the class. o and the unit vector e corresponding to the class p_1 ,e p_2 and the coordinates (b1, b2), a set of parameter values ​​p used to solve for instance A1 is calculated. pred Calculate.

[0057] Specifically, the processing unit 12 calculates the correction amount Δp. Δp=b1e p_1 +b2e p_2 The processing unit 12 calculates p by δp. o By modifying p pred Calculate p pred =p o +δp.

[0058] The processing unit 12 pred For example, the processing unit 12 outputs the instance A1 and p pred may be output to the search unit, causing the search unit to search for a solution. In the data processing system 10, the learning model M1 and the unit vector e p_1 ,e p_2 and a learning model M1 and a unit vector e p_1 ,e p_2 Using p pred The devices that calculate the data may be the same or different. Also, the functionality of data processing system 10 may be implemented by a single device, i.e., a data processing device.

[0059] According to the data processing system 10 of the first embodiment, a plurality of instances, each of which is information indicating a problem to be solved, are obtained, the plurality of instances being classified into the same class. A plurality of feature vectors corresponding to the plurality of instances are obtained. Class center coordinates corresponding to the class, calculated from the plurality of feature vectors, are obtained. A set of first generalized values ​​of a plurality of parameters obtained by a parameter search using a first group of the plurality of instances is obtained. A plurality of axis directions defining a class coordinate system having an origin corresponding to the class center coordinates are identified based on the plurality of feature vectors. A plurality of second group of instances corresponding to the plurality of axis directions are extracted from the plurality of instances based on the plurality of feature vectors and the plurality of axis directions. A set of second generalized values ​​of a plurality of parameters is obtained for each second group of instances by a parameter search using each of the plurality of second group of instances. A learning model is generated using the plurality of instances and a plurality of coordinates in the class coordinate system corresponding to the plurality of feature vectors, and outputs coordinates in the class coordinate system corresponding to the instance when an instance belonging to the class is input. A unit vector indicating a correction direction for the first generic value set is generated for each second instance group based on a first feature vector corresponding to the second instance group, the class center coordinate, the first generic value set, and the second generic value set. When a first instance classified into the class is input, a first coordinate corresponding to the first instance in the class coordinate system is obtained using the first instance and the learning model. Values ​​of multiple parameters used to solve for the first instance are calculated based on the first generic value set, the unit vector, and the first coordinate.

[0060] This allows the data processing system 10 to obtain appropriate values ​​for each of the multiple parameters used in solving for each individual instance. Specifically, the data processing system 10 modifies the generic values ​​of each parameter corresponding to a class based on the feature vector of the individual instance belonging to the class, thereby obtaining values ​​that are more appropriate for the individual instance than the generic values. By using the modified values ​​of each parameter in solving the instance, the data processing system 10 can improve the solution performance compared to using generic values ​​of each parameter.

[0061] Also, for example, when a user wants to solve instance A1, the data processing system 10 inputs the information of instance A1 into the learning model M1 and predicts the coordinates (b1, b2), thereby obtaining p by a relatively simple calculation. pred Therefore, the data processing system 10 can quickly obtain the values ​​of multiple parameters used to find a solution.

[0062] Here, the data processing system 10 does not directly regress the value of each parameter to the instance, but regresses the coordinates in the class coordinate system in the feature space (feature vectors transformed into the class coordinate system) to the instance for the following reasons.

[0063] For example, when creating a learning model that directly regresses the values ​​of each parameter, the learning model is created by finding the values ​​of each parameter corresponding to each existing instance through parameter search and using these as inputs. However, performing parameter search for each existing instance takes a very long time. Furthermore, the prediction dimensions are usually (number of dimensions in the feature space, K)<(number of dimensions in the parameter space, L). In one example, K=3 and L=10. Therefore, by using coordinates (number of dimensions, K) in the class coordinate system in the feature space as the regression quantities for the learning model, the data processing system 10 can speed up the learning process and prediction process compared to directly regressing the values ​​of each parameter.

[0064] [Second embodiment] Next, a second embodiment will be described. FIG. 2 illustrates an example of hardware of a data processing device according to the second embodiment.

[0065] The data processing device 100 searches for a solution to a combinatorial optimization problem using the TS method, SA method, PT method, or the like, and outputs the searched solution. The combinatorial optimization problem is formulated by an evaluation function E(x) of equation (1), and is converted into a problem of minimizing the value of the evaluation function E(x), for example. When solving the TS method, SA method, PT method, or the like, multiple parameters are used, as described above.

[0066] The data processing device 100 has a processor 101, a RAM 102, an HDD 103, a GPU 104, an input interface 105, a media reader 106, a communication interface 107, and an accelerator card 108. These units of the data processing device 100 are connected to a bus inside the data processing device 100. The processor 101 corresponds to the processing unit 12 of the first embodiment. The RAM 102 corresponds to the storage unit 11 of the first embodiment.

[0067] The processor 101 is an arithmetic device that executes program instructions. The processor 101 is, for example, a CPU. The processor 101 loads at least a portion of the program and data stored in the HDD 103 into the RAM 102 and executes the program. The processor 101 may include multiple processor cores. The data processing device 100 may also have multiple processors. The processes described below may be executed in parallel using multiple processors or processor cores. A set of multiple processors may also be referred to as a "multiprocessor" or simply as a "processor." A processor may also be referred to as a "processor circuitry." Each of the multiple processes executed by the data processing device 100 may be executed by a different processor, or at least some of the multiple processes may be executed by the same processor.

[0068] The RAM 102 is a volatile semiconductor memory that temporarily stores programs executed by the processor 101 and data used in calculations by the processor 101. Note that the data processing device 100 may include a type of memory other than RAM, or may include multiple memories.

[0069] The HDD 103 is a nonvolatile storage device that stores software programs such as an OS (Operating System), middleware, and application software, as well as data. The data processing device 100 may also include other types of storage devices, such as flash memory or an SSD (Solid State Drive), or may include multiple nonvolatile storage devices.

[0070] The GPU 104 outputs an image to a display 51 connected to the data processing device 100 in accordance with an instruction from the processor 101. The display 51 may be any type of display, such as a CRT (Cathode Ray Tube) display, a liquid crystal display (LCD: Liquid Crystal Display), a plasma display, or an organic EL (OEL: Organic Electro-Luminescence) display.

[0071] The input interface 105 acquires an input signal from an input device 52 connected to the data processing device 100 and outputs the signal to the processor 101. The input device 52 may be a pointing device such as a mouse, a touch panel, a touch pad, or a trackball, a keyboard, a remote controller, or a button switch. In addition, multiple types of input devices may be connected to the data processing device 100.

[0072] The medium reader 106 is a reading device that reads programs and data recorded on the recording medium 53. For example, a magnetic disk, an optical disk, a magneto-optical disk (MO: Magneto-Optical disk), a semiconductor memory, etc. can be used as the recording medium 53. Magnetic disks include flexible disks (FD: Flexible Disks) and HDDs. Optical disks include compact discs (CDs) and digital versatile discs (DVDs).

[0073] The medium reader 106 copies, for example, programs and data read from the recording medium 53 to another recording medium such as the RAM 102 or the HDD 103. The read programs are executed by, for example, the processor 101. The recording medium 53 may be a portable recording medium, which may be used to distribute programs and data. The recording medium 53 and the HDD 103 may also be referred to as computer-readable recording media.

[0074] The communication interface 107 is connected to the network 54 and communicates with other information processing devices via the network 54. The communication interface 107 may be a wired communication interface connected to a wired communication device such as a switch or a router, or may be a wireless communication interface connected to a wireless communication device such as a base station or an access point.

[0075] The accelerator card 108 is a hardware accelerator that searches for solutions to combinatorial optimization problems. The accelerator card 108 includes a processor 110 and a RAM 120. The processor 110 searches for solutions using a TS algorithm, an SA algorithm, a PT algorithm, or the like. The processor 110 is, for example, a GPU, a DSP, an ASIC, or an FPGA. The RAM 120 stores data used in the processing of the processor 110.

[0076] The RAM may be a dynamic random access memory (DRAM) or a static random access memory (SRAM). Here, combinatorial optimization problems are characterized by instances, which are information indicating the QUBO in formula (1). Multiple instances are clustered by the feature vector of each instance. The feature vector has multiple feature quantities as components. The types of feature quantities include, for example, the size and density of the instance, the solution performance when parameters are set to trial values ​​(default values), and the difficulty level, which is evaluated based on the type and number of constraints.

[0077] FIG. 3 is a diagram illustrating an example of the distribution of instances in the feature space. The feature space 60 is a two-dimensional feature space with the first feature on the horizontal axis and the second feature on the vertical axis. The feature space may have a dimension higher than two. A vector representing one point plotted in the feature space 60 is a feature vector corresponding to one instance. Each point in the feature space 60 represents the distribution of all instances in the feature space 60. Each instance is clustered based on its feature vector and classified into one of multiple classes. For example, HDBSCAN is used as a clustering method. Alternatively, clustering may be performed using a relatively simple method such as determining the identity of the user, the identity of the type of question, or which region the feature space 60 belongs to when it is divided into multiple regions. By clustering, instances with similar features are classified into the same class.

[0078] In the example of feature space 60, each instance is classified into one of six classes. Six sub-distributions 61, 62, 63, 64, 65, and 66 of the instances correspond to the six classes. One sub-distribution is the instance distribution of the corresponding class. The geometric center of each sub-distribution is the class center coordinate of the class. For example, the class center coordinate of a certain class is obtained as the arithmetic mean of each feature vector belonging to the class.

[0079] The data processing device 100 holds, for each class, a set of generalized values ​​of parameters used for solving the problem obtained by the parameter search. The data processing device 100 provides a function of modifying, before solving an individual instance, the set of generalized values ​​of parameters corresponding to the class to which the instance belongs, in accordance with the instance.

[0080] FIG. 4 is a diagram illustrating an example of functions of the data processing device. The data processing device 100 has an instance storage unit 120, a trained model storage unit 130, a general-purpose parameter storage unit 140, and a learning processing unit 150. The instance storage unit 120, the trained model storage unit 130, and the general-purpose parameter storage unit 140 use storage areas of the RAM 102 and the HDD 103. The learning processing unit 150 is realized by the processor 101 executing a program stored in the RAM 102.

[0081] The instance storage unit 120 stores an existing instance group 121. The existing instance group 121 is, for example, a collection of instances that have been created in the past. The trained model storage unit 130 stores training models created by the training processing unit 150 using machine learning techniques. Training models include training models for classification and training models for feature regression. A training model for classification is a model that, in response to an input of an instance, outputs the class into which the instance is classified. A training model for feature regression is a model that, in response to an input of an instance, outputs coordinates corresponding to the instance in a class coordinate system identified for each class. A training model for feature regression is created for each class. The training model for classification and the training model for feature regression are, for example, GNNs.

[0082] The generalized parameter storage unit 140 stores, for each class, a set of generalized values ​​of parameters used in solving. It also stores, for each class, a unit vector indicating the direction of correction in parameter space when correcting the set of generalized values ​​for each individual instance. This unit vector is called a correction unit vector.

[0083] Here, in the following, we will use the set p of generalized values ​​of parameters corresponding to the class of the class center coordinate o in the feature space. o =(p o1 ,p o2 ,…,p oL ) with the generalized parameter p o The general parameter p o The number of dimensions L corresponds to the number of types of parameters.

[0084] The learning processing unit 150 generates a classification learning model, a feature regression learning model, and a corrected unit vector. The learning processing unit 150 includes a feature extraction unit 151, a clustering unit 152, a class coordinate generation unit 153, an input conversion unit 154, a classification model learning unit 155, a feature position calculation unit 156, a feature regression model learning unit 157, a generalized parameter search unit 158, an axial generalized parameter search unit 159, a generalized parameter correction amount calculation unit 160, and a corrected unit vector calculation unit 161.

[0085] The feature extraction unit 151 extracts the feature of each instance in the existing instance group 121 for each type of feature, and generates a feature vector for each instance. The clustering unit 152 clusters each instance based on the feature vector of each instance. Through clustering, each instance in the existing instance group 121 is classified into multiple classes. The class center coordinates of a certain class are the geometric center of the feature vectors of each instance belonging to that class.

[0086] The class coordinate generation unit 153 generates a class coordinate system for each class based on the instance distribution of each class in the feature space. The origin of the class coordinate system corresponds to the class center coordinate of the corresponding class. For example, the class coordinate generation unit 153 performs PCA on the instance distribution in the feature space to obtain unit vectors indicating the principal component directions of the instance distribution, and identifies the principal component directions as the axis directions of the class coordinate system. The class coordinate generation unit 153 may also identify the axis directions of the class coordinate system using other methods such as SVD.

[0087] The input conversion unit 154 converts each instance of the existing instance group 121 into an input to the learning model. For example, if the learning model is a GNN, each instance is converted into graph data. The graph data of the converted instance is ij is represented as a graph structure with nodes and edges connecting the nodes.

[0088] The classification model learning unit 155 generates a classification learning model by machine learning using the class classification results of each instance by the clustering unit 152 and the data of each instance after conversion by the input conversion unit 154. The classification learning model is a model that outputs the class into which an instance is classified when an instance is input. The classification model learning unit 155 stores the classification learning model in the learned model storage unit 130.

[0089] The feature amount position calculation unit 156 converts the feature vector of each instance into coordinates in the class coordinate system of the class to which the instance belongs. The feature regression model learning unit 157 generates a feature regression learning model for each class by machine learning using the coordinates of each instance in the class coordinate system and the data of each instance after conversion by the input conversion unit 154. The feature regression model learning unit 157 stores the generated feature regression learning model for each class in the learned model storage unit 130.

[0090] The generalized parameter search unit 158 ​​identifies a group of instances having feature vectors that belong to an area within a predetermined range from the class center coordinates of the corresponding class in the feature space. The predetermined range is, for example, an area within a certain distance from the class center coordinates. The distance is predetermined by the user. The generalized parameter search unit 158 ​​searches for generalized parameters for each class by performing a parameter search using the identified group of instances. Parameter search methods include grid search, random search, and TPE. Details of the parameter search will be described later. The generalized parameter search unit 158 ​​stores the generalized parameters for each class in the generalized parameter storage unit 140.

[0091] The axial generalized parameter search unit 159 acquires the coordinates of the edge of the instance distribution in the feature space for each class. The coordinates of the edge of the instance distribution for a certain class are called the "cluster edge coordinates" of that class. For example, the axial generalized parameter search unit 159 identifies an area in the feature space corresponding to the instance distribution, and, starting from the class center coordinate, traces along the axial direction (principal component direction) of the class coordinate system to find the coordinates of the boundary between the inside and outside of that area as the cluster edge coordinates. The number of cluster edge coordinates obtained is equal to the number of principal component directions of the instance distribution.

[0092] The axial general-purpose parameter search unit 159 identifies a group of instances having feature vectors that belong to a region within a predetermined range from the cluster end coordinates of the corresponding class in the feature space. The predetermined range is, for example, a region within a certain distance from the cluster end coordinates. The distance is predetermined by the user. Here, the cluster end coordinates are coordinates in the feature space, and their components are various features. Therefore, the cluster end coordinates can also be considered as feature vectors corresponding to the group of instances.

[0093] Then, the axial direction generalized parameter search unit 159 searches for generalized parameters corresponding to the instance group by performing a parameter search using the instance group identified for the cluster end coordinates. The axial direction generalized parameter search unit 159 searches for the generalized parameters for each class as many times as the number of cluster end coordinates. That is, the axial direction generalized parameter search unit 159 obtains generalized parameters for each axis direction of the class coordinate system for a certain class.

[0094] The general-purpose parameter correction amount calculation unit 160 calculates the difference between the general-purpose parameters searched for by the general-purpose parameter search unit 158 ​​and the general-purpose parameters for each axial direction searched for by the axial general-purpose parameter search unit 159 for each class.

[0095] The correction unit vector calculation unit 161 calculates a correction unit vector used to correct the generalized parameters for each class. The difference in the generalized parameters corresponding to a certain axis direction calculated by the generalized parameter correction amount calculation unit 160 is denoted as Δp, the cluster edge coordinate in the feature space corresponding to that axis direction is denoted as a, and the class center coordinate is denoted as o. Then, the correction unit vector e of the generalized parameters corresponding to the component of the coordinate in the class coordinate system in that axis direction is expressed as e = Δp / |ao|. For example, when the feature vector is two-dimensional, the correction unit vector can be obtained by equations (5) and (6). The correction unit vector calculation unit 161 associates the correction unit vector e with the class and stores it in the generalized parameter storage unit 140.

[0096] FIG. 5 is a diagram showing an example of data held by the data processing device. 5(A) illustrates an example of data stored in the trained model storage unit 130. The trained model storage unit 130 stores a trained model for classification 131 generated by the trained model training unit 155. The trained model storage unit 130 also stores a trained model for feature regression 132 generated by the feature regression model training unit 157 in association with class identification information. "Class C1" is an example of class identification information.

[0097] 5B illustrates an example of data stored in the generalized parameter storage unit 140. The generalized parameter storage unit 140 stores the generalized parameters and the corrected unit vectors in association with the class identification information. As described above, the generalized parameters p o =(p o1 ,p o2 ,…) is L. Also, the dimension of the feature vector is the correction unit vector e p_1 ,e p_2 Here, the subscripts p_1, p_2, ... (= p1, p2, ...) of e identify the generic parameters obtained by the axial generic parameter search unit 159 for the cluster end coordinates a1, a2, ... of the class in the feature space.

[0098] FIG. 6 is a diagram illustrating an example of functions of the data processing device. The data processing device 100 includes a prediction processing unit 170. The prediction processing unit 170 is realized by the processor 101 executing a program stored in the RAM 102.

[0099] The prediction processing unit 170 predicts and outputs a set of parameter values ​​to be used for solving a new instance 122 obtained from the instance storage unit 120. The prediction processing unit 170 includes an input conversion unit 171, a classification processing unit 172, a feature regression processing unit 173, and a general-purpose parameter correction unit 174.

[0100] The input conversion unit 171 converts the new instance 122 into an input to the learning model. For example, if the learning model is a GNN, each instance is converted into graph data. The classification processing unit 172 inputs the converted data of the new instance 122 by the input conversion unit 171 into the classification learning model 131 stored in the learned model storage unit 130, thereby obtaining a class into which the new instance 122 is to be classified. Here, the class into which the new instance 122 is to be classified is represented as class Cx.

[0101] The feature regression processing unit 173 identifies the feature regression learning model 132 for class Cx stored in the learned model storage unit 130. The feature regression processing unit 173 performs regression processing by inputting the data of the new instance 122 converted by the input conversion unit 171 to the feature regression learning model 132. Specifically, the feature regression processing unit 173 obtains coordinates corresponding to the new instance 122 in the class coordinate system of class Cx as the output of the feature regression learning model 132.

[0102] The generalized parameter correction unit 174 acquires the generalized parameters of the class Cx stored in the generalized parameter storage unit 140 and the corrected unit vector of the generalized parameters. The generalized parameter correction unit 174 calculates a set of parameter values ​​(corrected parameters) to be used in solving the new instance 122 by correcting the generalized parameters based on the coordinates, the generalized parameters, and the corrected unit vector obtained by the feature regression processing unit 173.

[0103] Here, the number of dimensions of the feature vector is K. The coordinates of the new instance 122 in the class coordinate system are expressed as b = (b1, b2, ..., b K ) and the corrected unit vector is e p_1 ,e p_2 ,…,e p_K Then, the correction amount δp=b1e p_1 +b2e p_2 +…+b K e p_K The general parameter p o The corrected parameter p' for p'=p o +δp.

[0104] The general parameter modification unit 174 outputs the modified parameters. For example, the general parameter modification unit 174 may input the modified parameters together with the new instance 122 to the accelerator card 108, causing the accelerator card 108 to search for a solution for the new instance 122. The general parameter modification unit 174 may display the modified parameters on the display 51 or transmit them to another device via the network 54.

[0105] FIG. 7 is a diagram showing the direction of deviation in the feature space. For example, a two-dimensional class coordinate system Q1 is defined for an instance distribution 70 of a certain class in a two-dimensional feature space. The origin of the class coordinate system Q1 corresponds to the class center coordinate o in the feature space. The two axis directions of the class coordinate system Q1 are, for example, the directions of two principal components of the instance distribution 70. These two principal components are obtained as the eigenvectors and eigenvalues ​​of a covariance matrix based on each feature vector of the instance distribution 70.

[0106] The two axis directions of the class coordinate system Q1 correspond to the deviation directions D1 and D2 of the feature vector of each instance relative to the class center coordinate o. The deviation directions D1 and D2 in the feature space are converted into each component of the coordinates of the class coordinate system Q1.

[0107] Here, the elliptical region surrounding the instance distribution 70 shown in FIG. 7 can also be considered to be a region covering points indicated by the feature vectors of each instance included in the instance distribution 70. The boundary point between the inside and outside of the region, located in the shift direction D1 as viewed from the class center coordinate o, is the edge 71 of the region. The boundary point between the inside and outside of the region, located in the shift direction D2 as viewed from the class center coordinate o, is the edge 72 of the region. The edges 71 and 72 correspond to the cluster edges of the class. Here, the coordinates of the edges 71 and 72 in the feature space (cluster edge coordinates) are a1 and a2, respectively. The coordinates a1 and a2 indicate points in the feature space. Therefore, the position vector corresponding to the coordinates a1 and a2 can be considered to be a feature vector.

[0108] The coordinates a1 and a2 may be obtained as follows. For example, the coordinate a1 may be obtained by adding a vector obtained by multiplying the product of an eigenvector indicating the principal component corresponding to the deviation direction D1 and its eigenvalue by a positive constant α to the class center coordinate o. Similarly, the coordinate a2 may be obtained by adding a vector obtained by multiplying the eigenvector indicating the principal component corresponding to the deviation direction D2 and its eigenvalue by a positive constant α to the class center coordinate o.

[0109] FIG. 8 is a diagram illustrating the corrected unit vector. For example, the generalized parameter search unit 158 ​​searches for a generalized parameter p corresponding to the class by searching for a parameter using a plurality of existing instances corresponding to the instance distribution 70. o The general-purpose parameter search unit 158 ​​obtains the general-purpose parameter p by performing a parameter search using a group of instances having feature vectors that belong to an area within a predetermined range from the class center coordinate o. o may be obtained.

[0110] Furthermore, the axial direction generalized parameter search unit 159 extracts a group of instances having feature vectors that belong to an area 73 within a predetermined range centered on the coordinate a1, from among the instances corresponding to the instance distribution 70. The axial direction generalized parameter search unit 159 obtains a generalized parameter p1 by performing a parameter search using the extracted group of instances.

[0111] Generalized parameter p1 and generalized parameter p o The deviation direction D1a in the parameter space, which is indicated by the difference between p_1 is expressed by equation (5).

[0112] Furthermore, the axial direction generalized parameter search unit 159 extracts a group of instances having feature vectors that belong to an area 74 within a predetermined range centered on the coordinate a2 from among the instances corresponding to the instance distribution 70. The axial direction generalized parameter search unit 159 obtains a generalized parameter p2 by performing a parameter search using the extracted group of instances.

[0113] Generalized parameter p2 and generalized parameter p o The deviation direction D2a in the parameter space, which is indicated by the difference between p_2 is expressed by equation (6).

[0114] Next, definitions of variables used in the following explanation will be explained. FIG. 9 is a diagram for explaining the definition of variables. Legend 80 lists each variable and its definition.

[0115] Z={z1,...,z N} is the universal set of instances z1,...,z N Each of the is an instance. There are N instances. V={v1,...,v N} is a set of feature vectors for each instance. The number of dimensions of the feature vector is K.

[0116] C={c1,...,c M} is the universal set of classes c1,...,c M Each of the is a class. There are M classes. Z C ={Z c_1 ,...,Z c_M} is a set whose elements are the set of instances for each class. c_1 ,...,Z c_M Each of these is a set of instances of the corresponding class.

[0117] Zc_m ={z c_m 1 ,...z c_m N_c_m} is a class c m is the set of instances in z c_m 1 ,...z c_m N_c_m Each of the class c m class c is an instance of m The number of instances in N c_m There are individuals.

[0118] V C ={V c_1 ,...,V c_M} is a coordinate system for each class, i.e., a set of class coordinate systems. The class coordinate system is K-dimensional because it is included in the feature space. V c_1 ,...,V c_M Each of the above is class coordinate system information for each class. The class coordinate system information may include, for example, information on a vector representing the axis direction of the class coordinate system in the feature space. The class coordinate system information may also include information indicating an operation for converting a feature vector in the feature space into coordinates in the class coordinate system.

[0119] o C ={o c_1 ,...,o c_M} are the center coordinates (class center coordinates) of each class in the feature space. a c_m ={a c_m 1 ,...,a c_m K} is the class c in the feature space. m are the cluster edge coordinates in the K axis directions.

[0120] b C ={b c_1 ,...,b c_M} is a set whose elements are the set of coordinates of instances for each class coordinate system. c_1 ,...,b c_MEach of the is a set of coordinates of instances in the class coordinate system within each class.

[0121] b c_m ={b c_m 1 ,...,b c_m N_c_m} is a class c m is the set of coordinates of the instances in the class coordinate system within b c_m 1 ,...,b c_m N_c_m Each of the class c m The coordinates of the instance in the class coordinate system within the

[0122] p C ={p c_1 (·) ,...,p c_M (·)} is a set of parameters for each class. c_1 (·) ,...,p c_M (·) are parameters for each class. The number of dimensions of the parameters is L. The superscript "(·)" indicates the difference between the instances used in the parameter search.

[0123] e C ={e c_1 ,...,e c_m} is a set of correction unit vectors of generic parameters for each class. e c_1 ,...,e c_m Each of the is a correction unit vector for each class.

[0124] e c_m ={e c_m a^1 ,...,e c_m a^K} is a class c m Here, the subscript "a^i" means "a i " indicates. e c_m a^1 ,...,e c_ma^K is class C m Cluster edge coordinate a c_m 1 ,...,a c_m K is the corrected unit vector corresponding to

[0125] modified unit vector e c_m a^i (i=1,...,K) is expressed by equation (7).

[0126]

number

[0127] FIG. 10 is a diagram for explaining the definition of variables. Legend 81 lists each variable and its definition. W C ={W c_1 ,...,W c_M} is the set of input data for each class of the learning model. c_1 ,...,W c_M Each of these is input data to the learning model for each class.

[0128] M C indicates a learning model for classification. M R ={M R1 ,...,M RM} is a set of feature regression learning models for each class. M R1 ,...,M RM is a learning model for feature regression for each class. The numbers following the R in the subscripts R1,...,RM represent the classes c1,...,c M corresponds to the numerical subscript of c in

[0129] Next, a description will be given of the processing procedure of the data processing device 100. First, the learning processing by the learning processing unit 150 will be described. FIG. 11 is a flowchart illustrating an example of the learning process.

[0130] (S10) The feature extraction unit 151 receives an input of an instance set Z corresponding to the existing instance group 121. (S11) The feature extraction unit 151 calculates a set V of feature vectors corresponding to each instance in Z.

[0131] (S12) The clustering unit 152 clusters each instance based on the feature vector. As a result of the clustering, each instance is classified into classes c1,...,c M It is classified into one of the following.

[0132] (S13) The learning processing unit 150 m Step S14 is repeatedly executed for (m=1, . . . , M). (S14) The learning processing unit 150 m The details of the sub-processing will be described later.

[0133] (S15) When the learning processing unit 150 executes step S14 for all classes, it ends the repetition and proceeds to step S16. (S16) The learning processing unit 150 calculates the general-purpose parameter set p obtained as a result of repeating the sub-processing of step S14. C (o) , the corrected unit vector set e C , the instance set for each class Z C , the coordinate set of instances for each class b C These pieces of information are stored, for example, in the general-purpose parameter storage unit 140. Then, the process proceeds to steps S17 and S20.

[0134] FIG. 12 is a continuation of the flowchart illustrating an example of the learning process. (S17) The input conversion unit 154 converts Z C Get. (S18) The input conversion unit 154 converts Z C The input data for the learning model is W C Convert the input data W Cis used in step S19 and steps S21 to S23.

[0135] (S19) The classification model learning unit 155 uses the input data W C and the class to which each instance is classified, a classification learning model M C Then, the process proceeds to step S24.

[0136] (S20) The feature regression model learning unit 157 C Get. (S21) The feature regression model learning unit 157 m Step S22 is repeatedly executed for (m=1, . . . , M).

[0137] (S22) The feature regression model learning unit 157 uses the input data W c_m and b c_m By machine learning using Rm Learn about the following. (S23) When the feature quantity regression model learning unit 157 has executed step S22 for all classes, it ends the repetition and proceeds to step S24.

[0138] (S24) The classification model learning unit 155 uses the classification learning model M generated in step S19. C The feature regression model learning unit 157 stores the feature regression learning model M for each class generated in step S22. Rm are associated with the classes and stored in the trained model storage unit 130. Then, the training process ends.

[0139] Here, the procedure of steps S17 to S19 and the procedure of steps S20 to S23 may be executed in parallel by different processors or processor cores. Figure 13 shows class c m 10 is a flowchart illustrating an example of a sub-process for the

[0140] Class Cm The sub-processing for this corresponds to step S14. (S30) The class coordinate generation unit 153 c_m Based on the feature vector of each instance belonging to class c m Class coordinate system V c_m As described above, the class coordinate generation unit 153 calculates Z c_m The class coordinate system V in the feature space is calculated by PCA or SVD based on the feature vectors of each instance belonging to c_m It is possible to specify each axis of the class coordinate system V c_m The information of is used to represent the feature vector in the feature space in the class coordinate system V c_m The information may include information indicating an operation for converting the coordinates into coordinates of the object. Then, the process proceeds to steps S31, S32, and S36.

[0141] (S31) The axial general-purpose parameter search unit 159 searches for V c_m The cluster edge coordinate a in the feature space corresponding to the axis of c_m A group of instances having feature vectors in the vicinity of the target point is identified. The axial direction general parameter search unit 159 searches for parameters using the group of instances to find the general parameter p c_m (a^k) is calculated, where k=1,...,K. Then, the process proceeds to step S33.

[0142] (S32) The general-purpose parameter search unit 158 ​​calculates the class center coordinates o c_m A group of instances having feature vectors in the vicinity of each other is identified. The general-purpose parameter search unit 158 ​​searches for parameters using the group of instances to find the general-purpose parameter p c_m (o) Then, the process proceeds to step S33.

[0143] (S33) The general-purpose parameter correction amount calculation unit 160 c_m and a c_m k and the generalized parameter p c_m (o) and p c_m (a^k)Calculate the difference. (S34) The corrected unit vector calculation unit 161 calculates the general-purpose parameter p c_m (o) The modified unit vector e c_m Calculate.

[0144] (S35) The corrected unit vector calculation unit 161 calculates e c_m Then, the process proceeds to step S38. (S36) The class coordinate generation unit 153 generates the class coordinate system V c_m In the class instance Z c_m Coordinate b of c_m Calculate each.

[0145] (S37) The class coordinate generation unit 153 c_m Then, the process proceeds to step S38. (S38) The corrected unit vector calculation unit 161 calculates p c_m (o) ,e c_m The class coordinate generation unit 153 returns Z c_m ,b c_m And return class c m The sub-processing for

[0146] Here, steps S31 and S32 may be executed in parallel by different processors or processor cores. Furthermore, the procedures of steps S36 and S37 may be executed in parallel with steps S31 to S35 by a processor or processor core separate from the processor or processor core that executes steps S31 to S35.

[0147] Steps S20 to S23 in FIG. 12 are for class c m b against c_m Therefore, steps S20 to S23 are executed if class c m It may be executed before steps S31 to S35 or in parallel with steps S31 to S35. In that case, Z c_m The input data W c_mThe feature regression learning model M Rm It is used to learn.

[0148] Next, the prediction processing by the prediction processing unit 170 will be described. FIG. 14 is a flowchart illustrating an example of the prediction process. (S40) The input conversion unit 171 acquires a new instance z'.

[0149] (S41) The input conversion unit 171 converts z' into input data w' for a learning model. (S42) The classification processing unit 172 uses the classification learning model M C The classification process is performed on w' using the above formula. The classification process will be described in detail later. As a result of the classification process, a class c' into which w' is classified is obtained. Then, the process proceeds to steps S43 and S44.

[0150] (S43) The general-purpose parameter correction unit 174 corrects the general-purpose parameter p corresponding to the classification target class c′. c’ (o) and the modified unit vector e c’ Then, the process proceeds to step S45.

[0151] (S44) The feature regression processing unit 173 calculates the feature regression learning model M corresponding to the classification target class c′. Rc’ The feature regression process for w' is performed using the feature regression process. The details of the feature regression process will be described later. As a result of the feature regression process, the class coordinate system V corresponding to w' is c_m Coordinate b at c’ Then, the process proceeds to step S45.

[0152] (S45) The general-purpose parameter correction unit 174 calculates the amount of correction Δp. Δp is expressed by equation (8).

[0153]

number

[0154] bc’i is the coordinate b c’ The i-th component of class c' is a i is the component corresponding to (S46) The general-purpose parameter correction unit 174 calculates the corrected parameter p' and outputs p'. p' is expressed by equation (9). Then, the prediction process ends.

[0155]

number

[0156] Here, step S43 and step S44 may be executed in parallel by different processors or processor cores. FIG. 15 is a flowchart showing an example of the classification process.

[0157] The classification process corresponds to step S42. (S50) The classification processing unit 172 receives the classification learning model M from the learned model storage unit 130. C Get.

[0158] (S51) The classification processing unit 172 C The input data w' obtained by converting the new instance z' is classified using the above formula, and the resulting class c' is obtained. (S52) The classification processing unit 172 outputs the class c', and the classification processing ends.

[0159] FIG. 16 is a flowchart illustrating an example of feature regression processing. The feature regression process corresponds to step S44. (S60) The feature regression processing unit 173 calculates the feature regression learning model M corresponding to the class c′. Rc’ is obtained from the trained model storage unit 130.

[0160] (S61) The feature regression processing unit 173 Rc’ As a result, the regression quantity b c’ is obtained. (S62) The feature regression processing unit 173 calculates the regression quantity b c’ The regression quantity b is output. c’ is the class coordinate system V corresponding to the new instance z'. c_m Then, the feature regression process ends.

[0161] Next, the general-purpose parameter search performed in steps S31 and S32 will be described. FIG. 17 is a flowchart illustrating an example of a generalized parameter search. In the following, the general-purpose parameter search unit 158 ​​is mainly exemplified, but the general-purpose parameter search by the axial general-purpose parameter search unit 159 follows the same procedure. However, in steps S31 and S32, different instance groups are used in the general-purpose parameter search.

[0162] (S70) The general-purpose parameter search unit 158 ​​performs new parameter proposal. In the new parameter proposal, a new set of parameter values ​​is generated by grid search, random search, and TPE for the range of values ​​that each of the multiple parameters can take. The new set of parameter values ​​is referred to as a new parameter.

[0163] (S71) The general-purpose parameter search unit 158 ​​solves each of the multiple instances using the new parameters generated in step S70, and evaluates the new parameters based on the energy of the obtained solution and the history of state transitions during the solution-finding process. For example, the better the energy of the finally obtained solution, the higher the evaluation of the new parameters. Also, for example, the shorter the time it takes to obtain the final solution, the higher the evaluation of the new parameters. The solution for each instance may be performed using the accelerator card 108.

[0164] (S72) The general-purpose parameter search unit 158 ​​determines whether or not to end the repeated evaluation of the new parameter. If it is to be ended, the process proceeds to step S73. If it is not to be ended, the process proceeds to step S70. For example, in the case of a grid search, the repeated evaluation ends when all combinations of values ​​of multiple parameters have been exhausted. Also, in the case of a random search, the repeated evaluation ends when the number of repetitions specified in advance by the user has been reached. The repeated evaluation of the new parameter may be performed, for example, several hundred times.

[0165] (S73) The general-purpose parameter search unit 158 ​​outputs the new parameter with the best evaluation result (best parameter) from among the new parameters evaluated in step S71 as the general-purpose parameter, and the general-purpose parameter search ends.

[0166] Next, examples of parameters to be tuned in solving a combinatorial optimization problem will be described. FIG. 18 is a diagram illustrating an example of tuning target parameters.

[0167] Parameters to be tuned include tabu tenure in the TS method, and maximum and minimum temperature values ​​in the SA and PT methods. Other examples of parameters include the following: The following parameters can be used in any of the TS, SA, and PT methods.

[0168] Table 90 shows examples of tuning target parameters, and includes the parameter name, value range, default value, and description. The parameter "num_group" is the number of groups for which the solution search is performed in parallel. The search is performed independently between groups. The value range of "num_group" is, for example, "1-16". The default value of "num_group" is, for example, "1".

[0169] The parameter "num_run" is the number of times a search is executed within one group. The value range of "num_run" is, for example, "1-1024". The default value of "num_run" is, for example, "16".

[0170] The parameter "gs_level" is a coefficient for the maximum number of iterations of one search. The "maximum number of iterations of one search" is the upper limit allowed for the number of state transitions caused by changing one state variable per search. The "maximum number of iterations" is determined, for example, by the formula "maximum number of iterations = instance size × gs_level". The instance size is, for example, W ij The value range of "gs_level" is, for example, "0-1000". The default value of "gs_level" is, for example, "5".

[0171] The parameter "gs_cutoff" is the number of times to check convergence in one search. The number of times to check convergence is the number of times to repeat the state transition before the search is terminated if there is no improvement in the solution. The value range of "gs_cutoff" is, for example, "0-1000000". The default value of "gs_cutoff" is, for example, "8000".

[0172] FIG. 19 is a diagram illustrating tuning target parameters. FIG. 19(A) illustrates the parameters "num_group" and "num_run." Chart 91 shows the relationship between "num_group" and "num_run." Note that "num_group" = N. In other words, the number of groups that perform independent searches is N. Each group independently searches for a solution using the TS method and the SA method the number of times specified by "num_run." Alternatively, the PT method can be performed using multiple groups in parallel. In this case, one group may be called a replica.

[0173] FIG. 19(B) illustrates a graph 92. The graph 92 shows an example of the relationship between the number of repetitions of a state transition in one search and the energy of the resulting state. The horizontal axis of the graph 92 is the number of repetitions of the state transition. The vertical axis of the graph 92 is the energy of the state calculated based on equation (1). The maximum number of repetitions of the state transition allowed in one search is determined by "instance size x gs_level". In addition, the number of times of convergence determination used to determine whether to terminate the search when there is no improvement in the solution is determined by the parameter "gs_cutoff".

[0174] However, the above tuning target parameters are just examples, and the data processing device 100 may tune parameters other than those above. In this way, the data processing device 100 can obtain appropriate values ​​for each of the multiple parameters used in solving for each individual instance. Specifically, the data processing device 100 can obtain values ​​for each individual instance that are more appropriate than the generic values ​​by correcting the generic values ​​of each parameter corresponding to a class based on the feature vector of the individual instance belonging to the class. By using the corrected values ​​of each parameter in solving the corresponding instance, the data processing device 100 can improve the solution performance compared to using generic values ​​of each parameter.

[0175] Furthermore, the data processing device 100 inputs information about the new instance into the feature regression learning model, and calculates the coordinate b c’ By predicting p', the corrected parameter p' for solving the new instance can be obtained by simple calculations such as those in equations (8) and (9). This allows the data processing device 100 to quickly obtain the values ​​of multiple parameters used for solving the new instance.

[0176] Here, the data processing device 100 does not directly regress the value of each parameter to the instance, but regresses the coordinates in the class coordinate system in the feature space (feature vectors transformed into the class coordinate system) to the instance for the following reasons.

[0177] For example, when creating a learning model that directly regresses the values ​​of each parameter, the learning model is created by finding the values ​​of each parameter corresponding to each existing instance through parameter search and using these as inputs. However, performing parameter search for each existing instance takes a very long time. Furthermore, the prediction dimensions are usually (number of dimensions in the feature space, K)<(number of dimensions in the parameter space, L). In one example, K=3 and L=10. Therefore, by using coordinates (number of dimensions, K) in the class coordinate system in the feature space as the regression quantities of the learning model, the data processing device 100 can speed up the learning process and prediction process compared to directly regressing the values ​​of each parameter.

[0178] As described above, the data processing device 100 executes the following processes. The learning processing unit 150 acquires multiple instances, each of which is information indicating a problem to be solved and which are classified into the same class. The learning processing unit 150 acquires multiple feature vectors corresponding to the multiple instances. The learning processing unit 150 acquires class center coordinates corresponding to the class, calculated from the multiple feature vectors. The learning processing unit 150 acquires a set of first generalized values ​​of multiple parameters obtained by parameter search using a first instance group of the multiple instances. Based on the multiple feature vectors, the learning processing unit 150 identifies multiple axis directions that define a class coordinate system having an origin corresponding to the class center coordinates. Based on the multiple feature vectors and the multiple axis directions, the learning processing unit 150 extracts multiple second instance groups from the multiple instances that correspond to the multiple axis directions. The learning processing unit 150 acquires a set of second generalized values ​​of multiple parameters for each second instance group by parameter search using each of the multiple second instance groups. The learning processing unit 150 generates, for each second instance group, a unit vector indicating a correction direction for the first generic value set based on a first feature vector corresponding to the second instance group, a class center coordinate, a first generic value set, and a second generic value set. The learning processing unit 150 generates a learning model that outputs coordinates corresponding to an instance in the class coordinate system in response to an input of an instance belonging to a class, using multiple instances and multiple coordinates in the class coordinate system corresponding to the multiple feature vectors. When a first instance classified into the class is input, the prediction processing unit 170 uses the first instance and the learning model to obtain first coordinates corresponding to the first instance in the class coordinate system. The prediction processing unit 170 calculates values ​​of multiple parameters used to solve for the first instance based on the first generic value set, the unit vector, and the first coordinates.

[0179] This allows the data processing apparatus 100 to obtain appropriate values ​​for each of the multiple parameters used in solving for each individual instance. For example, the multiple axis directions are multiple principal component directions identified based on multiple feature vectors, which allows the data processing device 100 to easily obtain the axis directions of the class coordinate system of the corresponding class using techniques such as PCA or SVD.

[0180] Furthermore, the learning processing unit 150 can extract multiple second instance groups corresponding to multiple axial directions as follows: The learning processing unit 150 acquires a first feature vector indicating the edge of the distribution of points represented by multiple feature vectors for each of the multiple axial directions. The learning processing unit 150 extracts, from the multiple instances, instances having feature vectors that are included in an area within a predetermined range centered on a point represented by the first feature vector corresponding to each of the multiple axial directions as a second instance group corresponding to that axial direction.

[0181] This allows the data processing device 100 to appropriately acquire the second instance group so as to obtain the direction of deviation from the first set of generic values ​​in the parameter space for the coordinates of the instance in the class coordinate system. Furthermore, the data processing device 100 can improve the accuracy of the unit vectors for generic parameter correction obtained for each component of the coordinates of the instance in the class coordinate system. Here, "the feature vector is included in a predetermined region in the feature space" means that the point indicated by the feature vector is within the predetermined region.

[0182] Furthermore, the first instance group used to obtain the first set of generic values ​​may be a set of instances having feature vectors included in a region within a predetermined range centered on the class center coordinates, thereby enabling the data processing device 100 to improve the accuracy of the unit vectors for generic parameter correction obtained for each component of the coordinates of the instance in the class coordinate system.

[0183] The data processing device 100 may use the calculated values ​​of the plurality of parameters to solve the first instance. For example, the data processing device 100 may input the first instance and the calculated values ​​of the plurality of parameters to a search unit realized by the processor 101 or the processor 110, and solve the first instance using a TS algorithm, an SA algorithm, a PT algorithm, or the like. This allows the data processing device 100 to improve its performance in solving the first instance. For example, the data processing device 100 can increase the likelihood of arriving at a better solution in a short time when solving the first instance.

[0184] The device having the learning processing unit 150 and the device having the prediction processing unit 170 may be different. In this case, functions equivalent to those of the data processing device 100 are realized by a data processing system including a device having the learning processing unit 150 and a device having the prediction processing unit 170.

[0185] Furthermore, the plurality of processes by the learning processing unit 150 may be executed by different processors or processor cores, or at least some of the processes may be executed by the same processor or processor core. The plurality of processes by the prediction processing unit 170 may be executed by different processors or processor cores, or at least some of the processes may be executed by the same processor or processor core.

[0186] Moreover, the information processing of the first embodiment can be realized by causing the processing unit 12 to execute a program. The information processing of the second embodiment can be realized by causing the processor 101 to execute a program. The program can be recorded on a computer-readable recording medium 53.

[0187] For example, the program can be distributed by distributing recording medium 53 on which the program is recorded. Alternatively, the program may be stored in another computer and distributed via a network. For example, a computer may store (install) a program recorded on recording medium 53 or a program received from another computer in a storage device such as RAM 102 or HDD 103, and then read and execute the program from the storage device. [Explanation of symbols]

[0188] 10 Data Processing System 11 Storage section 12 Processing section 20 Feature Space 30 Instance Distribution 31 Class Coordinate System 40 Parameter Space A1 instances M1 Learning Model d1,d2 Axial direction

Claims

1. The data processing system acquiring a plurality of instances, each of which is information indicating a problem to be solved, classified into the same class, a plurality of feature vectors corresponding to the plurality of instances, class center coordinates corresponding to the class calculated from the plurality of feature vectors, and a set of first generalized values ​​of a plurality of parameters obtained by a parameter search using a first instance group of the plurality of instances; identifying, based on the plurality of feature vectors, a plurality of axis directions that define a class coordinate system having an origin corresponding to the class center coordinates; extracting, from the plurality of instances, a plurality of second instance groups that correspond to the plurality of axis directions based on the plurality of feature vectors and the plurality of axis directions; and obtaining, for each of the second instance groups, a set of second generic values ​​of the plurality of parameters by the parameter search using each of the plurality of second instance groups; generating, for each of the second instance groups, a unit vector indicating a correction direction for the set of first generic values ​​based on a first feature vector corresponding to the second instance group, the class center coordinates, the set of first generic values, and the set of second generic values; generating a learning model that outputs, in the class coordinate system, coordinates corresponding to the instances in response to input of the instances belonging to the class, using the instances and the coordinates in the class coordinate system corresponding to the instances; When a first instance classified into the class is input, a first coordinate corresponding to the first instance in the class coordinate system is obtained using the first instance and the learning model; calculating values ​​of the plurality of parameters used in solving for the first instance based on the first set of generic values, the unit vector, and the first coordinates; Data processing methods.

2. the plurality of axis directions are a plurality of principal component directions identified based on the plurality of feature vectors; 2. The data processing method according to claim 1.

3. In the extraction of the second instance groups corresponding to the axis directions, obtaining the first feature vector indicating an end of a distribution of points represented by the plurality of feature vectors for each of the plurality of axial directions; extracting, from among the plurality of instances, instances having feature vectors that are included in an area within a predetermined range centered on a point indicated by the first feature vector corresponding to each of the plurality of axis directions, as the second instance group corresponding to the axis direction; 2. The data processing method according to claim 1.

4. the first instance group is a set of instances having the feature vectors included in an area within a predetermined range centered on the class center coordinates; 2. The data processing method according to claim 1.

5. the data processing system performs a solution for the first instance using the calculated values ​​of the plurality of parameters.

2. The data processing method according to claim 1.

6. The plurality of instances are classified into the same class based on the similarity of the plurality of feature vectors.

2. The data processing method according to claim 1.

7. a storage unit that stores a plurality of instances, each of which is information indicating a problem to be solved, classified into the same class, a plurality of feature vectors corresponding to the plurality of instances, class center coordinates corresponding to the class calculated from the plurality of feature vectors, and a set of first generalized values ​​of a plurality of parameters obtained by a parameter search using a first instance group of the plurality of instances; a processing unit that identifies, based on the plurality of feature vectors, a plurality of axis directions that define a class coordinate system having an origin corresponding to the class center coordinates; extracts, from the plurality of instances, a plurality of second instance groups that correspond to the plurality of axis directions based on the plurality of feature vectors and the plurality of axis directions; obtains, for each second instance group, a set of second generic values ​​of the plurality of parameters by the parameter search using each of the plurality of second instance groups; generates, for each second instance group, a unit vector that indicates a correction direction for the set of first generic values ​​based on a first feature vector corresponding to the second instance group, the class center coordinates, the set of first generic values, and the set of second generic values; generates, using the plurality of instances and a plurality of coordinates in the class coordinate system that correspond to the plurality of feature vectors, a learning model that outputs coordinates in the class coordinate system that correspond to an input of the instance belonging to the class; when a first instance classified into the class is input, obtains a first coordinate corresponding to the first instance in the class coordinate system using the first instance and the learning model; and calculates values ​​of the plurality of parameters used to solve for the first instance based on the set of first generic values, the unit vector, and the first coordinate; A data processing device having:

8. On the computer, acquiring a plurality of instances, each of which is information indicating a problem to be solved, classified into the same class, a plurality of feature vectors corresponding to the plurality of instances, class center coordinates corresponding to the class calculated from the plurality of feature vectors, and a set of first generalized values ​​of a plurality of parameters obtained by a parameter search using a first instance group of the plurality of instances; identifying, based on the plurality of feature vectors, a plurality of axis directions that define a class coordinate system having an origin corresponding to the class center coordinates; extracting, from the plurality of instances, a plurality of second instance groups that correspond to the plurality of axis directions based on the plurality of feature vectors and the plurality of axis directions; and obtaining, for each of the second instance groups, a set of second generic values ​​of the plurality of parameters by the parameter search using each of the plurality of second instance groups; generating, for each of the second instance groups, a unit vector indicating a correction direction for the set of first generic values ​​based on a first feature vector corresponding to the second instance group, the class center coordinates, the set of first generic values, and the set of second generic values; generating a learning model that outputs, in the class coordinate system, coordinates corresponding to the instances in response to input of the instances belonging to the class, using the instances and the coordinates in the class coordinate system corresponding to the instances; A program that executes a process.

9. On the computer, the information indicating the problem to be solved, and when a first instance classified into a predetermined class is input, a first coordinate corresponding to the first instance in the class coordinate system is acquired based on a learning model that outputs, in response to an input of an instance belonging to the class, coordinates in the predetermined class coordinate system that correspond to a feature vector of the instance; calculating values ​​of the plurality of parameters used to find a solution for the first instance based on a set of first generalized values ​​of the plurality of parameters obtained by a parameter search using a first group of instances among the plurality of instances belonging to the class, a predetermined unit vector indicating a correction direction for the set of first generalized values, and the first coordinates; A program that executes a process.

Images