Automatic coordination of replica exchange

Abstract

To provide an automatic adjustment of a replica exchange.SOLUTION: In a form of one embodiment, an operation may acquire a first fixed temperature and a second fixed temperature of a replica exchange Markov chain Monte-Carlo (MCMC) process for use in a dissolution of an optimized question associated with a system, and may acquire a plurality of replicas of the system. The operation may further acquire a target swap acceptance probability related to a swap between the replicas which are adjacently and sequentially ordered out of a temperature set between the first fixed temperature and the second fixed temperature between the replica exchange MCMC processes. The operation may decide average swap acceptance probabilities related to one or more adjacent temperature pairs. Also, the operation may adjust one or more variable temperatures on the basis of a relationship between one or more of the target swap acceptance probabilities and the swap acceptance probabilities.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This patent application claims the benefit of and priority to U.S. Provisional Application No. 62 / 985,655, filed March 5, 2020, which is incorporated by reference in its entirety into this disclosure.

[0002] [Technical field] The embodiments discussed herein relate to automatic coordination of replica exchange. [Background technology]

[0003] Combinatorial optimization problems are often classified as NP problems (Nondeterministic Polynomial time Problems), such as NP-hard or NP-complete problems, and often there are no known algorithms for solving such problems in polynomial time. Such combinatorial optimization problems may appear in many applications, such as minimizing the number of vias in layout design, maximizing returns from stock portfolios, airline routing and scheduling, and wireless sensor networks. Replica exchange processes may be used to help solve such optimization problems.

[0004] The subject matter claimed herein is not limited to embodiments that solve any shortcomings or that operate only in environments such as those described above. Rather, this background is provided only to illustrate one example technology area where some embodiments described herein may be practiced. Summary of the Invention

[0005] According to an aspect of an embodiment, the operations may include obtaining a first fixed temperature of a replica exchange Markov Chain Monte Carlo (MCMC) process used to solve an optimization problem associated with the system, obtaining a second fixed temperature of the replica exchange MCMC process, and obtaining a plurality of replicas of the system to be used in the replica exchange MCMC process. The operations may further include assigning to each replica of the plurality of replicas a different corresponding temperature of a set of temperatures of the replica exchange MCMC process. The set of temperatures may include the first fixed temperature, the second fixed temperature, and a plurality of variable temperatures numerically ordered between the first fixed temperature and the second fixed temperature. The operations may further include obtaining a target swap acceptance probability for swapping between replicas corresponding to adjacently ordered variable temperatures of the plurality of variable temperatures during the replica exchange MCMC process. Additionally, the operations may include determining a respective average swap acceptance probability for one or more respective adjacent pairs of the plurality of variable temperatures. Each respective swap acceptance probability may be for swapping between replicas assigned to one or more respective adjacent pairs. Additionally, the operations may include adjusting one or more of the variable temperatures during the replica exchange MCMC process based on a relationship between a target swap acceptance probability and each of the one or more of the respective swap acceptance probabilities while maintaining the first fixed temperature and the second fixed temperature. Additionally, the operations may include adjusting a number of replicas used during the MCMC process during the replica exchange MCMC process. The adjustment of the number of replicas may be based on a difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature. The operations may further include identifying a particular state of the system as a solution to an optimization problem associated with the system based on an output of the replica exchange MCMC process.

[0006] The object and advantages of the embodiments will be realized and attained at least by the elements, features, and combinations particularly pointed out in the claims.

[0007] Both the foregoing general description and the following detailed description are provided by way of example and are exemplary, and are not limitations of the invention as claimed. [Brief description of the drawings]

[0008] Example embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings. [Figure 1A] FIG. 2 illustrates an example environment configured to perform a replica exchange process. [Figure 1B] 1 illustrates different representations of an example state based on different temperatures. [Diagram 2] 1 illustrates a block diagram of an exemplary computing system configured to perform a replica exchange process. [Diagram 3] 1 shows a flow chart of an exemplary method for adjusting temperature during a replica exchange process. [Figure 4] 1 shows a flowchart of an exemplary method for adjusting the number of replicas during a replica exchange process. [Diagram 5] 1 illustrates a flowchart of an exemplary method for performing a replica exchange process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Combinatorial optimization problems may include a class of optimization problems that may be used to determine a maximum or minimum of a system. For example, combinatorial optimization may be used to minimize the number of vias in a circuit layout design, maximize stock returns, improve airline routing and scheduling, and configure wireless sensor networks, among other applications. Additionally, simulated annealing may obtain solutions to optimization problems using a heuristic approach. Thus, annealing systems may be useful in aiding in the solution of combinatorial optimization problems, or any other applicable type of optimization problem.

[0010] Additionally, in some examples, a replica exchange Markov Chain Monte Carlo (MCMC) process (the "replica exchange process" is also referred to as a parallel tempering process) may be used as a simulated annealing technique to determine a maximum or minimum value of a cost function as a solution to the optimization function. In accordance with one or more embodiments of the present disclosure, as described in more detail below, operations may be performed that may improve the efficiency of performing replica exchange, which may also improve the efficiency of solving the optimization problem. The improved efficiency may improve such computing systems by reducing the amount of processing that may be performed by a computing system configured to solve an optimization problem. The improved efficiency may also reduce the amount of time used by a computing system in solving an optimization problem using the replica exchange technique.

[0011] Embodiments of the present disclosure will now be described with reference to the accompanying drawings.

[0012] 1A is a diagram depicting an example environment 100 configured to perform a replica exchange process, arranged in accordance with at least one embodiment described in this disclosure. The environment 100 may include a replica exchange engine 102 ("engine 102") configured to output a system state 104, which may be a solution to an optimization problem associated with a system 106.

[0013] System 106 may include any suitable representation of an optimization problem that may be solved. For example, in some embodiments, system 106 may include a state vector X, which may include a set of variables, each of which may represent a characteristic related to the optimization problem. Thus, state vector X may represent different states of system 106. For example, a first state vector X1, each of which has variables having a first value, may represent a first state of system 106, and a second state vector X2, each of which has variables having a second value, may represent a second state of system 106. In these or other embodiments, the difference between state vectors X1 and X2 may be anywhere from only one corresponding variable in both X1 and X2 having a different value to every variable in X1 and X2 having a different value.

[0014] As an example, system 106 may be a neural network that may include any suitable number of nodes (also referred to as "neurons"). In these or other embodiments, a state vector X of system 106 may represent the state of each of the neurons of the neural network. For example, each neuron may be a bit that may have a value of "0" or "1", and the state vector X may include a "1" value or a "0" value for each neuron of the neural network. In these or other embodiments, the neural network may be configured to solve one or more different types of optimization problems in any suitable manner.

[0015] Additionally or alternatively, system 106 may include an Ising model that is mapped to an optimization problem to represent the Ising energy of the optimization problem corresponding to system 106. For example, the Ising energy may be represented by the following equation (1):

number

[0016] Engine 102 may include code and routines configured to enable a computing system to perform one or more of the operations described therein. Additionally or alternatively, engine 102 may be implemented using hardware including any number of processors, microprocessors (e.g., performing or controlling the execution of one or more operations), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or any suitable combination of two or more of these.

[0017] Additionally or alternatively, engine 102 may be implemented using a combination of hardware and software. In this disclosure, operations described as being performed by engine 102 may include operations that engine 102 may direct a corresponding system to perform. In some embodiments, engine 102 may be included in or be part of an annealing system (e.g., a digital annealing system or a quantum annealing system).

[0018] The engine 102 may be configured to perform a replica exchange MCMC process ("replica exchange" or "parallel tempering") on the system 106. For example, the engine 102 may be configured to perform a replica exchange to find a state vector Xmin that may minimize the Ising energy of the system 106. As another example, the engine 102 may be configured to perform a replica exchange to find a state vector Xmax that may maximize the Ising energy of the system 106.

[0019] Replica exchange may involve running M copies of system 106 simultaneously, but with different scaling factors that affect whether changes to the system occur between runs of the copies of system 106. For example, "running" a copy of system 106 may include performing a set of multiple iterations, where each iteration updates the current state vector "X i ” is generated randomly (e.g., via a stochastic process) and the proposal sample “X i+1 ". Furthermore, in each iteration, i X i+1 It may be determined whether to replace the first and second copies of the system 106 with a first scaling factor. The decision to replace may be a function of the scaling factor. Thus, running the system 106 with different scaling factors may lead to different results because the replacement decision may change. For example, a first scaling factor set to a first value may result in replacement most of the time, and a second scaling factor set to a second value may result in replacement less of the time. Thus, running a first copy of the system 106 with a first scaling factor may produce significantly different results compared to running a second copy of the system 106 with a second scaling factor.

[0020] As an example, in some embodiments, in a type of problem that can be described as a set of interacting particles, the system 106 is in a particular state X i The probability of being present in can be given as the Boltzmann distribution function (2) as follows:

number

[0021] Furthermore, in this example, the proposed sample X i+1 The transition probability T for the probability that is accepted can be expressed as follows in equation (3):

number

[0022] Combining equation (3) and equation (2) can yield the following equation (4):

number

[0023] In equation (4), ΔE = E(X i+1 )-E(X i ) and the Boltzmann constant k B Furthermore, when the transition probability T approaches "1", the proposed sample X i+1 The probability of acceptance of approaches 100%. Furthermore, as shown in equation (4), the larger the value of the scaling factor T, the

number

[0024] In this disclosure, the term "temperature" is used as a synonym for "scaling factor" due to the common use of the term "temperature" as a synonym for scaling factor in the fields of simulated and digital annealing. In traditional annealing and parallel tempering, physical systems may be run at different actual temperatures to solve a particular problem, and the term "temperature" is carried over to approaches that use the same principles for solving optimization problems of other types of "systems". Thus, the use of the term "temperature" in this disclosure is used for consistency in the use of the term in the fields of simulated and digital annealing, even though it does not necessarily refer to an actual physical temperature. Instead, the term "temperature" may refer to any factor that may affect the Ising energy of a system in a way that may affect the probability of transition from one state to another, which may be used as a basis for accepting or rejecting a proposed state of the system as described above. Similarly, in this disclosure, references to the Ising energy of a system (e.g., system 106) are used more broadly as a representation of the state of the system, not necessarily the actual energy of the system. For example, for a system, an Ising energy value may be returned, which may represent the state of the system, such as the cost of the system, the profit of the system, etc. As a further example, if the cost is being analyzed, the optimization problem may be such as to minimize the Ising energy. Conversely, if the profit is being analyzed, the optimization problem may be such as to maximize the Ising energy.

[0025] As described above, the engine 102 may be configured to run M replicas of the system 106, each at a different temperature. The engine 102 may thus be configured to take the M replicas of the system 106 and assign each of the replicas to one of the M different temperatures of the set of temperatures. In some embodiments, each replica of the system may be or may include the state vector X of the system 106. In some embodiments, the initial state vector of each replica may be the same. Additionally or alternatively, the engine 102 may be configured to randomly generate the initial state vector of each replica, such that the state vectors of two or more replicas may be different. The set of temperatures is often referred to as a temperature ladder. The temperatures of the temperature ladder may be numerically ordered between low and high temperatures. Thus, assigning the replicas to the respective temperatures may be such that the M different replicas are ordered according to the ordering of the respective temperatures to which the replicas are assigned.

[0026] Due to the scaling effect of temperature on the state of the system 106, replicas at higher temperatures may sample a larger portion of the state, while replicas at lower temperatures take multiple samples in a localized region of the state. Replica swapping may include swapping replicas corresponding to adjacent temperatures, whereby a more representative view of the state may be achieved by swapping replicas compared to standard MCMC processes. For example, a first replica assigned to a first temperature may be swapped with a second replica assigned to a second temperature, whereby the first replica may be assigned to the second temperature and the second replica may be assigned to the first temperature. The first temperature and the second temperature may be adjacent to each other in the temperature ladder. Furthermore, the first replica and the second replica may be considered adjacent replicas by being assigned to adjacent temperatures (e.g., the first temperature and the second temperature).

[0027] In some embodiments, after each certain number of MCMC iterations during the replica exchange process, engine 102 can make a decision regarding the swap configuration (or, equivalently, swap temperature) of two or more replicas. In some embodiments, the decision may be made with respect to neighboring replicas. Additionally or alternatively, the decision may be made with respect to each possible set of neighboring replicas.

[0028] In these or other embodiments, the decision to swap can be based on a swap acceptance probability (SAP), which in some embodiments may be determined based on the temperature and energy differences between adjacent replicas. For example, in some embodiments, the SAP may be determined using the Metropolis solution according to Equation (5) below.

number

[0029] Equation (5) shows that when swapping, replicas with lower energies are more likely to drop to lower temperatures, and vice versa, because E i <E j and T i >T j , SAP is equal to 1. Furthermore, if a particular replica at a low temperature gets stuck in a local minimum due to a non-zero SAP among all temperatures, it can eventually rise to a higher temperature during the replica exchange process if enough determination is made, and thus escape from that local minimum (since the depth of the local minimum is smaller at higher temperatures).

[0030] The temperature selection can have a significant impact on the efficiency of sampling and the time to reach a solution. For example, at lower temperatures, more details about the state may be identified due to lower transition probabilities, as shown in equation (4) above. However, the additional details may come at the cost of running multiple iterations to obtain such information and break out of the local minimum. Conversely, at higher temperatures, details of the state may be masked due to higher transition probabilities. Thus, the selection of a temperature range that includes high and low temperatures, as well as the number of temperatures and the spacing between temperatures, can affect the efficiency and performance accuracy of the replica exchange process.

[0031] As an example, Figure IB shows different representations of an exemplary state 150 based on different temperatures. In particular, Figure IB shows a first representation 152 at a first temperature T1 of "1", a second representation 154 at a second temperature T2 of "2", a third representation 156 at a third temperature T3 of 4, and a fourth representation 158 at a fourth temperature T4 of 16. As shown by a comparison between the different representations, in Figure IB, the detail of the state may be reduced as the temperature increases, such that the fourth representation 158 may be significantly "flatter" than the first representation 152.

[0032] In some embodiments, the engine 102 can be configured to obtain a fixed temperature 108, which may include a first fixed temperature and a second fixed temperature. In some embodiments, the engine 102 can obtain the fixed temperature 108 as an input. Additionally or alternatively, the engine 102 can be configured to determine the fixed temperature 108. The first fixed temperature and the second fixed temperature can define a range of a temperature ladder, whereby they can be the highest and lowest temperatures of the temperature ladder (a "fixed high" and a "fixed low").

[0033] In some embodiments, the fixed low temperature can be set such that the effects of temperature scaling do not mask details of the state. For example, the fixed low temperature may be based on a relationship between transition probabilities and the fixed low temperature and how such a relationship affects details about the state that may be obtained at the fixed low temperature. The fixed low temperature may be set according to any suitable approach and may depend on the particular formulation of the Ising energy function used for the optimization problem. Additionally, the fixed low temperature may be based on an error tolerance that may be associated with a particular implementation.

[0034] The fixed high temperature can be set high enough that the replicas are not trapped in a local minimum and most of the transition proposals are accepted, which means that the state representation at the fixed high temperature is approximately flat. In these or other embodiments, the fixed high temperature may be further based on the relationship between the transition probability and the fixed high temperature and how such relationship affects the details about the state and the possibility of being trapped in a local minimum. For example, the fixed high temperature may be selected until the transition probability reaches a certain level (e.g., 80% or more). Based on equation (4), the probability of the transition is changed by a scaling factor. By increasing the maximum temperature and recording the transition probability, a value of the maximum temperature according to a specified target transition probability for the maximum temperature may be obtained.

[0035] During replica exchange, if the SAP is uniform across all adjacent temperatures, the replicas can perform a random walk in temperature space where each replica spends the same amount of time at each temperature, since at any time, any replica at any temperature has an equal probability of transitioning to a higher or lower temperature. Furthermore, the selection of temperature intervals in the temperature ladder to produce a uniform SAP is commonly used as a mechanism for optimizing the replica exchange process.

[0036] According to one or more embodiments of the present disclosure, the engine 102 can be configured to automatically adjust the temperatures of the temperature ladder between the fixed temperatures 108 to achieve a uniform SAP between all adjacent temperatures. In this disclosure, the temperatures between the fixed temperatures 108 and that can be adjusted may be referred to as "variable temperatures." In these and other embodiments, the engine 102 can be configured to add or remove replicas and corresponding temperatures during the replica exchange process to help avoid running more replicas than necessary while also helping to run enough replicas to converge to a uniform SAP. Thus, adding and / or removing replicas can improve resource execution time and efficiency by running operations on enough replicas to obtain a sufficiently acceptable solution while also avoiding running operations on more replicas than are necessary to obtain such a solution.

[0037] In some embodiments, engine 102 can be configured to obtain target SAP value 110 as the uniform SAP to be achieved for all adjacent temperatures. Target SAP value 110 may be determined or based on one or more factors, such as experimental results on the system of interest or systems having a similar structure. Additionally or alternatively, target SAP value may be based on a balance between the accuracy of the resulting solution and the amount of resources that may be used to determine the solution, which may be empirically determined. Thus, target SAP value may vary according to different implementations.

[0038] The engine 102 may be configured to adjust one or more of the variable temperatures during the replica exchange process to adjust the SAP between adjacent pairs. This adjustment may be based on a target SAP to attempt to have the SAP between adjacent pairs reach the target SAP or within a threshold of the target SAP. However, in some embodiments, as described in further detail below, the SAP between a first fixed temperature and its adjacent variable temperature, or between a second fixed temperature and its corresponding adjacent variable temperature (e.g., between a fixed high temperature and the highest variable temperature in a minimization scenario, or between a fixed low temperature and the lowest variable temperature in a maximization scenario) may not be adjusted to allow for replica insertion or removal. Note that a particular temperature and corresponding replica may be part of more than one adjacent pair. For example, in an ordered set of temperatures [T1, T2, T3], a first adjacent pair may include temperatures T1 and T2, and a second adjacent pair may include temperatures T2 and T3. Thus, the same temperature (and corresponding replica) may be included in two different adjacent pairs. In some embodiments, the temperature adjustment may be performed according to one or more operations described below with respect to FIG. 3.

[0039] In these or other embodiments, engine 102 can be configured to adjust the number of replicas (and, consequently, the number of temperatures) that may be executed during the replica exchange process. In some embodiments, as described above and in further detail below, the adjustment of the number of replicas may be based on the temperature difference between one of the fixed temperatures 108 and its adjacent variable temperature. For example, in a minimization problem, the adjustment may be based on the temperature difference between a fixed high temperature and the highest variable temperature, and in a maximization problem, the adjustment may be based on the temperature difference between a fixed low temperature and the lowest variable temperature. In some embodiments, the adjustment of the number of replicas may be related to temperature adjustment, as described in further detail below. Additionally or alternatively, the adjustment of the number of replicas may be performed according to one or more operations described below with respect to FIG. 4.

[0040] The engine 102 may be configured to perform the replica exchange, including temperature adjustments and / or replica number adjustments, until the replica exchange process converges and the swap acceptance probability for each of the temperature pairs is equal to or within a particular threshold of the target SAP. Additionally or alternatively, the engine 102 may output a system state 104 in response to the convergence of the replica exchange process. The system state 104 may indicate a state of the system 106 corresponding to the convergence of the replica exchange process and may be a solution to an optimization problem associated with the system 106. For example, the system state 104 may include one or more state vectors X in which the Ising energy may be minimized or maximized depending on whether the optimization problem is a maximization problem or a minimization problem.

[0041] Modifications, additions, or omissions may be made to Figure 1A without departing from the scope of the present disclosure. For example, environment 100 may include more or fewer elements than illustrated and described in this disclosure. Additionally, the particular arrangement, association, or inclusion of elements in a particular device or system may vary depending on the particular implementation.

[0042] 2 illustrates a block diagram of an exemplary computing system 202 configured to perform a replica exchange process in accordance with at least one embodiment of the present disclosure. The computing system 202 may be configured to implement or direct one or more operations associated with the replica exchange engine 102 of FIG. 1A in some embodiments. In some embodiments, the computing system 202 may be included in or form part of an annealing system. The computing system 202 may include a processor 250, a memory 252, and a data storage device 254. The processor 250, the memory 252, and the data storage device 254 may be communicatively coupled to

[0043] In general, the processor 250 may include any suitable special purpose or general purpose computer, computing entity, or processing device, including various computer hardware or software modules, and may be configured to execute instructions stored on any applicable computer-readable storage medium. For example, the processor 250 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or any other digital or analog circuitry configured to interpret and / or execute program instructions and / or process data. Although shown as a single processor in FIG. 2, the processor 250 may include any number of processors configured to individually or collectively perform or direct the execution of any number of operations described in this disclosure. Additionally, one or more of the processors may reside in one or more different electronic devices, such as different servers.

[0044] In some embodiments, processor 250 may be configured to interpret and / or execute program instructions and / or process data stored in memory 252, data storage device 254, or memory 252 and data storage device 254. In some embodiments, processor 250 may retrieve program instructions from data storage device 254 and load the program instructions into memory 252. After the program instructions are loaded into memory 252, processor 250 may execute the program instructions.

[0045] Memory 252 and data storage 254 may include computer-readable storage media that carry or have stored computer-executable instructions or data structures. Such computer-readable storage media may include any available non-transitory media that can be accessed by a general-purpose or special-purpose computer, such as processor 250. By way of example and not limitation, such computer-readable storage media may include tangible or non-transitory computer-readable storage media including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid-state memory devices), or any other storage medium that can be used to carry or store specific program code in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer. In these and other embodiments, the term "non-transitory" as described in this disclosure should be construed to exclude only those types of transitory media found to be outside the scope of patentable subject matter in In re Nuijten, 500 F.3d 1346 (Fed. Cir. 2007). Combinations of the above may also be included within the scope of computer-readable media.

[0046] Combinations of the above may also be included within the scope of computer-readable storage media.Computer-executable instructions may include, for example, instructions and data configured to cause processor 250 to perform a certain operation or group of operations.

[0047] Modifications, additions, or omissions may be made to computing system 202 without departing from the scope of the present disclosure. For example, in some embodiments, computing system 202 may include any number of other components that may not be explicitly illustrated or described. Furthermore, references in this disclosure to hardware or operations performed by hardware may refer to any applicable operation, configuration, or combination of one or more of the elements of computing system 202.

[0048] 3 illustrates a flowchart of an example method 300 of adjusting temperature during a replica exchange process, according to at least one embodiment described in this disclosure. Method 300 may be performed by any suitable system, apparatus, or device. For example, engine 102 of FIG. 1 or computing system 202 of FIG. 2 may perform one or more of the operations associated with method 300. Although shown in discrete blocks, the steps and operations associated with one or more of the blocks of method 300 may be divided into further blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0049] In block 302, N of the MCMC replica exchange process is used to solve the optimization problem of the system. sweep N MCMC iterations may be performed. sweep can be set to any value. In some embodiments, N sweepmay be set to the smallest possible value such that the execution time for the hardware to perform the swap is masked by the execution time of the MCMC iterations. The MCMC iterations may be performed for each of M replicas of the system, each of which is assigned to, and therefore corresponds to, one of M different temperatures. Each replica may include a state vector X of the system. In some embodiments, the temperatures may be numerically ordered as a temperature ladder having values ​​between a fixed low temperature and a fixed high temperature. The temperatures in the temperature ladder that are between the fixed high temperature and the fixed low temperature may be adjusted according to method 300, as described in more detail below, and may be referred to as "variable temperatures."

[0050] In these or other embodiments, N sweep For each of the MCMC iterations, a proposed sample (e.g., a proposed state vector) for each respective replica may be accepted or rejected as a replacement for the respective replica. In some embodiments, the acceptance or rejection of a proposed sample may be based on a transition probability T, as described above with respect to equation (4). In some embodiments, the generation and acceptance or rejection of two or more proposed replicas may be performed in parallel. Additionally or alternatively, the generation and acceptance or rejection of proposed replicas may be performed in parallel for all replicas.

[0051] N sweep In response to MCMC iterations being performed, method 300 may proceed to block 304. At block 304, a swap attempt operation may be performed for each adjacent replica pair and corresponding temperature. In some embodiments, a swap between each adjacent pair may be accepted or rejected based on a SAP decision made using equation (5) above.

[0052] In block 306, for each adjacent pair, adjust It may be determined whether N swap attempts have been performed. adjustis a value large enough to reduce sampling noise, and may be determined empirically in some embodiments. By way of example, 50 to 100 may be an appropriate number of samples in many cases. adjust In response to N swap attempts not yet being performed, the method 300 may return to block 302. In contrast, adjust In response to swap attempts being performed, the method 300 may proceed to block 308.

[0053] At block 308, an average SAP may be determined for each of one or more of the adjacent temperature pairs. The average SAP for each adjacent temperature pair may be N adjust The average SAP of those temperatures over the N swap attempts may be shown. For example, adjust If N is equal to '1', the average SAP of each pair may be equal to the SAP of each pair's swap attempt. adjust If is equal to "100", then the average SAP of each pair may be equal to the average SAP of the pair across 100 different swap trials.

[0054] In some embodiments, the average SAP can be determined using standard averaging techniques. For example, the average SAP for each respective adjacent pair may be determined by determining the SAP for each swap attempt for each respective adjacent pair (e.g., via equation (5)), adding each determined SAP together, and dividing by the number of swap attempts.

[0055] Additionally or alternatively, in some embodiments, the average SAP of each respective adjacent pair is N adjust The swap acceptance rate may be determined based on a swap acceptance rate that may indicate the rate at which a swap is accepted for each adjacent pair in the N swap attempts. adjust The swap acceptance rate over swap attempts is N adjustThe swap acceptance rate for each respective adjacent pair may be based on an average of the SAPs used to decide whether to swap over swap attempts. Thus, the swap acceptance rate for each respective adjacent pair may be equal to or approximately equal to the average SAP for the respective adjacent pair.

[0056] The swap acceptance rate for a particular adjacent temperature pair may be equal to the number of accepted swaps for that adjacent pair in the number of attempted swaps. For example, the swap acceptance rate for a particular pair “i” may be expressed in equation (6) as follows:

number

[0057] In some embodiments, the average SAP may be determined for every adjacent pair except for adjacent pairs that include a fixed high temperature or a fixed low temperature. For example, in a minimization scenario, the average SAP of the adjacent pair of a fixed high temperature and a highest variable temperature may not be determined. As another example, in a maximization scenario, the average SAP of the adjacent pair of a fixed low temperature and a lowest variable temperature may not be determined.

[0058] At block 310, the average SAP for each of the one or more adjacent pairs may be compared to a target swap acceptance probability (target SAP). In response to the average SAP being outside the target SAP threshold for the respective adjacent pair, method 300 may proceed to block 312. In response to the average SAP being within the target SAP threshold for the respective adjacent pair, method 300 may proceed to block 314.

[0059] The threshold may be based on multiple factors, including a target error margin for the solution to the corresponding optimization problem, a target processing efficiency, etc. For example, the threshold may be a percentage variation from a target SAP (e.g., within a particular percentage of the target SAP), where a larger percentage variation may correspond to a higher error margin but better processing efficiency. In contrast, a smaller percentage variation may correspond to a lower error margin but less efficient processing. Thus, the threshold may have a value within a relatively broad range, such as anywhere between 0.1% and 20% of the target SAP.

[0060] At block 312, a temperature adjustment may be made for the temperature difference between each adjacent temperature pair having an average SAP that is not within a threshold of the target SAP. The temperature adjustment may be made such that the swap acceptance probability of each adjacent pair may be moved toward the target SAP.

[0061] For example, as shown by equation (5), an increase in the temperature difference between a particular adjacent pair may result in a decrease in the swap acceptance probability for the particular adjacent pair. Thus, if the average SAP is greater than the target SAP and outside the threshold, the temperature difference may be decreased. Conversely, as further shown by equation (5), a decrease in the temperature difference between a particular adjacent pair may result in an increase in the swap acceptance probability for the particular adjacent pair. Thus, if the average SAP is less than the target SAP and outside the threshold, the temperature difference may be increased.

[0062] The temperature adjustment may hinder the convergence of the replica exchange process. Therefore, the magnitude of the temperature adjustment may be reduced over time to help ensure that convergence of the replica exchange process can occur. For example, a damping factor α may be applied to the temperature adjustment, which may be a function of the number of attempted swaps. In some embodiments, the damping factor may be determined according to Equation (7) as follows:

number

[0063] Further by way of example, the temperature adjustments for each adjacent temperature pair may, in some embodiments, follow equation (8) below:

number

[0064] A temperature adjustment between one adjacent pair may affect the temperature difference between adjacent pairs adjacent to the adjacent pair. For example, a temperature adjustment may be made to a first adjacent pair including a first temperature T1 and a second temperature T2. The first adjacent pair may be considered adjacent to a second adjacent pair including T2 and a third temperature T3. In response to the SAR of the first adjacent pair not being within the target SAP threshold, an adjustment may be made to T2. However, an adjustment to T2 may also change the swap acceptance probability between the second adjacent pair of T2 and T3 by changing the difference between T2 and T3. Thus, in some embodiments, in response to making a temperature adjustment for a particular adjacent temperature pair, an adjustment may be made to one or more additional temperatures. The adjustment may be such that the temperature difference between other adjacent temperature pairs may be maintained.

[0065] For example, in response to a temperature adjustment that increases a particular variable temperature by ΔT, all of the variable temperatures higher than the particular variable temperature may also be increased by ΔT. Similarly, in response to a temperature adjustment that decreases a particular variable temperature by ΔT, all of the variable temperatures lower than the particular variable temperature may also be decreased by ΔT.

[0066] Additionally or alternatively, one or more of the adjustments to one or more of the other variable temperatures may be less than ΔT in some instances. For example, an increase by ΔT to a first variable temperature may be such that the first variable temperature is now greater than a second variable temperature that is higher in the temperature ladder than the first variable temperature. In some embodiments, to avoid such a scenario, an increase in the first variable temperature may be such that the difference between the first and second variable temperatures may be reduced by a certain percentage. For example, the temperature difference between the first and second variable temperatures may be reduced by 5%, so that the temperature difference may be adjusted to be 95% compared to its original state. A similar approach may also be used in scenarios where the temperature is being reduced. In these and other embodiments, the changes may be made iteratively, starting with the adjusted variable temperatures of each adjacent pair and moving sequentially up the temperature ladder.

[0067] The fixed temperature may not be adjusted as part of the temperature adjustment process, which may affect the temperature difference between the fixed high temperature and the highest variable temperature or between the fixed low temperature and the lowest variable temperature, depending on whether the replica exchange process is performed as a minimization process or a maximization process. For example, in a minimization process, an increase in the temperature difference between a pair of adjacent variable temperatures and a subsequent adjustment to one or more higher temperatures may result in a decrease in the temperature difference between the fixed high temperature and the highest variable temperature. Similarly, in a minimization process, a decrease in the temperature difference between a pair of adjacent variable temperatures and a subsequent adjustment to one or more higher temperatures may result in an increase in the temperature difference between the fixed high temperature and the highest variable temperature. The opposite may be true for a maximization process. As described in more detail below with respect to FIG. 4, the temperature difference associated with the fixed temperature may be used to determine whether to add or remove a replica.

[0068] For pairs of adjacent temperatures that are within a threshold of the target SAP, the temperature difference between such adjacent pairs may be maintained in block 314. "Maintaining" this difference may include adjusting the difference by a particular percentage during the temperature adjustment of block 312, for the scenario described above regarding avoiding the lower temperature of a particular adjacent temperature pair becoming the higher temperature of that particular pair.

[0069] Modifications, additions, or omissions may be made to method 300 without departing from the scope of the present disclosure. For example, the operations of method 300 may be performed in a different order. Additionally, or alternatively, two or more operations may be performed simultaneously. Moreover, the outlined operations and actions are provided merely as examples, and some of the operations and actions may be optional, combined into fewer operations and actions, or expanded into additional operations and actions without detracting from the essence of the disclosed embodiments.

[0070] For example, in some embodiments, after blocks 312 and 314, method 300 may return to block 302 to repeat one or more of the actions of method 300. Additionally or alternatively, this process may be repeated until the average SAP between all adjacent temperatures is within a threshold of the target SAP, which may indicate convergence of the replica replacement process. In these or other embodiments, a solution to an optimization problem associated with the system corresponding to the replica may be based on the state vector of the replica at convergence.

[0071] 4 illustrates a flowchart of an example method 400 of adjusting the number of replicas during a replica exchange process, according to at least one embodiment described in this disclosure. Method 400 may be performed by any suitable system, apparatus, or device. For example, engine 102 of FIG. 1 or computing system 202 of FIG. 2 may perform one or more of the operations associated with method 400. Although shown in discrete blocks, the steps and operations associated with one or more of the blocks of method 400 may be divided into further blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0072] As described above, the temperature difference associated with a fixed high or low temperature in the temperature ladder of the replica exchange process can be used to determine whether to add or remove a replica. If the replica exchange process is performed for maximization, the difference between the fixed low temperature and the lowest variable temperature may be used. Conversely, if the replica exchange process is performed for minimization, the difference between the fixed high temperature and the highest variable temperature may be used. The operation of method 400 is described for a minimization process for ease of explanation, but the same principles can be applied for a maximization process, just with a fixed low temperature and the lowest variable temperature.

[0073] In block 402, a temperature difference may be determined between the fixed high temperature and the highest variable temperature. The temperature difference determined in block 402 may be referred to as the "last temperature difference" to indicate that it is the last temperature difference in the temperature ladder. Additionally, the fixed high temperature and the highest variable temperature may be referred to as the "last adjacent pair." In some embodiments, the determination of the last temperature difference may be performed after each temperature adjustment iteration, as described with respect to method 300, or after a certain number of temperature adjustment iterations have been performed. A "temperature adjustment iteration" in this context may refer to an adjustment of the temperature difference between a particular adjacent temperature pair and a corresponding adjustment to other variable temperatures to help maintain the swap acceptance probability of those respective pairs. Additionally or alternatively, a "temperature adjustment iteration" in this context may refer to an adjustment of each Nth temperature difference between all adjacent pairs that are not within the target swap acceptance probability threshold. adjust It may also refer to performing a temperature adjustment after swap attempts.

[0074] At block 404, it may be determined whether the final temperature difference meets a temperature difference threshold. If the number of replicas is greater than necessary to achieve the target SAP, the temperature adjustments of method 300 may be such that the gap between the variable temperatures may be larger, which may result in the final temperature difference being reduced. When the final temperature difference becomes smaller, the replicas associated with the last adjacent pair may approach agreement with each other, such that the swap acceptance probability of the last adjacent pair may approach one. Thus, the temperature difference threshold may be set such that once the final difference becomes smaller than the temperature difference threshold, the highest variable temperature may be removed from the temperature ladder and its corresponding replica may also be removed.

[0075] In some embodiments, the temperature difference threshold may be based on the total temperature range between a fixed low temperature and a fixed high temperature. Additionally or alternatively, the temperature difference threshold may be based on the maximum number of replicas that may be used, and consequently, the maximum number of temperatures that may be included in the temperature ladder. The maximum number of replicas may in some embodiments be based on the constraints of the hardware used to implement the replica exchange process. Additionally or alternatively, the maximum number of replicas may be set based on processing efficiency and timing considerations. By way of example, in some embodiments, the temperature difference threshold Threshold r can be determined according to the fraction of the temperature range and the maximum number of replicas according to the following equation (9):

number

[0076] In response to the last temperature difference satisfying (e.g., being less than or equal to) the temperature difference threshold, method 400 may proceed from block 404 to block 406. At block 406, the highest variable temperature and its corresponding replica may be removed. Removal of the highest variable temperature may result in the second-highest variable temperature becoming the highest variable temperature.

[0077] In response to the last temperature difference not meeting the temperature difference threshold (e.g., being equal to or greater than the temperature difference threshold), method 400 may proceed from block 404 to block 408. At block 408, a swap acceptance probability may be determined based on the last temperature difference. In some embodiments, the swap acceptance probability may be determined using a fixed high temperature and a maximum variable temperature according to equation (5) described above with respect to FIG. 1A. The swap acceptance probability determined at block 408 may be referred to as the “last SAP.”

[0078] At block 410, it may be determined whether the last SAP is within a target SAP threshold, such as those described above with respect to method 300. If the number of replicas is too low, the temperature adjustments of method 300 may be such that the gap between the variable temperatures may become smaller, which may result in the last temperature difference increasing, and consequently, the last SAP may decrease to a point where it may no longer be within the target SAP threshold. Thus, in response to the last SAP not being within the target SAP (which may be based on the last temperature difference), method 400 may proceed from block 410 to block 412, where a replica may be added.

[0079] For example, a new variable temperature may be added between what was previously the highest variable temperature and the fixed high temperature. The new variable temperature may then become the highest variable temperature. Furthermore, a new replica may be assigned to the new variable temperature, thereby adding a new replica to the replica exchange process.

[0080] Returning to block 410, in response to the last SAP being within the target SAP, the method 400 may proceed from block 410 to block 414. At block 414, the number of replicas may be maintained.

[0081] Modifications, additions, or omissions may be made to method 400 without departing from the scope of the present disclosure. For example, the operations of method 400 may be performed in a different order. Additionally, or alternatively, two or more operations may be performed simultaneously. Moreover, the outlined operations and actions are provided merely as examples, and some of the operations and actions may be optional, combined into fewer operations and actions, or expanded into additional operations and actions without detracting from the essence of the disclosed embodiments.

[0082] For example, in some embodiments, after blocks 412 or 414, method 400 may return to block 402 to repeat one or more of the operations of method 400. Additionally or alternatively, this process may be repeated any number of times until method 300 is completed. Additionally or alternatively, the decisions made at blocks 404 and 410 to add or remove replicas may be performed after the same number of temperature tuning iterations have been performed or after different numbers of temperature tuning iterations have been performed.

[0083] Additionally, in some embodiments, the initial number of replicas may be set as the maximum number of replicas that can be implemented according to the particular hardware used to execute the replica exchange process. In these or other embodiments, operations associated with adding replicas may be omitted.

[0084] 5 illustrates a flowchart of an example method 500 of performing a replica exchange process in accordance with at least one embodiment described in this disclosure. Method 400 may be performed by any suitable system, apparatus, or device. For example, engine 102 of FIG. 1 or computing system 202 of FIG. 2 may perform one or more of the operations associated with method 400. Although shown in discrete blocks, the steps and operations associated with one or more of the blocks of method 400 may be divided into further blocks, combined into fewer blocks, or eliminated depending on the particular implementation.

[0085] At block 502, a first fixed temperature of a replica exchange Markov Chain Monte Carlo (MCMC) process may be obtained. The replica exchange process may be used to solve an optimization problem associated with the system. The first fixed temperature may be a fixed high temperature or a fixed low temperature. In some embodiments, the first fixed temperature may be obtained as an input or may be determined. Additionally or alternatively, the first fixed temperature may be determined according to the description of determining a fixed high temperature or determining a fixed low temperature as described above with respect to FIG. 1A.

[0086] At block 504, a second fixed temperature of the replica exchange process may be obtained. The second fixed temperature may be a fixed high temperature or a fixed low temperature. For example, the second fixed temperature may be a fixed low temperature if the first fixed temperature is a fixed high temperature, or vice versa. In some embodiments, the second fixed temperature may be obtained as an input or may be determined. Additionally or alternatively, the second fixed temperature may be determined according to the description of determining a fixed high temperature or determining a fixed low temperature as described above with respect to FIG. 1A.

[0087] At block 506, a number of replicas of the system to be used in the replica exchange process may be obtained. In some embodiments, the replicas may be received as an input. Additionally or alternatively, the system may be received as an input and replicas may be created from the system. The replicas may be obtained and / or constructed, in some embodiments, as described above with respect to FIG. 1A. Additionally or alternatively, the number of initial replicas may be based on the maximum number of replicas that a computing system configured to perform the replica exchange process may be able to handle. In these or other embodiments, the number of initial replicas may be determined somewhat arbitrarily, as replica additions or removals as described above and performed in method 500 may automatically add or remove replicas to the number that may be needed, as described below.

[0088] At block 508, each replica of the plurality of replicas may be assigned a different corresponding temperature of the temperature set. The number of temperatures in the temperature set may be the same number of replicas. Furthermore, the temperature set may range between a first fixed temperature and a second fixed temperature. Furthermore, the temperatures between the first fixed temperature and the second fixed temperature may be variable such that they may be adjusted as described above. Furthermore, the variable temperatures may be numerically ordered between the first fixed temperature and the second fixed temperature. In some embodiments, the difference between the temperatures of the temperature set may be uniform upon initialization. Additionally or alternatively, the difference may vary. Initialization of the temperature set may occur at any suitable time between obtaining the fixed temperature and obtaining the replica.

[0089] At block 510, a target swap acceptance probability (target SAP) may be obtained. As detailed in the above description, the target SAP may be the probability of swapping between replicas corresponding to adjacently ordered variable temperatures of the plurality of variable temperatures during the replica exchange process. The target SAP may be obtained as an input or may be determined.

[0090] At block 512, a respective average SAP may be determined for one or more respective adjacent pairs of variable temperatures. Each respective average SAP may relate to swaps between replicas assigned to one or more respective adjacent pairs. In some embodiments, one or more operations of the method 300 of FIG. 3 may be performed in connection with determining the average SAP.

[0091] At block 514, one or more of the variable temperatures may be adjusted based on a relationship between the target swap acceptance probability and each of one or more of the respective average SAPs. The temperature adjustment may be performed during the replica exchange process, and the fixed first temperature and the fixed second temperature may be maintained, and the fixed first temperature and the fixed second temperature may not be adjusted. In some embodiments, one or more operations of the method 300 of FIG. 3 may be performed with respect to adjusting the temperatures.

[0092] At block 516, the number of replicas may be adjusted based on the difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature. In a minimization process, the second fixed temperature may be a high fixed temperature. In a maximization process, the second fixed temperature may be a low fixed temperature. The replica number adjustment may be performed during the replica exchange process. In some embodiments, one or more operations of the method 400 of FIG. 4 may be performed with respect to adjusting the temperature.

[0093] A particular state of the system may be identified as a solution to an optimization problem associated with the system at block 518. The particular state may include one or more state vectors of the replicas after the replica exchange process has converged.

[0094] Modifications, additions, or omissions may be made to method 500 without departing from the scope of the present disclosure. For example, the operations of method 500 may be performed in a different order. Additionally, or alternatively, two or more operations may be performed simultaneously. Moreover, the outlined operations and actions are provided merely as examples, and some of the operations and actions may be optional, combined into fewer operations and actions, or expanded into additional operations and actions without detracting from the essence of the disclosed embodiments.

[0095] One or more embodiments of the present disclosure relating to enhancing the replica exchange technique with automatic temperature and / or replica number adjustment may be implemented in any number of different scenarios. For example, the replica exchange technique detailed above may be integrated with a parallel MCMC trial and update technique in some embodiments. For example, the automatic temperature and / or replica number adjustment may be implemented in the context of a parallel MCMC trial and update technique ("parallel technique") applied to a neural network that may be used to solve an optimization problem. An example of implementing automatic temperature and / or replica number adjustment of replica exchange using a parallel technique may be found in the paper "Replica Exchange MCMC Hardware with Automatic Temperature Selection and Parallel Trial" by Dabiri et al., IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, available for online viewing from June 20, 2020, which is incorporated by reference in its entirety in this disclosure.

[0096] Additionally or alternatively, as described above, one or more embodiments of the present disclosure may be implemented through software programming. An example of pseudocode that may be used for a software implementation may also be found in the Dabiri et al. paper.

[0097] Additionally or alternatively, as discussed above, one or more embodiments of the present disclosure may be implemented in hardware, such as with an FPGA. For example, an exemplary FPGA implementation of the integration of automatic temperature and / or replica number adjustment of replica exchange using a parallel approach may also be found in Dabiri et al.

[0098] As used herein, the term "module" or "component" may refer to a specific hardware implementation configured to perform the operations of the module or component, and / or a software object or software routine that may be stored and / or executed on general-purpose hardware of a computing system (e.g., computer-readable medium, processing device, etc.). In some embodiments, different components, modules, engines, and services described herein may be realized as objects or processes (e.g., as separate threads) executing on a computing system. Although some of the systems and methods described herein are generally described as being implemented in software (stored and / or executed on general-purpose hardware), specific hardware implementations or combinations of software and specific hardware implementations are also possible and contemplated. In this description, a "computing entity" may be any computing system as previously defined herein, or a combination of any module or modulate operating on a computing system.

[0099] Terms used in this disclosure, particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "including" should be interpreted as "including but not limited to," etc.).

[0100] Furthermore, where a particular number of introduced claim recitations is intended, such intent is expressly set forth in the claim, and where such recitation is absent, such intent does not exist. For example, as an aid to understanding, the appended claims below may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation with the indefinite article "a" or "an" limits any particular claim that includes such introduced claim recitation to embodiments that include only one such recitation, even when the same claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" (e.g., "a" and / or "an" should be interpreted to mean "at least one" or "one or more"). The same applies to the use of definite articles used to introduce claim recitations.

[0101] Moreover, even when a particular number of introduced claim recitations is explicitly recited, one of ordinary skill in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the sole recitation of "two recitations" without other modifiers means at least two recitations, or more than two recitations). Furthermore, in instances where a definition similar to "at least one of A, B, and C, etc." or "one or more of A, B, and C, etc." or "at least one of A, B, or C, etc." is used, such a structure is generally intended to include A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, etc. Furthermore, use of the term "and / or" is intended to be interpreted in this manner.

[0102] Additionally, any disjunctive word or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to contemplate the possibility of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B" or "A and B," even if the term "and / or" is used elsewhere.

[0103] All examples and conditional language described in this disclosure are intended for educational purposes to aid the reader in understanding the concepts contributed by the inventor to advance the art, and should not be considered as being limited to the examples and conditions specifically described in this manner. Although embodiments of the present disclosure have been described in detail, various changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present disclosure.

[0104] The following notes are provided regarding the above embodiment. (Appendix 1) obtaining a first fixed temperature of a replica exchange Markov chain Monte Carlo (MCMC) process used to solve an optimization problem associated with the system; obtaining a second fixed temperature of the replica exchange MCMC process; obtaining a number of replicas of the system to be used in the replica-exchange MCMC process; assigning to each replica of the plurality of replicas a different corresponding temperature of a set of temperatures of the replica exchange MCMC process, the set of temperatures including the first fixed temperature, the second fixed temperature, and a plurality of variable temperatures numerically ordered between the first fixed temperature and the second fixed temperature; obtaining a target swap acceptance probability for swapping between replicas corresponding to adjacently ordered variable temperatures among the plurality of variable temperatures during the replica swap MCMC process; determining a respective average swap acceptance probability for one or more respective adjacent pairs of the plurality of variable temperatures, each respective swap acceptance probability being for a swap between replicas assigned to the one or more respective adjacent pairs; adjusting one or more of the variable temperatures based on a relationship between the target swap acceptance probability and each of one or more of the respective swap acceptance probabilities while maintaining the first fixed temperature and the second fixed temperature during the replica exchange MCMC process; identifying a particular state of the system as a solution to the optimization problem associated with the system based on an output of the replica-exchange MCMC process; The method includes: (Appendix 2) 2. The method of claim 1, further comprising: during the replica exchange MCMC process, adjusting a number of replicas used during the MCMC process, wherein the adjustment of the number of replicas is based on a difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature. (Appendix 3) 3. The method of claim 2, wherein adjusting the number of replicas includes removing the particular variable temperature and removing a particular replica corresponding to the particular variable temperature. (Appendix 4) 3. The method of claim 2, wherein adjusting the number of replicas includes inserting a new variable temperature between the second fixed temperature and the particular variable temperature, and assigning a new replica to the new variable temperature. (Appendix 5) 3. The method of claim 2, wherein adjusting the number of replicas based on the difference between the second fixed temperature and the particular variable temperature is in response to the difference between the second fixed temperature and the particular variable temperature satisfying a temperature difference threshold. (Appendix 6) Adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature includes: determining a particular swap acceptance probability for a particular swap between a first replica corresponding to the particular variable temperature and a second replica corresponding to the second fixed temperature based on a difference between the second fixed temperature and the particular variable temperature; adjusting the number of replicas based on a relationship between the specific swap acceptance probability and the target swap acceptance probability; The method of claim 2, comprising: (Appendix 7) 2. The method of claim 1, wherein obtaining the second fixed temperature includes determining the second fixed temperature based on a relationship between a particular swap acceptance probability and the second fixed temperature. (Appendix 8) One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause the system to perform an action, the action including: obtaining a first fixed temperature of a replica exchange Markov chain Monte Carlo (MCMC) process used to solve an optimization problem associated with the system; obtaining a second fixed temperature of the replica exchange MCMC process; obtaining a number of replicas of the system to be used in the replica-exchange MCMC process; assigning to each replica of the plurality of replicas a different corresponding temperature of a set of temperatures of the replica exchange MCMC process, the set of temperatures including the first fixed temperature, the second fixed temperature, and a plurality of variable temperatures numerically ordered between the first fixed temperature and the second fixed temperature; obtaining a target swap acceptance probability for swapping between replicas corresponding to adjacently ordered variable temperatures among the plurality of variable temperatures during the replica swap MCMC process; determining a respective average swap acceptance probability for one or more respective adjacent pairs of the plurality of variable temperatures, each respective swap acceptance probability being for a swap between replicas assigned to the one or more respective adjacent pairs; adjusting one or more of the variable temperatures based on a relationship between the target swap acceptance probability and each of one or more of the respective swap acceptance probabilities while maintaining the first fixed temperature and the second fixed temperature during the replica exchange MCMC process; during the replica exchange MCMC process, adjusting a number of replicas used during the MCMC process, the adjustment of the number of replicas being based on a difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature; identifying a particular state of the system as a solution to the optimization problem associated with the system based on an output of the replica-exchange MCMC process; A non-transitory computer readable storage medium comprising: (Appendix 9) 9. The non-transitory computer-readable storage medium of claim 8, wherein adjusting the number of replicas includes removing the particular variable temperature and removing a particular replica corresponding to the particular variable temperature. (Appendix 10) 9. The non-transitory computer-readable storage medium of claim 8, wherein adjusting the number of replicas includes removing the particular variable temperature and removing a particular replica corresponding to the particular variable temperature. (Appendix 11) 9. The non-transitory computer-readable storage medium of claim 8, wherein adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature is in response to the difference between the second fixed temperature and the particular variable temperature satisfying a temperature difference threshold. (Appendix 12) Adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature includes: determining a particular swap acceptance probability for a particular swap between a first replica corresponding to the particular variable temperature and a second replica corresponding to the second fixed temperature based on a difference between the second fixed temperature and the particular variable temperature; adjusting the number of replicas based on a relationship between the specific swap acceptance probability and the target swap acceptance probability; 9. The non-transitory computer-readable storage medium of claim 8, comprising: (Appendix 13) 9. The non-transitory computer-readable storage medium of claim 8, wherein obtaining the second fixed temperature includes determining the second fixed temperature based on a relationship between a particular swap acceptance probability and the second fixed temperature. (Appendix 14) 9. The non-transitory computer-readable storage medium of claim 8, wherein the first fixed temperature is a fixed low temperature and the second fixed temperature is a fixed high temperature. (Appendix 15) 1. A system comprising: hardware configured to perform the operations; The operation includes: obtaining a first fixed temperature of a replica exchange Markov chain Monte Carlo (MCMC) process used to solve an optimization problem associated with the system; obtaining a second fixed temperature of the replica exchange MCMC process; obtaining a number of replicas of the system to be used in the replica-exchange MCMC process; assigning to each replica of the plurality of replicas a different corresponding temperature of a set of temperatures of the replica exchange MCMC process, the set of temperatures including the first fixed temperature, the second fixed temperature, and a plurality of variable temperatures numerically ordered between the first fixed temperature and the second fixed temperature; obtaining a target swap acceptance probability for swapping between replicas corresponding to adjacently ordered variable temperatures among the plurality of variable temperatures during the replica swap MCMC process; determining a respective average swap acceptance probability for one or more respective adjacent pairs of the plurality of variable temperatures, each respective swap acceptance probability being for a swap between replicas assigned to the one or more respective adjacent pairs; adjusting one or more of the variable temperatures based on a relationship between the target swap acceptance probability and each of one or more of the respective swap acceptance probabilities while maintaining the first fixed temperature and the second fixed temperature during the replica exchange MCMC process; identifying a particular state of the system as a solution to the optimization problem associated with the system based on an output of the replica-exchange MCMC process; Including, the system. (Appendix 16) The operation includes: during the replica exchange MCMC process, adjusting a number of replicas used during the MCMC process, the adjustment of the number of replicas being based on a difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature. 16. The system of claim 15, further comprising: (Appendix 17) 17. The system of claim 16, wherein adjusting the number of replicas includes removing the particular variable temperature and removing a particular replica corresponding to the particular variable temperature. (Appendix 18) 17. The system of claim 16, wherein adjusting the number of replicas includes inserting a new variable temperature between the second fixed temperature and the particular variable temperature and assigning a new replica to the new variable temperature. (Appendix 19) 17. The system of claim 16, wherein adjusting the number of replicas based on the difference between the second fixed temperature and the particular variable temperature is in response to the difference between the second fixed temperature and the particular variable temperature satisfying a temperature difference threshold. (Appendix 20) Adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature includes: determining a particular swap acceptance probability for a particular swap between a first replica corresponding to the particular variable temperature and a second replica corresponding to the second fixed temperature based on a difference between the second fixed temperature and the particular variable temperature; adjusting the number of replicas based on a relationship between the specific swap acceptance probability and the target swap acceptance probability; 17. The system of claim 16, comprising:

Claims

1. A method performed by a computing system, comprising: Obtaining a first fixed temperature of a replica exchange Markov chain Monte Carlo (MCMC) process used to solve an optimization problem associated with the system; Obtaining a second fixed temperature of the replica exchange MCMC process; and obtaining a number of replicas of the system to be used in the replica-exchange MCMC process; assigning each replica of the plurality of replicas a different corresponding temperature of a set of temperatures of the replica exchange MCMC process, the set of temperatures including the first fixed temperature, the second fixed temperature, and a plurality of variable temperatures numerically ordered between the first fixed temperature and the second fixed temperature; obtaining a target swap acceptance probability for swapping between replicas corresponding to adjacently ordered variable temperatures of the plurality of variable temperatures during the replica swapping MCMC process; determining a respective average swap acceptance probability for one or more respective adjacent pairs of the plurality of variable temperatures, each respective swap acceptance probability being for a swap between replicas assigned to the one or more respective adjacent pairs; adjusting one or more of the variable temperatures based on a relationship between the target swap acceptance probability and each of one or more of the respective swap acceptance probabilities while maintaining the first fixed temperature and the second fixed temperature during the replica exchange MCMC process; identifying a particular state of the system as a solution to the optimization problem associated with the system based on an output of the replica-swapping MCMC process; The method includes:

2. 2. The method of claim 1, further comprising: adjusting, during the replica exchange MCMC process, a number of replicas used during the replica exchange MCMC process, the adjustment of the number of replicas being based on a difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature.

3. The method of claim 2 , wherein adjusting the number of replicas comprises removing the particular variable temperature and removing a particular replica that corresponds to the particular variable temperature.

4. 3. The method of claim 2, wherein adjusting the number of replicas comprises inserting a new variable temperature between the second fixed temperature and the particular variable temperature, and assigning a new replica to the new variable temperature.

5. 3. The method of claim 2, wherein adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature is in response to the difference between the second fixed temperature and the particular variable temperature satisfying a temperature difference threshold.

6. Adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature includes: determining a particular swap acceptance probability for a particular swap between a first replica corresponding to the particular variable temperature and a second replica corresponding to the second fixed temperature based on a difference between the second fixed temperature and the particular variable temperature; adjusting the number of replicas based on a relationship between the specific swap acceptance probability and the target swap acceptance probability; The method of claim 2 , comprising:

7. 2. The method of claim 1, wherein obtaining the second fixed temperature comprises determining the second fixed temperature based on a relationship between a particular swap acceptance probability and the second fixed temperature.

8. A computer program for causing a system to perform operations, the operations comprising: Obtaining a first fixed temperature of a replica exchange Markov chain Monte Carlo (MCMC) process used to solve an optimization problem associated with the system; Obtaining a second fixed temperature of the replica exchange MCMC process; and obtaining a number of replicas of the system to be used in the replica-exchange MCMC process; assigning each replica of the plurality of replicas a different corresponding temperature of a set of temperatures of the replica exchange MCMC process, the set of temperatures including the first fixed temperature, the second fixed temperature, and a plurality of variable temperatures numerically ordered between the first fixed temperature and the second fixed temperature; obtaining a target swap acceptance probability for swapping between replicas corresponding to adjacently ordered variable temperatures of the plurality of variable temperatures during the replica swapping MCMC process; determining a respective average swap acceptance probability for one or more respective adjacent pairs of the plurality of variable temperatures, each respective swap acceptance probability being for a swap between replicas assigned to the one or more respective adjacent pairs; adjusting one or more of the variable temperatures based on a relationship between the target swap acceptance probability and each of one or more of the respective swap acceptance probabilities while maintaining the first fixed temperature and the second fixed temperature during the replica exchange MCMC process; during the replica exchange MCMC process, adjusting a number of replicas used during the replica exchange MCMC process, the adjustment of the number of replicas being based on a difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature; identifying a particular state of the system as a solution to the optimization problem associated with the system based on an output of the replica-swapping MCMC process; A computer program comprising:

9. 9. The computer program product of claim 8, wherein adjusting the number of replicas comprises removing the particular variable temperature and removing a particular replica that corresponds to the particular variable temperature.

10. 9. The computer program product of claim 8, wherein adjusting the number of replicas comprises removing the particular variable temperature and removing a particular replica that corresponds to the particular variable temperature.

11. 9. The computer program product of claim 8, wherein adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature is in response to the difference between the second fixed temperature and the particular variable temperature satisfying a temperature difference threshold.

12. Adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature includes: determining a particular swap acceptance probability for a particular swap between a first replica corresponding to the particular variable temperature and a second replica corresponding to the second fixed temperature based on a difference between the second fixed temperature and the particular variable temperature; adjusting the number of replicas based on a relationship between the specific swap acceptance probability and the target swap acceptance probability; 9. The computer program of claim 8, comprising:

13. 9. The computer program product of claim 8, wherein obtaining the second fixed temperature comprises determining the second fixed temperature based on a relationship between a particular swap acceptance probability and the second fixed temperature.

14. 9. The computer program product of claim 8, wherein the first fixed temperature is a fixed low temperature and the second fixed temperature is a fixed high temperature.

15. 1. A system comprising: hardware configured to perform the operations; The operation includes: Obtaining a first fixed temperature of a replica exchange Markov chain Monte Carlo (MCMC) process used to solve an optimization problem associated with the system; Obtaining a second fixed temperature of the replica exchange MCMC process; and obtaining a number of replicas of the system to be used in the replica-exchange MCMC process; assigning each replica of the plurality of replicas a different corresponding temperature of a set of temperatures of the replica exchange MCMC process, the set of temperatures including the first fixed temperature, the second fixed temperature, and a plurality of variable temperatures numerically ordered between the first fixed temperature and the second fixed temperature; obtaining a target swap acceptance probability for swapping between replicas corresponding to adjacently ordered variable temperatures of the plurality of variable temperatures during the replica swapping MCMC process; determining a respective average swap acceptance probability for one or more respective adjacent pairs of the plurality of variable temperatures, each respective swap acceptance probability being for a swap between replicas assigned to the one or more respective adjacent pairs; adjusting one or more of the variable temperatures based on a relationship between the target swap acceptance probability and each of one or more of the respective swap acceptance probabilities while maintaining the first fixed temperature and the second fixed temperature during the replica exchange MCMC process; identifying a particular state of the system as a solution to the optimization problem associated with the system based on an output of the replica-swapping MCMC process; Including, the system.

16. The operation includes: during the replica exchange MCMC process, adjusting a number of replicas used during the replica exchange MCMC process, the adjustment of the number of replicas being based on a difference between the second fixed temperature and a particular variable temperature ordered adjacent to the second fixed temperature; The system of claim 15 further comprising:

17. The system of claim 16 , wherein adjusting the number of replicas comprises removing the particular variable temperature and removing the particular replica that corresponds to the particular variable temperature.

18. 17. The system of claim 16, wherein adjusting the number of replicas includes inserting a new variable temperature between the second fixed temperature and the particular variable temperature and assigning a new replica to the new variable temperature.

19. 17. The system of claim 16, wherein adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature is in response to the difference between the second fixed temperature and the particular variable temperature satisfying a temperature difference threshold.

20. Adjusting the number of replicas based on a difference between the second fixed temperature and the particular variable temperature includes: determining a particular swap acceptance probability for a particular swap between a first replica corresponding to the particular variable temperature and a second replica corresponding to the second fixed temperature based on a difference between the second fixed temperature and the particular variable temperature; adjusting the number of replicas based on a relationship between the specific swap acceptance probability and the target swap acceptance probability; The system of claim 16, comprising:

Images