An extensible bell inequality violation strategy optimization method and device and storage medium

Abstract

The application discloses an extensible Bell inequality violation strategy optimization method and device and a storage medium, and particularly relates to the technical field of quantum strategy optimization. The method comprises the following steps: receiving a game description comprising local observation sets, local decision sets, joint observation distributions and utility matrices of each participant, and constructing a weighted utility tensor; constructing a differentiable parameterized object; performing tensor contraction operation on the weighted utility tensor and the differentiable parameterized object without explicitly constructing a global score matrix, so as to solve an unbiased estimation of an average score or a strategy gradient of all participants; based on the gradient of the average score relative to the differentiable parameterized object, using a first-order optimization algorithm to iteratively update the differentiable parameterized object, so that the average score converges to the maximum direction; and outputting the optimized differentiable parameterized object. The application realizes extensible solution of a medium / large-scale game quantum optimal strategy in a communication-restricted multi-party collaborative decision-making scene.

Description

Technical Field

[0001] This invention relates to the field of quantum policy optimization technology, and more specifically, to a scalable method, apparatus, and storage medium for optimizing policies that violate Bell's inequality. Background Technology

[0002] In engineering scenarios such as distributed computing, high-frequency trading, aerospace communications, and inter-enterprise collaboration, a common multi-party collaborative decision-making problem exists: due to physical latency or information access limitations, multiple spatially separated decision-makers cannot or are not allowed to communicate with each other in time. They need to make local decisions under strict communication constraints, and each party's decisions must satisfy specific cooperation relationships to obtain benefits. This scenario can be uniformly abstracted as follows: each party receives local observations and makes independent local decisions. The system calculates scores based on whether each party's decisions satisfy cooperation relationships. Since the decision-makers cannot communicate in real time during the decision-making period, they can only pre-agree on a set of strategies, the goal of which is to maximize the long-term average score under a given observation distribution.

[0003] When participants share a quantum entangled state, their optimal average score can strictly exceed the upper limit achievable by any pre-agreed classical strategy; this exceedance is known in physics as a violation of Bell's inequality. This cooperative advantage can be directly translated into latency advantages, security advantages, or interference resistance advantages in engineering scenarios.

[0004] However, existing technologies suffer from several drawbacks in terms of efficiency. The brute-force enumeration method used is not scalable; both the number of observation and decision combinations grow exponentially with the number of participants. Explicitly constructing large scoring matrices further exacerbates storage and computational overhead, increasing quadratically with the total system dimension. Many existing tools default to fixed dimensions and projective measurements, limiting the family of policies that can be represented and failing to approximate the true quantum upper bound. Furthermore, existing reinforcement learning methods are primarily geared towards sequential decision-making and lack low-variance gradient estimators for "one-off average payoff" structures, especially when the size of the observation-decision combination exceeds a certain threshold. At the order of magnitude, even without explicitly constructing a large scoring matrix, it is difficult to traverse all combinations to obtain an accurate value. Therefore, there is an urgent need for an efficient, scalable, and verifiable quantum policy optimization method. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, the present invention provides a scalable Bell inequality violation strategy optimization method, apparatus, and storage medium to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A scalable Bell inequality violation optimization method includes the following steps: S1: Receive the game description including the local observation set, local decision set, joint observation distribution and utility matrix of each participant, and construct a weighted utility tensor based on the joint observation distribution and the utility matrix; S2: Construct a differentiable parameterizable object based on the local observation set and the local decision set; S3: Without explicitly constructing a large global score matrix, perform tensor shrinkage operation on the weighted utility tensor and the differentiable parameterizable object to solve for an unbiased estimate of the average score or policy gradient of all participants. S4: Based on the gradient of the average score relative to the differentiable parameterizable object, use a first-order optimization algorithm to iteratively update the differentiable parameterizable object, so that the average score converges in the direction of maximization; S5: Output the optimized differentiable parameterized object.

[0007] In a preferred embodiment, the receiving step includes a game description comprising the local observation set, local decision set, joint observation distribution, and utility matrix of each participant, and constructing a weighted utility tensor based on the joint observation distribution and the utility matrix, including: The local information visible to each participant is taken as the local observation set, and the local decisions that each participant can make are taken as the local decision set. The probability distribution of the observed combinations of each participant is taken as the joint observation distribution; The scores corresponding to different observations and different decisions are combined into a utility matrix; The weighted utility tensor is obtained by multiplying the joint observation distribution element-wise with the utility matrix.

[0008] In a preferred embodiment, the differentiable parameterizable object includes: a local differentiable unitary operator for each participant under each local observation, and a differentiable parameter vector representing the entangled state shared by all participants, and optionally includes a local differentiable stochastic post-processing matrix from measurement results to output decisions.

[0009] In a preferred embodiment, the tensor shrinkage operation includes: Using the weighted utility tensor and the differentiable parameterizable object as tensor operands, the average score of all participants is calculated in one go through a single tensor contraction operation. By introducing a batch dimension into the first dimension of the differentiable parameterizable object, the average score and gradient of multiple independently and randomly initialized differentiable parameterizable objects can be solved in parallel during a single tensor shrinkage operation.

[0010] In a preferred embodiment, the unbiased estimation of the policy gradient includes: Based on the joint observation distribution, random sampling is performed to obtain sampled observations; By tensor contraction, for each sampled observation, a corresponding decision action is extracted to obtain an observation-decision pair sample. For each observation-decision pair sample, determine the conditional probability of the decision action under the corresponding sampled observation, and the differentiable logarithm of the conditional probability relative to the differentiable parameterized object; Query the weighted utility tensor to obtain the sample utility of each observation-decision pair. Maintain a baseline, which is obtained by bias correction of the exponentially weighted moving average of historical sample utility; Subtract the baseline value from the utility of each sample to obtain the advantage of each observation-decision pair for the sample. The unbiased estimate of the policy gradient is obtained by averaging the products of the advantages of all observation-decision pairs and the gradients of their corresponding differentiable logarithms.

[0011] In a preferred embodiment, the step of iteratively updating the differentiable parameterizable object using a first-order optimization algorithm based on the gradient of the average score relative to the differentiable parameterizable object, so that the average score converges in the direction of maximization, includes: The first-order optimization algorithm used is an adaptive optimizer that maintains the second-order moment estimate of the differentiable parameterizable object; When the game size is within a traversable range, the adaptive optimizer is driven to update the differentiable parameterized object by the gradient of the average score relative to the differentiable parameterized object; When the game size exceeds the traversable range, the gradient of the average score relative to the differentiable parameterized object is replaced with an unbiased estimate of the strategy gradient. Before each update of the differentiable parameterized object, the second moment estimate of the differentiable parameterized object is read from the internal state of the adaptive optimizer, and a preconditioning factor vector is constructed based on the second moment estimate with a preset small constant. The global norm of the preconditioning factor vector is injected into the learning rate as a multiplicative factor so that the preconditioning factor vector neither explodes nor returns to zero when the gradient magnitude tends to zero.

[0012] In a preferred embodiment, the first-order optimization algorithm further includes: According to the preset window length, count the number of times the average score is improved within each window; When the number of improvements reaches the preset first threshold, the step size is increased by the preset improvement factor and the rollback snapshot is cleared. When there is no improvement within the window, the step size is reduced by a preset shrinkage factor, and the differentiable parameterized object and the internal state of the adaptive optimizer are rolled back to the snapshot saved at the beginning of the window, wherein the internal state includes at least the second moment estimate.

[0013] In a preferred embodiment, the output optimized differentiable parameterized object includes: The optimized differentiable parameterized object is used as the local unitary operator, shared entangled state target value, and optional local random post-processing matrix for each participant in the physical deployment; The shared entangled state target value is distributed to each participant through means such as optical fiber, satellite, or mobile quantum storage; Write each participant’s local unitary operator and optional local random post-processing matrix into each participant’s local quantum controller and classical post-processor; In each decision cycle, after receiving local observations, each participant performs corresponding unitary operations and measurements according to the local unitary operator, and then outputs the final decision based on the local stochastic post-processing matrix. The entire process does not involve classical or quantum communication between the participants.

[0014] On the other hand, the present invention provides a scalable Bell inequality violation strategy optimization apparatus, comprising the following modules: The game definition module is used to receive a game description including the local observation set, local decision set, joint observation distribution and utility matrix of each participant, and to construct a weighted utility tensor based on the joint observation distribution and the utility matrix. The parameterized quantum strategy module is used to construct a differentiable parameterizable object based on the local observation set and the local decision set. The average score calculation module is used to perform tensor shrinkage operations on the weighted utility tensor and the differentiable parameterizable object without explicitly constructing a large global score matrix, so as to solve for an unbiased estimate of the average score or policy gradient of all participants. An optimization control module is used to iteratively update the differentiable parameterizable object using a first-order optimization algorithm based on the gradient of the average score relative to the differentiable parameterizable object, so that the average score converges in the direction of maximization. The physical deployment module is used to output the optimized differentiable parameterized object.

[0015] On the other hand, the present invention provides a storage medium on which a program or instruction is stored, and when the program or instruction is executed by a processor, a scalable Bell inequality violation strategy optimization method is implemented.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention unifies the parameterization of quantum strategies into automatically differentiable local unitary operators, shared entangled states, and optional local random post-processing matrices. Combined with a computational shortcut for directly calculating the average score through tensor contraction, all evaluations and gradient calculations can be completed in a single tensor contraction without constructing a large scoring matrix whose size expands quadratically with the total dimension of the system. Compared with the explicit construction method, this achieves quadratic improvements in memory and time, enabling medium-to-large-scale games to be optimized with conventional computing power.

[0017] 2. When the game scale is huge and the combination of observation and decision exceeds the traversable range, this invention performs unbiased policy gradient estimation based only on the sampled observation-decision pairs, without having to traverse all combinations. This reduces the cost of evaluation from an exponential increase with the number of combinations to a linear increase with the number of samples, so that games of an originally unbearable scale can also obtain non-trivial lower bounds within reasonable computing power.

[0018] 3. This invention employs an adaptive optimizer that maintains the second-order moment estimate. The second-order moment is reinterpreted into a stable preconditioner. Through global norm multiplicative back-reference, the preconditioner is guaranteed to neither explode nor return to zero when the gradient magnitude approaches zero. It restores linear convergence near the optimal value, eliminating the terminal oscillation of the standard adaptive optimizer. At the same time, it is combined with a staged adaptive line search mechanism, which counts the number of improvements in a window unit. When progress is smooth, the step size is automatically increased; when stuck, the step size is automatically reduced and the parameters and optimizer state are rolled back. This makes the invention ready to use out of the box under a large number of different games and dimensional configurations, without the need for manual adjustment of the learning rate and convergence threshold for each problem. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of a scalable Bell inequality violation strategy optimization method provided in an embodiment of the present invention; Figure 2 This is a functional block diagram of a scalable Bell inequality violation strategy optimization device provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 The diagram shown is a flowchart illustrating a scalable Bell inequality violation optimization method according to an embodiment of the present invention. In this embodiment, the scalable Bell inequality violation optimization method includes: S1: Receive the game description including the local observation set, local decision set, joint observation distribution and utility matrix of each participant, and construct a weighted utility tensor based on the joint observation distribution and the utility matrix; In this embodiment of the invention, receiving a game description including the local observation set, local decision set, joint observation distribution, and utility matrix of each participant, and constructing a weighted utility tensor based on the joint observation distribution and the utility matrix, includes: The local information visible to each participant is taken as the local observation set, and the local decisions that each participant can make are taken as the local decision set. Specifically, the problem scenario addressed by this invention is as follows: multiple spatially separated decision-makers make local decisions under strict communication constraints, and each party's decisions must satisfy a specific cooperative relationship to obtain benefits. Let there be a total... The first participant, the first The local observation set of each participant is denoted as The local decision set is denoted as .

[0023] Each observation in the local observation set represents the first... The local information that each participant can obtain during the decision-making cycle, such as in a frequency distribution scenario for countering interference, where each communication node can see which frequency bands are currently safe, and the set of safe frequency bands is the node's local observation.

[0024] Each decision value in the local decision set represents the first decision. The local choices that each participant can make, such as in a frequency distribution scenario to combat interference, where each communication node selects one of several frequency bands as its working frequency band, and the selected frequency band is the node's local decision.

[0025] In practical applications, the size of the observation set and the decision set depends on the complexity of the specific scenario and the capabilities of the participants. For example, in a distributed load balancing scenario, the observation set can encode local rate fluctuations, and the decision set can encode the target channels to be selected.

[0026] In practice, the aforementioned observation set and decision set are configured by those skilled in the art according to the actual situation of the specific application scenario, and input into the system through the game description interface.

[0027] The probability distribution of the observed combinations of each participant is taken as the joint observation distribution.

[0028] Specifically, the joint observation distribution represents the probability distribution of the occurrence of observation combinations from all participating parties. The observation distribution reflects the frequency of different observation combinations occurring during long-term operation and is the basis for calculating the long-term average time-sharing weighting of various observation combinations.

[0029] In one implementation of a frequency distribution scenario to combat interference, the observation distribution is a uniform distribution across combinations of subsets. In practical applications, the joint observation distribution is determined by those skilled in the art based on prior knowledge of the specific scenario or historical statistical data. For example, in high-frequency trading scenarios, it is given by the joint distribution of historical signals from each node, and in load balancing scenarios, it is obtained from historical fluctuation statistics.

[0030] The scores corresponding to different observations and different decisions are combined into a utility matrix.

[0031] Specifically, the utility matrix represents the scoring rules for a given observation and decision.

[0032] In an exemplary implementation of a frequency distribution scenario to combat interference, the utility matrix is ​​defined as follows: 1 point is awarded when both parties select the same physical frequency band, and 0 points are awarded otherwise.

[0033] In practical engineering scenarios, the utility matrix is ​​designed according to specific business objectives. For example, in a distributed load balancing scenario, the utility function is defined as follows: when the load on all channels does not exceed the preset capacity limit, the benefit is the number of idle channels; when any channel exceeds the capacity limit, a larger negative value is taken as a penalty. In high-frequency trading collaboration scenarios, the utility matrix can be composed of a weighted sum of multiple indicators such as net trading revenue, slippage loss, inventory penalty, transaction fees, risk control conflict penalty, and relative return from cross-market arbitrage. Those skilled in the art can flexibly design the specific form and parameters of the utility matrix according to the business needs of specific application scenarios.

[0034] The weighted utility tensor is obtained by multiplying the joint observation distribution element-wise with the utility matrix.

[0035] The weighted utility tensor is the object directly manipulated in subsequent optimization steps. It contains the weight information of the observation distribution. When calculating the average score, the weighted utility tensor can be used directly for weighted summation without needing to multiply by the observation distribution.

[0036] In practice, the weighted utility tensor is a multidimensional array with the following number of dimensions: That is, it includes the observation dimension and decision dimension of all participants.

[0037] In a specific implementation, the specific dimensionality of the weighted utility tensor can be determined based on the actual number of participants and the number of observations and decisions made by each participant. Those skilled in the art can use conventional array processing tools (such as NumPy, PyTorch, etc.) to implement the above element-wise multiplication and tensor storage operations.

[0038] By constructing a weighted utility tensor, the probability information and scoring information in the game description are integrated into a unified tensor data structure, enabling subsequent steps to access all game information through a unified tensor interface.

[0039] S2: Construct a differentiable parameterizable object based on the local observation set and the local decision set; In this embodiment of the invention, the differentiable parameterizable object includes: a local differentiable unitary operator for each participant under each local observation, and a differentiable parameter vector representing the entangled state shared by all participants, and optionally includes a local differentiable stochastic post-processing matrix from measurement results to output decisions.

[0040] Each locally differentiable unitary operator has an arbitrary positive integer dimension. The complex unitary matrix, and Not limited to 2, thus supporting arbitrary dimensions of each party's local Hilbert space.

[0041] Unitary operators are constructed from ordinary parameters in any way that is differentiable everywhere. They can be constructed using any of the following differentiable parameterization methods: orthogonal / unitary parameterization based on matrix exponents; concatenated construction based on angle parameters (including rotation angle and phase angle); unitary parameterization based on Cayley transform; or any other parameterization method that maps Euclidean space parameters to unitary matrices and is differentiable everywhere.

[0042] Specifically, a complex unitary matrix is ​​assigned to each (participant, observer) combination, representing the invertible linear transformation performed by the participant on the local quantum system under the observation. The unitary matrix means an invertible linear transformation that preserves the vector length and the total probability.

[0043] In the context of counter-jamming frequency distribution, the local differentiable unitary operator corresponds to the rotation transformation performed by the communication node on the quantum resources it holds locally after seeing the current set of secure frequency bands.

[0044] A differentiable parameter vector representing the entangled state shared by all participants, namely the shared state vector, represents the joint quantum pure state shared by all participants in advance. It is a "correlated resource" shared by all parties in advance. It is not a means of communication, does not transmit any information, and neither party can use it to send messages to the other party.

[0045] In one embodiment of the invention, the shared state vector adopts a fully free entangled state mode. In this mode, the shared state vector is a global vector, normalized before each forward evaluation, covering the entire pure state space. This mode is suitable for scenarios where quantum hardware can flexibly prepare arbitrary entangled states and can search the broadest quantum policy space.

[0046] In another embodiment of the invention, the shared state vector adopts a fixed structured state mode. In this mode, the caller provides a fixed shared state, optimizing only the local unitary operator and the local random post-processing matrix. The fixed shared state is selected from a family of pure states with structured generation paths, including but not limited to maximally entangled states, Dicke states, GHZ states, W states, cluster states, matrix product states, or symmetric pure states (all of which are standard structured entangled state templates that can be natively prepared by the corresponding physical hardware; their specific definitions can be found in the literature on quantum information). This mode is suitable for practical deployment scenarios where quantum hardware can only prepare specific types of entangled states.

[0047] In another embodiment of the invention, the shared-state vector employs a classical strategy reduction model. In this model, the local dimension of all participants is uniformly set to 1, causing each party's unitary matrix to degenerate into a trivial form. Scalars and shared-state vectors degenerate into trivial unit vectors. In this mode, all quantum degrees of freedom are completely masked, while the rest of the parameterization methods, tensor shrinkage calculations, and optimization mechanisms of this invention remain completely unchanged, enabling the entire infrastructure to run on classical policy families without requiring separate implementation.

[0048] Local random post-processing matrix representation of the first Each participating party in the observation The following table will be used to translate the measured readings into a reference table for the final decision based on probability.

[0049] Specifically, the local random post-processing matrix is ​​a conditional probability matrix, where the rows correspond to different measurement results of the computational basis measurement, and the columns correspond to different final output decisions. For example, in a frequency distribution scenario against interference, the local random post-processing matrix corresponds to the communication node deciding which operating frequency band to ultimately select according to a preset probability table after reading a certain measurement result.

[0050] In one embodiment of the invention, the local random post-processing matrix is ​​obtained by row-wise Softmax normalization of learnable log probabilities.

[0051] In another embodiment of the invention, the local random post-processing matrix is ​​obtained by a non-learnable hard assignment function. In this embodiment, the mapping from measurement results to the final decision is deterministic, that is, each measurement result uniquely corresponds to one output decision. This mapping relationship is pre-set by those skilled in the art according to the specific application scenario and does not participate in gradient updates.

[0052] In another embodiment of the invention, the local random post-processing matrix is ​​provided externally by the caller. In this embodiment, the random post-processing matrix is ​​directly input by an external system or user, and the optimization process of the invention does not modify it, but only uses it as a fixed component of the strategy in the calculation of the average score.

[0053] In a preferred embodiment of the invention, the differentiable parameterizable object also supports a unified expression of projective nonprojective measurement (POVM). Specifically, unified support for POVM is achieved by adding an extra dimension to the unitary operator of each participant. Fangyou Calculation Additional An auxiliary dimension, with the auxiliary subsystem initialized in a fixed reference state. This approach equates the more flexible POVM reading method to ordinary projective measurements in large spaces plus random post-processing, thus eliminating the need to write a separate set of code for POVM to uniformly support both projective and non-projective measurement strategies. At this point, the scheme degenerates into ordinary projective measurement. The implementation of this additional dimension is based on the Naimark extension theorem, which mathematically guarantees that any POVM can be equivalently implemented using the above method.

[0054] The three types of parameters mentioned above—local unitary operators, shared-state vectors, and local stochastic post-processing matrices—are all stored in the form of learnable parameters within a differentiable tensor framework that supports automatic backpropagation.

[0055] Specifically, in one embodiment of the present invention, a differentiable tensor computation framework such as PyTorch or JAX is used to store and automatically differentiate the aforementioned parameters. In this framework, the underlying parameters of the local unitary operator, the complex vector elements of the shared state vector, and the learnable logits of the local stochastic post-processing matrix are all registered as trainable tensors. The computation framework automatically tracks all operations on these parameters and constructs a computation graph. During forward propagation, these parameters participate in tensor shrinkage operations to calculate the average score; during backpropagation, the computation framework automatically calculates the gradient of the average score relative to all the aforementioned parameters. Through this parameterization method, the quantum strategy of either party under any observation is uniformly described as a set of numerical parameters that can be automatically differentiated by the computer.

[0056] In practice, the specific values ​​of the above three types of parameters are set by those skilled in the art during the initialization phase according to the specific application scenario.

[0057] In one embodiment of the present invention, a random initialization method is adopted, the underlying parameters of the local unitary operator are sampled from the standard normal distribution, the elements of the shared state vector are sampled from the standard normal distribution and then normalized, and the logits of the local random post-processing matrix are sampled from the standard normal distribution.

[0058] In another embodiment of the present invention, a preset initialization method is adopted, in which the local unitary operator is initialized to an identity matrix, the shared state vector is initialized to a maximally entangled state, and the local random post-processing matrix is ​​initialized to a uniform distribution.

[0059] Those skilled in the art can select an appropriate initialization strategy based on the scale and characteristics of the specific game.

[0060] Once the aforementioned differentiable parameterizable object is constructed, it will serve as the core operand for tensor contraction operations, participating in the calculation of the average score together with the weighted utility tensor. The local unitary operator and the shared state vector are mandatory parameters, while the local random post-processing matrix is ​​an optional parameter. Whether it is enabled depends on whether a random mapping from the measurement results to the final decision is needed in the specific game scenario.

[0061] Through the above process, the construction of a differentiable parameterizable object is completed. This parameterization method describes the quantum policy of either party under any observation with a unified, automatically differentiable parameter, without limiting the dimensionality of the quantum system or the measurement method, thus ensuring that the optimal quantum policy can be searched in the subsequent optimization process without sacrificing the policy degrees of freedom.

[0062] S3: Without explicitly constructing a large global score matrix, perform tensor shrinkage operation on the weighted utility tensor and the differentiable parameterizable object to solve for an unbiased estimate of the average score or policy gradient of all participants. In this embodiment of the invention, two optional evaluation paths are provided depending on the size of the game: when the game size is within the traversable range, the exact evaluation path is adopted, and the average score of all participants is directly calculated through tensor contraction operation; when the game size exceeds the traversable range and the number of observation-decision combinations is too large to sum one by one, the probability sampling path is adopted, and the exact average score is replaced by the unbiased estimation of the policy gradient.

[0063] Specifically, the tensor shrinkage operation in the precise evaluation path includes the following process.

[0064] Using the weighted utility tensor and the differentiable parameterizable object as tensor operands, the average score of all participants is calculated in one operation through a single tensor contraction.

[0065] Specifically, the tensor contraction operation takes the weighted utility tensor and the differentiable parameterized object as inputs and directly calculates the scalar result of the average score through a single einsum (Einstein summation) operation.

[0066] In one embodiment of the present invention, the tensor shrinkage operation introduces a batch dimension into the first dimension of the differentiable parameterizable object, so that in one tensor shrinkage operation, multiple sets of independently and randomly initialized differentiable parameterizable objects can solve for the average score and gradient in parallel.

[0067] Specifically, a batch dimension is inserted into the first dimension (i.e., the 0th dimension) of the local unitary operator, shared state vector, and optional local stochastic post-processing matrix in the differentiable parameterizable object, so that each set of parameters is stored independently as different samples in a batch. In actual implementation, the parameters with the batch dimension are evaluated and their gradients are calculated in parallel during a single tensor shrinkage call.

[0068] In a preferred embodiment of the present invention, the batch dimension size is approximately... At this order of magnitude, the batch multiplication between the unitary matrix and the state vector in tensor shrinkage is implemented using batch matrix multiplication. This operation is a fundamental operation natively highly optimized on parallel hardware such as GPUs, ensuring that the wall-clock time for a single forward evaluation remains essentially unchanged relative to a batch dimension of 1. Therefore, the batching mechanism of this invention achieves parallel exploration in approximately the same wall-clock time as a single optimization. By taking the local extrema corresponding to independent random initializations of the order of magnitude and selecting the best one, the probability of convergence to the global optimum or near-global optimum is significantly increased.

[0069] In one embodiment of the present invention, when the tensor shrinking operation involves a large tensor size, the selectable shrinking order is automatically selected by the einsum optimizer to minimize the memory usage and computation time of intermediate results.

[0070] Through the tensor shrinkage operation in the aforementioned precise evaluation path, the accurate solution for the average score of all participants is obtained. This average score will serve as the target value for gradient calculation and parameter updates.

[0071] Furthermore, in this embodiment of the invention, when the game size exceeds the traversable range, the average score is replaced by an unbiased estimate of the policy gradient.

[0072] The unbiased estimate of the policy gradient is obtained through the following probabilistic sampling optimization sub-method.

[0073] Based on the joint observation distribution, random sampling is performed to obtain sampled observations.

[0074] Specifically, based on the joint observation distribution, independent sampling is performed. One observation tuple.

[0075] In practice, if the observation distribution is uniform, then each observation tuple has an equal probability of being sampled; if the observation distribution is non-uniform, then standard random sampling methods such as inverse transformation sampling or accept-reject sampling are used to make the sampling results conform to the joint observation distribution.

[0076] Those skilled in the art can select an appropriate sampling algorithm based on the size and specific distribution of the observation space. Simultaneously, the exact logarithmic probability of each sampled observation is recorded for subsequent baseline calculation and gradient estimation.

[0077] By tensor shrinking, for each sampled observation, a corresponding decision action is extracted to obtain an observation-decision pair sample.

[0078] Specifically, for each sampled observation, the rotated state is obtained through tensor contraction using the constructed shared state vector and unitary operators of each aspect. Then, a measurement result is randomly selected according to the squared distribution of the computational basis amplitude. If a local random post-processing matrix is ​​configured in step S2, the final output decision is further randomly selected according to the normalized distribution of the corresponding row. When no local random post-processing matrix exists, the measurement result is directly used as the output decision.

[0079] In one embodiment of the present invention, the above-mentioned forward computation process of multiple samples adopts a vectorized batch processing method, that is, all samples are processed simultaneously in a single tensor shrinkage call. One sampling observation.

[0080] After the above process, we obtain One observation-decision pair sample.

[0081] For each observation-decision pair, determine the conditional probability of the decision action under the corresponding sampled observation, and the differentiable logarithm of the conditional probability relative to the differentiable parameterized object.

[0082] Specifically, for the sampled observation-decision pairs, the conditional probability of the decision under a given observation is calculated again through tensor shrinking. This calculation only requires computing the required tensor elements, and the computational complexity of a single sample is of the same order as the aforementioned forward computation. Then, the logarithm is taken to obtain the differentiable logarithmic probability. Since all parameters are differentiable tensors, the gradient of this logarithmic probability with respect to the differentiable parameterized object can be obtained through an automatic differentiation mechanism.

[0083] Query the weighted utility tensor to obtain the sample utility of each observation-decision pair.

[0084] Specifically, for each observation-decision pair obtained from sampling, the utility matrix is ​​queried using the observation and decision as indexes to obtain the sample utility.

[0085] In one embodiment of the present invention, since the sampling process only generates For each sample, the utility query operation only needs to be performed on the utility matrix. This allows for random index access without traversing the entire utility matrix. Utility queries for multiple samples can be completed in one go using a vectorized approach.

[0086] Maintain a baseline, which is obtained by adjusting the bias of the exponentially weighted moving average of historical sample utility.

[0087] Specifically, the baseline is updated using an exponential moving average (EMA), and bias correction is used to eliminate the estimation bias of the EMA in the initial stage.

[0088] In one embodiment of the invention, the baseline is initially set to 0. After each round of sampling, the EMA is updated using the mean sample utility of the current batch.

[0089] The baseline value is subtracted from the utility of each sample to obtain the advantage of each observation-decision pair for that sample.

[0090] Specifically, advantage represents the degree to which the utility of a sample deviates from the historical average: when advantage is greater than 0, it means that the utility of the sample is higher than the average level, and the corresponding decision action should be strengthened; when advantage is less than 0, it means that the utility of the sample is lower than the average level, and the corresponding decision action should be weakened.

[0091] The unbiased estimate of the policy gradient is obtained by averaging the products of the advantages of all observation-decision pairs and the gradients of their corresponding differentiable logarithms.

[0092] The estimate is unbiased, meaning its expected value equals the true policy gradient.

[0093] In one embodiment of the present invention, when the variance of the observation-decision pair utility of the sample is large, the variance of the gradient estimation is reduced by subtracting the historical average utility from the baseline, thereby accelerating convergence.

[0094] In a preferred embodiment of the present invention, the exponentially weighted decay coefficient of the baseline and the second-order moment decay coefficient of the adaptive optimizer in step S4 are the same value, so that their time constants are coordinated.

[0095] In one embodiment of the present invention, the probability sampling optimized sampling backend can be implemented in any of the following ways: (1) Classical simulation of the complete state vector, that is, using a classical computer to completely simulate the joint quantum state of all participants, calculate the measurement distribution and sample; (2) Classical simulation based on the stabilizer subcircuit; (3) Classical simulation based on tensor networks or matrix product states; (4) Classical simulation based on near-Clifford+T decomposition; (5) The sampling output of the real hardware quantum computer and the matching conditional probability estimation module, wherein the real hardware quantum computer performs fast sampling of the parameterized quantum circuit and serves as part of the environment for the probability optimization, and the circuit parameters are updated by the outer policy gradient. (6) A combination of the above methods.

[0096] The specific implementation methods of the above-mentioned sampling backends are general technologies in this field. Those skilled in the art can choose any one of the backends to implement sampling and probability estimation based on the available computing resources and quantum hardware conditions.

[0097] Through the above probability sampling process, the unbiased estimate of the policy gradient is calculated. This unbiased estimate will serve as the gradient input for the first-order optimization algorithm in step S4, driving the update of the differentiable parameterizable object.

[0098] When using the probabilistic sampling mode, the staged line search mechanism in step S4 needs to be turned off as needed, because the improvement of a single-point target value cannot be accurately determined under sampling noise. However, the second-order moment precondition and the stability scaling factor are still applicable and significantly speed up the process.

[0099] Through the above steps, without explicitly constructing a large global scoring matrix, the unbiased estimation of the average score (precise evaluation path) or policy gradient of all participants (probability sampling path) is completed. These results will serve as the basis for parameter updates in step S4.

[0100] In one embodiment of the present invention, when the shared state and the weighted utility tensor have a sum of tensor products structure, the tensor contraction operation can be accelerated by superimposing the sum of tensor products structure decomposition to further reduce computational complexity.

[0101] Specifically, the tensor product sum structure means that the shared state and weighted utility tensors can each be represented as the sum of a finite number of tensor products. When the shared state and weighted utility tensors have the above structure, the entire tensor contraction in step S3 is linearly distributed into a set of sub-contractions, and the complexity of each sub-contraction depends only polynomially on each local dimension.

[0102] In a preferred embodiment of the invention, the sum of the tensor products is derived from the sum of intersecting mini-games. Specifically, a network of relationships is established among multiple participants, where each edge represents a two-way mini-game, and the overall score is a weighted sum of the scores of these mini-games. The two-way mini-game contribution of each edge only applies to the first... and the The participating parties and other participating parties are identical tensor integrals; the entire relational network gives a weighted utility tensor. Tensor integral solutions. When evaluating a participant shared by multiple edges, its local unitary operator is reused by multiple components, which is precisely the manifestation of a single strategy serving multiple edges at the tensor level. Each edge can be independently assigned an output label permutation, making the reused minigames distinct game components, avoiding degeneration into separable cases.

[0103] In another embodiment of the invention, the sum of the tensor products originates from the fractal decomposition of a vast decision space. When a participant has a large number of decision options (e.g., choosing one from thousands of stocks), "which one to choose" is reformulated as a combination of "making a small choice in several dimensions or subgames." The key is that while many options may appear similar in any single dimension, across a sufficient number of dimensions, each option becomes distinct in at least one dimension. Thus, the vast decision space is compactly and unambiguously encoded by a set of minigames.

[0104] Many real-world games have weighted utility tensors that naturally possess low tensor product-rank structures, such as CHSH-type games, interference-resistant games, and decomposable load balancing utilities. The optimal shared state also approximates a low-rank tensor network state in various scenarios (e.g., matrix product state MPS). Through the aforementioned tensor product summation structure decomposition, even with extremely large game sizes, tensor contraction operations can still be performed quickly.

[0105] S4: Based on the gradient of the average score relative to the differentiable parameterizable object, use a first-order optimization algorithm to iteratively update the differentiable parameterizable object, so that the average score converges in the direction of maximization; In this embodiment of the invention, the step of iteratively updating the differentiable parameterizable object using a first-order optimization algorithm based on the gradient of the average score relative to the differentiable parameterizable object, so that the average score converges in the direction of maximization, includes the following process.

[0106] Obtain the gradient of the average score relative to the differentiable parameterizable object.

[0107] Specifically, in step S3, after calculating the average score (precise evaluation path) or the unbiased estimate of the policy gradient (probabilistic sampling path) through tensor shrinkage operations, if the precise evaluation path is used, the automatic differentiation mechanism of the differentiable tensor computation framework is utilized to call the backpropagation function to calculate the gradient of the average score relative to each trainable parameter in the differentiable parameterized object (including the underlying parameters of each participating unitary operator, the elements of the shared state vector, and the learnable logits of the local stochastic post-processing matrix). If the probabilistic sampling path is used, the unbiased estimate of the policy gradient is directly used as a substitute for the gradient of the average score relative to the differentiable parameterized object.

[0108] The first-order optimization algorithm used is an adaptive optimizer that maintains the second-order moment estimate of the differentiable parameterizable object.

[0109] Specifically, the adaptive optimizer maintains the gradient second moment estimate for each parameter dimension during each parameter update, which is used to adaptively adjust the learning rate for each parameter dimension.

[0110] In one embodiment of the present invention, the adaptive optimizer is selected from any of the following: Adam, AdamW, RMSProp, AdaGrad, or variations thereof. These optimizers are all well-known first-order gradient optimization algorithms in the art, and their core idea is to adaptively adjust the update step size of each parameter dimension using the first moment (momentum) and second moment (exponentially weighted moving average of the squared gradient). Those skilled in the art can select an appropriate type of adaptive optimizer based on specific application scenarios and computational resources.

[0111] In a preferred embodiment of the present invention, the Adam optimizer is employed.

[0112] When the game size is within an traversable range, the adaptive optimizer is driven to update the differentiable parameterized object by the gradient of the average score relative to the differentiable parameterized object.

[0113] Specifically, in each iteration, the gradient of the average score obtained by the exact evaluation path is input into the adaptive optimizer. The optimizer calculates the update amount of each parameter dimension based on the first-order moment estimate and second-order moment estimate maintained internally, and updates the differentiable parameterizable object.

[0114] When the game size exceeds the traversable range, the gradient of the average score relative to the differentiable parameterized object is replaced with an unbiased estimate of the policy gradient.

[0115] Specifically, after each round of sampling, the unbiased estimate of the policy gradient calculated by the probability sampling path in step S3 is input into the adaptive optimizer. The optimizer processes the unbiased estimate in the same way as the exact gradient, driving the update of the differentiable parameterizable object.

[0116] In one embodiment of the present invention, before each update of the differentiable parameterizable object, the second moment estimate of the differentiable parameterizable object is read from the internal state of the adaptive optimizer, and a preconditioning factor vector is constructed based on the second moment estimate with a preset small constant. The global norm of the preconditioning factor vector is injected into the learning rate as a multiplicative factor so that the gradient magnitude neither explodes nor returns to zero when it approaches zero.

[0117] Specifically, the internal state of the adaptive optimizer maintains a second-moment estimate (i.e., an exponentially weighted moving average of the squared gradient) for each parameter dimension.

[0118] Before each parameter update, read the second-moment estimates for all parameter dimensions from the optimizer state. With a preset small constant Constructing the preconditioning factor vector : ; In the formula, This is a preset small constant used to prevent division by zero errors.

[0119] Calculate the global norm of the preconditioning factor vector, i.e., its overall length or size. Inject the global norm as a multiplicative factor into the learning rate. Equivalently, the parameter update is controlled by multiplying the gradient by a stable preconditioner obtained by element-wise dividing the global norm by the preconditioning factor vector.

[0120] Specifically, the preconditioner possesses the following stability characteristics. When the estimated second moments of each parameter dimension are similar, the preconditioner approximates the square root of the total number of parameters, degenerating into gradient descent updates with uniform scaling. When the curvature in a certain parameter direction is high, the corresponding estimated second moment is large, and the value of the preconditioner in that direction automatically decreases, resulting in a correspondingly smaller step size in that direction. When the gradients of all parameter dimensions tend to zero, the numerator and denominator of the preconditioner factor synchronously approach a stable ratio determined by the preset small constant. This prevents step size explosion due to an excessively small denominator or complete disappearance of the step size due to the numerator returning to zero, thus ensuring stable and controllable convergence behavior near the optimal solution.

[0121] In one embodiment of the present invention, the first-order optimization algorithm further includes a staged adaptive line search mechanism.

[0122] The mechanism counts the number of times the average score is improved within each window, according to a preset window length.

[0123] Specifically, maintain a global multiplicative factor with an initial value of 1.

[0124] Set window length (For example (20 steps), counting the number of improvements to the target value (i.e., average score) within each window, using the window length as the unit. When the average score improves relative to the previous window, it is counted as one improvement.

[0125] When the number of improvements reaches the preset first threshold, the step size is increased by the preset improvement factor and the rollback snapshot is cleared.

[0126] Specifically, if the number of improvements in the current window reaches or exceeds a preset first threshold... (For example If each step within the window represents an improvement, it indicates that the current step size is too conservative, and a preset improvement factor should be used. (For example Increase the effective step size. At the same time, clear previously saved rollback snapshots and reset the count.

[0127] When there is no improvement within the window, the step size is reduced by a preset shrinkage factor, and the differentiable parameterized object and the internal state of the adaptive optimizer are rolled back to the snapshot saved at the beginning of the window, wherein the internal state includes at least the second moment estimate.

[0128] Specifically, if the average score within the current window does not improve, it indicates that the current step size is too large, causing optimization to stall, and a preset reduction factor is used. (For example The effective step size is reduced. At the same time, the differentiable parameterizable object and the internal state of the adaptive optimizer are rolled back to the snapshot saved at the beginning of the window.

[0129] Synchronous rollback of parameters and state is a key technical point. Rolling back only the parameters without rolling back the optimizer state will cause the second-order moment estimates maintained internally by the optimizer to continue drifting on the wrong trajectory, leading to subsequent convergence anomalies. Therefore, during rollback, all parameter values ​​of the differentiable parameterizable object and all internal states of the adaptive optimizer (including first-order moment estimates, second-order moment estimates, iteration steps, etc.) must be restored to the snapshot saved at the beginning of the window.

[0130] If there are improvements within the window but the amplification threshold has not been reached, only the count is refreshed; the step size is not adjusted and no rollback is performed.

[0131] In one embodiment of the invention, the staged adaptive line search mechanism can be superimposed on a conventional learning rate scheduler. The conventional learning rate scheduler (e.g., cosine annealing scheduler or warm-up scheduler) provides a basic learning rate change curve, and the line search mechanism and preconditioning factor act on the scheduler output.

[0132] In one embodiment of the present invention, the target value may also be subjected to utility normalization.

[0133] Specifically, the global or moving mean and variance of the average score are calculated, and the target value is normalized to a standard score before further judgment and line search decision-making. This normalization method allows the same set of hyperparameter templates to be reused across different games without needing to readjust the window length and threshold for each new game. When a rollback is triggered, the normalization statistics are reset synchronously.

[0134] In an alternative embodiment of the invention, the first-order optimization algorithm may also employ an optimizer that does not maintain second-order moment estimation but has equivalent stable step size scheduling (e.g., Lion or symbolic momentum optimizer), whose effective step size is guaranteed by an explicit scheduling mechanism to neither explode nor return to zero when the gradient magnitude tends to zero.

[0135] In one embodiment of the invention, when a probabilistic sampling path is used (i.e., the unbiased estimate of the policy gradient is calculated in step S3 instead of the exact average), the staged line search mechanism needs to be turned off as needed. This is because, in the presence of sampling noise, it is impossible to accurately determine whether there is a real improvement in a single target value. However, the second-moment preconditioning and the stability scaling factor are still applicable in the sampling mode and can significantly speed up the process.

[0136] Those skilled in the art can choose whether to enable the staged line search mechanism based on the specific level of sampling noise: when the sample size When the sample size is large enough and the estimated variance is small enough, line search can be enabled; when the sample size is small and the estimated variance is large, line search should be disabled to implement a rollback mechanism.

[0137] Through the above steps, the differentiable parameterized object is iteratively updated using a first-order optimization algorithm, causing the average score to converge towards maximization. The second-order moment preconditioning mechanism of the adaptive optimizer eliminates the oscillation problem near the optimal value in the standard adaptive optimizer; the staged adaptive line search mechanism eliminates the need to manually set the learning rate and convergence threshold.

[0138] The above update process continues until the preset convergence condition is met (e.g., the change in the target value is less than the preset tolerance or the preset maximum number of iterations is reached). At this point, the differentiable parameterizable object has converged to the numerical approximation of the optimal quantum strategy, and the converged differentiable parameterizable object will be used as the output object of step S5.

[0139] S5: Output the optimized differentiable parameterized object.

[0140] In this embodiment of the invention, the output of the differentiable parameterized object after optimization includes the following process.

[0141] The optimized differentiable parameterized object is used as the local unitary operator, shared entangled state target value, and optional local random post-processing matrix for each participant in the physical deployment.

[0142] Specifically, through iterative optimization in step S4, the differentiable parameterizable object has converged to a numerical approximation of the optimal quantum strategy. This differentiable parameterizable object contains three core types of parameters: local unitary operators for each party under each local observation, a target value representing the shared entangled state among all participants, and an optional local stochastic post-processing matrix. These three types of parameters correspond to three different elements that each participant needs to load and execute during physical deployment.

[0143] In one embodiment of the present invention, the physical distribution method of the shared entangled state target value is as follows. Specifically, the physical entangled state corresponding to the optimized shared entangled state target value is distributed to the local systems of each participating party through any of the following methods.

[0144] In one embodiment of the present invention, an optical fiber entanglement distribution method is employed.

[0145] Specifically, entangled photon pairs or photon groups are generated by an entanglement source located at a central position and transmitted to each participating party's local location via low-loss optical fiber. This method is suitable for scenarios where all participating parties are located in the same city or region, such as scheduling servers in a multi-datacenter distributed system or trading nodes across exchanges within the same city. In this embodiment, the entanglement source can be selected based on the actual distance (such as a spontaneous parametric down-conversion source or a quantum dot source), and the optical fiber link uses low-loss single-mode fiber supplemented by appropriate quantum repeaters to extend the transmission distance.

[0146] In another embodiment of the invention, a satellite entanglement distribution method is employed.

[0147] Specifically, low-Earth orbit or medium-Earth orbit satellites transmit entangled photon pairs or groups of photons to multiple ground stations. This method is suitable for scenarios where participating parties are geographically distant, such as intercontinental exchange docking, orbit-to-ground communication, and interplanetary probe collaboration. In this embodiment, the satellite carries an entangled light source module, which distributes entangled photons through free-space optical links between the satellite and various ground stations.

[0148] In another embodiment of the present invention, a mobile quantum memory distribution method is adopted.

[0149] Specifically, when the participants are not far apart (e.g., in the same building or park), a quantum storage device with error correction capabilities is used to temporarily connect the quantum computers of each participant. The target shared state is written into the quantum storage device, and then the physical connection is broken and the quantum storage device is physically transported to the location of each participant. This method is suitable for deployment scenarios where the requirements for entanglement distribution time window are not high but high fidelity is required.

[0150] In a supplementary embodiment of the invention, a reuse and on-demand regeneration method is also included.

[0151] Specifically, optimized shared entangled states are pre-generated in batches and stored in a quantum memory. Before each round of the game, the stored entangled states are used as the distribution source for distribution. Alternatively, before each round of the game, the target shared state is used as the preparation target, and the entanglement preparation system is used to re-prepare and distribute the entangled states in real time.

[0152] Those skilled in the art can choose any of the above distribution methods or combinations thereof based on the actual physical distance between the participants, available hardware resources, and latency requirements. All of the above distribution methods are existing physical implementation technologies in the field of quantum information.

[0153] Each participant's local unitary operator and optional local random post-processing matrix are written into each participant's local quantum controller and classical post-processor.

[0154] Specifically, after the distribution of the shared entangled state is completed, the optimized local unitary operators and local random post-processing matrices of each participant under each local observation are loaded into the local quantum controller and classical post-processor of each participant, respectively.

[0155] In one embodiment of the present invention, the quantum controller is a quantum gate controller capable of performing unitary operations on a single or two qubit. The unitary operator is compiled into a series of quantum gate pulse sequences and stored in the waveform memory of the controller. The classical post-processor is a classical computing unit (e.g., an FPGA or microcontroller) with random number generation and probability sampling capabilities. The local random post-processing matrix is ​​stored in the post-processor's memory in the form of a lookup table.

[0156] After loading is complete, the system enters the online game execution phase.

[0157] In each decision cycle, after receiving local observations, each participant performs corresponding unitary operations and measurements according to the local unitary operator, and then outputs the final decision based on the local stochastic post-processing matrix. The entire process does not involve classical or quantum communication between the participants.

[0158] Specifically, each decision cycle includes the following sub-steps.

[0159] Each participating party receives its own local observations. These local observations can include indicators of safe frequency band sets in anti-jamming frequency distribution scenarios, local request rates in load balancing scenarios, local market signals in high-frequency trading scenarios, or local detector status in aerospace communication scenarios.

[0160] Each participant selects the unitary operator corresponding to the current observation from the set of local unitary operators loaded into the quantum controller based on the received local observation, and executes the unitary operation in the local shared state subsystem.

[0161] Specifically, if the shared entangled state has been distributed to each participant via optical fiber, satellite, or quantum storage, each participant holds a local subsystem of that shared entangled state. The participant then applies a selected unitary operator to its local subsystem to achieve a rotation of the measurement basis.

[0162] Each participant performs computational basis measurements on the local subsystem after the unitary operation to obtain basis vector labels (i.e., measurement results). Computational basis measurement is the most basic and direct way of reading the data, directly indicating which reference lattice the quantum state falls into. In actual physical implementation, computational basis measurement is performed through a single-qubit projection measurement module in the quantum controller, and the measurement results are output in the form of classical bits.

[0163] If a local stochastic post-processing matrix exists, each participant performs secondary sampling according to the rows of the stochastic post-processing matrix corresponding to the current observation (corresponding to the measurement results), transforming the measurement results into the final output decision. If no local stochastic post-processing matrix exists, the results of the calculated basis measurements are directly used as the final output decision.

[0164] Each participating party outputs the final decision.

[0165] In anti-interference frequency allocation scenarios, this decision refers to the selected working frequency band, and each node subsequently attempts to establish a chain according to the selected frequency band. In distributed load balancing scenarios, this decision refers to the selected target channel, and each server subsequently distributes requests to the selected channel. In high-frequency trading collaboration scenarios, this decision is sent to the respective exchange gateway via the local order management and execution module.

[0166] During the online game process described above, no classical or quantum communication is conducted between the participants.

[0167] Specifically, each participant relies solely on local observations and pre-distributed shared entangled states to generate decisions throughout the entire decision-making cycle, without sending any decision-related information to other participants. This ensures the applicability of the invention in communication-constrained environments—whether due to physical latency limitations (such as light-speed delay) or information access restrictions (such as compliance or anti-eavesdropping), the deployment scheme of the invention can operate effectively.

[0168] Through the above steps, the output and physical deployment of the optimized differentiable parameterizable object were completed. The optimized local unitary operator, the shared entangled state objective value, and the local stochastic post-processing matrix together constitute a complete policy configuration that can be directly loaded and executed by each participant, realizing a complete closed loop from offline optimization to online deployment.

[0169] In one embodiment of the present invention, the method further includes a parallel classical benchmark link for providing classical optimal values. The classical benchmark chain provides provable upper and lower bounds, thus offering a quantitative and provable verification of quantum advantage. This classical benchmark chain operates independently of the main quantum policy optimization process, retrieving the game description from the game definition module and outputting the classical value bounds in parallel.

[0170] In one embodiment of the present invention, the classical reference link provides the classical optimal value in the following manner. That is, the provable upper bound of any classical strategy.

[0171] Specifically, the expected utility of a classic mixed strategy is modeled as a function of the local conditional probability variables of each party. For strategies containing... In a game involving multiple participants, an integer constraint is imposed that the sum of the probabilities of all decisions under each observation is 1, such that each participant chooses a deterministic decision under each observation.

[0172] For each combination of observations and decisions, a choice variable is introduced, and a marginal consistency constraint is imposed such that the choice variable equals the product of the deterministic decisions made by each party. The feasible solution of this MILP precisely constitutes the convex hull of the deterministic strategies of each party, and its objective function is to maximize the sum of the element-wise products of the weighted utility tensor and the choice variable.

[0173] The MILP is solved using an existing linear MILP global solver, and the global optimum output by the solver is taken as the solution. Since the feasible solution of this MILP covers all classical deterministic policies and their convex combinations, and since convexity indicates that the optimal value of the hybrid policy is equal to the optimal value of the deterministic policy, this global optimum is... It is both the exact value of the classical optimal value and a provable upper bound for any classical strategy.

[0174] In one embodiment of the present invention, the classical benchmark link further includes a differentiable optimizeable branch of the classical policy reduction mode, used to give the classical optimal value. The lower bound estimate.

[0175] Specifically, the differentiable parameterizable object in step S2 is switched to classical policy reduction mode. In this mode, the local dimension of all participants is uniformly set to 1, causing the unitary matrix of each participant to degenerate into a trivial scalar, and the shared state vector to degenerate into a trivial unit vector. At this point, the local stochastic post-processing matrix directly corresponds to the conditional probability distribution in the classical local stochastic policy family. All quantum degrees of freedom are completely masked, while the remaining parameterization methods, tensor shrinkage operations, adaptive optimizers, and line search mechanisms of this invention remain completely unchanged. The tensor shrinkage, second-order moment preconditioning, line search, and batch parallel mechanisms, which are exactly the same as those used in quantum optimization, are reused to perform differentiable optimization on the classical policy family, outputting... The lower bound estimate is then obtained. This lower bound is used in conjunction with the upper bound given by the MILP reconstruction above to... To put pressure on them.

[0176] In one embodiment of the invention, the classical reference link further includes a quantum upper bound verification link as an optional bypass diagnostic.

[0177] Specifically, the link calls an existing quantum upper bound solver based on an NPA semidefinite programming hierarchy or its equivalent hierarchy, and outputs a quantum value. The provable upper bound is given. This solver only outputs the numerical upper bound, does not generate deployable policies, and does not participate in the policy search of this invention. The quantum optimization output of this invention... The difference between the upper bound and the NPA is taken as the provable approximate accuracy of the optimized output of this invention within the quantum policy family. When this upper bound and the lower bound of the quantum optimized output of this invention are closed, the result is obtained. The exact value. It should be noted that the solution cost of the NPA solver is also affected by the expansion of the large scoring matrix with the square of the total dimension of the system. The higher the level, the tighter the upper bound, but the more severe the expansion. Therefore, this approach is mainly suitable for diagnostic scenarios of small to medium-sized games.

[0178] In one embodiment of the invention, when the NPA quantum upper bound solver is not feasible due to the expansion of the large scoring matrix, a retrospective self-evaluation of the quantum maximum value is used as a cheap alternative or supplement.

[0179] Specifically, using the output of the batch parallel mechanism in step S3, a large number of independent random seeds are used to solve the problem at once. The distribution of these converged values ​​is examined to determine whether the true quantum maximum has been found. If a steep plateau appears near the highest value, meaning that a large number of initializations converge to the same highest value, it suggests that the optimal value is relatively easy to find, and the current batch is likely to be the true quantum maximum. If a slow tail is observed, meaning that the extreme tail is still slowly climbing and the optimal value continues to increase slightly with the number of seeds, it suggests that the optimal value is relatively difficult to find, and the current batch may not have reached the true maximum. The seed size or the number of iterations needs to be increased. This criterion is a retrospective criterion and does not replace the strict upper bound of NPA, but it provides a cheap "whether the top has been reached" indicator when NPA is not feasible due to the expansion of the large scoring matrix.

[0180] In one embodiment of the present invention, the verification output of the classical reference link is: a comparison of the quantum optimization value. With provable classical upper bound ,when At that time, quantum advantage is provably guaranteed, and the difference This is the provable lower bound of the advantage.

[0181] This is the basis for engineering decisions: when the quantum advantage significantly outweighs the overhead of shared state generation and distribution, it is recommended to deploy a quantum solution; when classical methods can approach quantum levels, it is not necessary to bear the cost of shared state generation and distribution.

[0182] By using the aforementioned classical benchmark link and quantum upper bound verification, this invention not only outputs the optimal quantum strategy, but also provides a quantitative and provable verification of quantum advantage, thus providing a reliable quantitative basis for decisions from offline optimization to engineering deployment.

[0183] In one embodiment of the present invention, an anti-interference collaborative scenario under adversarial communication interference is used as a specific operational example to illustrate the complete implementation process of the present invention.

[0184] Scene display: Each friendly communication node operates at frequency bands, of which The line is interfered with; each node locally observes one. A subset of the safe frequency bands (the visible subsets may differ between different nodes) must be selected for local operation; if the selected frequencies of different nodes are inconsistent, they will be unable to communicate with each other and will be unable to effectively negotiate on the disturbed spectrum.

[0185] Game modeling: Number of participants (Can be expanded to multiple parties); The Fang Local Observation Index one A subset of the security frequency band; the first Local decision-making The index is located within the locally visible subset of the operating frequency band; the observation distribution is a uniform distribution across all subset combinations; the utility matrix... If all parties choose the same physical frequency band, then ,otherwise .

[0186] Connection with combinatorial modeling: In multi-party promotion, "every two adjacent nodes must choose the same frequency" constitutes a two-party mini-game, and the whole is the weighted sum of these mini-games on the node relationship network (such as the spanning tree); a node shared by multiple neighbors serves all adjacent mini-games with a single local strategy, making the whole indivisible, and tensor integral decomposition can be used to accelerate the evaluation.

[0187] Deployment: Shared entangled state. Each party within a communication window uses local interference scan results. Measure and output Attempt to establish a link by selecting the frequency band.

[0188] Numerical Examples ( strip frequency band, (The bar is interfered with, 2 participants): At this time, each party observes one locally. A subset of the security frequency bands, taken from the local dimension Running the optimized link of this invention yields: (1) Classical Optimal The solution is obtained by reconstructing the linear MLP with an off-the-shelf solver (COPT is used in this example; other equivalent solvers such as Gurobi can also be used), and is marked as globally optimal (as opposed to the solution). (The configuration is consistent with the classic reference value). (2) Quantum optimal It is obtained by differentiable optimization (with a relative error of approximately [missing value] compared to the known quantum reference value). (that is, it has converged to the quantum supremum of the game). (3) Quantum advantage can be proven The improvement relative to the classic optimal is approximately That is: the pure classic (local random) strategy allows at most two nodes to be randomly selected. The probability of selecting the same safe frequency band is low, while sharing entanglement resources increases that probability to high. Furthermore, this advantage can be proven by classical analysis to have an undeniable upper bound.

[0189] (4) Self-evaluation of the batch distribution of quantum maximum values, in one... In the parallel optimization, the distribution of the final values ​​of each random initialization is as follows: 865 copies in the interval. There are 126 intervals and 33 other intervals; the optimal value is 0.5833, the mean is 0.5795, and the worst is 0.5555. The vast majority (approximately 84.5%) of the initializations converge to the intervals containing... The highest interval shows a steep plateau rather than a slow tail – a heuristic signal that “the quantum optimum is easier to find and the current value is likely the true quantum maximum.”

[0190] In addition to numerical values, this invention also directly outputs the deployable strategy itself.

[0191] The following is a "behavior table". Presentation, behavior It is a probability table: given the safe frequency band subsets observed by each party. Both sides selected the first... The joint probability of the visible frequency bands (denoted as...) ; This means "select the lower of the two bands I observe" (meaning "choose the one with the lower frequency among the two bands I observe"). Each of the three observations corresponds to a 2-band subset: , , .

[0192] Table 1. Quantum Optimal Behavior Table Table 2 Classic Optimal Behavior Table The quantum optimal behavior table is shown in Table 1, and the classical optimal behavior table is shown in Table 2. The intuitive comparison of the two behavior tables reveals the source of quantum advantage: when both parties see the same safe frequency band subset (the first 3 rows), the quantum strategy concentrates all probabilities on... and In the quantum strategy, there is no chance of a mismatch; both sides will always choose the same frequency band (win rate 1). When both sides see different subsets (with only one common frequency band), the quantum strategy tilts the probability towards "choosing the same common frequency band" (win rate 0.375). The classical optimal strategy, on the other hand, can only make each side mechanically choose the frequency band with the lower number, resulting in only 5 out of 9 observation combinations hitting the same frequency band (win rate 5 / 9). It is the "perfect coordination when seeing the same subset" plus the "probability tilt when seeing different subsets" that raises the overall win rate from 5 / 9 to 7 / 12.

[0193] The numerical examples above are only minimal demonstrations. For larger-scale... In configuration and multi-party promotion scenarios, the same set of links of this invention can be superimposed with tensor product summation structure decomposition acceleration or probability sampling optimization when necessary, so as to obtain the optimal quantum strategy in a scalable manner.

[0194] This numerical example demonstrates the comparison between quantum values ​​and provable classical values, the method of measuring quantum advantage, and the readable form of the discovered strategy. The specific values ​​of the local unitary operators and shared states used by each party for measurement are derived by the optimizer, and there are multiple equivalent solutions achieving the same optimal value. In actual deployment, each party performs local operations and measurements for each decision cycle according to the local unitary operators and shared entangled state target values ​​output in step S5, without involving communication between the participants throughout the process.

[0195] In one embodiment of the present invention, the implementation process of the present invention is described using a low-latency load balancing scenario of a distributed computing system as a specific example.

[0196] Scenario: n front-end scheduling servers distribute external requests from a unified resource pool. Each server independently decides within a microsecond window which of m shared back-end channels to allocate the request to. The network round-trip latency between servers is higher than the decision window, making it too late to exchange scheduling intentions: if all requests are directed to the same channel, congestion and packet loss will occur; if they are distributed too widely, idle channels and average latency will increase.

[0197] Game modeling: Number of participants Equal to the number of scheduling servers; the first Fang Local Observation Local encoding rate fluctuation ;No. Local decision-making Select the target channel; the observed distribution is obtained from historical fluctuation statistics (which can be approximated as uniform); utility matrix. If the load of all channels does not exceed the capacity The profit is the number of idle channels; if any channel exceeds its capacity, a larger negative penalty is applied.

[0198] Deployment: Servers share entangled states; each server within a window receives data at its local rate. Execute the unitary operation And provide channels according to local random mapping .

[0199] Available in multiple Output quantum synergistic advantage under configuration This study verifies the provable advantages of entangled resources over pure local randomization in low-latency load balancing.

[0200] In one embodiment of the present invention, the implementation process of the present invention is described using a high-frequency trading collaboration scenario across remote exchanges as a specific example.

[0201] Scenario: Two or more high-frequency trading nodes are deployed in geographically separated exchanges / data centers. Each node performs market making or arbitrage on several related assets (such as cross-market quotes for the same asset, or combinations of related stocks / options / derivatives). The physical communication latency between nodes exceeds the decision window, preventing nodes from synchronizing their intentions within the window. If multiple nodes simultaneously place orders in the same direction, it will lower each other's queue priority or drive unfavorable price movements; if they place orders in opposite directions, it may lead to self-trading or risk control conflicts.

[0202] Game modeling: Number of participants Equal to the number of deployed nodes; the Fang Local Observation This represents the discretization results of local market signals (such as order book imbalance, recent trading direction, volatility levels, price spread levels, etc.); Local decision-making It can perform discrete actions (such as placing buy orders, placing sell orders, canceling orders, changing prices, cross-market hedging, waiting, etc.); the observation distribution is given by the joint distribution of historical signals of each node; the utility function includes the weighted sum of net transaction profit, slippage cost, inventory penalty, transaction fee, risk control conflict penalty, and relative profit of cross-market arbitrage.

[0203] When the decision of a node is not "buy / sell" but "choose one from many related targets", its decision space is huge. The decision space can be encode "which target to choose" into several smaller choices on several dimensions by decomposing the decision space (many targets are similar on any dimension, and each target is different after crossing a sufficient number of dimensions). This allows for a compact modeling of the massive decision space and rapid evaluation using tensor integral decomposition.

[0204] Deployment: Nodes share entangled states. Within each window, each node independently executes local operations and measurements. Decisions are sent to their respective exchange gateways via the local order management and execution module.

[0205] The above embodiments illustrate the proposed scalable Bell inequality violation strategy optimization method using typical application scenarios such as anti-interference collaboration under adversarial communication interference, low-latency load balancing in distributed computing systems, and high-frequency trading collaboration across remote exchanges. It is understood that this invention is not limited to the above three implementation scenarios, and those skilled in the art can expand the scenarios based on the core ideas proposed in this invention.

[0206] like Figure 2 The diagram shown is a functional block diagram of a scalable Bell inequality violation policy optimization device according to an embodiment of the present invention. In this embodiment, a scalable Bell inequality violation policy optimization device includes the following modules: The game definition module is used to receive a game description including the local observation set, local decision set, joint observation distribution and utility matrix of each participant, and to construct a weighted utility tensor based on the joint observation distribution and the utility matrix. The parameterized quantum strategy module is used to construct a differentiable parameterizable object based on the local observation set and the local decision set. The average score calculation module is used to perform tensor shrinkage operations on the weighted utility tensor and the differentiable parameterizable object without explicitly constructing a large global score matrix, so as to solve for an unbiased estimate of the average score or policy gradient of all participants. An optimization control module is used to iteratively update the differentiable parameterizable object using a first-order optimization algorithm based on the gradient of the average score relative to the differentiable parameterizable object, so that the average score converges in the direction of maximization. The physical deployment module is used to output the optimized differentiable parameterized object.

[0207] In this embodiment, a storage medium stores a program or instruction, which, when executed by a processor, implements a scalable Bell inequality violation strategy optimization method.

[0208] All calculations involved in the embodiments are dimensionless numerical calculations, and the preset parameters and thresholds in the calculations are set by those skilled in the art according to the actual situation.

[0209] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0210] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A scalable Bell inequality violation optimization method, characterized in that, Includes the following steps: S1: Receive the game description including the local observation set, local decision set, joint observation distribution and utility matrix of each participant, and construct a weighted utility tensor based on the joint observation distribution and the utility matrix; S2: Construct a differentiable parameterizable object based on the local observation set and the local decision set; S3: Without explicitly constructing a large global score matrix, perform tensor shrinkage operation on the weighted utility tensor and the differentiable parameterizable object to solve for an unbiased estimate of the average score or policy gradient of all participants. S4: Based on the gradient of the average score relative to the differentiable parameterizable object, use a first-order optimization algorithm to iteratively update the differentiable parameterizable object, so that the average score converges in the direction of maximization; S5: Output the optimized differentiable parameterized object.

2. The scalable Bell inequality violation strategy optimization method according to claim 1, characterized in that, The receiving includes a game description comprising the local observation set, local decision set, joint observation distribution, and utility matrix of each participant, and constructs a weighted utility tensor based on the joint observation distribution and the utility matrix, including: The local information visible to each participant is taken as the local observation set, and the local decisions that each participant can make are taken as the local decision set. The probability distribution of the observed combinations of each participant is taken as the joint observation distribution; The scores corresponding to different observations and different decisions are combined into a utility matrix; The weighted utility tensor is obtained by multiplying the joint observation distribution element-wise with the utility matrix.

3. The scalable Bell inequality violation strategy optimization method according to claim 1, characterized in that, The differentiable parameterized object includes: a local differentiable unitary operator for each participant under each local observation, and a differentiable parameter vector representing the entangled state shared by all participants, and optionally includes a local differentiable stochastic post-processing matrix from measurement results to output decisions.

4. The scalable Bell inequality violation strategy optimization method according to claim 1, characterized in that, The tensor shrinkage operation includes: Using the weighted utility tensor and the differentiable parameterizable object as tensor operands, the average score of all participants is calculated in one go through a single tensor shrinkage operation. By introducing a batch dimension into the first dimension of the differentiable parameterizable object, the average score and gradient of multiple independently and randomly initialized differentiable parameterizable objects can be solved in parallel during a single tensor shrinkage operation.

5. The scalable Bell inequality violation strategy optimization method according to claim 1, characterized in that, The unbiased estimation of the policy gradient includes: Based on the joint observation distribution, random sampling is performed to obtain sampled observations; By tensor contraction, for each sampled observation, a corresponding decision action is extracted to obtain an observation-decision pair sample. For each observation-decision pair sample, determine the conditional probability of the decision action under the corresponding sampled observation, and the differentiable logarithm of the conditional probability relative to the differentiable parameterized object; Query the weighted utility tensor to obtain the sample utility of each observation-decision pair. Maintain a baseline, which is obtained by bias correction of the exponentially weighted moving average of historical sample utility; Subtract the baseline value from the utility of each sample to obtain the advantage of each observation-decision pair for the sample. The unbiased estimate of the policy gradient is obtained by averaging the products of the advantages of all observation-decision pairs and the gradients of their corresponding differentiable logarithms.

6. The scalable Bell inequality violation strategy optimization method according to claim 1, characterized in that, The step of iteratively updating the differentiable parameterizable object using a first-order optimization algorithm based on the gradient of the average score relative to the differentiable parameterizable object, so that the average score converges in the direction of maximization, includes: The first-order optimization algorithm used is an adaptive optimizer that maintains the second-order moment estimate of the differentiable parameterizable object; When the game size is within a traversable range, the adaptive optimizer is driven to update the differentiable parameterized object by the gradient of the average score relative to the differentiable parameterized object; When the game size exceeds the traversable range, the gradient of the average score relative to the differentiable parameterized object is replaced with an unbiased estimate of the strategy gradient. Before each update of the differentiable parameterized object, the second moment estimate of the differentiable parameterized object is read from the internal state of the adaptive optimizer, and a preconditioning factor vector is constructed based on the second moment estimate with a preset small constant. The global norm of the preconditioning factor vector is injected into the learning rate as a multiplicative factor so that the preconditioning factor vector neither explodes nor returns to zero when the gradient magnitude tends to zero.

7. A scalable Bell inequality violation strategy optimization method according to claim 6, characterized in that, The first-order optimization algorithm further includes: According to the preset window length, count the number of times the average score is improved within each window; When the number of improvements reaches the preset first threshold, the step size is increased by the preset improvement factor and the rollback snapshot is cleared. When there is no improvement within the window, the step size is reduced by a preset shrinkage factor, and the differentiable parameterized object and the internal state of the adaptive optimizer are rolled back to the snapshot saved at the beginning of the window, wherein the internal state includes at least the second moment estimate.

8. A scalable Bell inequality violation strategy optimization method according to claim 1, characterized in that, The output optimized differentiable parameterizable object includes: The optimized differentiable parameterized object is used as the local unitary operator, shared entangled state target value, and optional local random post-processing matrix for each participant in the physical deployment; The shared entangled state target value is distributed to each participant through means such as optical fiber, satellite, or mobile quantum storage; Write each participant’s local unitary operator and optional local random post-processing matrix into each participant’s local quantum controller and classical post-processor; In each decision cycle, after receiving local observations, each participant performs corresponding unitary operations and measurements according to the local unitary operator, and then outputs the final decision based on the local stochastic post-processing matrix. The entire process does not involve classical or quantum communication between the participants.

9. A scalable Bell inequality violation policy optimization apparatus, used to implement the scalable Bell inequality violation policy optimization method according to any one of claims 1-8, characterized in that, Includes the following modules: The game definition module is used to receive a game description including the local observation set, local decision set, joint observation distribution and utility matrix of each participant, and to construct a weighted utility tensor based on the joint observation distribution and the utility matrix. The parameterized quantum strategy module is used to construct a differentiable parameterizable object based on the local observation set and the local decision set. The average score calculation module is used to perform tensor shrinkage operations on the weighted utility tensor and the differentiable parameterizable object without explicitly constructing a large global score matrix, so as to solve for an unbiased estimate of the average score or policy gradient of all participants. An optimization control module is used to iteratively update the differentiable parameterizable object using a first-order optimization algorithm based on the gradient of the average score relative to the differentiable parameterizable object, so that the average score converges in the direction of maximization. The physical deployment module is used to output the optimized differentiable parameterized object.

10. A storage medium, characterized in that, A program or instruction is stored on a storage medium, and when the program or instruction is executed by a processor, it implements a scalable Bell inequality violation strategy optimization method as described in any one of claims 1-8.

Images