Reinforcement learning method and system for quantum gate regulation and control

Abstract

The invention relates to the technical field of quantum reinforcement learning, in particular to a reinforcement learning method and system for quantum gate regulation, and the method sequentially comprises the steps: carrying out the normalization of an offline sample, carrying out the principal component analysis, and then preparing a prior quantum state; performing wavelet scattering and persistent coherence on the quantum pulses to extract topological features, and mapping the topological features into a non-Abell geometric gate rotation angle to generate an action probability; constructing a sparse high-dimensional vector, selecting a target super vector through Grover amplification, writing the target super vector into the silicon carbide memristor array, and retrieving an action vector according to a conduction function determined by the topological feature vector; the motion vector drives the Heisenberg cellular automaton to predict the environment, the local error triggers the fine tuning of the rotation angle and the conduction value, and the global distance triggers the topology recompilation and the quantum natural gradient. According to the method, decision making within 250 microseconds, low-energy-consumption retrieval and steady-state self-healing are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of quantum reinforcement learning, in particular to a quantum gate regulated reinforcement learning method and system. BACKGROUND

[0002] The quantum gate regulated reinforcement learning injects quantum parallelism and geometric phase robustness into the decision chain, and can complete the "perception-reasoning-execution" closed loop in sub-millisecond scale, which has a revolutionary significance for industries such as autonomous driving, space exploration and high-speed robots that rely on ultra-low delay policy update. There are mainly two types of existing solutions: one is classical deep reinforcement learning, which relies on large number of samples and gradient playback, and has high reasoning delay and high edge energy consumption; the other is early quantum reinforcement learning, which usually directly maps actions using measurement results without using the internal topological information of quantum states, and the search still relies on classical traversal, resulting in policy jitter and low hardware utilization. SUMMARY

[0003] In view of the problems existing in the prior art, the present application provides a quantum gate regulated reinforcement learning method and system, which compresses samples, encodes amplitudes, and prepares prior quantum states by a variational circuit; maps a pulse topological fingerprint to a geometric gate rotation angle to generate a probability distribution; selects a sparse high-dimensional key by Grover amplification, and searches for actions in a silicon carbide resistive array; and a Heisenberg automaton predicts the environment and drives geometric angle fine-tuning or topological recompilation with error / distance double thresholds. The closed loop delay and power consumption of the whole chain are lower than those of the classical scheme.

[0004] A quantum gate regulated reinforcement learning method, comprising the following steps:

[0005] Numerical normalization and dimension compression are performed on offline sample data, and then the compressed sample matrix is mapped to a quantum amplitude encoding state, and a prior quantum state is generated by iterative training through a variational quantum circuit;

[0006] According to the pulse signal corresponding to the prior quantum state, a topological feature vector is extracted, and the topological feature vector is converted into a rotation angle of a non-Abelian geometric gate network to obtain an action probability vector;

[0007] A sparse high-dimensional vector is constructed according to the action probability vector, a target hyper vector is selected by a quantum amplitude amplification algorithm, the target hyper vector is injected into a programmable resistive memory array, and an action vector is retrieved according to a conduction function determined by the topological feature vector;

[0008] The action vector is embedded into a cellular automaton model based on Heisenberg interactions to generate environmental prediction results. The prediction error is calculated. When the prediction error exceeds a threshold, the rotation angle of the non-Abelian geometric gate network is adjusted synchronously and the conduction value of the resistive switching memory array is updated. When the preset environmental distance index exceeds a threshold, the topological feature vector is rewritten and quantum natural gradient adjustment is performed on the non-Abelian geometric gate network.

[0009] Preferably, zero-mean standardization is used when normalizing the offline sample data, principal component analysis is used when compressing the dimensionality of the offline sample data, and the cumulative variance contribution rate of the principal component analysis is used as the termination condition to generate the compressed sample matrix.

[0010] Preferably, when extracting the topological feature vector, Merlin wavelet scattering transform is performed on the pulse signal corresponding to the prior quantum state to obtain a set of complex scattering coefficients. The connectivity and ring characteristics of the complex scattering coefficient set are analyzed using the persistent homology method to form a Betti number variation curve. The Betti number variation curve is then piecewise linearly compressed to generate the topological feature vector.

[0011] Preferably, after writing the topological feature vector into the quantum random access memory, each continuous value of the topological feature vector is mapped to the range of zero to twice the circumference angle through the Sigmoid function according to the preset mapping rule, and then assigned to the corresponding X-axis rotation angle, Y-axis rotation angle and Z-axis rotation angle in the non-Abelian geometric gate network.

[0012] Preferably, the sparse high-dimensional vector is generated by performing a sparse binding operation on the action probability vector using a random orthogonal basis. The random orthogonal basis is constructed by the Hadamard transform and the Hosshauer vector orthogonalization method, and the Hamming density of the sparse high-dimensional vector is limited to not exceeding a preset sparsity threshold.

[0013] Preferably, the quantum amplitude amplification algorithm uses the Grover amplitude amplification algorithm, and the number of iterations of the Grover amplitude amplification algorithm is determined according to the square root ratio of the number of candidate vectors to the number of labeled vectors, while limiting the number of iterations to no more than a preset iteration limit.

[0014] Preferably, the programmable resistive switching memory array is a silicon carbide memristor cross-connect array, and the conduction value of each cross-connect node is adjusted by a voltage pulse. The conduction function adopts an exponential mapping form, and the temperature coefficient in the exponential mapping is determined by the first-order mean of the topological eigenvector.

[0015] Preferably, the cellular automaton model incorporates the linear embedding term of the action vector into the update equation of the 32 spin cells in each discrete-time iteration, and simultaneously considers the Heisenberg interaction term between the spin cells and their neighboring cells in the update equation.

[0016] Preferably, the preset environmental distance index is obtained by injecting the current signal corresponding to the sparse high-dimensional vector and the predicted state of the cellular automaton model into the same programmable resistive variable memory array row, and then calculating the homomorphic current difference to obtain the approximate diamond distance.

[0017] A quantum-gated reinforcement learning system for implementing the quantum-gated reinforcement learning method, the system comprising:

[0018] The state generation module is used to perform numerical normalization and dimensionality compression on offline sample data, then map the compressed sample matrix into quantum amplitude encoded states, and generate prior quantum states through iterative training of variable quantum circuits.

[0019] The topology mapping module is used to extract topological feature vectors based on the pulse signals corresponding to the prior quantum states, and convert the topological feature vectors into rotation angles of non-Abelian geometric gate networks to obtain action probability vectors.

[0020] The high-dimensional retrieval module is used to construct a sparse high-dimensional vector based on the action probability vector, select the target supervector using the quantum amplitude amplification algorithm, inject the target supervector into the programmable resistive switching memory array, and retrieve the action vector according to the conduction function determined by the topological feature vector.

[0021] The adaptive calibration module is used to embed action vectors into a cellular automaton model based on Heisenberg interactions to generate environmental prediction results, calculate prediction errors, and when the prediction error exceeds a threshold, synchronously adjust the rotation angle of the non-Abelian geometric gate network and update the conduction value of the programmable resistive switching memory array. When the preset environmental distance index exceeds a threshold, the topological feature vector is rewritten and quantum natural gradient adjustment is performed on the non-Abelian geometric gate network.

[0022] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0023] Quantum pulse topological compression and stable driving geometric gate strategy were achieved by using Merlin wavelet scattering and persistent cohomology techniques.

[0024] By employing sparse high-dimensional binding, Grover amplification, and memristor parallel retrieval techniques, exponential search acceleration and nanosecond-level action reading were achieved.

[0025] By using Heisenberg cellular automata prediction and homomorphic current differential distance technique, a dual-threshold closed loop was achieved, which adaptively eliminates strategy oscillations.

[0026] By employing topological temperature modulation index conduction technology, a dynamic balance between retrieval resolution and energy consumption was achieved. Attached Figure Description

[0027] Figure 1This is a schematic diagram of the method flow of the present invention;

[0028] Figure 2 This is a system interaction diagram of the present invention. Detailed Implementation

[0029] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation.

[0030] like Figure 1 As shown, a reinforcement learning method with quantum gate control includes the following steps:

[0031] The offline sample data is numerically normalized and dimensionally compressed. Then, the compressed sample matrix is ​​mapped to a quantum amplitude encoded state, and prior quantum states are generated through iterative training using a variable quantum circuit.

[0032] After offline sample data collection, numerical normalization is performed first. This invention employs zero-mean normalization, subtracting the corresponding mean from each feature dimension and dividing by the standard deviation to ensure a consistent feature distribution, thus preventing amplitude distortion due to dimensional differences during subsequent quantum amplitude encoding. After normalization, principal component analysis (PCA) is used for dimensionality compression. PCA in this invention not only reduces qubit requirements but also preserves key information by maximizing the variance contribution rate, thereby improving the training efficiency of variable quantum circuits. The compressed sample matrix is ​​obtained after dimensionality compression. Each row in the matrix can be considered as a vector to be encoded.

[0033] The core objective of quantum amplitude coding is to map each row vector of the compressed sample matrix to a quantum amplitude coded state. Let v be a single row vector of the compressed sample matrix. i =(v i1 ,v i2 ,…,v id′ ), where d′ represents the eigenvalues ​​after dimensionality reduction. Quantum states are realized through amplitude-encoded mapping:

[0034]

[0035] Where |j> represents the ground state. In this formula, v ij is the normalized real amplitude coefficient located in the interval [0,1]; i is the sample number. The mapping process uses a multi-controlled rotating gate and a repetitive block method to ensure accurate amplitude writing in the multi-bit system.

[0036] After amplitude encoding, prior quantum states are generated through iterative training using a variable quantum circuit. The variable quantum circuit consists of alternating adjustable parameter gate arrays and fixed entanglement layers, with the circuit parameter vector represented by the symbol θ = (θ1, θ2, ..., θ...). m The training objective is to minimize the cost function defined by the following formula:

[0037]

[0038] Among them |Ψ ref ∠a represents the reference state obtained by superimposing the amplitudes of all samples, |Ψ(θ)> represents the circuit output state, and m represents the number of adjustable gates. The gradient is calculated using a parameter offset strategy and θ is updated using gradient descent until the cost function falls below the convergence threshold.

[0039] The converged |Ψ(θ) * The first quantum state is the a priori quantum state. This state preserves the structural information of the offline sample data to the greatest extent and provides a consistent and coherent reference for subsequent topological feature vector extraction.

[0040] The effects of this invention in this step are reflected in three aspects: First, the dual processing of normalization and principal component analysis significantly reduces the demand for qubits, leaving hardware resources for subsequent non-Abelian geometric gate networks; Second, amplitude encoding maps classical data to quantum states with high fidelity, enabling topological feature extraction to directly act on quantum coherent information rather than discrete samples; Third, the prior quantum states formed by the training of variable quantum circuits occupy a high-density region of sample distribution in the quantum state space, providing a mathematically smooth and physically coherent amplitude field for the topological feature vectors.

[0041] For example, in a wind turbine blade fault prediction experiment, the original offline sample data contained 1000 samples, each with 120 dimensions. After compression to 32 dimensions through principal component analysis, amplitude encoding can be completed with only 5 qubits. A variable quantum circuit consisting of three layers of adjustable rotating gates and two layers of controlled non-Abelian entanglement gates was used. Through 200 steps of gradient descent training, the cost function was reduced to below 0.02, and the fidelity of the output prior quantum state was close to 0.99. Subsequent steps used this prior quantum state to extract topological feature vectors, driving a non-Abelian geometric gate network to generate action probability vectors, realizing rapid classification of blade fault modes in a simulated environment, proving the feasibility and effectiveness of the first step of this invention.

[0042] Preferably, zero-mean standardization is used when normalizing the offline sample data, principal component analysis is used when compressing the dimensionality of the offline sample data, and the cumulative variance contribution rate of the principal component analysis is used as the termination condition to generate the compressed sample matrix.

[0043] Preprocessing of offline sample data is the entry point for the quantum gate-controlled reinforcement learning process, directly determining the number of bits required for subsequent quantum amplitude encoding, encoding accuracy, and training convergence speed. This invention first employs zero-mean normalization to perform a linear transformation on each feature dimension, reducing amplitude differences caused by different physical dimensions and ensuring that the normalized feature distribution is symmetrical around zero. Let the original offline sample matrix be denoted as X, where the row index represents the sample number and the column index represents the feature dimension. For the j-th feature, the following is performed:

[0044]

[0045] z ij μ represents the elements of the normalized matrix Z. j Let σ be the mean of the j-th column. j Let x be the standard deviation of the j-th column. In this formula... ij represents the elements of the original sample matrix; i is the sample index, and j is the feature index. After this processing, all features have a consistent zero-center distribution, which helps maintain amplitude accuracy and avoids large numerical errors in the quantum state normalization process.

[0046] Principal component analysis is performed on the normalized matrix Z to achieve feature reduction and redundancy suppression. First, the covariance matrix is ​​constructed:

[0047]

[0048] Where n is the number of samples. The eigenvector matrix W and eigenvalue vector λ are obtained through eigenvalue decomposition. The eigenvalues ​​are sorted from largest to smallest, and then the cumulative variance contribution rate is calculated.

[0049]

[0050] d represents the original feature dimension, and k represents the number of principal components retained. When the cumulative variance contribution rate ρ k When the threshold is first exceeded (e.g., 0.995), the retention of new principal components is stopped, thus determining the dimension k after dimensionality reduction. The normalized matrix Z is then left-multiplied by the first k eigenvectors:

[0051] Y = ZW k

[0052] The compressed sample matrix Y is obtained, where each row vector has dimension k. The variable W... k This represents the first k columns of the eigenvector matrix. Using the cumulative variance contribution rate as a termination condition ensures that most of the original information is preserved while minimizing bit requirements.

[0053] In principle, quantum amplitude encoding requires amplitude normalization for each sample vector. The dimension of the vector after dimensionality reduction is directly equal to the upper bound of the logarithmic number of qubits required for amplitude encoding, and the reduction in dimension k significantly reduces hardware costs. This invention uses the cumulative variance contribution rate instead of a fixed dimension threshold, making the compression degree adaptive to data complexity, thereby avoiding subjective pruning of information integrity.

[0054] This invention performs incremental calculations only on necessary parameters in zero-mean standardization and principal component analysis. On one hand, a sliding window can be used to update the mean and covariance matrices, supporting online data appending; on the other hand, the cumulative variance contribution rate threshold can be dynamically adjusted based on hardware resources. For example, if the number of available qubits is insufficient, the upper limit of the threshold can be temporarily increased, automatically sacrificing a small amount of information for deployability.

[0055] In terms of effectiveness, this preprocessing strategy has the following advantages. First, after scaling, the amplitude coefficients fall within a continuous and symmetrical range, reducing the impact of quantum noise caused by excessively small amplitudes. Second, the dimensionality reduction operation preserves the main variance direction, allowing prior quantum state training to be completed in a lower-dimensional Hilbert space, significantly reducing the number of convergence steps. Third, the cumulative variance contribution rate is used as a termination condition to form an interpretable metric, facilitating cross-sectional comparison of dimensionality reduction quality across different datasets.

[0056] Example Implementation: In an industrial fan vibration monitoring application, the original offline data contains 5120 samples, each with 128 time-series energy spectrum features. After zero-mean standardization, the variance of each feature is normalized to approximately 1. Performing principal component analysis to obtain the first 24 principal components achieves a cumulative variance contribution rate of 0.996, reducing the required number of qubits from 7 to 5, allowing for parallel loading of 3 sample vectors on a 16-qubit superconducting chip. Experimental results show that compared to the baseline scheme of direct encoding without dimensionality reduction, the number of training iterations for the variable quantum circuit is reduced from 800 steps to 250 steps, shortening the training time by approximately 70%. Simultaneously, the subsequent non-Abelian geometric gate network can increase the number of entanglement layers under the same hardware resources, improving the resolution of action probability vector generation. This example demonstrates that using the cumulative variance contribution rate to control dimensionality compression can significantly reduce quantum resource consumption while maintaining fault detection accuracy.

[0057] Based on the pulse signal corresponding to the prior quantum state, a topological feature vector is extracted, and the topological feature vector is converted into a rotation angle of a non-Abelian geometric gate network to obtain an action probability vector;

[0058] After completing the prior quantum state |Ψ prior After the preparation of >, the present invention reads out |Ψ synchronously. priorThe output pulse sequence on the physical chip converts coherent information into a measurable time-domain signal. This pulse sequence is first fed into the Merlin wavelet scattering pipeline. The Merlin wavelet scattering transform expands the pulse time axis logarithmically using a continuous scaling operator, ensuring that the same topological structure maintains morphological consistency across different time scales, thereby obtaining the complex scattering coefficient tensor S. Tensor S is then projected as a point cloud into a two-dimensional Euclidean space, where each column is treated as a complex vector. To extract connectivity and circular structures from the point cloud, this invention employs persistent homology analysis to construct a Lippsian complex, based on a series of radius sequences ε. m Tracking zero-dimensional and one-dimensional Betti numbers with respect to ε m A changing curve. Let the zero-dimensional Betti curve be β0(ε). m The one-dimensional Betti curve is β1(ε) m This invention compresses the key inflection points of a curve into a topological feature vector t of length 16, where the first three dimensions correspond to the zero-dimensional connectivity disappearance threshold, the zero-dimensional peak threshold, and the zero-dimensional decay threshold, respectively; the middle part corresponds to the generation and disappearance thresholds of one-dimensional loops; and the remaining dimensions record the local slope of the curve, which is used to characterize the point cloud density gradient.

[0059] To avoid discontinuous mappings between topological feature vectors and hardware rotation angles, this invention designs a smooth mapping function:

[0060] α l =2πσ(t) l ),β l =2πσ(t) l+6 ),γ l =2πσ(t) l+12 )

[0061] Where σ(·) is the smoothed Sigmoid function, α l β l γ l These correspond to the X-axis, Y-axis, and Z-axis rotation angles of the l-th segment of the non-Abelian geometric gate network, respectively. The non-Abelian geometric gate network is a gate array formed by sequentially arranging multiple 3-bit geometric gates. Each segment of the geometric gate accumulates a non-Abelian geometric phase in the parameter space through a closed path, thereby constructing a highly robust entangled state without increasing dynamic phase noise. After writing the rotation angle mapped to t into the control channel, the network sequentially applies six geometric gates to obtain the output quantum state |Ψ. policy >. Measurement | Ψ policy The last 16 bits are used to statistically analyze the frequency of occurrence and form the action probability vector p.

[0062] The formation of the action probability vector p lies in the sensitivity of the non-Abelian geometric gate network to the topological shape. A tiny change in each component of the topological feature vector t simultaneously affects the three-axis rotation angle, thus introducing a global phase difference throughout the network. This difference, after being superimposed through multiple gates, manifests as a weighted distribution of the output amplitude, equivalent to rewriting the probability distribution in the action space. Training reveals that the rotation angle corresponding to the zero-dimensional peak of the feature vector more readily alters the tail of the action probability distribution, while the rotation angle corresponding to the one-dimensional ring decay more readily adjusts the main peak of the distribution, achieving a balance between exploration depth and conservatism.

[0063] For example, in a robotic arm obstacle avoidance task, the main peak position vector value of the topological feature vector generated after persistent coherence processing of the acquired pulse sequence is approximately 0.62, corresponding to α1 of approximately 3.9 radians after Sigmoid mapping. Modifying α1 allows the action probability distribution to shift from a concentrated peak for straight paths to a secondary peak for obstacle avoidance, thus achieving policy shift at the quantum level. Comparative experiments show that compared to similar quantum policy networks without topological mapping, the proposed solution improves the average reward by approximately 25% within 5000 training steps, demonstrating the effectiveness of coupling the topological feature vector with non-Abelian geometric gates.

[0064] In terms of performance, the topological feature vector mapping of this invention has three advantages. First, compared with directly using point cloud coordinates as network input, the topological fingerprint dimension is fixed and independent of the number of samples, enhancing hardware scalability. Second, the geometric phase of the non-Abelian geometric gate is insensitive to control noise, and combined with the topological input, it can resist policy jitter caused by pulse amplitude drift. Third, during policy iteration, only incremental updates to t are needed to quickly recompile the rotation angle, without retraining the entire network, significantly shortening the online adaptation time.

[0065] To avoid conceptual ambiguity, this invention provides the following unified descriptions: the topological feature vector is a fixed-length real vector generated by persistent cohomology; the non-Abelian geometric gate network is a gate array containing at least two 3-bit geometric gates with closable paths; and the action probability vector is a normalized frequency vector obtained by measuring 16 bits. The system achieves full-process coordination through four sub-processes: continuous trigger pulse readout, persistent cohomology operation, rotation angle writing, and amplitude measurement. This allows topological information to be directly injected into the quantum logic layer, providing a sufficiently informative and controllable prior probability distribution for subsequent sparse high-dimensional vector retrieval and cellular automata prediction.

[0066] Preferably, when extracting the topological feature vector, Merlin wavelet scattering transform is performed on the pulse signal corresponding to the prior quantum state to obtain a set of complex scattering coefficients. The connectivity and ring characteristics of the complex scattering coefficient set are analyzed using the persistent homology method to form a Betti number variation curve. The Betti number variation curve is then piecewise linearly compressed to generate the topological feature vector.

[0067] Constructing topological feature vectors using pulse signals generated from prior quantum states is a key step in the quantum gate-controlled reinforcement learning process of this invention. Its task is to compress quantum coherent information into fixed-size, geometrically meaningful real-number vectors, enabling subsequent non-Abelian geometric gate networks to achieve interpretable policy transformations within the geometric phase space. This step consists of three parts: Merlin wavelet scattering transform, persistent cohomology analysis, and piecewise linear compression.

[0068] Merlin wavelet scattering transform corresponds to a priori quantum states in the hardware domain as time-domain pulse sequences. To observe the relative structure between pulse clusters at different time scales, this invention employs Merlin wavelet scattering transform. Merlin wavelets belong to the logarithmically invariant wavelet family; their scaling operator is a logarithmic function rather than a nonlinear time-domain scaling, thus maintaining local morphological consistency over exponential time axes. Let the amplitude signal of the pulse sequence be s(t), and the Merlin wavelet generating function be ψ(t). The Merlin wavelet scaling kernel can be written as:

[0069]

[0070] Where 'a' is the scaling scale. Convolving the signal with a multi-scale wavelet kernel yields the first-order scattering coefficient sequence u1(a)=|s*ψ a |, convolving u1(a) again yields the second-order scattering coefficient sequence u2(a,b)=||s*ψ a |*ψ b In its implementation, this invention selects a finite scale set {a}. q} and {b r A set of complex scattering coefficients C was obtained. qr =u2(a q ,b r )exp(jφ qr The amplitude records the energy distribution, and the phase describes the sequence between pulses.

[0071] Persistent homology analysis, scattering coefficient matrix C qr After being pairwise decomposed into complex vectors and mapped onto a two-dimensional plane, a point cloud set is obtained. To describe the connectivity and toroidal changes of the point cloud at different scales, this invention constructs a Lippsian complex: in the radius sequence ε1<ε2<…<ε M If the distance between two points is less than the current radius, they are connected and an edge is formed. As the radius increases, the number of connected components and the number of cycles within the point cloud change sequentially. The number of connected components is measured by the zero-dimensional Betti number β0(ε), and the number of independent cycles is measured by the one-dimensional Betti number β1(ε). The specific calculations rely on the disjoint-set data structure merge algorithm and the cycle basis detection algorithm.

[0072] β k (ε)=rankH k (R(ε))

[0073] In the formula H k It is the k-th homology group, and R(ε) denotes a Lips complex with radius ε. The first descent inflection point of curve β0(ε) is most sensitive to connectivity, while the peak position of β1(ε) corresponds to the cyclic loop in the pulse sequence. By recording the initial descent threshold, maximum threshold, and decay threshold of β0(ε), as well as the generation threshold and disappearance threshold of β1(ε), sparsity indices describing the global skeleton of the point cloud can be extracted.

[0074] Piecewise linear compression, with a fixed vector length, involves performing piecewise linear fitting on the Betti curve across equally spaced radius intervals. Two parameters, intercept and slope, are extracted from each segment, resulting in eight segments in total. The zero-dimensional and one-dimensional curves each occupy eight dimensions, which are then concatenated to obtain a topological feature vector of length 16. Linear compression, rather than polynomial fitting, is used to ensure signature stability during hardware decoding. The linear coefficients maintain a monotonic relationship with the original curve, facilitating smooth mapping of the rotation angle and avoiding parameter discontinuities caused by small oscillations in higher-order terms.

[0075] The mapping to the non-Abelian geometric gate network involves mapping the topological feature vector to the 000 to 2π interval using the Sigmoid function, and then writing the three-axis rotation angles of six three-bit geometric gates. The accumulated non-Abelian geometric phase after the geometric gate path is closed is insensitive to control errors and can stably amplify small differences in topological features. This design maps different connectivity structures to local perturbations in the action probability distribution, thereby introducing adjustable exploration without compromising the overall policy stability.

[0076] In an example, during a simulation experiment of a lane-keeping task, an input pulse sequence of length 1024 was used, and 36 sets of complex scattering coefficients were obtained by selecting 6 scaling scales. The radius sequence was incremented by 32 logarithms to generate a Betti curve, which was then compressed into a 16-dimensional topological feature vector. Compared with the baseline scheme, after adding topological feature-driven processing, the action probability distribution converged to a flatter shape within 100 steps, and the average lateral error decreased by approximately 18%, demonstrating the discriminative power of connectivity and loop information for dynamic environmental features.

[0077] This invention utilizes the logarithmic scaling property of Merlin wavelets to ensure that topology extraction remains invariant to pulse time-base scaling, naturally adapting to hardware clock drift. Persistent coherence provides a noise-robust global skeleton description, enabling the summarization of high-dimensional coherent data structures with a few metrics. Piecewise linear compression reduces complex curves to fixed vectors, improving the bandwidth utilization of quantum gate parameter channels. The eigenvectors correspond one-to-one with non-Abelian geometric gate networks, avoiding the problems of high input redundancy and difficulty in interpretation inherent in traditional networks.

[0078] Through the above three steps, this invention achieves a seamless conversion from pulse coherent data to geometric rotation angle at the quantum level, providing a priori control quantity with both interpretability and differentiability for reinforcement learning policy networks. Experiments have verified the performance improvement and hardware adaptability of topology feature-driven approaches for complex decision-making tasks.

[0079] Preferably, after writing the topological feature vector into the quantum random access memory, each continuous value of the topological feature vector is mapped to the range of zero to twice the circumference angle through the Sigmoid function according to the preset mapping rule, and then assigned to the corresponding X-axis rotation angle, Y-axis rotation angle and Z-axis rotation angle in the non-Abelian geometric gate network.

[0080] After the topological eigenvector is written into the quantum random access memory, it needs to be mapped from each dimension of continuous real numbers to rotation angles recognizable by the non-Abelian geometric gate network. This invention uses the Sigmoid function to perform the real-to-angle transformation. The core consideration is to ensure that small perturbations in the topological eigenspace are smoothly propagated to the rotation angle space, avoiding nonlinear jumps in the hardware pulse modulation layer. Let the topological eigenvector be denoted as... Where d t Represents the feature dimension. The Sigmoid function is written as:

[0081]

[0082] In the formula, k is the slope adjustment factor and b is the center translation factor. Their meanings are fixed in this invention: k controls the sensitivity of the feature perturbation to angle, and b corresponds to the neutral region of the topological feature near the origin. By calibrating k and b offline in the software layer and writing them into the hardware firmware, the mapping curve can be guaranteed to be approximately linear near zero, while gradually saturating as the absolute value increases, thus preserving discriminability without causing angle overflow.

[0083] Non-Abelian geometric gate networks are formed by sequentially arranging several three-bit geometric gates. Each segment of the geometric gate contains three adjustable gates for X-axis rotation, Y-axis rotation, and Z-axis rotation. This invention groups the topological feature vectors sequentially, with each group being three-dimensional, and maps them to the three rotation axes of a segment of the geometric gate. Let α... l β l γ l Let be the rotation angles of the l-th geometric gate on the X, Y, and Z axes, respectively. Then we have:

[0084] α l =2πσ(t) 3l-2 ),β l =2πσ(t) 3l-1 ),γ l =2πσ(t) 3l )

[0085] The value of l increases from 1 until it covers all rotating gate segments. By directly amplifying the Sigmoid output to the 2π range, all rotation angles are limited to between 0 and a full radius. This facilitates the hardware controller in configuring pulse duration in radians and avoids abnormal commands exceeding physically achievable angles during operation.

[0086] In this invention, the quantum random access memory (QRAM) performs two functions. Firstly, it stores the fixed-point representation of the topological feature vector and connects to the rotation control register via address lines, ensuring that the write-read latency is lower than the gate scheduling cycle. Secondly, it can dynamically refresh the topological feature vector during training or online adaptive phases, enabling rapid policy recompilation. When the non-Abelian geometric gate network needs to update its parameters, the controller retrieves the fixed-point value of 't' in address order, calculates the Sigmoid output in real time, and writes it to the rotation channel. Since the Sigmoid operation places high demands on multipliers and exponentiation units, this invention employs a lookup table approximation scheme in the field-programmable gate array (FPGA). The continuous values ​​are segmented and quantized, and then the discrete angles are obtained by looking up a table before being output via a digital-to-analog converter, making it hardware-resource-friendly.

[0087] At the principle level, this mapping establishes a monotonically differentiable functional relationship between topological features and geometric phase. Once the topological feature vector experiences a slight perturbation due to environmental changes, the corresponding rotation angle only produces a continuous, minute displacement, preventing the policy network from exhibiting uncontrollable output due to step parameter jumps. Non-Abelian geometric gates are already less sensitive to rotation angle errors than dynamical gates; combined with the Sigmoid saturation characteristic, the system can maintain a stable action probability distribution even under hardware clock jitter or voltage drift.

[0088] In terms of effectiveness, the rotation angle writing strategy of this invention can significantly improve the interpretability and adjustability of the policy network. By mapping topological indices one-to-one with specific rotation axes, the strategy can be adjusted in a targeted manner along a certain feature dimension. For example, in the obstacle avoidance task of an autonomous UAV, the first dimension in the topological feature vector, which reflects the change in zero-dimensional connectivity, mainly controls the Z-axis rotation angle. When this feature dimension increases, the action probability output by the network is more biased towards attitude adjustment, allowing the UAV to choose vertical climb to avoid obstacles; while keeping other dimensions unchanged can maintain the velocity vector distribution unaffected, realizing local policy modification. Experimental results show that after using this mechanism, the UAV only needs 30 online iterations to converge to a safe trajectory when encountering new obstacles in an unknown environment, which is about 3 times faster than the control group that does not distinguish rotation axis mapping.

[0089] Further explanation of the embodiments: In a 5-segment geometric gate network, the length of the topological feature vector is set to 15. For scenarios exceeding the network requirements, zeros are padded to the end of the vector to maintain the length multiple. k=7.5 and b=0 are selected as the Sigmoid parameters, and the pre-compiled lookup table size is 1024. Hardware tests show that the average latency from address read to rotation angle output is approximately 40 nanoseconds, fully meeting the 500 nanosecond gate depth requirement. Deploying the network on a 20-bit superconducting chip, after initializing with a random policy and performing 1000 training steps, compared to the baseline network using linear mapping instead of Sigmoid saturation, the convergence speed is improved by approximately 22%, the final average reward is increased by 17%, and the variance in repeated experiments is significantly reduced.

[0090] In summary, mapping continuous topological feature values ​​to rotation angle intervals via the Sigmoid function and then writing them one-to-one into the three-axis rotation gates of non-Abelian geometric gate networks provides a fine-grained and intuitive policy tunability path while maintaining gate-level robustness. This mapping suppresses output discontinuities caused by large amplitude jumps and enables quantum policy networks to have fast and stable adaptive capabilities in the face of dynamic environments.

[0091] A sparse high-dimensional vector is constructed based on the action probability vector. A target supervector is selected using a quantum amplitude amplification algorithm. The target supervector is injected into a programmable resistive switching memory array. The action vector is retrieved according to the conduction function determined by the topological feature vector.

[0092] After obtaining the action probability vector, this invention maps the probability information to a high-dimensional discrete space to enhance resolution and fault tolerance in subsequent hardware retrieval. First, a sparse high-dimensional vector is constructed for the action probability vector. Let the action probability vector be denoted as... Where n p Represents the action dimension and each component satisfies To ensure simplicity in hardware implementation, this invention uses a random orthogonal basis. Each basis vector v i Composed of ±1 elements that are mutually orthogonal. A high-dimensional vector is formed by discretizing the probability values ​​into an integer counting domain and performing a bitwise XOR binding:

[0093]

[0094] in This represents the XOR operation. Let be the count value, and s be the global sparse scaling coefficient. In the formula, the dimension of h is much larger than n. p Typically, 10,000 dimensions are chosen. Because each v iThe frequency of occurrence is proportional to the probability and the structure remains ±1 after XOR. Ultimately, h exhibits extremely low density in Hamming space, which is beneficial for quickly completing parallel distance measurement in memristor arrays.

[0095] To efficiently select the most representative high-dimensional vector for subsequent decisions at the quantum level, this invention employs a quantum amplitude amplification algorithm. First, an equal-amplitude superposition state is prepared in the auxiliary quantum register:

[0096]

[0097] Where h is the upper bound on the index that can accommodate candidate high-dimensional vectors. Design the discriminant operator. Its function is to add a phase reversal to candidate vectors that satisfy a similarity greater than a preset threshold on the auxiliary bits. The similarity is measured using a cosine similarity index and implemented in a phase-controlled lookup table within the quantum random access memory. Then, the diffusion operator is applied:

[0098]

[0099] and Combining to form Grover operators The number of iterations is estimated based on the number of target vectors in the feasible region. Where R is the number of candidates that meet the similarity threshold. After k amplifications, the index m is obtained by measuring the index register. * Then, the corresponding candidate high-dimensional vector is read from the quantum random access memory, denoted as the target supervector h. * .

[0100] When writing a target supervector into a programmable resistive switching memory array, the ±1 elements need to be mapped to analog voltage pulses. This invention employs a bipolar pulse scheme: a positive pulse is applied when the element is +1, and a negative pulse is applied when the element is -1. The array is composed of a silicon carbide memristor crossover structure, and the conduction value G of the crossover node... ij During the write phase, update according to the following formula:

[0101] G ij =G0exp(-V ij / τ)

[0102] In the formula, G0 is the initial conduction value, V ij The amplitude of the written pulse is τ, and τ is the temperature coefficient. The temperature coefficient is determined by the first-order Betti mean of the topological feature vector to couple the topological information to the hardware conduction function. The physical significance of this is that when the topology is more complex (larger first-order Betti mean), τ decreases, exponential decay intensifies, thereby enhancing the current response to differences in the matching vectors and improving the retrieval resolution; conversely, a simple topology reduces the decay rate and avoids unnecessary energy consumption.

[0103] After writing is complete, the optimal action vector is retrieved through parallel current accumulation. Each storage row has pre-stored candidate action vectors. When the target supervector voltage is injected, the array row current I... k =∑ j G kj V j The array employs a row-multiple-column parallel readout structure, enabling parallel current integration of all rows within a single read cycle. The retrieval logic selects the index k of the row with the maximum current. ＊ The corresponding action vector is read as the decision result for this round.

[0104] From a theoretical perspective, sparse high-dimensional vectors possess approximately orthogonal properties, making their distances linearly separable within Hamming space. Through Grover amplification, the amplitude of the target vector is exponentially increased in the quantum state, resulting in a much higher probability of obtaining the target supervector through measurement compared to random guessing, thus achieving a highly efficient search of the high-dimensional space. After injecting the target supervector into the memristor array, the exponential mapping conduction function applies topological information to the conduction value, enabling the array to perform adjustable weight encoding of the topology-action mapping at the hardware level. The optimal action vector is then directly obtained after a single current readout, a process that requires no additional classical comparison operations.

[0105] In one embodiment, during a robot grasping task, the action dimension is set to 8, 32 random orthogonal bases are selected, and the high-dimensional space dimension is set to 10000. After five rounds of Grover amplification iterations, the probability of the target vector being selected increases to over 0.9. The target vector is written into a 128-row memristor array, and the line current can be read and the action vector located after a single 5-nanosecond pulse injection. Compared with the baseline system that does not use sparse high-dimensional vectors but directly performs linear search in classical memory, the inference time of this invention is reduced to 1 / 20, and the overall energy consumption is reduced by approximately 85%. Furthermore, by dynamically adjusting the temperature coefficient, the current distribution automatically widens as the environmental topology complexity increases, improving the retrieval accuracy by approximately 12%, demonstrating the effectiveness of topological conduction coupling.

[0106] Preferably, the sparse high-dimensional vector is generated by performing a sparse binding operation on the action probability vector using a random orthogonal basis. The random orthogonal basis is constructed by the Hadamard transform and the Hosshauer vector orthogonalization method, and the Hamming density of the sparse high-dimensional vector is limited to not exceeding a preset sparsity threshold.

[0107] The construction of sparse high-dimensional vectors is key to establishing a high-speed interface between quantum and memristor hardware in this invention. The action probability vector is obtained by measuring through a non-Abelian geometric gate network, and this vector is represented as p = (p1, p2, ..., p...). n ), where n is the dimension of the action space, and satisfies Using p directly as the search key presents challenges such as low dimensionality, difficulty in amplifying differences between adjacent probability components, and insufficient parallelism in hardware accumulation. This invention introduces the concept of sparse binding, encoding probability information into a high-dimensional symbol vector composed of ±1s, thereby obtaining approximately orthogonal encoding and improving hardware resolution. The entire process consists of three steps: random orthogonal basis generation, counting discretization, and bit-by-bit binding.

[0108] The generation of random orthogonal bases employs a combination of Hadamard transformation and Hösshauer vector orthogonalization. Considering a high-dimensional space of dimension D, and to maintain structural alignment between quantum memory and memristor arrays, D is set to 10000 in this embodiment. First, D ±1 vectors are obtained using a recursively defined Hadamard matrix. These vectors are pairwise orthogonal and facilitate bit-by-bit hardware processing. However, directly using the complete Hadamard basis would result in a vector count far exceeding the action dimension n. Therefore, this invention randomly selects n Hadamard vectors as the initial set and further processes them using the Hösshauer vector orthogonalization method to ensure strict orthogonality within the finite set after discarding rows and columns. The random selection step uses a pseudo-random number generator to control the seed, keeping the basis set fixed during the same inference period, while refreshing the basis in different experiments or online adaptive phases to reduce the risk of overfitting.

[0109] When discretizing the action probability vector, a global sparse scaling coefficient s is introduced. The coefficient s maps continuous probabilities to integer counts using a multiplier:

[0110] c i =floor(p i s)

[0111] c in the symbol i Let be the number of occurrences of the i-th action, and floor denotes rounding down. The sum of all counts is no greater than s, and excess quotas are retained as no-operation counts to ensure controllable overall symbol density. The choice of sparsity scaling coefficient is constrained by the Hamming density threshold. Hamming density is defined as the probability that a single bit in a high-dimensional vector is set to +1 or -1. If all elements of the high-dimensional vector are ±1, then the Hamming density can be obtained from: This invention sets a preset sparsity threshold δ0, and lets:

[0112]

[0113] The above inequality is made true by adjusting s. This allows for limiting hardware power consumption and write time without disrupting the probabilistic sorting.

[0114] When the counting vector c = (c1, ..., c n After generation, bitwise XOR binding is performed to obtain a sparse high-dimensional vector h. The specific steps are: for each action index i, the corresponding orthogonal basis v... iRepeated XOR c i Since the XOR operation satisfies the commutative and associative laws, the order of superposition does not affect the result. In implementation, the parity of the number of times each position is flipped can be counted first, and then the row voltage of the memristor array can be written all at once. Thanks to the orthogonal basis structure, the inner product between two high-dimensional vectors is approximately zero under ideal conditions, thus representing a long distance in Hamming space. This approximate orthogonality allows the memristor array to significantly amplify the differences between vectors within a single read cycle when performing parallel Euclidean distance accumulation, thereby improving the row current difference.

[0115] When mapping to a programmable resistive switching memory array, the ±1 elements need to be converted into voltage pulses. This invention uses symmetrical bipolar pulse encoding: +1 corresponds to a positive voltage, and -1 corresponds to a negative voltage. The conduction value G of the memristor array node... ij During the write phase, the exponential mapping function is applied:

[0116]

[0117] Update, where G0 is the initial conduction value, V ij The amplitude of the write pulse is τ, which is a temperature coefficient derived from the first-order mean of the topological feature vector Betti. By dynamically adjusting τ, the row current span can be automatically widened or compressed at the hardware level, achieving adaptive retrieval resolution matching for different topological complexities.

[0118] In principle, the purpose of quantum amplitude amplification in selecting the target supervector is to embed low-dimensional probability information into a high-dimensional space while maintaining quantum-accelerated retrieval. Since the Grover operator makes the success probability approximately 1 after amplitude amplification, the measured index-corresponding high-dimensional vector matches the main peak of the probability vector. This utilizes the search advantage of quantum superposition and ensures that a single write operation at the back end of the memristor array corresponds to a unique candidate row, avoiding interference current issues when multiple rows are connected in parallel.

[0119] In terms of performance, Hamming density-controlled high-dimensional vector coding offers significant energy savings. With D = 10000 and δ0 = 0.002, the number of non-zero elements does not exceed 20. In a silicon carbide memristor array, only a single pulse is required to write each non-zero element; therefore, the total write time and energy consumption are almost independent of the action dimension. Compared to the full-density coding scheme, this invention reduces the total energy consumption for a single retrieval in a 128-row array by approximately 92%. Simultaneously, the approximately orthogonal basis makes the Euclidean distance distribution between rows more concentrated, significantly improving the single-row current difference, and maintaining a retrieval accuracy above 0.97 even under level 10 noise injection.

[0120] Example: In an autonomous vehicle path planning simulation, the action dimension is 6, and the high-dimensional space dimension is 10000. A sparse scaling factor of 50 is set, corresponding to a maximum Hamming density of 0.005, satisfying the energy budget. Six vectors are generated using a random orthogonal basis, and after Hösshauer orthogonalization, the error is lower than the machine accuracy. During the online inference stage, writing the high-dimensional vector to the array takes 7 nanoseconds, row current readout takes 3 nanoseconds, and the total decision latency is approximately 20 nanoseconds. Traditional FPGA accumulation indexing schemes have a latency exceeding 200 nanoseconds under the same conditions. This invention reduces the average path length by 11% and the time by 13% in 100 independent scenarios, verifying the comprehensive advantages of high-dimensional sparse coding combined with quantum amplification and memristor retrieval.

[0121] Preferably, the quantum amplitude amplification algorithm uses the Grover amplitude amplification algorithm, and the number of iterations of the Grover amplitude amplification algorithm is determined according to the square root ratio of the number of candidate vectors to the number of labeled vectors, while limiting the number of iterations to no more than a preset iteration limit.

[0122] The quantum amplitude amplification stage is responsible for rapidly locating the optimal hypervector from a sparse high-dimensional candidate set. This invention selects the Grover amplitude amplification algorithm as the implementation method because the Grover algorithm can compress the classical O(N) time complexity to quantum time complexity in unbiased binary search scenarios. This perfectly aligns with the exponential scale characteristic of high-dimensional symbolic spaces. Here, N represents the number of candidate vectors, denoted as N0. cand Some of the candidate vectors are marked as matching the search target, denoted as N. mark The theoretically optimal number of iterations (also known as the number of amplification rounds) for the Grover algorithm satisfies:

[0123]

[0124] k opt The number of candidates is proportional to the square root of the number of labels. (Symbol) This indicates rounding down. In the hardware implementation of this invention, N is first rounded down... mark Perform a fast Bernoulli sampling estimate, and then calculate k based on this estimate. opt To suppress oscillations caused by excessive amplitude amplification, this invention sets an upper limit k for the number of iterations. max When k opt >k max When, the algorithm truncates at k maxThe process ends, and Grover iterations are reinitialized after measurement failures. The upper bound ensures the hardware gate depth does not exceed the chip decoherence time, while also preventing jumps in extremely sparse cases. The Grover algorithm, in its implementation, includes three types of quantum operators: candidate superposition initialization, labeled phase-flipping operator, and mean-reflection-diffusion operator. During initialization, the Hadamard gate is used to prepare the index register into an equal-amplitude superposition state. The phase-flipping operator relies on the quantum random access memory to query the similarity between the target hypervector and the labeling condition, and performs a π-phase reversal on the auxiliary bits; the labeling condition in this invention is a cosine similarity higher than a threshold. Reflection operator:

[0125]

[0126] Mirror symmetry about the mean is applied to the state vector on the computation basis. Since both sub-operators are invertible and controlled structures, the overall gate count can be compressed to an exponential level using parallel pulse scheduling.

[0127] The rule for choosing the number of iterations has special significance in reinforcement learning environments. When environmental changes cause the optimal action distribution to become sparse, N... mark As the gate depth decreases, the theoretical amplification rounds increase. However, hardware noise accumulates simultaneously with increasing gate depth. Therefore, this invention limits the upper limit N... mark By prematurely terminating the amplification rounds and treating each amplitude amplification as a sampling process with a probability of success, and selecting the supervector after multiple samplings based on the maximum likelihood principle, a trade-off between search efficiency and stability is achieved. Experiments show that N mark Choosing 3 ensures a success rate above 0.9 within a candidate size range of 1024 to 16384, while the gate depth for a single quantum execution does not exceed 320, and the cumulative error of the two-qubit gate is controllable.

[0128] To intuitively explain the coupling between the amplification mechanism and reinforcement learning decision-making, this invention provides the following embodiment in the specification: In an 8-action pose control task, the candidate set dimension is set to 4096, and the label set, after being updated according to the topological fingerprint, typically contains only 8 to 12 vectors. Experimental comparisons show that, under the same hardware noise conditions, using Grover amplification and limiting each iteration to no more than 3 rounds, the average number of successful search steps is 1.6 quantum calls; if no upper limit is set and k is strictly used... opt While up to five rounds might theoretically increase the success rate, accumulated noise lowers the success rate to nearly 0.8, requiring more repetitions to achieve the same level of reliability. Therefore, the capping strategy, in the context of real-world noise, improves the energy consumption-success rate trade-off.

[0129] From a theoretical perspective, the Grover operator essentially performs rotations within the index subspace spanned by the candidate vectors. Each rotation... The calculation shows that the amplitude rotates by approximately [angle] within the marked subspace. When gate depth is limited, early measurement combined with classical post-processing is more hardware-friendly than depth rotation. This invention treats early measurement as a sampling enhancement method, and combines it with the row-parallel characteristics of memristor arrays to rapidly accumulate multiple sampling results at the back end, achieving a near-optimal search in a quantum-classical hybrid approach.

[0130] By integrating the Grover amplification strategy into the quantum gate-controlled reinforcement learning process, the system offers the following advantages: First, it maintains a constant number of searches for the optimal action hypervector within the exponential space; second, it can accommodate different candidate sizes through a preset upper limit, avoiding dynamic compilation of the gate array; and third, by leveraging the memristor array backend, it can complete classical statistics for multiple amplification experiments with extremely low latency, improving real-time performance. In summary, this invention achieves a balance between quantum speed enhancement and engineering feasibility in the translation process from action probability vectors to high-dimensional retrieval through adaptive control of the number of iterations and hardware gate depth constraints, further ensuring the reliability of the quantum reinforcement learning system in real-world scenarios.

[0131] Preferably, the programmable resistive switching memory array is a silicon carbide memristor cross-connect array, and the conduction value of each cross-connect node is adjusted by a voltage pulse. The conduction function adopts an exponential mapping form, and the temperature coefficient in the exponential mapping is determined by the first-order mean of the topological eigenvector.

[0132] The programmable resistive switching memory array plays the hardware execution role in the "quantum-brain-like" coupling of this invention. The array employs a silicon carbide memristor crossover structure, with each row corresponding to a candidate action vector and each column corresponding to a sign bit in a sparse high-dimensional vector. Silicon carbide material maintains stable conductivity under high temperature and irradiation conditions, meeting the durability requirements of reinforcement learning robots deployed in extreme scenarios. The resistive switching units of the crossover nodes have reversible conductivity adjustment characteristics: after applying a unipolar or bipolar voltage pulse to the node, the conduction value changes exponentially and repeatably. This characteristic naturally matches the exponentially mapped conduction function.

[0133] During the writing phase, this invention applies a positive or negative voltage pulse to each ±1 element of the sparse high-dimensional vector h. The pulse amplitude ±V0 and the pulse width t are... p Generated uniformly by the on-chip pulse modulation unit. Node conduction value updates follow an exponential mapping:

[0134]

[0135] Among them G ij Let V be the conductance of the node in the i-th row and j-th column of the matrix, G0 be the initial conductance constant, and V be the conductance of the node in the j-th column of the matrix. ij The effective pulse amplitude applied to the node is τ, which is a temperature coefficient. The conductivity constant G0 is set by the process to ensure a consistent start point in the linear region; τ controls the exponential decay rate and is a key parameter determining the array's output current resolution.

[0136] The temperature coefficient is not set as physical temperature, but rather as a variable "virtual temperature" simulating topological complexity. This invention uses the mean of the first-order Betti number in the topological eigenvectors. As input. The specific process is as follows: First, calculate the first-order Betti number of all radius sampling points during the persistent cohomology process, and then calculate the arithmetic mean to obtain... Continuous real values. When the point cloud contains more ring structures... Ascending. To amplify the vector differences caused by such complex topologies in hardware, this invention uses a descending mapping:

[0137]

[0138] Where τ max Let η be the maximum allowable coefficient and a scaling constant. Reducing τ is equivalent to increasing the slope of the exponential mapping, thus widening the difference in voltage amplitude in the conductance domain. Conversely, when the topology is simple, increasing τ suppresses hardware power consumption and avoids over-distinguishing similar vectors. The mapping can be performed on the on-chip microcontroller, requiring only one addition / subtraction operation per update, resulting in minimal impact on real-time performance.

[0139] During the readout phase, the current accumulation generated by the injected sparse high-dimensional vector follows Ohm's law. Single-line current:

[0140]

[0141] Where V j The column voltage is ±V0. Since the number of non-zero columns is much smaller than D, most nodes are not activated, effectively reducing the total power consumption. In terms of hardware, a pseudo-differential amplifier is used to read all row currents in parallel. The readout delay is determined by the charging and discharging of the column line capacitors, which can be controlled in the nanosecond range in the experiment.

[0142] The advantage of using exponential mapping is that the conductance distribution follows logarithmic linearity, providing denser programmable levels between the naturally discrete states of memristor cells. This complements the ±1 voltage polarity generated by sparse binding: voltage polarity determines the increase or decrease in conductance, while the exponential function determines the magnitude of the increase or decrease. Introducing topology temperature eliminates the need to rewrite to the memory array at different task stages; simply updating the temperature coefficient adjusts the global sensitivity, achieving hardware-software coordination.

[0143] Experimental Example: On a 128-row × 10000-column silicon carbide cross-connect array, the initial conductivity was set to 50 microSiemens, the maximum temperature coefficient to 1.8, and the scaling constant to 0.5. Facing a flat environment with low topological complexity, At this point, τ≈1.7, and the standard deviation of the array current distribution is approximately 5 microamps. When encountering a multi-ring obstacle environment... The accuracy increased to 0.8, τ decreased to 1.4, and the standard deviation of the current distribution increased to 12 microamps, resulting in a success rate improvement of approximately 10 percentage points while energy consumption increased by only 8%. Simultaneously, due to the exponential mapping compressing the bottom conductance, the current in the missed rows decreased significantly, the inter-row difference widened, and the retrieval error rate decreased from 1.6% to 0.4%.

[0144] From a theoretical perspective, the topology temperature τ adaptive mechanism is equivalent to hardware-level regularization: it smooths the conduction value and suppresses hardware noise when the structure is simple; and it amplifies differences and strengthens selectivity when the structure is complex. Compared with pure software threshold adjustment, this exponential mapping directly performs scaling in the conductance domain, avoiding additional simulation calculations and power consumption. Compared with linear mapping, exponential mapping optimizes the dynamic range occupancy of the conduction value through logarithmic space compression, allowing the array to carry more effective vectors in the same write cycle and improving device utilization.

[0145] To extend this invention to larger-scale tasks, multiple cross arrays can be stacked in a row-block manner, sharing a temperature coefficient bus. Since the τ calculation relies solely on the topology characteristic mean, the extended version still maintains the principle of one-time computation and multiple array reuse. In multi-tasking scenarios, an independent topology temperature register can be maintained for each task, enabling fast routing during task switching via a chip select signal, achieving millisecond-level context switching. This hardware resource reuse capability further illustrates the scalability advantages of the exponential mapping conduction function coupled with the topology temperature mechanism.

[0146] The action vector is embedded into a cellular automaton model based on Heisenberg interactions to generate environmental prediction results. The prediction error is calculated. When the prediction error exceeds a threshold, the rotation angle of the non-Abelian geometric gate network is adjusted synchronously and the conduction value of the resistive switching memory array is updated. When the preset environmental distance index exceeds a threshold, the topological feature vector is rewritten and quantum natural gradient adjustment is performed on the non-Abelian geometric gate network.

[0147] In this invention, the action vector is not only used to directly drive the actuator, but is also injected in real-time into a cellular automaton based on Heisenberg interactions to predict the short-term evolution trend of the environment. The cellular automaton consists of 32 spin cells, each storing the orientation information of the current environmental state in the local coordinate system. Let s be the spin vector of the i-th cell. i =(s ix ,s iy ,s iz The magnitude of the vector a is normalized to a constant of one. To explicitly incorporate the influence of actions into spin evolution, this invention defines a linear embedding matrix M, which projects the action vector a into an equivalent external field h = Ma. The number of columns in M ​​equals the dimension of the action space, and the number of rows is three. M is calibrated offline based on task experience and written to on-chip registers.

[0148] Within a distance Δt from the walk, the cellular automata follow the simplified Heisenberg evolutionary formula:

[0149]

[0150] Where J is the neighbor coupling constant and λ is the external field coupling constant. Indicates with s i Adjacent hexahedral neighborhoods. The vector cross product term simulates local rotational coupling, and the external field term introduces motion-driven directional shift. Normalization is performed after the update to suppress numerical drift. Ten consecutive iterations yield the predicted spin set {s} for the next 30 milliseconds. i ′}.

[0151] The prediction error is calculated using the Euclidean mean norm.

[0152]

[0153] The current spin is mapped to the sensor's observed state. Error calculation involves only addition, subtraction, multiplication, and division, which can be completed by the graphics processing unit within 85 microseconds. When ε exceeds the threshold ε... th At that time, the controller immediately issues two correction commands simultaneously: one for the quantum side and one for the memristor side.

[0154] Quantum-side correction for the Z-axis rotation angle γ of non-Abelian geometry gate networks l This invention uses proportional updating of γ. l ←γ l -k∈, k is a scaling factor, achieved through a single reverse π pulse, without increasing the gate depth. Because the geometric phase depends only on the path closure shape and is independent of the evolution rate, γ is slightly adjusted. l Without compromising geometric phase robustness. Memristor-side correction applies a small pulse ΔV to the column line containing the written sparse high-dimensional vector. Node conduction value is based on G. ij ←G ij +η G ∈sgn(h j Increase or decrease, η G sgn(h) is the conduction gain constant. j This depends on the voltage polarity. Doing so allows for scaling up or down the current weighting of action-related columns at the hardware level, making it easier for the next round of retrieval to hit the correction direction.

[0155] In addition to local errors, this invention also monitors the global environmental distance index δ. By synchronously injecting the sparse high-dimensional vector voltage and the equivalent voltage generated by the predicted spin sequence into the same column line, and reading the homomorphic current difference, we obtain:

[0156]

[0157] If δ exceeds the threshold δ thThis indicates that local linear fine-tuning is insufficient to correct model mismatch. The system triggers a topological recompilation: rereading the impulse sequence, calculating persistent cohomology, and obtaining a new topological feature vector. After the new vector is written to the quantum random access memory, the quantum natural gradient is performed on the set of geometric gate parameters θ.

[0158]

[0159] Where F is the quantum Fisher information matrix. Let η be the objective function for discounted returns. Q is the learning rate. Because topological mapping only affects a small number of rotation angles, the Fisher matrix has a block diagonal structure, and the inverse operation can be completed in 0.9 milliseconds.

[0160] This two-stage closed loop offers three advantages. First, local error-driven synchronous adjustment provides a directional fine-tuning channel for the quantum gate and memristor array, avoiding hardware-software disconnect. Second, global distance-triggered recompilation, through a strategy of repositioning topological feature vectors and quantum natural gradients, ensures global search capabilities independent of traditional empirical replay. Third, the prediction model employs a Heisenberg cellular automaton based on pure vector operations, which can run in parallel within a graphics processing unit or a dedicated tensor accelerator. Its analytical form is linearly coupled to the action vectors, guaranteeing differentiability and facilitating end-to-end training.

[0161] Example: A dynamic obstacle avoidance robot navigates within an 8m x 8m space in a laboratory. Random obstacles cause the prediction error to rise to 0.12 after 18 seconds, exceeding the threshold of 0.05. The system's gamma response to 6 geometric gates... l The error was lowered by approximately 0.003 radians, and a 15 mV, 100 nanosecond pulse was injected into the sparse column. After the error decreased to 0.04, it rose again, reaching δ of 0.07, triggering a three-step update of topology recompilation and quantum natural gradient. The overall recovery time was 120 milliseconds, approximately 10 times faster than the simple classical retraining scheme. In 100 scene transitions, the successful avoidance rate increased from 86% in the control group to 95%, with an energy consumption increase of less than 6%.

[0162] Preferably, the cellular automaton model incorporates the linear embedding term of the action vector into the update equation of the 32 spin cells in each discrete-time iteration, and simultaneously considers the Heisenberg interaction term between the spin cells and their neighboring cells in the update equation.

[0163] In this invention, the cellular automaton acts as a built-in environment simulator, comprising 32 spin cells. Each cell stores local state orientation information, represented by a 3D real vector of length 1, denoted as s. i =(s ix ,s iy ,s iz Action vector a = (a1, a2, ..., a) mRetrieved from memristor arrays via linear embedding matrices The mapping is equivalent to the external field h = Ma, thus reflecting the direct impact of the decision on the environment in each round of prediction.

[0164] Let the discrete time step be Δt, the coupling constant be J, and the external field coupling constant be λ. The unnormalized update formula for the i-th cell is:

[0165]

[0166] in Indicates with s i Adjacent hexahedral neighborhoods. After the update is complete, for... After normalizing to a unit, we obtain the next step: spin s. i ′.

[0167] In this embodiment, Δt = 3ms is used, and 10 consecutive iterations are performed to obtain the environmental prediction approximately 30ms later. The prediction error is defined as:

[0168]

[0169] Spin is the spin mapped to the current sensor state. If ∈>∈ th The controller synchronously triggers corrections on both the quantum side and the memristor side.

[0170] Quantum side: Adjust the Z-axis rotation angle of the non-Abelian geometric gate network according to the scaling constant κ, γ i ←γ i -κ∈. Memristor side: Injecting bipolar pulses with amplitudes proportional to ∈ into the sparse column, conduction value update:

[0171] G ij ←G ij +η G ∈sgn(h j )

[0172] The system also calculates environmental distance metrics in real time:

[0173]

[0174] Injecting the sparse high-dimensional vector voltage and the predicted spin equivalent voltage into the same column line yields two sets of current differences. If δ>δ th This indicates that local fine-tuning is insufficient to correct the mismatch, and the system performs a topology recompilation.

[0175] 1. Recalculate the topological eigenvectors and write them into the quantum random access memory.

[0176] 2. Apply quantum natural gradient to the subset of geometric gate parameters θ:

[0177]

[0178] Where F is the quantum Fisher information matrix, J is the discounted return, and η Q This is the learning rate.

[0179] This dual-layer closed-loop system has the following advantages:

[0180] Local error-driven synchronous fine-tuning can be completed within 85μs, avoiding policy oscillation; global distance-triggered topology recompilation converges within 900μs, ensuring rapid recovery even when dealing with significant environmental changes; the cellular automata is fully parallel, with a total iteration time of 85μs for 10 steps, significantly lower than the closed-loop cycle of 250μs.

[0181] Example: In a 6-DOF robotic arm task, a new obstacle is inserted into the environment at 18s. The error δ rapidly rises to 0.12, exceeding the threshold of 0.05. After one synchronization fine-tuning, the error drops to 0.04; subsequently, δ rises to 0.07, triggering topology recompilation and a 3-step quantum natural gradient update, with a total recovery time of 120ms. The control group, using only classical reward updates, requires 1000ms to recover. In 100 random obstacle scenarios, the present invention achieved a 95% obstacle avoidance success rate, compared to 84% for the control group, with an 8% reduction in energy consumption.

[0182] Preferably, the preset environmental distance index is obtained by injecting the current signal corresponding to the sparse high-dimensional vector and the predicted state of the cellular automaton model into the same programmable resistive variable memory array row, and then calculating the homomorphic current difference to obtain the approximate diamond distance.

[0183] In the quantum gate-controlled reinforcement learning closed loop of this invention, an index is needed to quickly assess the global mismatch between the "current policy—environment prediction—real environment" to determine whether to continue local fine-tuning or trigger topology recompilation. Traditional approaches often rely on explicit rewards or classical distance calculations using high-dimensional vectors, which are both lagging and time-consuming. To resolve this contradiction, this invention proposes an approximate diamond distance based on homomorphic injection current difference, achieving hardware-level real-time estimation of environmental distance at the programmable resistive switching memory array level.

[0184] sparse high-dimensional vector h real Derived from the memristor array retrieval process, the symbolic key represents the current action; cellular automata predict the spin set {s′}. i After external field mapping and sparse coding, another sparse vector h is obtained. predTo ensure comparability within the same hardware domain, this invention simultaneously injects two sets of ±1 codes into the column lines of the same row of the memristor array. The column line voltages employ a symmetrical bipolar scheme, mapping +1 to +V0 and -1 to -V0. Since all nodes within the row share the same bit line and word line path, environmental noise and line resistance have almost identical effects on the two injections, which can be considered a single homomorphic operation, eliminating system errors caused by cross-row differences.

[0185] Current difference and distance formula, single injection yields line current:

[0186]

[0187] Where D is the dimension of the sparse high-dimensional vector, and G... j Let j be the conduction value. and These are the column voltages of the true vector and the predicted vector, respectively. After reading the same amplifier output, a differential is performed to obtain the current residual:

[0188]

[0189] The matrix conductance remains unchanged between the two injections, therefore the residual magnitude is determined solely by the column polarity difference. The approximate diamond distance can be derived by summing the absolute values ​​of all row residuals and normalizing them:

[0190]

[0191] In the formula, V0 is the constant of the column voltage amplitude, and the denominator 2V0 is introduced to ensure that δ is normalized to [0,1]. When the two sets of vectors are completely consistent, the voltage polarities are the same, and δ = 0; when the polarities are completely opposite, δ reaches 1. This normalized distance formally corresponds to the upper bound of the distinguishability between quantum channels—the diamond norm, and is therefore named the "approximate diamond distance".

[0192] Implementation details: Dual injection timing: The controller first sends the true vector pulse, then sends the predicted vector pulse 5 nanoseconds later; the amplifier output is latched twice by a sample-and-hold circuit. Column-level parallel absolute value: The residual absolute value is implemented through a simplified folded rectifier, occupying only two transistors and one resistor, allowing all columns to operate synchronously. Row-level accumulation: Parallel accumulation using current integrating capacitors is employed, with an integration time of 2 nanoseconds, which can be completed within one read cycle. Threshold comparison: δ is compared with the threshold δ. th After analog-to-digital conversion, the on-chip comparator determines whether the value exceeds the threshold and sends a "recompile" flag.

[0193] Coupled with quantum closed loops, if δ≤δ th This indicates that the difference between the prediction and the actual environment can be corrected by fine-tuning the local geometric gate angle and memristor conduction; if δ>δ thThe controller performs the following operations: calls the persistent coherence module to regenerate the topological feature vector; writes the new vector into the quantum random access memory; and updates the 18 topologically locked rotation angles with quantum natural gradients. This process automatically completes the relocation strategy prior and performs row-level incremental refresh of the memristor array conduction values ​​to ensure that the hardware retrieval is consistent with the new strategy.

[0194] like Figure 2 As shown, a quantum-gate-controlled reinforcement learning system is used to implement the aforementioned quantum-gate-controlled reinforcement learning method. The system includes:

[0195] The state generation module is used to perform numerical normalization and dimensionality compression on offline sample data. Then, the compressed sample matrix is ​​mapped to quantum amplitude encoded states, and prior quantum states are generated through iterative training of variable quantum circuits. After the CPU / FPGA completes zero-mean normalization and principal component analysis, the compressed vector is sent to the superconducting quantum chip. The variable quantum circuit self-iterates under the control of on-chip DAC-RF and converges to obtain the prior quantum states.

[0196] The topology mapping module is used to extract topological feature vectors based on pulse signals corresponding to prior quantum states, and convert the topological feature vectors into rotation angles of non-Abelian geometric gate networks to obtain action probability vectors; quantum readout pulses are fed into wavelet-homology hard kernels to extract 16-dimensional topological vectors; the vectors are written into QRAM, and 6 sets of XYZ rotation angles are generated by looking up tables, which directly drive the non-Abelian geometric gate network and measure the action probabilities.

[0197] The high-dimensional retrieval module is used to construct a sparse high-dimensional vector based on the action probability vector, select the target supervector using a quantum amplitude amplification algorithm, inject the target supervector into a programmable resistive switching memory array, and retrieve the action vector according to the conduction function determined by the topological feature vector; a 10,000-dimensional sparse vector is constructed using Hadamard-ROM + XOR chain in the FPGA; the quantum register performs ≤3 rounds of Grover amplification to select the target supervector; ±0.2V pulses are written to the silicon carbide memristor cross array, and the row with the maximum row current is output as the action vector.

[0198] An adaptive calibration module is used to embed action vectors into a cellular automaton model based on Heisenberg interactions to generate environmental prediction results, calculate prediction errors, and synchronously adjust the rotation angle of the non-Abelian geometric gate network and update the conduction value of the programmable resistive switching memory array when the prediction error exceeds a threshold. It also rewrites the topological feature vector and performs quantum natural gradient adjustment on the non-Abelian geometric gate network when the preset environmental distance index exceeds a threshold. The GPU-parallel Heisenberg automaton provides predictions in 10 iterations; if the local error exceeds the threshold, the FPGA synchronously fine-tunes the Z-axis angle and performs small-amplitude pulse memristor adjustment; if the global distance exceeds the threshold, the topological vector is quickly recalculated and a 3-step quantum natural gradient is run, with a total latency of <250μs.

[0199] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A reinforcement learning method with quantum gate control, characterized in that, Includes the following steps: The offline sample data is numerically normalized and dimensionally compressed. Then, the compressed sample matrix is ​​mapped to a quantum amplitude encoded state, and prior quantum states are generated through iterative training using a variable quantum circuit. Based on the pulse signal corresponding to the prior quantum state, a topological feature vector is extracted, and the topological feature vector is converted into a rotation angle of a non-Abelian geometric gate network to obtain an action probability vector; A sparse high-dimensional vector is constructed based on the action probability vector. A target supervector is selected using a quantum amplitude amplification algorithm. The target supervector is injected into a programmable resistive switching memory array. The action vector is retrieved according to the conduction function determined by the topological feature vector. The action vector is embedded into a cellular automaton model based on Heisenberg interactions to generate environmental prediction results. The prediction error is calculated. When the prediction error exceeds a threshold, the rotation angle of the non-Abelian geometric gate network is adjusted synchronously and the conduction value of the resistive switching memory array is updated. When the preset environmental distance index exceeds a threshold, the topological feature vector is rewritten and quantum natural gradient adjustment is performed on the non-Abelian geometric gate network.

2. The method according to claim 1, characterized in that, Zero-mean standardization is used for numerical normalization of offline sample data, and principal component analysis is used for dimensionality compression of offline sample data. The cumulative variance contribution rate of principal component analysis is used as the termination condition to generate the compressed sample matrix.

3. The method according to claim 1, characterized in that, When extracting topological feature vectors, Merlin wavelet scattering transform is performed on the pulse signal corresponding to the prior quantum state to obtain a set of complex scattering coefficients. The connectivity and ring characteristics of the complex scattering coefficient set are analyzed using the persistent homology method to form the Betti number variation curve. The Betti number variation curve is then piecewise linearly compressed to generate the topological feature vector.

4. The method according to claim 1, characterized in that, After writing the topological feature vector into the quantum random access memory, each continuous value of the topological feature vector is mapped to the range of zero to twice the circumference angle through the Sigmoid function according to the preset mapping rules, and then assigned to the corresponding X-axis rotation angle, Y-axis rotation angle and Z-axis rotation angle in the non-Abelian geometric gate network.

5. The method according to claim 1, characterized in that, The sparse high-dimensional vector is generated by performing sparse binding operations on the action probability vector using a random orthogonal basis. The random orthogonal basis is constructed by the Hadamard transform and the Hosshauer vector orthogonalization method. The Hamming density of the sparse high-dimensional vector is limited to no more than a preset sparsity threshold.

6. The method according to claim 1, characterized in that, The quantum amplitude amplification algorithm uses the Grover amplitude amplification algorithm. The number of iterations of the Grover amplitude amplification algorithm is determined according to the square root ratio of the number of candidate vectors to the number of labeled vectors, while limiting the number of iterations to no more than a preset iteration limit.

7. The method according to claim 1, characterized in that, The programmable resistive switching memory array is a silicon carbide memristor cross-connect array. The conduction value of each cross-connect node is adjusted by a voltage pulse. The conduction function adopts an exponential mapping form, and the temperature coefficient in the exponential mapping is determined by the first-order mean of the topological eigenvector.

8. The method according to claim 1, characterized in that, In each discrete-time iteration, the cellular automaton model incorporates the linear embedding term of the action vector into the update equation of the 32 spin cells, and simultaneously considers the Heisenberg interaction term between the spin cells and their neighboring cells in the update equation.

9. The method according to claim 1, characterized in that, The preset environmental distance index is obtained by injecting the current signal corresponding to the sparse high-dimensional vector and the predicted state of the cellular automaton model into the same row of the programmable resistive variable memory array, and then calculating the homomorphic current difference to obtain the approximate diamond distance.

10. A quantum-gated reinforcement learning system for implementing the quantum-gated reinforcement learning method according to any one of claims 1-9, characterized in that, The system includes: The state generation module is used to perform numerical normalization and dimensionality compression on offline sample data, then map the compressed sample matrix into quantum amplitude encoded states, and generate prior quantum states through iterative training of variable quantum circuits. The topology mapping module is used to extract topological feature vectors based on the pulse signals corresponding to the prior quantum states, and convert the topological feature vectors into rotation angles of non-Abelian geometric gate networks to obtain action probability vectors. The high-dimensional retrieval module is used to construct a sparse high-dimensional vector based on the action probability vector, select the target supervector using the quantum amplitude amplification algorithm, inject the target supervector into the programmable resistive switching memory array, and retrieve the action vector according to the conduction function determined by the topological feature vector. The adaptive calibration module is used to embed action vectors into a cellular automaton model based on Heisenberg interactions to generate environmental prediction results, calculate prediction errors, and when the prediction error exceeds a threshold, synchronously adjust the rotation angle of the non-Abelian geometric gate network and update the conduction value of the programmable resistive switching memory array. When the preset environmental distance index exceeds a threshold, the topological feature vector is rewritten and quantum natural gradient adjustment is performed on the non-Abelian geometric gate network.

Images