Rotating double-prism hybrid inverse solution method deeply fusing deterministic physical prior model and reinforcement learning

Abstract

This invention discloses a hybrid inverse kinematics method for rotating biprisms that deeply integrates deterministic physics prior models with reinforcement learning. It solves the long-standing problem of balancing computational efficiency and pointing accuracy in existing rotating biprism beam pointing inverse kinematics techniques. This invention employs a reinforcement learning compensation method based on the paraxial approximation method, preserving its computational efficiency. The paraxial approximation method solves the problem through basic arithmetic operations, a single inverse trigonometric function, and simple conditional judgments; the number of operations remains constant regardless of the input parameters, resulting in a time complexity of O(n log n). Spatially, the paraxial approximation method requires only a small number of scalar variables, resulting in a space complexity of O(n log n), lower memory usage, and better numerical stability.

Description

Technical Field

[0001] This invention relates to a rotating biprism hybrid inverse problem method, specifically a rotating biprism hybrid inverse problem method that deeply integrates a deterministic physics prior model with reinforcement learning. Background Technology

[0002] Rotating biprism ( The system consists of two coaxially rotating wedge prisms. By adjusting the rotation angle of the two prisms, the arbitrary direction of the beam can be achieved within a certain field of view, which can be described by the azimuth angle and deflection angle. This technology is widely used in laser communication, lidar, and optoelectronic countermeasures. One of the core challenges lies in solving for the rotation angle of the two prisms based on the azimuth angle and deflection angle of the desired beam direction, i.e., the inverse problem of beam direction determination.

[0003] Existing inverse kinematics techniques for rotating double prism systems mainly fall into three categories: approximate analytical methods, numerical methods, and data-driven methods.

[0004] The approximate analytical method usually adopts a first-order paraxial approximation model. This method simplifies the complex nonlinear refraction process into a linear algebraic formula based on the small angle assumption. Although it has extremely high computational efficiency and can provide a closed-loop solution in analytical form, under large field-of-view deflection, the pointing error will increase significantly due to the neglect of prism thickness effect and nonlinear change of refractive index, making it difficult to meet the requirements of high-precision applications.

[0005] Numerical methods are mostly based on rigorous non-paraxial vector ray tracing models, combined with iterative optimization algorithms such as the Newton-Raphson method for solving the problem. These methods use the Jacobian matrix to guide the gradient descent direction for repeated iterative corrections. Although they can theoretically achieve high-fidelity solutions, they are computationally complex, time-consuming, and highly dependent on the selection of initial values. They are prone to getting trapped in local extrema or even non-convergence near system singularities or blind zones, which severely restricts dynamic real-time performance.

[0006] In recent years, with the rise of artificial intelligence methods such as neural networks, some scholars have attempted to construct an end-to-end mapping from the pointing angle to the prism rotation angle using deep learning models such as neural networks, which is a method similar to that presented in this paper. Although this method avoids complex physical calculations to some extent, thereby improving inference speed, such purely data-driven models lack physical interpretability and heavily rely on massive amounts of full-field coverage data for training. They also exhibit poor generalization ability in non-training areas or extreme conditions, making it difficult to guarantee the robustness and determinism of the system across the entire domain. Summary of the Invention

[0007] To address the long-standing challenge of balancing computational efficiency and pointing accuracy in existing rotating biprism beam pointing inverse solving techniques—that is, while traditional analytical methods are computationally fast, they suffer from significant nonlinear errors in large fields of view, and numerical iterative methods, while highly accurate, are computationally time-consuming and prone to local extrema—this invention provides a rotating biprism hybrid inverse solving method that deeply integrates deterministic physics prior models with reinforcement learning.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A hybrid inverse problem method using a rotating biprism that deeply integrates deterministic physics prior models with reinforcement learning is characterized by the following steps:

[0010] Step 1: Calculate the reference rotation angle of the first prism in the rotating biprism. The reference angle of the second prism ; Calculate the azimuth error of the rotating biprism and deflection angle error ;

[0011] Step 2: Based on azimuth error and deflection angle error Define the state vector s, action vector a, and reward value r in the dual-delay deep deterministic policy gradient algorithm, and implement the dual-delay deep deterministic policy gradient algorithm for reinforcement learning; the dual-delay deep deterministic policy gradient algorithm includes a policy network. Target-Policy Network Dual evaluation network and dual-objective evaluation network ;

[0012] Step 3: Convert the state vector at time t. The policy network in the double-delay deep deterministic policy gradient algorithm Then the policy network Output actual action vector :

[0013] T ;

[0014] in, and For policy networks The output shows the first prism compensation value and the second prism compensation value; T is the transpose.

[0015] Step 4: Compare the actual motion vector with the reference angle of the first prism. The reference angle of the second prism By superimposing the values, we can obtain the actual rotation angles of the first and second prisms in the rotating biprism. :

[0016] ;

[0017] Complete the hybrid inverse solution of the rotating double prism system.

[0018] Further, step 1 includes:

[0019] Step 1.1: Set the target azimuth angle of the beam emitted after passing through the rotating double prism. and target deflection angle ;

[0020] Step 1.2: Calculate the reference rotation angle of the first prism in the rotating biprism using the paraxial approximation algorithm. The reference angle of the second prism ;

[0021] ;

[0022] in, This represents the paraxial approximation calculation method;

[0023] Step 1.3: Measure the actual azimuth angle of the emitted beam after passing through the rotating double prism system. Compared with the actual deflection angle ;

[0024] Step 1.4: Calculate the azimuth error and deflection angle error :

[0025] ;

[0026] .

[0027] Furthermore, in step 2, based on the azimuth error and deflection angle error In the dual-delay deep deterministic policy gradient algorithm, the state vector s, action vector a, and reward value r are defined as follows:

[0028] , T ;

[0029] ;

[0030] ;

[0031] in, It is a very small positive number. This is the actual compensation amount for the first prism. This is the actual compensation amount for the second prism.

[0032] Furthermore, in step 2, the dual-delay deep deterministic policy gradient algorithm is used for reinforcement learning, specifically including:

[0033] Step A: Convert the state vector at time t Input Policy Network State vector Feature vectors are obtained by sequentially extracting features through two fully connected layers. ;

[0034] Step B, Policy Network The final output layer will feature vector Mapped to action vectors and through The function limits the output range to 1000. Within the range; finally, the result is mapped to the actual physical compensation range through linear scaling. The comprehensive output formula is:

[0035]

[0036] in, For the output layer weights of the policy network, This indicates element-wise multiplication. For bias terms;

[0037] Step C: To the policy network Add exploration noise Obtain the state vector S at time t+1. t+1 ;

[0038] Step D, Target Policy Network Calculate the action vector at time t+1 ;

[0039] Step E: Dual-objective evaluation network Through the state vector S t+1 and action vectors Calculate the scalar Q value ;

[0040] Step F: Using the scalar Q value Updated dual-evaluation network Then, the gradient guidance policy network of the first evaluation network is used. renew;

[0041] Step G: Adjust the target policy network The parameters are then processed, and step A is returned until the reward value r converges.

[0042] Furthermore, step E specifically includes:

[0043] Step E1: Dual-objective evaluation network By processing state and action inputs in parallel through independent state branches and action branches, the state feature vector is obtained. and action feature vector ;

[0044] Step E2: Transfer the state feature vector and action feature vector The feature vectors are then concatenated and fused in the feature space. Entering the dual-objective evaluation network The output layer performs dimensionality compression and value mapping, ultimately outputting a scalar representing the expected long-term return. value .

[0045] Furthermore, step A specifically includes:

[0046] The state vector at time t Input Policy Network Policy Network The input layer receives the state vector. State vector After feature extraction through a fully connected layer with 512 neurons, feature vector h1 is obtained. Feature vector h1 then enters another fully connected layer with 512 neurons for feature extraction, resulting in a new feature vector. Each feature extraction process integrates linear transformation, normalization, and nonlinear activation. Its forward propagation logic is as follows:

[0047]

[0048] in, ; The first Layer weight matrix and bias vector; This is the ReLU activation function.

[0049] Furthermore, step E1 specifically involves:

[0050] Dual-objective evaluation network State and action inputs are processed in parallel through independent state branches and action branches; where: state vector The input state branch is processed through a 512-dimensional fully connected layer, a normalization layer, and a non-linear activation function to extract the state feature vector. Action vector The input action branch is processed through another 512-dimensional fully connected layer to extract the action feature vector. .

[0051] Furthermore, step E2 specifically involves:

[0052] The state feature vector and action feature vector The feature vectors are then concatenated and fused in the feature space. eigenvectors Entering the dual-objective evaluation network The output layer performs dimensionality compression and value mapping, ultimately outputting a scalar representing the expected long-term return. value :

[0053]

[0054] Where γ is the discount factor, r t Let t be the reward value at time t.

[0055] Furthermore, in step G, the target policy network is adjusted. The parameters are as follows:

[0056] Adjusting the target policy network using a moving average strategy The parameters.

[0057] Furthermore, step F specifically includes:

[0058] Using scalar Q value Updated dual-evaluation network Then, the gradient guidance policy network of the first evaluation network is used. Update, the goal is to maximize the scalar Q value. Policy Network The update frequency is lower than that of the double-evaluation network. The first evaluation network in the system.

[0059] The beneficial effects of this invention are:

[0060] The rotating double prism hybrid inverse calculation method proposed in this invention has significant technical advantages in terms of efficiency, accuracy and system stability.

[0061] 1. In terms of efficiency, the reinforcement learning compensation method based on the paraxial approximation method almost retains the computational efficiency of the paraxial approximation method: the paraxial approximation method solves the problem through basic arithmetic operations, a single inverse trigonometric function, and simple conditional judgments; the number of operations does not change with the input parameters, and the time complexity is O(n log n). Spatially, the paraxial approximation method requires only a small number of scalar variables to store, with a space complexity of O(n log n). It boasts lower memory footprint and superior numerical stability. Introducing reinforcement learning compensation into the paraxial approximation method adds only two addition operations, resulting in virtually no change in complexity. From a time complexity perspective, the added addition operations are scalar operations with a fixed frequency, preserving the core characteristic that the number of operations does not increase with the input parameters, thus maintaining the same level of complexity. In terms of space complexity, memory usage remains a constant, and the same applies. Experimental testing showed that the computation time of the reinforcement learning compensation first-order paraxial approximation method proposed in this invention is only that of the non-paraxial ray tracing method. .

[0062] 2. Regarding accuracy, this method effectively eliminates the nonlinear errors neglected by traditional first-order paraxial models by introducing a reinforcement learning intelligent compensation mechanism, significantly improving accuracy. Following the steps outlined above, the algorithm training process resulted in a reduction in the root mean square error and absolute error integral of the beam deflection angle and azimuth angle in the vast majority of experimental tests. This significantly narrowed the gap with the exact solution.

[0063] 3. In terms of stability and robustness, this invention uses physical priors as a solution reference, which effectively avoids the risk of output divergence that may occur in the area not covered by training data in a pure data-driven "black box" model, greatly enhancing the safety of engineering applications. In addition, thanks to the unique smoothing regularization mechanism in the dual-delay deep deterministic strategy gradient algorithm, the trained model shows good anti-interference ability against small perturbations of prism system parameters, ensuring the reliability of long-term operation. Attached Figure Description

[0064] Figure 1 This is a flowchart of an embodiment of the rotating biprism hybrid inverse problem method of the present invention, which deeply integrates deterministic physical prior models with reinforcement learning. Detailed Implementation

[0065] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] This invention provides a method for hybrid inverse problem solving using a rotating biprism that deeply integrates deterministic physics prior models with reinforcement learning. Figure 1 As shown, the rotating biprism hybrid inverse solution method includes the following steps:

[0067] Step 1: Calculate the reference rotation angle of the first prism in the rotating biprism. The reference angle of the second prism ; Calculate the azimuth error of the rotating biprism and deflection angle error Specifically, this includes:

[0068] Step 1.1: Set the target azimuth angle of the beam emitted after passing through the rotating double prism. and target deflection angle ;

[0069] Step 1.2: Calculate the reference rotation angle of the first prism in the rotating biprism using the paraxial approximation formula. The reference angle of the second prism ;

[0070] ;

[0071] in, This represents the paraxial approximation calculation method; the inverse solution reference angle obtained at this time There is a large nonlinear residual. Reference rotation angle. The first angle represents the coarse rotation angle of the two prisms. Subsequent reinforcement learning will compensate for this angle, and the resulting values ​​will be superimposed to obtain the rotation angle of the two prisms.

[0072] Step 1.3: Measure the actual azimuth angle of the emitted beam after passing through the rotating double prism system. Compared with the actual deflection angle ;

[0073] Step 1.4: Calculate the azimuth error and deflection angle error :

[0074] ;

[0075] .

[0076] Step 2: Based on azimuth error and deflection angle error In the Two-Delay Deep Deterministic Policy Gradient Algorithm (TD3), the state vector s, action vector a, and reward value r are defined as follows:

[0077] , T ;

[0078] ;

[0079] ;

[0080] in, For a very small positive number (such as 10)-6 This is used to prevent gradient explosion at zero error. This is the actual compensation amount for the first prism. This represents the actual compensation amount of the second prism. The reward value r is a value immediately fed back by the environment after the agent performs the compensation action based on the state vector s.

[0081] The dual-delay deep deterministic policy gradient algorithm includes a policy network. Target-Policy Network Dual evaluation network and dual-objective evaluation network Policy Network The Actor Network is responsible for establishing a direct mapping from the observed state to the prism angle compensation. Its input layer receives a 4-dimensional state vector *s*, and the feature extraction layer consists of two fully connected layers, each with 512 neurons. Normalization is applied after each fully connected layer to standardize the feature distribution, mitigating the internal covariate bias problem in deep network training and accelerating convergence. The feature extraction layer uses the ReLU function to enhance nonlinear fitting capability. The output layer outputs a 2-dimensional action vector *a*, corresponding to the compensation amounts of the two prisms. The Tanh activation function is used to strictly limit the output to the [-1, 1] interval, finally mapping it to the actual compensation range.

[0082] Dual Evaluation Network (Twin Critic Networks) are responsible for evaluating the value of taking an action in the current state. A dual-network structure is used to suppress overestimation. (Dual Evaluation Network) A dual-stream convergence structure is adopted: the state branch receives a 4-dimensional state vector *s*, which is processed through a 512-dimensional fully connected layer and normalized to extract state features. The action branch receives a 2-dimensional action vector *k*, which is processed through a 512-dimensional fully connected layer to extract action features. These two feature vectors are concatenated and fused in the feature space. The fused feature then enters a 256-dimensional fully connected layer, ultimately outputting a scalar Q-value representing the expected reward.

[0083] Then, the dual-delay deep deterministic policy gradient algorithm is used for reinforcement learning, specifically including:

[0084] Step A: Convert the state vector at time t Input Policy Network State vector Feature vectors are obtained by sequentially extracting features through two fully connected layers. Specifically:

[0085] The state vector at time t Input Policy Network Policy Network The input layer receives the state vector. State vector After feature extraction through a fully connected layer with 512 neurons, feature vector h1 is obtained. Feature vector h1 then enters another fully connected layer with 512 neurons for feature extraction, resulting in a new feature vector. Each feature extraction process integrates linear transformation, normalization, and nonlinear activation. Its forward propagation logic is as follows:

[0086]

[0087] in, ; The first Layer weight matrix and bias vector; This is the ReLU activation function.

[0088] Step B, Policy Network The final output layer will feature vector Mapped to action vectors and through The function limits the output range to 1000. Within the range; finally, the result is mapped to the actual physical compensation range through linear scaling. The comprehensive output formula is:

[0089]

[0090] in, For the output layer weights of the policy network, This indicates element-wise multiplication. For bias terms;

[0091] Step C: To the policy network Add exploration noise Obtain the state vector S at time t+1. t+1 ;

[0092] The agent, based on the policy network Select an action and add exploration noise. To explore the environment and execute action vector a t Afterwards, you will receive reward points. The state vector at time +1 , tuple Storage capacity is 10 6 Experience replay buffer , This represents a value with a mean of 0 and a standard deviation of . The normal distribution. From 512 samples were randomly selected as a batch to break the temporal correlation of the data and ensure training stability.

[0093] Step D, Target Policy Network Calculate the action vector at time t+1 Furthermore, cutoff normal noise is incorporated to enhance the robustness of the value assessment to motion disturbances.

[0094]

[0095] Here, `clip` is a constraint function that ensures the final output compensation instructions are strictly constrained within the specified range. It follows a truncated normal distribution. The range is limited to [-0.5, 0.5], and the algorithm aims to enhance its generalization ability in the continuous space of the prism corner by using a smooth target strategy.

[0096] Step E: Dual-objective evaluation network Through the state vector S t+1 and action vectors Calculate the scalar Q value Specifically, this includes:

[0097] Step E1: Dual-objective evaluation network State and action inputs are processed in parallel through independent state branches and action branches; where: state vector The input state branch is processed through a 512-dimensional fully connected layer, a normalization layer, and a non-linear activation function to extract the state feature vector. Action vector The input action branch is processed through another 512-dimensional fully connected layer to extract the action feature vector. .

[0098] Step E2: Transfer the state feature vector and action feature vector The feature vectors are then concatenated and fused in the feature space. eigenvectors Entering the dual-objective evaluation network The output layer performs dimensionality compression and value mapping, ultimately outputting a scalar representing the expected long-term return. value :

[0099]

[0100] Where γ is the discount factor, a weight used to balance "immediate reward" and "future long-term total return," with a value of γ = 0.99. t Let t be the reward value at time t.

[0101] Step F: Using the scalar Q value Updated dual-evaluation network Then, the gradient guidance policy network of the first evaluation network is used. Update, the goal is to maximize the scalar Q value. Policy Network The update frequency is lower than that of the double-evaluation network. The first evaluation network in the system.

[0102] Step G: Adjust the target policy network using a moving average strategy. The parameters are used to maintain the stability of the training target, among which =0.005:

[0103] ;

[0104] Return to step A until the reward value r converges.

[0105] After training, only the policy network is retained. .

[0106] Step 3: Convert the state vector at time t. The policy network in the double-delay deep deterministic policy gradient algorithm Then the policy network Output actual action vector :

[0107] T ;

[0108] in, and For policy networks The output shows the first prism compensation value and the second prism compensation value; T is the transpose.

[0109] Step 4: Compare the actual motion vector with the reference angle of the first prism. The reference angle of the second prism By superimposing the values, we can obtain the actual rotation angles of the first and second prisms in the rotating biprism. :

[0110] ;

[0111] Complete the hybrid inverse solution of the rotating double prism system.

[0112] The core technology of this invention lies in the "physical prior-guided residual compensation" framework. This framework possesses good versatility and scalability, allowing for reasonable variations and substitutions based on actual application scenarios. At the baseline model level, although this embodiment uses a first-order paraxial approximation method to provide initial values, the physical prior source can also be replaced by any other calculation method, such as a third-order paraxial approximation formula or a lookup table, which can still serve a guiding role. At the algorithm selection level, the reinforcement learning algorithm used for residual learning is not limited to TD3. Continuous control reinforcement learning algorithms such as DDPG, Flexible Action-Evaluation Algorithm (SAC), or Proximal Policy Optimization (PPO), or other neural network learning algorithms can all be used as alternatives. Although the specific convergence performance may differ slightly, the core inventive concept of correcting physical errors through data-driven approaches remains consistent. Furthermore, at the network architecture level, the fully connected neural network (DNN) described in this embodiment is only a preferred option. In practical applications, it can be replaced with a residual network or other deep network structures to further enhance the model's ability to extract and fit complex nonlinear features.

[0113] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A rotating biprism hybrid inverse problem method that deeply integrates deterministic physics prior models with reinforcement learning, characterized in that, Includes the following steps: Step 1: Calculate the reference rotation angle of the first prism in the rotating biprism. The reference angle of the second prism ; Calculate the azimuth error of the rotating biprism and deflection angle error ; Step 2: Based on azimuth error and deflection angle error Define the state vector s, action vector a, and reward value r in the dual-delay deep deterministic policy gradient algorithm, and implement the dual-delay deep deterministic policy gradient algorithm for reinforcement learning; the dual-delay deep deterministic policy gradient algorithm includes a policy network. Target-Policy Network Dual evaluation network and dual-objective evaluation network ; Step 3: Convert the state vector at time t. The policy network in the double-delay deep deterministic policy gradient algorithm Then the policy network Output actual action vector : T ； in, and For policy networks The output shows the first prism compensation value and the second prism compensation value; T is the transpose. Step 4: Compare the actual motion vector with the reference angle of the first prism. The reference angle of the second prism By superimposing the values, we can obtain the actual rotation angles of the first and second prisms in the rotating biprism. : ； Complete the hybrid inverse solution of the rotating double prism system.

2. The rotating biprism hybrid inverse problem method that deeply integrates deterministic physics prior models with reinforcement learning as described in claim 1, characterized in that, Step 1 includes: Step 1.1: Set the target azimuth angle of the beam emitted after passing through the rotating double prism. and target deflection angle ; Step 1.2: Calculate the reference rotation angle of the first prism in the rotating biprism using the paraxial approximation algorithm. The reference angle of the second prism ; ； in, This represents the paraxial approximation calculation method; Step 1.3: Measure the actual azimuth angle of the emitted beam after passing through the rotating double prism system. Compared with the actual deflection angle ; Step 1.4: Calculate the azimuth error and deflection angle error : ； 。 3. The rotating biprism hybrid inverse problem method that deeply integrates deterministic physics prior models with reinforcement learning according to claim 2, characterized in that, In step 2, based on the azimuth error and deflection angle error In the dual-delay deep deterministic policy gradient algorithm, the state vector s, action vector a, and reward value r are defined as follows: ， T ； ； ； in, It is a very small positive number. This is the actual compensation amount for the first prism. This is the actual compensation amount for the second prism.

4. The rotating biprism hybrid inverse problem method that deeply integrates deterministic physics prior models with reinforcement learning according to claim 3, characterized in that, Step 2 involves using the dual-delay deep deterministic policy gradient algorithm for reinforcement learning, specifically including: Step A: Convert the state vector at time t Input Policy Network State vector Feature vectors are obtained by sequentially extracting features through two fully connected layers. ; Step B, Policy Network The final output layer will feature vector Mapped to action vectors and through The function limits the output range to 1000. Within the range; finally, the result is mapped to the actual physical compensation range through linear scaling. The comprehensive output formula is: ； in, For the output layer weights of the policy network, This indicates element-wise multiplication. For bias terms; Step C: To the policy network Add exploration noise Obtain the state vector S at time t+1. t+1 ; Step D, Target Policy Network Calculate the action vector at time t+1 ; Step E: Dual-objective evaluation network Through the state vector S t+1 and action vectors Calculate the scalar Q value ; Step F: Using the scalar Q value Updated dual-evaluation network Then, the gradient guidance policy network of the first evaluation network is used. renew; Step G: Adjust the target policy network The parameters are then processed, and step A is returned until the reward value r converges.

5. The rotating biprism hybrid inverse problem solving method that deeply integrates deterministic physics prior models with reinforcement learning according to claim 4, characterized in that, Step E specifically includes: Step E1: Dual-objective evaluation network By processing state and action inputs in parallel through independent state branches and action branches, the state feature vector is obtained. and action feature vector ; Step E2: Transfer the state feature vector and action feature vector The feature vectors are then concatenated and fused in the feature space. Entering the dual-objective evaluation network The output layer performs dimensionality compression and value mapping, ultimately outputting a scalar representing the expected long-term return. value .

6. The rotating biprism hybrid inverse problem method for deeply fusing deterministic physics prior models with reinforcement learning as described in claim 5, characterized in that, Step A is as follows: The state vector at time t Input Policy Network Policy Network The input layer receives the state vector. State vector After feature extraction through a fully connected layer with 512 neurons, feature vector h1 is obtained. Feature vector h1 then enters another fully connected layer with 512 neurons for feature extraction, resulting in a new feature vector. Each feature extraction process integrates linear transformation, normalization, and nonlinear activation. Its forward propagation logic is as follows: ； in, ; The first Layer weight matrix and bias vector; This is the ReLU activation function.

7. The rotating biprism hybrid inverse problem method for deeply fusing deterministic physics prior models with reinforcement learning as described in claim 6, characterized in that, Step E1 is as follows: Dual-objective evaluation network State and action inputs are processed in parallel through independent state branches and action branches; where: state vector The input state branch is processed through a 512-dimensional fully connected layer, a normalization layer, and a non-linear activation function to extract the state feature vector. Action vector The input action branch is processed through another 512-dimensional fully connected layer to extract the action feature vector. .

8. The rotating biprism hybrid inverse problem method for deeply fusing deterministic physics prior models with reinforcement learning as described in claim 7, characterized in that, Step E2 is as follows: The state feature vector and action feature vector The feature vectors are then concatenated and fused in the feature space. eigenvectors Entering the dual-objective evaluation network The output layer performs dimensionality compression and value mapping, ultimately outputting a scalar representing the expected long-term return. value : ； Where γ is the discount factor, r t Let t be the reward value at time t.

9. The rotating biprism hybrid inverse problem method for deeply fusing deterministic physics prior models with reinforcement learning as described in claim 8, characterized in that, In step G, the target policy network is adjusted. The parameters are as follows: Adjusting the target policy network using a moving average strategy The parameters.

10. The rotating biprism hybrid inverse problem method for deeply fusing deterministic physics prior models with reinforcement learning as described in claim 9, characterized in that, Step F specifically includes: Using scalar Q value Updated dual-evaluation network Then, the gradient guidance policy network of the first evaluation network is used. Update, the goal is to maximize the scalar Q value. Policy Network The update frequency is lower than that of the double-evaluation network. The first evaluation network in the system.

Images