Reinforcement learning-based strong generalization capability accelerator orbit correction method and system

Abstract

The application aims to provide a reinforcement learning-based strong generalization capability accelerator orbit correction method and system, which comprises the following steps: establishing a simulation accelerator according to the data of a real accelerator; measuring BPM values in the simulation accelerator and constructing a feature matrix according to the BPM values; inputting the feature matrix into a reinforcement learning model, and outputting a current change value from the reinforcement learning model; and correcting the current value of the real accelerator according to the current change value and the magnet current value in the real accelerator. The application can greatly save online debugging time and cost and improve the beam debugging efficiency.

Description

Technical Field

[0001] This invention relates to the field of accelerator beam control technology, and more particularly to a method and system for accelerator trajectory correction based on reinforcement learning with strong generalization ability. Background Technology

[0002] The beam in the accelerator is affected by factors such as collimation accuracy and installation errors. When the beam passes through the accelerator, the trajectory may deviate from the center position. Therefore, beam trajectory correction plays a crucial role in the commissioning and operation of the accelerator.

[0003] The basic method of beam calibration involves observing the horizontal and vertical readings of the beam position monitor (BPM) and adjusting the current in the correction iron to bring the BPM readings as close to zero as possible. Existing methods primarily rely on human experience and reinforcement learning-based approaches for in-line accelerators. Accelerator beam calibration time is expensive, especially in regions with solenoids. Human experience-based methods are time-consuming, labor-intensive, complex, and difficult to apply. Reinforcement learning-based in-line accelerator calibration schemes require collecting online accelerator operational data, consuming significant beam calibration time, and are therefore costly.

[0004] However, due to the unpredictable differences between simulated and real accelerators, such as magnetic fields, electric fields, installation errors, and collimation errors, reinforcement learning models trained in a simulated environment are difficult to directly apply to real accelerators. Summary of the Invention

[0005] The purpose of this invention is to provide a strong generalization method for accelerator orbit correction based on reinforcement learning, which can significantly save online debugging time and costs and improve beam debugging efficiency.

[0006] This invention proposes a strong generalization ability accelerator orbit correction method based on reinforcement learning, comprising:

[0007] A simulated accelerator was built based on data from a real accelerator.

[0008] BPM values ​​are measured in a simulated accelerator, and a characteristic matrix is ​​constructed based on the BPM values;

[0009] The feature matrix is ​​input into the reinforcement learning model, which outputs the current change value. The current value of the real accelerator is corrected based on the current change value and the magnet current value in the real accelerator.

[0010] Preferably, the step of building a simulated accelerator based on data from a real accelerator includes:

[0011] Accelerator simulation software was selected as the basic environment, and the beam position detector of the simulated accelerator was set according to the beam position monitor on the real accelerator.

[0012] The values ​​of the quadrupole current at the corresponding location in the simulated accelerator are set according to the values ​​of the quadrupole current in the real accelerator.

[0013] Replace the location for correcting the ferrocurrent in the accelerator's configuration file with a replaceable variable.

[0014] Preferably, the BPM value is:

[0015] B i Let BPM be the readings at time i for all BPMs. Assume there are N sets of BPMs, each set divided into X and Y directions. i =[ b x1 i b y1 i b x2 i b y2 i , ...b xn 1 b yn i ..., b xN 1 b yN i ], where b xn 1 This represents the BPM reading in the X direction for the nth group at time i, where b yn 1 This represents the reading of the nth group of BPM in the Y direction at time i.

[0016] Preferably, the step of inputting the feature matrix into the reinforcement learning model, the reinforcement learning model outputting the current change value, and correcting the current value of the real accelerator based on the current change value and the magnet current value in the real accelerator specifically includes: defining ΔB i =B i -B i-1 Then the observed values ​​during the simulation of the accelerator operation are:

[0017] M i To determine the current value of the correcting iron at time i, assuming there are a total of K sets of correcting irons, then M i =[m x1 i m y1 i m x2 i m y2 i ,...m xk i myk i ..., m xK i m yK i ], m xk i To determine the current value in the X direction of the corrective iron at time i, m yk i Let M be the current value of the k-th group of corrective iron in the Y direction at time i. Let M be the current of the corrective iron at time i-1. i-1 The current increment predicted by the reinforcement learning model is ΔM i-1 Then the current M of the corrector iron at time i i =M i-1 +ΔM i-1 .

[0018] Preferably, measuring the BPM value in the simulated accelerator includes:

[0019] Random component errors are added to the accelerator simulation software to simulate the deviation of the beam's center of mass from that of a real accelerator.

[0020] The beam position monitor readings are split into x-direction and y-direction readings, and the result after splitting is B. ix ={b x1 i b x2 i , ..., b xN i}, B iy ={b y1 i b y2 i , ..., b yN i For regions where there is coupling in the x and y directions—that is, adjusting the corrector in the x direction would affect the beam position monitors in both the x and y directions simultaneously—this region is not separated.

[0021] The observation OB is designed based on whether there is coupling in the x and y directions. For the case where there is no coupling in the x and y directions, the observation design is: OB = [OB...]. x i OB y i At time i, OB x OB y The definition is as follows:

[0022] OB x i =[ΔB x i-1 ΔB xi B x i ] T , where: ΔB x i =B x i -B x i-1 ΔB x i-1 =B x i-1 -B x i-2 OB y i =[ΔB y i-1 ΔB y i B y i ] T , where: ΔB y i =B y i -B y i-1 ΔB y i-1 =B y i-1 -B y i-2 .

[0023] When there is coupling in the X and Y directions, OB = [ΔB] i-1 ΔB i B i ] T , where ΔB i =B i -B i-1 ΔB i-1 =B i-1 -B i-2 , where T is the transpose operator in the matrix operation.

[0024] Preferably, the reinforcement learning model includes:

[0025] Input layer, hidden layer, output layer;

[0026] A convolutional layer is added between the input layer and the hidden layer, and the convolutional layer takes the constructed feature matrix as input.

[0027] For cases where there is no coupling between the X and Y directions, S 3*1 convolutional kernels are set for each direction to extract the variation features in the X and Y directions respectively;

[0028] When there is coupling in the X and Y directions, the changes in the X and Y directions are considered simultaneously, as well as the influence of the X and Y direction correction iron on the BPM. Three convolutional kernels of different sizes are used for feature extraction.

[0029] Preferably, before the BPM value is input into the reinforcement learning model, the reinforcement learning model outputs a current change value, and the current change value is added to the current magnet current value to obtain the corrected current value, the reinforcement learning model is further trained, specifically as follows:

[0030] Set the reward function to include: distance reward, change reward, numerical reward, task completion reward, and remaining steps reward.

[0031] The reinforcement learning model is trained based on distance reward, change reward, numerical reward, task completion reward, and remaining steps reward.

[0032] A reinforcement learning-based accelerator orbit correction system with strong generalization ability, the system comprising:

[0033] Modeling unit, used to build simulated accelerators based on data from real accelerators;

[0034] The processing unit is used to measure the BPM value in the analog accelerator and construct a feature matrix based on the BPM value.

[0035] The prediction unit is used to input the feature matrix into the reinforcement learning model, and the reinforcement learning model outputs the current change value. The current value of the real accelerator is corrected based on the current change value and the magnet current value in the real accelerator.

[0036] A computer device includes a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the above-described reinforcement learning-based accelerator orbit correction method with strong generalization ability.

[0037] A storage medium, which is a computer-readable storage medium, stores a computer program, which, when executed by a processor, implements the above-described reinforcement learning-based accelerator orbit correction method with strong generalization ability.

[0038] This invention establishes a simulated accelerator based on data from a real accelerator; measures the BPM value in the simulated accelerator and constructs a feature matrix based on the BPM value; inputs the feature matrix into a reinforcement learning model, which outputs a current change value; and corrects the current value of the real accelerator based on the current change value and the magnet current value in the real accelerator. This is an offline accelerator beam trajectory correction based on reinforcement learning. The readings of the beam position detector of the real accelerator are read, and then observations are constructed. These observations are used as input to the trained reinforcement learning model, which outputs a current change value. This current change value is added to the current value of the current correction iron to obtain the current correction current value, which is then set on the real accelerator through a data interaction interface. Due to the unknown differences between the simulated and real accelerators and the high cost of beam commissioning, this invention establishes a simulated accelerator based on real accelerator data; observes the BPM value in the simulated accelerator; inputs the BPM value into the reinforcement learning model, which outputs a current change value; and adds the current change value to the current magnet current value to obtain the correction current value. This significantly reduces online commissioning time and cost and improves beam commissioning efficiency. This method can bridge the gap between simulated and real accelerators, allowing models to be trained on simulated accelerators and directly applied to real accelerators. This can significantly save online beam time and reduce beam commissioning costs. This method can be applied not only to the mid-energy range of accelerators, but also to the superconducting and high-energy ranges. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of the accelerator orbit correction method based on reinforcement learning with strong generalization ability proposed in this invention;

[0042] Figure 2 This diagram illustrates the method for constructing observations when there is no coupling between the X and Y directions in the strong generalization accelerator orbit correction method based on reinforcement learning proposed in this invention.

[0043] Figure 3 This is a diagram of the convolutional layer network structure when the X and Y directions are decoupled in the strong generalization capability accelerator orbit correction method based on reinforcement learning proposed in this invention.

[0044] Figure 4 This is a diagram of the convolutional layer network structure when the X and Y directions are coupled in the strong generalization capability accelerator orbit correction method based on reinforcement learning proposed in this invention.

[0045] Figure 5 This diagram illustrates the steps of offline reinforcement learning for accelerator orbit correction, as proposed in this invention, using a reinforcement learning-based accelerator orbit correction method with strong generalization capabilities. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0047] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0048] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, such a combination should be considered non-existent and not within the scope of protection claimed by this invention.

[0049] Due to unpredictable differences in magnetic fields, electric fields, installation errors, and collimation errors between simulated and real accelerators, reinforcement learning models trained in a simulated environment are difficult to directly apply to real accelerators. This invention constructs a simulated accelerator based on data from a real accelerator; measures the BPM value in the simulated accelerator and constructs a feature matrix based on the BPM value; inputs the feature matrix into a reinforcement learning model, which outputs current change values; and corrects the current value of the real accelerator based on these current change values ​​and the magnet current values ​​in the real accelerator. This method achieves offline accelerator beam trajectory correction based on reinforcement learning. This approach bridges the gap between simulated and real accelerators, allowing models trained on simulated accelerators to be directly applied to real accelerators, significantly saving online beam time and reducing beam commissioning costs. This method can be applied not only to the mid-energy range of accelerators but also to the superconducting and high-energy ranges.

[0050] Example 1

[0051] This invention proposes a strong generalization ability accelerator orbit correction method based on reinforcement learning, comprising:

[0052] Step 100: Build a simulated accelerator based on data from the real accelerator;

[0053] Step 200: Measure the BPM value in the simulated accelerator and construct a feature matrix based on the BPM value;

[0054] Step 300: Input the feature matrix into the reinforcement learning model, the reinforcement learning model outputs the current change value, and correct the current value of the real accelerator based on the current change value and the magnet current value in the real accelerator.

[0055] This invention establishes a simulated accelerator based on data from a real accelerator; measures the BPM value in the simulated accelerator and constructs a feature matrix based on the BPM value; inputs the feature matrix into a reinforcement learning model, which outputs a current change value; and corrects the current value of the real accelerator based on the current change value and the magnet current value in the real accelerator. This is an offline accelerator beam trajectory correction based on reinforcement learning. The readings of the beam position detector of the real accelerator are read, and then observations are constructed. These observations are used as input to the trained reinforcement learning model, which outputs a current change value. This current change value is added to the current value of the current correction iron to obtain the current correction current value, which is then set on the real accelerator through a data interaction interface. Due to the unknown differences between the simulated and real accelerators and the high cost of beam commissioning, this invention establishes a simulated accelerator based on real accelerator data; observes the BPM value in the simulated accelerator; inputs the BPM value into the reinforcement learning model, which outputs a current change value; and adds the current change value to the current magnet current value to obtain the correction current value. This significantly reduces online commissioning time and cost and improves beam commissioning efficiency. This method can bridge the gap between simulated and real accelerators, allowing models to be trained on simulated accelerators and directly applied to real accelerators. This can significantly save online beam time and reduce beam commissioning costs. This method can be applied not only to the mid-energy range of accelerators, but also to the superconducting and high-energy ranges.

[0056] Preferably, the step of building a simulated accelerator based on data from a real accelerator includes:

[0057] Accelerator simulation software was selected as the basic environment, and the beam position detector of the simulated accelerator was set according to the beam position monitor on the real accelerator.

[0058] The values ​​of the quadrupole current at the corresponding location in the simulated accelerator are set according to the values ​​of the quadrupole current in the real accelerator.

[0059] Replace the location for correcting the ferrocurrent in the accelerator's configuration file with a replaceable variable.

[0060] Preferably, the BPM value is:

[0061] B i Let BPM be the readings at time i for all BPMs. Assume there are N sets of BPMs, each set divided into X and Y directions. i =[b x1 i b y1 i b x2 i b y2 i , ...bxn i b yn i ..., b xN i b yN i ], where b xn i This represents the BPM reading in the X direction for the nth group at time i, where b yn i This represents the reading of the nth group of BPM in the Y direction at time i.

[0062] Preferably, the step of inputting the feature matrix into the reinforcement learning model, the reinforcement learning model outputting the current change value, and correcting the current value of the real accelerator based on the current change value and the magnet current value in the real accelerator specifically includes: defining ΔB i =B i -B i-1 Then the observed values ​​during the simulation of the accelerator operation are:

[0063] M i To determine the current value of the correcting iron at time i, assuming there are a total of K sets of correcting irons, then M i =[m x1 i m y1 i m x2 i m y2 i ,...m xk i m yk i ..., m xK i m yK i ], m xk i To determine the current value in the X direction of the corrective iron at time i, m yk i Let M be the current value of the k-th group of corrective iron in the Y direction at time i. Let M be the current of the corrective iron at time i-1. i-1 The current increment predicted by the reinforcement learning model is ΔM i-1 Then the current M of the corrector iron at time i i =M i-1 +ΔM i-1 .

[0064] Preferably, measuring the BPM value in the simulated accelerator includes:

[0065] Random component errors are added to the accelerator simulation software to simulate the deviation of the beam's center of mass from that of a real accelerator.

[0066] The beam position monitor readings are split into x-direction and y-direction readings, and the result after splitting is B. ix ={b x1 i b x2 i , ..., b xN i}, B iy ={b y1 i b y2 i , ..., b yN i For regions where there is coupling in the x and y directions—that is, adjusting the corrector in the x direction would affect the beam position monitors in both the x and y directions simultaneously—this region is not separated.

[0067] The observation OB is designed based on whether there is coupling in the x and y directions. For the case where there is no coupling in the x and y directions, the observation design is: OB = [OB... x i OB y i At time i, OB x OB y The definition is as follows:

[0068] OB x i =[ΔB x i-1 ΔB x i B x i ] T , where: ΔB x i =B x i -B x i-1 ΔB x i-1 =B x i-1 -B x i-2 OB y i =[ΔB y i-1 ΔB y i B y i ] T , where: ΔBy i =B y i -B y i-1 ΔB y i-1 =B y i-1 -B y i-2 .

[0069] When there is coupling in the X and Y directions, OB = [ΔB] i-1 ΔB i B i ] T , where ΔB i =B i -B i-1 ΔB i-1 =B i-1 -B i-2 , where T is the transpose operator in the matrix operation.

[0070] Preferably, the reinforcement learning model includes:

[0071] Input layer, hidden layer, output layer;

[0072] A convolutional layer is added between the input layer and the hidden layer, and the convolutional layer takes the constructed feature matrix as input.

[0073] For cases where there is no coupling between the X and Y directions, S 3*1 convolutional kernels are set for each direction to extract the variation features in the X and Y directions respectively;

[0074] When there is coupling in the X and Y directions, the changes in the X and Y directions are considered simultaneously, as well as the influence of the X and Y direction correction iron on the BPM. Three convolutional kernels of different sizes are used for feature extraction.

[0075] Preferably, before the BPM value is input into the reinforcement learning model, the reinforcement learning model outputs a current change value, and the current change value is added to the current magnet current value to obtain the corrected current value, the reinforcement learning model is further trained, specifically:

[0076] Set the reward function to include: distance reward, change reward, numerical reward, task completion reward, and remaining steps reward.

[0077] The reinforcement learning model is trained based on distance reward, change reward, numerical reward, task completion reward, and remaining steps reward.

[0078] Example 2

[0079] A reinforcement learning-based accelerator orbit correction system with strong generalization ability, the system comprising:

[0080] Modeling unit, used to build simulated accelerators based on data from real accelerators;

[0081] The processing unit is used to measure the BPM value in the analog accelerator and construct a feature matrix based on the BPM value.

[0082] The prediction unit is used to input the feature matrix into the reinforcement learning model, and the reinforcement learning model outputs the current change value. The current value of the real accelerator is corrected based on the current change value and the magnet current value in the real accelerator.

[0083] This invention establishes a simulated accelerator based on data from a real accelerator; measures the BPM value in the simulated accelerator and constructs a feature matrix based on the BPM value; inputs the feature matrix into a reinforcement learning model, which outputs a current change value; and corrects the current value of the real accelerator based on the current change value and the magnet current value in the real accelerator. This is an offline accelerator beam trajectory correction based on reinforcement learning. The readings of the beam position detector of the real accelerator are read, and then observations are constructed. These observations are used as input to the trained reinforcement learning model, which outputs a current change value. This current change value is added to the current value of the current correction iron to obtain the current correction current value, which is then set on the real accelerator through a data interaction interface. Due to the unknown differences between the simulated and real accelerators and the high cost of beam commissioning, this invention establishes a simulated accelerator based on real accelerator data; observes the BPM value in the simulated accelerator; inputs the BPM value into the reinforcement learning model, which outputs a current change value; and adds the current change value to the current magnet current value to obtain the correction current value. This significantly reduces online commissioning time and cost and improves beam commissioning efficiency. This method can bridge the gap between simulated and real accelerators, allowing models to be trained on simulated accelerators and directly applied to real accelerators. This can significantly save online beam time and reduce beam commissioning costs. This method can be applied not only to the mid-energy range of accelerators, but also to the superconducting and high-energy ranges.

[0084] Example 3

[0085] This embodiment provides a computer device, including a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the above-described reinforcement learning-based accelerator orbit correction method with strong generalization ability.

[0086] Example 4

[0087] This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the program is executed by a processor, the processor executes the computer program stored in the memory to implement the above-described reinforcement learning-based accelerator orbit correction method with strong generalization ability.

[0088] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein are implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A strong generalization-based accelerator orbit correction method based on reinforcement learning, characterized in that, include: A simulated accelerator was built based on data from a real accelerator. BPM values ​​are measured in a simulated accelerator, and a characteristic matrix is ​​constructed based on the BPM values; The feature matrix is ​​input into the reinforcement learning model, and the reinforcement learning model outputs the current change value. The current value of the real accelerator is corrected based on the current change value and the magnet current value in the real accelerator. The BPM value is specifically: Let N be the BPM readings at time i. Assume there are N sets of BPMs, each set divided into X and Y directions. ,in This represents the BPM reading in the X direction for the nth group at time i. This represents the reading of the nth group of BPM in the Y direction at time i; The process of inputting the feature matrix into the reinforcement learning model, the reinforcement learning model outputting current change values, and correcting the current value of the real accelerator based on the current change values ​​and the magnet current values ​​in the real accelerator specifically includes: definition Then the observed values ​​during the simulation of the accelerator operation are: , To determine the current value of the corrector at time i, assuming there are a total of K sets of correctors, then... , Let be the current value of the k-th group of corrective iron in the X direction at time i. Let the current value of the corrector in the Y direction at time i be the value of the k-th group of correctors. The corrector current at time i-1 is defined as... The current increment predicted by the reinforcement learning model is Then the current of the corrector iron at time i ; The measurement of BPM values ​​in the simulated accelerator includes: Random component errors are added to the accelerator simulation software to simulate the deviation of the beam's center of mass from that of a real accelerator. The beam position monitor readings are split into x-direction and y-direction readings. The result after splitting is: , For regions where there is coupling between the x and y directions, i.e., adjusting the corrector in the x direction will affect the beam position monitors in both the x and y directions simultaneously, they are not separated. The observation OB is designed based on whether there is coupling in the x and y directions. For the case where there is no coupling in the x and y directions, the observation design is as follows: At time i, The definition is as follows: ,in: , , ,in: , , When there is coupling in the X and Y directions ,in , , where T is the transpose operator in the matrix operation.

2. The accelerator orbit correction method based on reinforcement learning with strong generalization ability according to claim 1, characterized in that, The process of building a simulated accelerator based on data from a real accelerator includes: Accelerator simulation software was selected as the basic environment, and the beam position detector of the simulated accelerator was set according to the beam position monitor on the real accelerator. The values ​​of the quadrupole current at the corresponding location in the simulated accelerator are set according to the values ​​of the quadrupole current in the real accelerator. Replace the location for correcting the ferrocurrent in the accelerator's configuration file with a replaceable variable.

3. The accelerator orbit correction method based on reinforcement learning with strong generalization ability according to claim 1, characterized in that, The reinforcement learning model includes: Input layer, hidden layer, output layer; A convolutional layer is added between the input layer and the hidden layer, and the convolutional layer takes the constructed feature matrix as input. For cases where there is no coupling between the X and Y directions, S 3*1 convolutional kernels are set for each direction to extract the variation features in the X and Y directions respectively; When there is coupling in the X and Y directions, the changes in the X and Y directions are considered simultaneously, as well as the influence of the X and Y direction correction iron on the BPM. Three convolutional kernels of different sizes are used for feature extraction.

4. The accelerator orbit correction method based on reinforcement learning with strong generalization ability according to claim 1, characterized in that, The BPM value is input into the reinforcement learning model, and the reinforcement learning model outputs the current change value. Before adding the current change value to the current magnet current value to obtain the corrected current value, the reinforcement learning model is trained, specifically as follows: Set the reward function to include: distance reward, change reward, numerical reward, task completion reward, and remaining steps reward. The reinforcement learning model is trained based on distance reward, change reward, numerical reward, task completion reward, and remaining steps reward.

5. An accelerator orbit correction system with strong generalization ability based on reinforcement learning, characterized in that, The system includes: Modeling unit, used to build simulated accelerators based on data from real accelerators; The processing unit is used to measure the BPM value in the analog accelerator and construct a feature matrix based on the BPM value. The prediction unit is used to input the feature matrix into the reinforcement learning model, and the reinforcement learning model outputs the current change value. The current value of the real accelerator is corrected based on the current change value and the magnet current value in the real accelerator. The BPM value is specifically: Let N be the BPM readings at time i. Assume there are N sets of BPMs, each set divided into X and Y directions. ,in This represents the BPM reading in the X direction for the nth group at time i. This represents the reading of the nth group of BPM in the Y direction at time i; The process of inputting the feature matrix into the reinforcement learning model, the reinforcement learning model outputting current change values, and correcting the current value of the real accelerator based on the current change values ​​and the magnet current values ​​in the real accelerator specifically includes: definition Then the observed values ​​during the simulation of the accelerator operation are: , To determine the current value of the corrector at time i, assuming there are a total of K sets of correctors, then... , Let be the current value of the k-th group of corrective iron in the X direction at time i. Let the current value of the corrector in the Y direction at time i be the value of the k-th group of correctors. The corrector current at time i-1 is defined as... The current increment predicted by the reinforcement learning model is Then the current of the corrector iron at time i ; The measurement of BPM values ​​in the simulated accelerator includes: Random component errors are added to the accelerator simulation software to simulate the deviation of the beam's center of mass from that of a real accelerator. The beam position monitor readings are split into x-direction and y-direction readings. The result after splitting is: , For regions where there is coupling between the x and y directions, i.e., adjusting the corrector in the x direction will affect the beam position monitors in both the x and y directions simultaneously, they are not separated. The observation OB is designed based on whether there is coupling in the x and y directions. For the case where there is no coupling in the x and y directions, the observation design is as follows: At time i, The definition is as follows: ,in: , , ,in: , , When there is coupling in the X and Y directions ,in , , where T is the transpose operator in the matrix operation.

6. A computer device, characterized in that, The computer device includes a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, the computer device performs the method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, A stored program, which, when executed by a processor, performs the method according to any one of claims 1 to 4.

Images