Quantitative filling oil way control method and system based on reinforcement learning

Abstract

The invention relates to the technical field of fluid automation control, and discloses a quantitative filling oil way control method and system based on reinforcement learning, and the method comprises the steps: firstly generating a physical reference opening instruction based on a fluid mechanics model; calculating a rheological hysteresis factor to identify the viscosity characteristic of the fluid by superposing a micro-disturbance signal and analyzing the downstream pressure frequency response; and constructing a flow velocity adaptive state vector by adopting equal-volume interval sampling, and inputting the flow velocity adaptive state vector into the reinforcement learning network to output residual correction. Meanwhile, dynamic transmission time lag is calculated according to the rheological hysteresis factor and the real-time flow velocity, and time axis causal alignment of flow rewards and historical actions is executed. And finally, synthesizing the reference instruction, the correction amount and the micro-disturbance signal, driving the regulating valve after water attack suppression logic constraint, and executing end point braking at the tail section. The problems of time-delay matching and sample distortion under the long-pipeline variable-viscosity working condition are solved, and the quantitative filling precision and the system stability are improved.

Description

Technical Field

[0001] This invention relates to the field of fluid automation control technology, specifically to a quantitative oil filling circuit control method and system based on reinforcement learning. Background Technology

[0002] In aerospace, petrochemical, and large-scale industrial production, the quantitative dispensing of fluid media is a core component, and its control accuracy and stability directly affect the system's operational safety and the success or failure of the mission. Such systems typically employ electro-hydraulic proportional control valves as actuators, working in conjunction with flow meters and pressure sensors to form a closed-loop control circuit, thereby achieving precise control over the total dispensing volume.

[0003] However, in real-world scenarios involving wide-temperature-range and long-pipeline refueling, existing control technologies face numerous challenges. First, the physical properties of the fluid medium are not static; fluctuations in ambient temperature or differences between batches of the medium can cause significant drifts in parameters such as fluid viscosity and density. Traditional control methods typically rely on offline-calibrated fixed fluid dynamics models or fixed PID parameters, failing to detect and compensate for damping changes caused by fluid rheological characteristics in real time. When actual operating conditions deviate from the calibration point, control commands based on fixed parameters exhibit significant execution deviations, making it difficult to maintain steady-state control accuracy.

[0004] Secondly, large-scale dispensing systems often feature long intermediate transmission pipelines, introducing significant transmission hysteresis. Unlike constant-flow-rate systems, the flow rate during quantitative dispensing varies dynamically over a wide range, resulting in a nonlinear, time-varying transmission hysteresis between valve actuation and flowmeter response. Existing data-driven control methods or conventional reinforcement learning strategies typically assume a fixed response delay and often employ equal-interval sampling based on absolute time. Under variable-flow-rate conditions, this approach leads to a causal misalignment between control actions and feedback rewards on the time axis. Furthermore, the time-sampled data fails to accurately reflect the physical volumetric process of fluid delivery, causing distortion in the training sample distribution and making it difficult for the control strategy to converge or resulting in oscillations over long time delays.

[0005] Furthermore, to ensure the accuracy of the filling endpoint, the control system needs to rapidly adjust the valve opening at the end. However, in long pipeline systems, the fluid has significant inertia, and excessively rapid valve action can easily trigger water hammer, causing a sudden surge in pipeline pressure and resulting in physical damage to valves, sensors, and pipeline seals. Existing control schemes often struggle to strike a balance between pursuing high response speed and suppressing pressure surges, lacking an effective mechanism to dynamically constrain the increment of control commands based on the real-time pressure status of the pipeline, thus limiting the system's extreme control capabilities while ensuring safety. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a quantitative oil filling circuit control method and system based on reinforcement learning, aiming to solve the problem that existing quantitative filling control technologies struggle to simultaneously ensure filling accuracy and system stability when faced with fluid viscosity variations, long pipeline transmission delays, and water hammer effects.

[0007] To achieve the above objectives, the present invention provides a quantitative oil filling circuit control method and system based on reinforcement learning. The first aspect of the invention provides a quantitative oil filling circuit control method based on reinforcement learning, which is applied to a control system including a physical fluid loop unit and a control processing unit. In terms of control logic, this method combines physical model-based baseline control with data-driven residual correction, and introduces active rheological identification and spatiotemporal alignment mechanisms. Specifically, at the beginning of the control cycle, the control processing unit initializes system parameters and reads the target filling amount of the filling task, acquires real-time differential pressure data of the pipeline through a sensor group, and calculates the physical baseline opening command that satisfies the current physical constraints using a preset fluid dynamics model.

[0008] After the filling process enters the steady-state phase, a micro-perturbation signal of a specific frequency is superimposed on the physical reference opening command, and the frequency response data of the downstream pressure to this signal is collected. Based on this, the rheological hysteresis factor characterizing the current viscosity of the fluid is calculated. To address the time-scale distortion caused by variable flow rate, historical flow data is resampled at preset equal volume intervals to construct a flow rate adaptive state vector. This vector is then input into a reinforcement learning policy network, which outputs a residual correction amount for the physical reference opening command.

[0009] To address pipeline transmission delay, a dynamic transmission delay is calculated based on the rheological hysteresis factor and the current real-time flow rate. This delay is then used to causally align the current flow reward signal with the action commands generated in the past on the time axis, constructing a tuple containing the correct causal relationship and storing it in the experience replay pool for network training.

[0010] During execution, the physical reference opening command, residual correction amount, and micro-disturbance signal are synthesized, and the synthesized command is dynamically constrained by water hammer suppression logic to generate the final control command to drive the electro-hydraulic proportional control valve. At the end of the filling stage, the remaining filling amount is monitored in real time. When the remaining amount is lower than the preset threshold, the system switches to the endpoint braking strategy until the valve is completely closed.

[0011] Preferably, the generation of the physical reference opening command adopts the reverse physical modeling method: first, the fluid specific gravity is calculated based on the fluid temperature, and the flow coefficient corresponding to the valve opening is obtained; a Bernoulli equation model describing the mapping relationship between instantaneous flow rate and valve opening, real-time pipeline pressure difference and fluid specific gravity is constructed; by performing inverse operation on the model, the target set flow rate and real-time pressure difference data are converted into the corresponding opening value.

[0012] Preferably, the solution of the rheological hysteresis factor is based on the principle of frequency domain analysis: the response component with the same frequency as the micro-disturbance signal is extracted from the frequency response data of the downstream pressure, and the phase hysteresis and amplitude attenuation ratio of the response component relative to the micro-disturbance signal are calculated. Subsequently, the phase hysteresis, amplitude attenuation ratio and fluid temperature are weighted and summed using pre-calibrated weighting coefficients, thereby quantifying the comprehensive influence of fluid viscosity distribution and pipe wall elasticity on the flow field.

[0013] In one specific embodiment, the calculation of dynamic transmission delay combines physical volume and viscosity correction: first, the ratio of the physical volume of the intermediate transmission pipeline downstream of the electro-hydraulic proportional control valve to the current instantaneous flow rate is calculated to obtain the basic physical transmission time; then, the basic physical transmission time is multiplied and corrected using a correction coefficient based on the rheological hysteresis factor to compensate for the uneven flow velocity distribution caused by fluid viscosity.

[0014] Furthermore, the construction of the flow rate adaptive state vector aims to eliminate the uncertainty in the time dimension: the instantaneous flow rate is integrated, and data is collected every time the accumulated flow volume reaches a preset threshold to form a historical flow sequence vector. This vector is then fused with the normalized remaining injection volume, the rate of change of pressure after the valve, the real-time differential pressure in the pipeline, and the rheological hysteresis factor as the input to the reinforcement learning network.

[0015] For sample generation in reinforcement learning, the causal alignment logic on the time axis is as follows: the calculated dynamic transmission delay is converted into the number of discrete control cycles, and the historical data buffer is backtracked based on the number of cycles to retrieve the historical actions and states that generated the current traffic result. These are then recombined with the current traffic reward signal to ensure the physical causality of the training data.

[0016] Preferably, the control command is synthesized using a linear superposition method, adding the physical reference opening command, the residual correction amount, and the micro-perturbation signal to generate the original command. The residual correction amount is a value output by the reinforcement learning policy network and processed by a hyperbolic tangent function to limit it within a preset correction range, preventing the policy network from outputting excessively.

[0017] To protect the pipeline system, the water hammer suppression logic is configured in dynamic limiting mode: real-time acquisition of inlet valve pressure data and calculation of the maximum single-step opening change that is inversely proportional to the inlet valve pressure. The increment of the original command relative to the final command of the previous control cycle is calculated. If the absolute value of this increment exceeds the maximum permissible single-step opening change, the increment is truncated, and the command of the previous cycle is added to the truncated increment as the final output.

[0018] During the endpoint control phase, the endpoint braking strategy executes the following logic: It calculates the target flow limit value, which monotonically decreases as the remaining fill volume decreases, based on the square root braking law, and then solves for the braking opening command. Simultaneously, it estimates the fluid tail volume in real time based on the valve closing response time and the fluid damping coefficient. When the remaining fill volume is less than or equal to the tail volume, or when the braking opening command is less than the valve's physical minimum controllable opening, the control command is forcibly reset to zero to achieve precise cutoff.

[0019] A second aspect of the present invention provides a quantitative oil filling circuit control system based on reinforcement learning. The system includes: a sensing and preprocessing module configured to acquire raw signals from sensors and perform filtering processing, outputting system state observations including differential pressure, flow rate, and temperature; an active rheological identification module configured to generate micro-perturbation signals and calculate the rheological hysteresis factor based on the frequency response characteristics of the downstream pressure to the micro-perturbation signals; a reinforcement learning decision module configured to receive the flow velocity adaptive state vector after equal-volume interval sampling processing, and output a residual correction amount for the physical reference opening command using a policy network; a spatiotemporal aligned data management module configured to align the time axis of historical actions and current rewards based on dynamic transmission delays, construct causal aligned tuples, and store them in an experience replay pool; and an execution control module configured to calculate the physical reference opening command, combine the residual correction amount and the micro-perturbation signal to synthesize the final control command, and execute water hammer suppression and endpoint braking operations to drive the electro-hydraulic proportional control valve.

[0020] This invention provides baseline control through a physical model, uses reinforcement learning to handle nonlinear residuals, and combines active rheological identification and dynamic time delay compensation to effectively improve the accuracy of quantitative injection and the robustness of the system under complex working conditions.

[0021] This invention provides a quantitative fuel injection circuit control method and system based on reinforcement learning. It has the following beneficial effects:

[0022] 1. This invention, by superimposing micro-disturbance signals and calculating the rheological hysteresis factor during the steady-state filling phase, can identify the viscosity characteristics of the fluid and the elastic effect of the pipe wall in real time. This active identification mechanism solves the problem of fluid dynamic parameter drift caused by ambient temperature fluctuations or batch differences in the medium. By introducing the rheological hysteresis factor into the flow model correction and time delay calculation, it effectively suppresses control errors caused by changes in fluid viscosity, ensuring the accuracy of the reference control command without requiring shutdown and recalibration.

[0023] 2. This invention employs a strategy of equal-volume interval sampling and dynamic time-delay causal alignment, overcoming the defects of uneven data density and time axis distortion caused by traditional time sampling in variable flow velocity scenarios. By calculating the dynamic transmission delay based on real-time flow velocity and rheological characteristics, the current flow reward signal is accurately backtracked and matched with the action commands at historical moments, ensuring that the training samples stored in the experience replay pool have strict physical causal consistency, significantly improving the convergence speed and decision stability of reinforcement learning policy networks in long pipelines and large time-delay systems.

[0024] 3. This invention constructs a hybrid control architecture that includes a physical model baseline, reinforcement learning residual correction, and water hammer suppression logic. The physical model provides a fast basic response based on Bernoulli's equation, the reinforcement learning network performs minor corrections for nonlinear disturbances to improve final accuracy, and the water hammer suppression logic based on inlet valve pressure feedback dynamically limits the increment of commands. This layered architecture ensures high-precision quantitative dispensing while effectively preventing pipeline pressure surges caused by excessively rapid valve action, achieving a balance between operational efficiency and equipment safety. Attached Figure Description

[0025] Figure 1 This is a system framework diagram of the present invention;

[0026] Figure 2 This is a schematic diagram of the method flow of the present invention.

[0027] The components include: 1. Oil storage tank; 2. Variable frequency pump; 3. Oil delivery pipeline; 301. Intermediate transmission pipeline; 4. Electro-hydraulic proportional control valve; 5. High-precision flow meter; 601. Inlet pressure sensor; 602. Outlet pressure sensor; 603. Temperature sensor; 200. Control processing unit; 201. Sensing and preprocessing module; 202. Active rheological identification module; 203. Spatiotemporal aligned data management module; 204. Reinforcement learning decision-making module; 205. Execution control module. Detailed Implementation

[0028] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] See attached document Figure 1 This invention provides a quantitative oil filling control system based on reinforcement learning. The system mainly consists of a physical fluid loop unit and a control processing unit, and is used to achieve high-precision quantitative filling of fluids with varying viscosity under long pipeline transportation conditions.

[0030] The physical fluid loop unit includes an oil storage tank 1, a variable frequency pump 2, an oil delivery pipeline 3, an electro-hydraulic proportional control valve 4, and a high-precision flow meter 5. The oil storage tank 1 stores the liquid medium to be added, and its outlet is connected to the suction port of the variable frequency pump 2 via a pipeline. The variable frequency pump 2 serves as the system's power source, used to establish pipeline pressure, and its outlet is connected to the main section of the oil delivery pipeline 3. The electro-hydraulic proportional control valve 4, acting as the flow control actuator, is located on the oil delivery pipeline 3 and receives control commands to continuously adjust the valve core opening to change the flow area. The high-precision flow meter 5, serving as the system's feedback mechanism, is installed downstream of the electro-hydraulic proportional control valve 4 and is used to collect the instantaneous flow rate of the fluid in real time. and cumulative injection volume An intermediate transmission pipe 301 connects the outlet of the electro-hydraulic proportional control valve 4 to the inlet of the high-precision flow meter 5. This intermediate transmission pipe 301 has a physical length... Since the fluid needs time to travel from the electro-hydraulic proportional control valve 4 through the intermediate transmission pipeline 301 to the high-precision flow meter 5, and this transmission time depends on the real-time flow rate and rheological characteristics of the fluid, the intermediate transmission pipeline 301 constitutes a time-varying pure delay element in the system control loop.

[0031] The system further includes sensor arrays distributed at key nodes along the pipeline. A pressure sensor 601 is located at the inlet side of the electro-hydraulic proportional control valve 4 to collect the pressure upstream of the valve. The downstream pressure sensor 602 is located on the outlet side of the electro-hydraulic proportional control valve 4, i.e., at the beginning of the intermediate transmission line 301, and is used to collect the downstream pressure. and high-frequency pressure fluctuation signals; temperature sensor 603 is installed inside oil pipeline 3 and immersed in the fluid to collect the real-time temperature of the fluid. The aforementioned sensor group and high-precision flow meter 5 are all connected to the control processing unit 200 via signal cables.

[0032] The control processing unit 200 is a computing device that includes a microprocessor and storage medium, including a sensing and preprocessing module 201, an active rheology identification module 202, a spatiotemporal aligned data management module 203, a reinforcement learning decision module 204, and an execution control module 205.

[0033] The sensing and preprocessing module 201 is communicatively connected to the aforementioned sensor group and is configured to acquire raw analog signals at a preset sampling frequency, perform low-pass filtering and noise reduction processing, and output the system's state observation values, including pressure difference. Instantaneous flow ,temperature and pressure change rate .

[0034] The active rheology identification module 202 is configured to generate a micro-perturbation signal with a specific frequency and amplitude during the steady-state phase of the filling process, and transmit this micro-perturbation signal to the execution control module 205. Simultaneously, the active rheology identification module 202 receives pressure response data from the downstream pressure sensor 602, and calculates the fluid's rheological parameters based on the phase difference and amplitude ratio between the micro-perturbation signal and the pressure response signal. These rheological parameters characterize the fluid's current equivalent viscosity and pressure wave propagation velocity.

[0035] The spatiotemporal alignment data management module 203 is configured to maintain a causal alignment experience replay pool (STCARB). This module calculates the dynamic transmission delay based on the rheological parameters calculated by the active rheological identification module 202 and the current real-time flow velocity. The spatiotemporal alignment data management module 203 executes data reorganization logic, setting the current time... Flow reward signal fed back by high-precision flow meter 5 With historical moments Action instructions generated by reinforcement learning decision module 204 Perform backtracking matching on the timeline, construct aligned tuples containing the correct causal relationships, and store them in the causal alignment experience replay pool.

[0036] The reinforcement learning decision module 204 internally constructs a policy network (Actor) and a value network (Critic) based on neural networks. This module is configured to obtain a flow-weighted state vector from the perception and preprocessing module 201 and train it using data stored in the causal alignment experience replay pool. The reinforcement learning decision module 204 does not directly output the total valve opening; instead, based on a residual learning architecture, it outputs an opening correction amount for the physical baseline model. .

[0037] The execution control module 205 is configured to calculate the physical reference opening degree and synthesize the final command. This module internally stores a physical flow model based on Bernoulli's equation and calculates the reference opening degree based on the target flow rate and the current pressure difference. The execution control module 205 sets the reference opening degree. The opening correction amount output by reinforcement learning decision module 204 The signals are superimposed, along with the micro-perturbation signals generated by the active rheology identification module 202, to generate the final control command. The execution control module 205 is also equipped with water hammer protection logic to handle the final control commands. After limiting the amplitude and rate of change, the electro-hydraulic proportional control valve 4 is activated.

[0038] See attached document Figure 2 This invention provides a quantitative oil filling circuit control method based on reinforcement learning. This method is executed by a control processing unit and includes the following steps:

[0039] S100 initializes system parameters and reads the target injection volume, collects real-time differential pressure data of the pipeline, and calculates the physical reference opening command based on the fluid dynamics model;

[0040] S200 superimposes micro-perturbation signals onto the physical reference opening command during the steady-state filling phase, collects frequency response data of the downstream pressure, and calculates the rheological hysteresis factor and equivalent viscosity parameters of the fluid.

[0041] S300 resamples historical flow data at preset equal volume intervals to construct a flow velocity adaptive state vector, inputs the flow velocity adaptive state vector into a reinforcement learning policy network, and outputs a residual correction amount for the physical reference opening command.

[0042] S400 calculates the dynamic transmission delay based on the rheological hysteresis factor and the current flow velocity, aligns the current flow reward signal with the action instructions generated in the past time on the time axis, constructs a causal alignment tuple and stores it in the experience replay pool.

[0043] The S500 synthesizes the physical reference opening command, residual correction amount and micro-disturbance signal, and generates the final control command after water hammer suppression logic processing, which drives the electro-hydraulic proportional control valve to operate.

[0044] The S600 monitors the remaining refueling amount in real time. When the remaining refueling amount is less than the preset threshold, it executes the endpoint braking strategy until the valve is completely closed.

[0045] The following section will elaborate on each of the above control steps in conjunction with specific computational models and logical algorithms.

[0046] Step S100 is mainly completed by the execution control module 205 in conjunction with the sensing and preprocessing module 201. At the initial stage of the control cycle or at the beginning of each time step, the execution control module 205 constructs a feedforward control loop based on the principles of fluid mechanics. The purpose is to provide a rough action reference for the electro-hydraulic proportional regulating valve 4 that satisfies the basic physical laws, thereby reducing the action search space and exploration difficulty of the subsequent reinforcement learning module.

[0047] Specifically, the execution control module 205 first receives real-time status data collected and filtered by the sensing and preprocessing module 201. The status data includes the pressure difference across the valve obtained by the upstream pressure sensor 601 and the downstream pressure sensor 602. and the fluid temperature obtained through temperature sensor 603. Based on Bernoulli's equation and its engineering application variants, a clear physical coupling relationship exists between the instantaneous flow rate through the electro-hydraulic proportional control valve 4 and the valve opening and differential pressure. The execution control module 205 internally stores a flow model describing this physical coupling relationship, which is expressed as:

[0048] ;

[0049] in, This represents the instantaneous flow rate predicted by the model; This command represents the normalized valve opening degree, with a value range of 0 to 1. Indicates real-time differential pressure; Indicates the fluid at the current temperature The proportion of the lower; Indicates valve opening degree The corresponding flow coefficient function.

[0050] This embodiment implements the functional parameters involved in the above flow model using specific lower-level technical features. Regarding the proportion... Considering that the density of the oil medium changes significantly with temperature, the execution control module 205 is equipped with linear temperature correction logic, based on the formula... Calculate the current proportion, where Reference temperature The standard proportion below, Let be the coefficient of thermal expansion of the fluid. For the flow coefficient function... Its specific form depends on the valve core structure characteristics of the electro-hydraulic proportional control valve 4, and it usually exhibits linear or equal percentage characteristics. In this embodiment, the execution control module 205 obtains the current opening degree by querying a pre-stored flow characteristic calibration table or calling a high-order polynomial fitting function. The value represents the flow characteristics. This calibration table is built based on offline test data and covers the flow coefficient distribution from fully closed to fully open valve stroke.

[0051] Based on the established flow model, the execution control module 205 performs a reverse calculation to generate a physical reference opening command. Specifically, the execution control module 205 reads the target set flow rate for the current control cycle. Substituting this into the inverse function of the aforementioned flow model, the physical baseline opening is calculated. The calculation process is as follows:

[0052] ;

[0053] in, This represents the inverse mapping relationship of the flow coefficient function. If... If the features are stored in the form of a lookup table, the execution control module 205 obtains the corresponding opening value through table lookup and interpolation algorithms; if they are stored in the form of a function, they are obtained through numerical solution.

[0054] Through the above steps, the execution control module 205 calculates in real time the flow rate that can theoretically meet the current target flow rate. Basic opening The physical reference opening As a feedforward control variable, it directly reflects the fluid physical constraints under the current operating conditions and can respond quickly to pressure differences. Significant fluctuations. However, due to the static error in the flow characteristic calibration table and the lack of consideration for nonlinear factors such as pipeline dynamics and equipment wear, relying solely on... High-precision control is difficult to achieve. Therefore, this physical reference opening degree... It will then be fed into the subsequent synthesis logic as a benchmark for reinforcement learning residual correction.

[0055] Simultaneously or immediately after establishing the physical reference control model in step S100, the active rheology identification module 202 executes step S200, actively injecting micro-disturbance signals into the fluid system and analyzing their frequency response to identify the rheological characteristics of the fluid online and quantify the dynamic transmission delay accordingly. The active rheology identification module 202 first performs the generation and injection of micro-disturbance signals. In the non-stop state during the injection process, the active rheology identification module 202 generates a high-frequency, low-amplitude sinusoidal jitter signal and superimposes it onto the physical reference opening command generated by the execution control module 205. Finally, a composite control command is applied to the electro-hydraulic proportional control valve 4. The expression is as follows:

[0056] ;

[0057] in, The physical reference opening obtained from the aforementioned steps; The amplitude of the micro-disturbance is set to be 0.5% to 1.0% of the full stroke of the electro-hydraulic proportional control valve 4. This amplitude is set to be able to induce detectable pressure fluctuations while avoiding macroscopic flow oscillations. The frequency is the micro-disturbance frequency. The selection of this frequency must meet the condition of avoiding the inherent frequency of the oil pipeline 3 to prevent mechanical resonance. It is usually set to a specific value that is higher than the system closed-loop control bandwidth and lower than the sensor sampling frequency.

[0058] While applying the composite control command, the active rheology identification module 202 simultaneously acquires the downstream pressure signal fed back by the downstream pressure sensor 602. The signal was analyzed for frequency response. Due to the viscous damping and compressibility of the fluid medium, the downstream pressure... The response to micro-perturbation signals exhibits phase lag on the time axis and amplitude attenuation in terms of energy. The active rheology identification module 202 is internally configured with a bandpass filter and an orthogonal correlation operation unit (or fast Fourier transform unit) to extract the frequency of the pressure signal. The component, and calculate the component relative to the input signal. Phase lag and amplitude attenuation ratio .

[0059] Based on the analyzed frequency domain characteristics, the active rheological identification module 202 calculates the rheological hysteresis factor of the fluid using a preset mapping model. This factor is used to comprehensively characterize the correction effect of fluid viscosity distribution and pipe wall elasticity on the flow velocity field. Its calculation formula is as follows:

[0060] ;

[0061] in, , and The weighting coefficients determined by the experiment through the offline system correspond to the phase lag contribution, amplitude attenuation contribution, and temperature, respectively. The thermal correction contribution, the above coefficients are constants determined by fitting the response data of fluids with different viscosities using the least squares method through offline system identification experiments.

[0062] Based on the rheological hysteresis factor identified above The active rheological identification module 202 combines real-time flow velocity information to perform quantitative calculations of dynamic transmission time delay. In long pipeline systems, this refers to the physical time required for the fluid unit to flow from the electro-hydraulic proportional control valve 4 to the high-precision flow meter 5. It is not a fixed value, but rather changes dynamically with flow velocity and fluid flow regime. The active rheology identification module 202 updates the system time delay in real time based on the following dynamic time delay model:

[0063] ;

[0064] in, The physical length of the intermediate transmission pipeline 301 between the electro-hydraulic proportional control valve 4 and the high-precision flow meter 5; This is the cross-sectional area of ​​the pipeline; The instantaneous flow rate is the current flow rate collected by the high-precision flow meter 5 and filtered. This term constitutes a nonlinear correction to the ideal plug flow model. The formula shows that the transport delay is not only inversely proportional to the volumetric flow rate, but also subject to the rheological hysteresis factor. The modulation allows for accurate reflection of the actual transmission delay deviation caused by differences in laminar / turbulent velocity distribution and fluid compressibility. The calculated dynamic transmission delay... It will be transmitted to the spatiotemporal aligned data management module 203 as a time reference for subsequent reinforcement learning data construction.

[0065] In executing step S300, in order to overcome the nonlinear distortion caused by large fluctuations in flow velocity to the extraction of temporal features, the reinforcement learning decision module 204 does not directly input the raw data collected by the sensor at fixed time intervals into the neural network, but instead adopts an equal volume interval sampling strategy based on the physical fluid volume to construct an adaptive flow velocity state space.

[0066] The reinforcement learning decision module 204 continuously receives instantaneous flow data uploaded by the perception and preprocessing module 201. Internally, it maintains a flow accumulation register. This module performs time integration on the instantaneous flow rate, calculating the cumulative flow volume since the last sampling moment in real time. When the cumulative flow volume reaches a preset volume sampling threshold... At that time, the reinforcement learning decision module 204 triggers a state sampling event, records the system state at the current moment, and resets the cumulative register to zero to start the integration of the next cycle. This process realizes the mapping of the physical process from the "time domain" to the "volume domain," and its mathematical expression is: the timestamp of the sampling point sequence. The following integral constraints must be satisfied:

[0067] ;

[0068] in, Indicates the first Each volume sampling time, Indicates the first Each volume sampling time, It is a continuous flow function. The sampling interval is constant. In practical discrete control systems, since sensor data is uploaded discretely with fixed time steps, the reinforcement learning decision module 204 uses a linear interpolation algorithm to accurately calculate the time that satisfies the above integration condition. The corresponding state values ​​are used to eliminate discretization errors.

[0069] Based on the above sampling strategy, the reinforcement learning decision module 204 constructs a historical traffic sequence vector. This vector is used to characterize the historical trend of fluid flow. It is composed of the most recent... The flow data consists of flow data from sampling points of equal volume, defined as follows:

[0070] ;

[0071] in, Representative at the The average flow rate or the instantaneous flow rate at each sampling point within a volume sampling window. The sequence length is given.

[0072] The historical sequence constructed in this way corresponds physically to the flow state of a fixed-length (i.e., fixed-volume) liquid column in a pipeline, unaffected by the fluid velocity. Regardless of whether the current system is in a high-flow-rate rapid injection phase or a low-flow-rate slow approximation phase, the feature vector input to the neural network... Maintaining consistency across spatial scales eliminates the time-axis scaling effect caused by velocity variations during data preprocessing, allowing the neural network to focus on learning the essential characteristics of fluid dynamics rather than adapting to changes in time scale. This processed vector It is then incorporated into the complete state vector. In this context, it serves as the input for the policy network to generate actions.

[0073] In reinforcement learning decision-making module 204, historical traffic sequence vectors were completed. After its construction, the module then performs a fusion operation of multi-source physical features to generate a complete system state vector. This serves as the input tensor for both the Actor Network and the Critic Network. (State vector) The design aims to provide the neural network with complete information about the filling process, pipeline safety status, fluid dynamics characteristics, and energy boundary conditions, thereby ensuring that the output action commands are physically interpretable and safe.

[0074] Reinforcement learning decision module 204 will use the state vector Defined as a multidimensional vector composed of remaining injection capacity, pressure change rate, historical flow sequence, valve differential pressure, and rheological hysteresis factor. The mathematical expression for this state vector is as follows:

[0075] ;

[0076] in, This represents the normalized remaining bet amount. The reinforcement learning decision module 204 first reads the target bet amount. Compared with the current cumulative injection volume Calculate the physical surplus Then divide by Normalization is performed. This feature component provides the agent with global guidance information about task completion, which is directly related to the control objective of injection accuracy.

[0077] This represents the normalized rate of change of the downstream pressure. The sensing and preprocessing module 201 processes the acquired downstream pressure signal. Perform discrete differential operations to calculate... and map it to a preset interval. This feature component is used to characterize the dynamic fluctuation of pipeline pressure. As a core indicator of pipeline dynamic safety, it enables neural networks to sense the severity of pressure fluctuations and thus suppress water hammer effects by minimizing this value during valve closure.

[0078] This is the historical flow sequence vector generated by sampling at equal volume intervals in the preceding steps. This vector preserves the flow trend characteristics of the fluid within the most recent physical volume and eliminates the time axis scaling distortion caused by flow velocity changes, providing the agent with the spatiotemporal context information needed to predict fluid inertia.

[0079] This represents the normalized pressure difference across the valve. This data directly reflects the potential energy difference across the valve at the current moment, determining the valve's flow gain characteristics. Incorporating this data into the state space allows the neural network to perceive the current operating point and adaptively adjust the valve opening correction based on different oil supply pressures.

[0080] This represents the normalized rheological hysteresis factor, enabling the neural network to incorporate the current viscosity characteristics of the fluid into its decision-making. This parameter is calculated in real time by the active rheological identification module 202 and transmitted to the reinforcement learning decision module 204. Introducing this feature component allows the neural network to explicitly perceive the current viscosity and hysteresis characteristics of the fluid, thereby automatically adjusting the control strategy under different oil temperatures or different batches of oil, without the need for manual readjustment of the PID parameters.

[0081] The reinforcement learning decision module 204 concatenates the above physical components into a vector. Beforehand, standardized Z-Score processing or Min-Max normalization is performed to eliminate the order-of-magnitude differences between different physical units, ensuring the stability of gradient descent during neural network training. The completed... It is then fed into the input layer of the deep neural network to infer subsequent residual actions.

[0082] After the reinforcement learning decision module 204 generates action instructions and the execution control module 205 executes them, the system enters step S400, which is the data sampling and storage stage. At this time, the spatiotemporal aligned data management module 203 does not directly use the traditional synchronous sampling method to simply combine the current state, action and reward, but executes a causal reconstruction logic based on physical time delay to solve the inherent observation lag problem of long pipeline transportation systems.

[0083] In typical pipeline control scenarios, there is a physical distance between the electro-hydraulic proportional control valve 4 and the high-precision flow meter 5. Valve opening action The control effect produced by the fluid requires transmission time. Only then can it be detected by the downstream flow meter. If the time is directly... valve action With time Detected traffic rewards Pairing neural networks for training can cause reinforcement learning algorithms to incorrectly attribute current flow fluctuations to current valve actions, resulting in incorrect gradient update directions under variable time-delay conditions.

[0084] To correct this physical causal misalignment, the spatiotemporal alignment data management module 203 uses the dynamic transmission delay calculated in real time by the active rheology identification module 202. Perform discretization mapping and backtracking matching operations.

[0085] The spatiotemporal alignment data management module 203 first determines the continuous dynamic transmission delay. This is mapped to the number of lag steps in a discrete control cycle. Let the system control cycle be... For the current discrete time step Discrete lag steps Calculated using the following formula:

[0086] ;

[0087] Among them, round This indicates the rounding operation. This calculation process ensures that the discretization granularity of the time axis is consistent with the sampling frequency of the control system.

[0088] Based on a determined number of discrete lag steps The spatiotemporal alignment data management module 203 performs data backtracking within its internally maintained circular buffer. This buffer temporarily stores historical state sequences and historical action sequences over a past period. When the system is at the current time step... Flow data was collected using a high-precision flow meter 5, and the instant reward was calculated. At that time, the spatiotemporal alignment data management module 203 does not read the current action. Instead, based on the index Backtracking read Historical actions performed before a certain time step and the historical situation at that time At this point, the module constructs an aligned tuple that has undergone causal repair. Its mathematical structure is defined as follows:

[0089] ;

[0090] In this tuple, and This represents the result that led to the current traffic. The real physical reason, and This represents the delayed result caused by the physical transmission of the cause. The completed aligned tuple. The data is then stored in the Causal Alignment Experience Replay Pool (STCARB). Unlike a typical first-in, first-out queue, this pool stores physically calibrated data samples. When the reinforcement learning decision module 204 randomly samples data from the STCARB to update network parameters, the data relationships seen by the neural network are presented in an ideal form with no time delay or fully compensated time delay. This processing method allows the Critic network in the reinforcement learning algorithm to directly approximate the true action-value function. This eliminates time delay misalignment in data samples, enabling reinforcement learning algorithms to directly learn the true causal relationship between actions and results.

[0091] In reinforcement learning decision module 204, a state vector containing multi-dimensional physical features is constructed. Then, the system proceeds to step S500. The reinforcement learning decision module 204 internally employs a deep neural network based on an Actor-Critic (policy-value) architecture to fit the nonlinear control law. The policy network (Actor) is responsible for generating action commands, while the value network (Critic) is responsible for evaluating the quality of the actions to guide the parameter updates of the policy network. Unlike traditional end-to-end reinforcement learning that directly outputs the full opening command of the valve, the policy network in this embodiment is configured to output a residual correction amount for the aforementioned physical baseline opening command, thus forming a physical model-assisted residual learning mechanism.

[0092] Specifically, the policy network receives a normalized state vector. The network uses fully connected layers as input data. Hidden layers consist of several fully connected layers, with ReLU (Modified Linear Unit) or Swish activation functions used between layers to introduce non-linear feature transformation capabilities. The output layer of the policy network is designed to generate a bounded correction magnitude, using the hyperbolic tangent (tanh) function as the activation function to map the original network output to the interval [-1, 1]. The reinforcement learning decision module 204 calculates the final residual correction amount according to the following formula. :

[0093] ;

[0094] in, The parameter is The forward propagation function of the policy network; It is the hyperbolic tangent activation function; This is the preset residual weighting coefficient or the maximum correction range limit. This is a key physical constraint parameter, and its value is typically set to 5% to 15% of the full stroke of the electro-hydraulic proportional control valve 4. This is achieved by introducing parameters... The system forcibly restricts the control permissions of the neural network, allowing it to only calculate the opening degree from the physical baseline model. It allows for localized fine-tuning but cannot output large-scale mutation instructions. This design ensures that even in the early stages of neural network training or when encountering unseen conditions, the system's output remains guided by physical benchmarks, thus guaranteeing basic security.

[0095] In this architecture, the value network (Critic) operates in parallel, with its input being a state vector. The system outputs the synthesized action (or only the residual action) and a value assessment (Q-value) for the current state-action pair. This Q-value does not directly participate in real-time valve control; instead, it is used to calculate the timing difference error (TDError), which is then used to update the parameters of the policy network via backpropagation. Through this residual architecture, the neural network does not need to spend a lot of training time relearning basic fluid dynamics formulas (such as Bernoulli's equation). Instead, it focuses on learning nonlinear dynamic characteristics that are difficult for physical models to describe, such as the dynamic changes in pipeline friction coefficients, transient fluid shocks, and valve dead zone characteristics, thereby significantly improving the algorithm's convergence speed and control accuracy. The generated residual correction... It is then transmitted to the execution control module 205 for subsequent instruction synthesis.

[0096] The execution control module 205 receives the residual correction amount output by the policy network. Subsequently, the linear synthesis and safety constraint processing of multi-source commands are performed to generate the final control signal driving the electro-hydraulic proportional control valve 4. This process aims to organically combine the steady-state prediction capability of the physical model, the dynamic compensation capability of reinforcement learning, and the micro-disturbance requirements of active identification, while ensuring that the executed actions comply with the physical safety boundaries of the pipeline system.

[0097] The execution control module 205 first performs a linear superposition operation of the instructions. This module reads the reference opening degree calculated by the physical reference model. The residual correction amount output by the reinforcement learning decision module 204 And the micro-perturbation signal generated by the active rheology identification module 202. In the non-identification phase, micro-perturbation signals Set to zero. The synthesized original instruction. Follow the following mathematical model:

[0098] ;

[0099] in, This indicates a composite aperture command that has not undergone constraint processing. This is a feedforward component that reflects the current pressure difference and the physical constraints of the target flow rate; This is an intelligent compensation component used to eliminate the effects of model errors and time delays; This is a probe signal used to excite the system's frequency response.

[0100] After obtaining the original instructions, the execution control module 205 applies physical saturation limits and dynamic rate limits to them to generate the final control instructions. Physical saturation limiting ensures the command value remains within the effective stroke range of the control valve (i.e., 0% to 100%), preventing control divergence caused by actuator saturation or mechanical dead zone. Dynamic rate limiting suppresses pipeline water hammer effects caused by sudden command changes, particularly preventing pressure surges caused by rapid valve closure under high pressure differential conditions. The execution control module 205 calculates the final command based on the following multi-objective constraint formula:

[0101] ;

[0102] in, This is the amplitude truncation function; This indicates the final instruction executed in the previous control cycle; This is the rate-of-change constraint function.

[0103] Rate of change constraint function The specific definitions are as follows:

[0104] ;

[0105] In this formula, Represents the currently calculated instruction increment; The maximum allowable single-step opening change, It is not a fixed value, but is determined by the execution control module 205 based on the current inlet pressure of the valve. A dynamically adjusted function. To prevent water hammer under high pressure differential, the execution control module 205 adjusts the current inlet pressure based on the current valve pressure. Dynamic adjustment The two are inversely proportional, and the specific calculation formula is as follows:

[0106] ;

[0107] in, The safety constant related to the pressure rating of its pipeline, To prevent small positive numbers with a denominator of zero, the system automatically reduces the allowable rate of change in valve opening when the pressure before the valve increases, thereby smoothing pressure fluctuations.

[0108] When a high pressure is detected before the valve, the control module 205 automatically reduces it. The value of this constraint is adjusted to forcibly reduce the valve's operating rate, thereby actively smoothing pressure fluctuations and protecting pipeline hardware. After the above constraint treatment... It is converted into a standard current or voltage signal and sent to the servo driver of the electro-hydraulic proportional control valve 4 for execution.

[0109] In the process of model training and policy update, the reinforcement learning decision module 204 employs a composite reward function mechanism that integrates physical safety constraints. This mechanism guides the policy network to actively explore and learn control boundaries that suppress pipeline water hammer while pursuing flow control accuracy. The design of this reward function overcomes the shortcomings of traditional reinforcement learning, which focuses only on error convergence and neglects process dynamic safety. By transforming the pressure gradient constraint in fluid dynamics into a numerical penalty term, it forces the agent to avoid areas in the action space that may induce dangerous conditions.

[0110] Reinforcement learning decision-making module 204 at each control time step Calculate instant reward value The immediate reward value is not a single scalar, but a multi-objective evaluation function composed of weighted terms for flow tracking, water hammer suppression, and motion smoothing, and its mathematical expression is as follows:

[0111] ;

[0112] in, This represents the total reward value; a higher value indicates a better current action strategy. , and These are the corresponding weighting coefficients, used to balance the relative importance of each sub-objective in the overall optimization objective. In this embodiment, the weighting coefficients... It is set as a dynamic variable that increases with the remaining injection volume. That is, in the valve closing stage at the end of the injection, the weight of the water hammer suppression term is significantly increased to enhance the system's safety awareness.

[0113] The specific definitions and physical meanings of each term in the formula are as follows:

[0114] This is a traffic tracking item used to assess the current instantaneous traffic. For target traffic The approximation degree. To prevent flow error from numerically masking other features, the reinforcement learning decision module 204 defines it using a Gaussian kernel function:

[0115] ;

[0116] in, Let be the desired flow control tolerance. This functional form ensures that the reward is at most 1 when the error is zero, and the reward decays non-linearly to close to 0 as the error increases, thus encouraging fine-tuning of the model in the steady-state phase.

[0117] The water hammer suppression term is the core constraint component of this control strategy. The reinforcement learning decision module 204 utilizes the pressure change rate provided by the perception and preprocessing module 201. (Right now This is used to quantify the risk of water hammer. This term aims to penalize excessive pressure gradients, and its calculation formula is as follows:

[0118] ;

[0119] in, The threshold for the allowable safe pressure change rate of the pipeline system. For linear rectified functions (i.e. This formula shows that when the actual pressure gradient... When the pressure gradient is below the safety threshold, the penalty term is zero, and the system does not intervene in normal pressure fluctuations. Once the pressure gradient exceeds the threshold, the system applies a strong penalty that increases quadratically. Through this mechanism, the Critic network can learn the temporal correlation between rapid and large valve-closing actions and high negative rewards, thereby forming a virtual pressure gradient wall in the gradient update direction of the Actor network, forcing the agent to automatically output a smooth shutdown curve.

[0120] This is a motion smoothing term used to suppress high-frequency jitter in control commands and reduce mechanical wear on the actuator. This term addresses the residual correction amount. Penalize the amplitude and rate of change:

[0121] ;

[0122] in, This is the rate-of-change penalty coefficient. The existence of this term ensures that the correction action of the reinforcement learning output is continuous and smooth, avoiding system oscillations caused by excessive pursuit of response speed.

[0123] By maximizing the aforementioned cumulative rewards The reinforcement learning decision module 204 eventually converges to an optimal policy. This strategy manifests at the physical level as follows: in the early and middle stages of betting, with... The system is primarily responsible for quickly eliminating flow deviations; during the valve-closing phase at the end of refueling, it automatically senses the upward trend of the pipeline pressure gradient and is influenced by... By constraining and actively slowing down the valve closing rate, high-precision quantitative control and low water hammer impact are adaptively achieved without the need for manually setting complex piecewise curves.

[0124] During the residual control process of the system performing flow rate adaptation, step S600 executes the control module 205 to continuously monitor the cumulative injection volume fed back by the high-precision flow meter 5. And calculate the distance to the target injection volume in real time. Remaining amount of fuel .when Reduce to the preset braking threshold At this point, the system automatically triggers step S600, entering the endpoint braking control phase. In this phase, to prevent overfilling due to the exploratory behavior of the reinforcement learning strategy under extremely low flow conditions, the execution control module 205 switches control from the probabilistic strategy of the reinforcement learning decision module 204 to the deterministic endpoint braking logic, implementing variable gain braking and predictive shutdown based on the remaining volume.

[0125] The execution control module 205 first constructs a dynamic flow decay trajectory, i.e., a braking curve, based on the remaining injection volume. This braking curve aims to establish a monotonically converging relationship between instantaneous flow rate and remaining volume, ensuring that the flow velocity decreases smoothly and forcefully as the remaining volume decreases. The execution control module 205 calculates the target flow limit value for the braking phase based on the following square root braking law. :

[0126] ;

[0127] in, This is the flow correction factor, typically ranging from 0.8 to 1.0; The maximum permissible deceleration (i.e., the negative value of the rate of change of flow rate) is preset. This parameter is set based on the water hammer tolerance of the pipeline system to ensure that the peak pressure wave generated during braking does not exceed the safety threshold. This formula physically simulates the uniform deceleration braking process of rigid body motion, so that the fluid momentum is dissipated according to physical laws as the volume decreases, avoiding the flow cutoff effect generated at the end of linear deceleration.

[0128] Based on the calculation The execution control module 205 calls the inverse function of the aforementioned physical baseline model to solve for the corresponding valve braking opening command. At this time, the water hammer suppression logic described in the previous steps is still in effect, clamping the rate of change of the braking opening command to ensure that the actual valve closing action does not violate the pressure gradient constraint.

[0129] While following the braking curve for flow convergence, the execution control module 205 performs predictive shutdown calculations based on dead-zone compensation in parallel. This takes into account the inherent mechanical response time of the electro-hydraulic proportional control valve 4 from receiving the shutdown command to the valve core fully closing. And the "tailing flow" caused by fluid inertia, if only Issuing a shutdown command at the wrong time will inevitably lead to the final refueling volume exceeding the tolerance. Therefore, the execution control module 205 estimates the current tail volume in real time. The calculation formula is as follows:

[0130] ;

[0131] in, This represents the current measured instantaneous flow rate; The equivalent damping coefficient of the fluid system is obtained by the active rheological identification module 202 through micro-perturbation response during the steady-state filling phase. The first term of the formula represents the volume flowing through during the valve action delay, and the second term represents the sliding volume converted from the remaining kinetic energy in the pipeline after the valve is fully closed.

[0132] The execution control module 205 will add the remaining amount With the calculated tail volume Perform real-time comparisons. Once the judgment criteria are met... The execution control module 205 immediately blocks the current flow regulation command, forcibly issues a "hard shutdown" command, and switches the control signal to the control module 205. Set it directly to zero.

[0133] Furthermore, to protect the actuator and avoid flow nonlinearity (such as cavitation or flow interruption squealing) at extremely small openings, the actuator control module 205 introduces physical small signal cutoff logic. When the calculated braking opening command is less than the physical minimum controllable opening of the electro-hydraulic proportional control valve 4... When the refill amount is 3% of the total stroke, regardless of whether the remaining refill amount is zero, the control module 205 will force a shut-off operation to avoid the valve operating in the non-linear region, and ultimately achieve a smooth shut-off of the refilling process.

[0134] This logic ensures that the valve will not operate in the valve core flutter zone for a long time. Combined with the predictive shutdown algorithm mentioned above, it achieves physical limiting protection of pipeline hardware while ensuring filling accuracy, and finally achieves a soft landing in the filling process.

Claims

1. A quantitative oil filling circuit control method based on reinforcement learning, characterized in that, Includes the following steps: S100. At the start of the control cycle of the control processing unit, initialize the system parameters and read the target filling amount of the filling task. Collect real-time differential pressure data of the pipeline of the physical fluid loop unit through the sensor group. Calculate the physical reference opening command that can meet the current fluid physical constraints based on the preset fluid dynamics model. S200. During the steady-state filling phase of the physical fluid loop unit, the control processing unit superimposes a micro-perturbation signal with a specific frequency onto the physical reference opening command, collects frequency response data of the downstream pressure in the physical fluid loop unit, and calculates the rheological hysteresis factor characterizing the current viscosity characteristics of the fluid based on the micro-perturbation signal and the frequency response data. S300: The control processing unit resamples the historical flow data fed back by the flow meter according to a preset equal volume interval, constructs a flow velocity adaptive state vector to eliminate time axis distortion, inputs the flow velocity adaptive state vector into a pre-constructed reinforcement learning policy network, and outputs the residual correction amount for the physical reference opening command. S400, The control processing unit calculates the dynamic transmission delay based on the rheological hysteresis factor and the current real-time flow rate, and uses the dynamic transmission delay to perform causal alignment on the time axis between the flow reward signal obtained at the current moment and the action command generated at the historical moment, constructs a causal alignment tuple and stores it in the experience playback pool. S500, The control processing unit synthesizes the physical reference opening command, the residual correction amount and the micro-disturbance signal, and performs constraint processing on the synthesized command through water hammer suppression logic to generate the final control command to drive the electro-hydraulic proportional regulating valve in the physical fluid loop unit to operate. S600: The control processing unit monitors the remaining amount of fuel to be added from the target amount of fuel in real time. When the remaining amount of fuel is less than the preset braking threshold, the end-point braking strategy is executed until the electro-hydraulic proportional regulating valve is completely closed.

2. The quantitative oil injection circuit control method based on reinforcement learning according to claim 1, characterized in that, In step S100, the physical reference opening command is generated in the following manner: Calculate the current fluid specific gravity based on the fluid temperature, and obtain the flow coefficient corresponding to the valve opening of the electro-hydraulic proportional regulating valve; Construct a Bernoulli equation model to describe the mapping relationship between instantaneous flow rate and valve opening, real-time pipeline differential pressure data, and fluid specific gravity; By performing the inverse operation of the Bernoulli equation model, the set flow rate corresponding to the target injection volume and the real-time pipeline differential pressure data are converted into the physical reference opening command.

3. The quantitative oil injection circuit control method based on reinforcement learning according to claim 1, characterized in that, In step S200, the calculation of the rheological hysteresis factor characterizing the current viscosity properties of the fluid includes: Extract the response component with the same frequency as the micro-perturbation signal from the frequency response data; The phase hysteresis and amplitude attenuation ratio of the response component relative to the micro-perturbation signal are obtained; The rheological hysteresis factor is obtained by weighting and summing the phase hysteresis, the amplitude attenuation ratio, and the fluid temperature using pre-calibrated weighting coefficients.

4. The quantitative oil injection circuit control method based on reinforcement learning according to claim 1, characterized in that, In step S400, the dynamic transmission delay is determined by the control processing unit based on the physical volume transmission time and the viscosity correction term; wherein, the physical volume transmission time is the ratio of the volume of the intermediate transmission pipeline downstream of the electro-hydraulic proportional regulating valve to the instantaneous flow rate corresponding to the real-time flow rate, and the viscosity correction term is a correction coefficient based on the rheological hysteresis factor.

5. The quantitative oil injection circuit control method based on reinforcement learning according to claim 1, characterized in that, In step S300, the process of constructing the flow velocity adaptive state vector is as follows: Integrate the instantaneous flow rate corresponding to the real-time flow rate, and perform sampling once whenever the accumulated flow volume reaches a preset volume sampling threshold to construct a historical flow sequence vector composed of the flow rates of multiple consecutive sampling points; The normalized remaining injection volume, the rate of change of pressure after valve, the real-time differential pressure data of the pipeline, and the rheological hysteresis factor are fused. The historical flow sequence vector is combined with the fused features to generate the flow rate adaptive state vector.

6. The quantitative oil injection circuit control method based on reinforcement learning according to claim 1, characterized in that, In step S400, the causal alignment on the time axis is performed through the following logic: The dynamic transmission delay is converted into the number of discrete control cycles of the control processing unit; Based on the discrete control cycle number, backtracking is performed in the historical data buffer to retrieve the historical action and historical status that generated the current flow result; The traffic reward signal is combined with the retrieved historical moment action and the historical moment state to form the causal alignment tuple.

7. The quantitative oil injection circuit control method based on reinforcement learning according to claim 1, characterized in that, In step S500, the physical reference opening command, the residual correction amount, and the micro-perturbation signal are synthesized, specifically as follows: The physical reference opening command, the residual correction amount, and the micro-perturbation signal are added together by linear superposition to generate the original command; The residual correction amount is a value output by the reinforcement learning policy network and limited to a preset correction range by the hyperbolic tangent function.

8. The quantitative oil injection circuit control method based on reinforcement learning according to claim 7, characterized in that, In step S500, the water hammer suppression logic is configured as follows: Calculate the instruction increment between the original instruction and the final control instruction generated in the previous control cycle; Acquire the current inlet valve pressure data and calculate the maximum allowable single-step opening change, wherein the maximum single-step opening change is configured to be inversely proportional to the inlet valve pressure data; If the absolute value of the instruction increment exceeds the maximum single-step opening change, the instruction increment is truncated to the maximum single-step opening change, and the truncated increment is added to the final control instruction generated in the previous control cycle to generate the final control instruction.

9. The quantitative oil injection circuit control method based on reinforcement learning according to claim 1, characterized in that, In step S600, the execution of the endpoint braking strategy specifically includes: The target flow limit value that decreases as the remaining injection amount decreases is calculated using the square root braking law, and the braking opening command is derived accordingly. Based on the closing response time and fluid damping coefficient of the electro-hydraulic proportional control valve, the fluid tail volume is estimated in real time. When the remaining refueling volume is less than or equal to the tail volume, or when the braking opening command is less than the physical minimum controllable opening, the final control command is forcibly set to zero.

10. A quantitative fuel injection circuit control system based on reinforcement learning, applied to the quantitative fuel injection circuit control method based on reinforcement learning as described in any one of claims 1-9, characterized in that, include: The sensing and preprocessing module is configured to acquire raw signals from sensors and perform filtering processing, outputting system status observations including differential pressure, flow rate, and temperature. An active rheological identification module is configured to generate a micro-perturbation signal and calculate the rheological hysteresis factor based on the frequency response characteristics of the micro-perturbation signal to the downstream pressure. The reinforcement learning decision module is configured to receive the flow velocity adaptive state vector after sampling at equal volume intervals, and use the policy network to output the residual correction amount for the physical reference opening command. The spatiotemporal alignment data management module is configured to align the timeline of historical actions with the current reward based on dynamic transmission delay, construct causal alignment tuples and store them in the experience replay pool; The execution control module is configured to calculate the physical reference opening command, combine the residual correction amount with the micro-disturbance signal to synthesize the final control command, and perform water hammer suppression and endpoint braking operations to drive the electro-hydraulic proportional control valve.

Images