Multi-environmental factor coordinated regulation system for mariculture based on deep reinforcement learning

Abstract

The application discloses a seawater culture multi-environment factor coordinated control system based on deep reinforcement learning, and relates to the technical field of intelligent culture control, and comprises the following steps: acquiring dissolved oxygen, dissolved carbon dioxide, pH, total ammonia nitrogen, temperature, salinity and equipment running state before and after feeding events, determining free ammonia concentration and identifying free ammonia lag peak amplification window; using a deep reinforcement learning control strategy to generate target running intensity of open aeration equipment, low decarburization oxygen supply equipment and ammonia nitrogen treatment circulating equipment, and preferentially using low decarburization oxygen supply and ammonia nitrogen treatment adjustment in the amplification window; separating lag ammonia nitrogen exposure and decarburization amplification exposure after the amplification window ends, distributing the decarburization amplification exposure to corresponding open aeration actions, correcting joint action rewards and updating the control strategy. The application can take into account oxygen supply safety and free ammonia risk, and avoid reverse optimization caused by immediate dissolved oxygen benefits.

Description

Technical Field

[0001] This invention relates to the field of intelligent aquaculture control technology, and in particular to a multi-environmental factor collaborative regulation system for marine aquaculture based on deep reinforcement learning. Background Technology

[0002] In marine aquaculture, environmental factors such as dissolved oxygen, dissolved carbon dioxide, pH, total ammonia nitrogen, temperature, and salinity directly affect the feeding, growth, and survival of farmed organisms. To maintain suitable water conditions, open aeration systems, pure oxygen supply systems, biological filtration systems, and circulating water pumps are typically installed, and the operating intensity of each system is adjusted based on water quality monitoring results. Current marine aquaculture environmental control methods often employ threshold control for single environmental factors, such as increasing aeration intensity when dissolved oxygen is below a set value or increasing water circulation when total ammonia nitrogen is above a set value. This type of control fails to reflect the differences in the timing, direction, and duration of changes in various environmental factors, and it is also difficult to achieve synergistic control based on the continuous impact of multiple systems on the aquatic environment. While some aquaculture systems incorporate deep reinforcement learning to generate control actions based on environmental conditions and equipment operating results, the rewards are usually determined by immediate improvements in dissolved oxygen, water quality compliance, or equipment energy consumption after the control action is executed. This makes it difficult to address environmental risks that gradually emerge after multiple control cycles following feeding.

[0003] In marine aquaculture waters after feeding, open aeration can rapidly increase dissolved oxygen and remove some dissolved carbon dioxide through gas-liquid exchange, thus raising the pH of the water. The total ammonia nitrogen load formed by the metabolism of cultured organisms and the decomposition of uneaten feed is usually delayed. The proportion of free ammonia in total ammonia nitrogen is affected by pH, temperature, and salinity. When the delayed rise in total ammonia nitrogen overlaps with the higher pH state formed by open aeration carbon removal, it may cause the free ammonia concentration to rise rapidly, while dissolved oxygen is still at a high level. Existing control systems tend to give positive evaluations to open aeration actions based on the immediate improvement in dissolved oxygen, but cannot trace the additional risk of free ammonia formed later back to the corresponding preceding aeration actions. This leads to deep reinforcement learning control strategies continuously increasing the intensity of open aeration, resulting in reverse optimization where dissolved oxygen index improves but free ammonia risk increases simultaneously. It is difficult to coordinate the control of oxygen supply safety, carbon dioxide removal impact, and delayed ammonia nitrogen load. Summary of the Invention

[0004] The purpose of this invention is to address the problems in existing technologies, such as the difficulty in identifying the additional risk of free ammonia caused by the combined effects of open aeration decarbonization and delayed total ammonia nitrogen rise, and the susceptibility of deep reinforcement learning to adverse effects of immediate dissolved oxygen gains, leading to reverse optimization of multiple environmental factors. Therefore, this invention proposes a multi-environmental factor synergistic regulation system for marine aquaculture based on deep reinforcement learning.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution:

[0006] A multi-environmental factor synergistic regulation system for marine aquaculture based on deep reinforcement learning includes:

[0007] The synchronization unit acquires water environment data and the operating status of open aeration equipment, low decarbonization oxygen supply equipment and ammonia nitrogen treatment circulation equipment to form an environmental equipment synchronization sequence.

[0008] The identification unit determines the state of the free ammonia late peak magnification window based on the synchronization sequence of the environmental equipment.

[0009] The control unit generates a target operating intensity based on the synchronization sequence of the environmental equipment and the state of the free ammonia late peak magnification window using a deep reinforcement learning control strategy.

[0010] The collaborative control unit, based on the target operating intensity within the free ammonia late peak magnification window, increases the operating intensity of the low decarbonization oxygen supply equipment and adjusts the ammonia nitrogen treatment circulation equipment, and appends the controlled data to the environmental equipment synchronization sequence;

[0011] After the free ammonia late peak amplification window ends, the risk attribution unit determines the late ammonia nitrogen exposure and decarbonization amplification exposure by the additional environmental equipment synchronization sequence, and allocates the decarbonization amplification exposure to the open aeration action.

[0012] The strategy iteration unit adjusts the reward for the open aeration action based on the allocation result, updates the control strategy, and uses it for the next feeding event.

[0013] Preferably, the synchronization unit forms the environmental device synchronization sequence, including:

[0014] Dissolved oxygen, dissolved carbon dioxide, pH, total ammonia nitrogen, temperature, and salinity of the seawater aquaculture body are obtained as the water body environmental data;

[0015] The operating intensity of the open aeration device, the low decarbonization oxygen supply device, and the ammonia nitrogen treatment circulation device, as well as the moment when the operating intensity changes, are obtained as the operating state.

[0016] The last complete sampling time before the start of a feeding cycle is determined as the baseline time, and baseline dissolved oxygen, baseline dissolved carbon dioxide, baseline pH, baseline total ammonia nitrogen, baseline temperature, and baseline salinity are determined.

[0017] The water environment data and operational status after the start of feeding are correlated over time according to a unified control cycle.

[0018] The unified control cycle that simultaneously measures dissolved oxygen, dissolved carbon dioxide, pH, total ammonia nitrogen, temperature, and salinity is defined as the effective control time, and the water environment data and operating status corresponding to the effective control time are written into the synchronization sequence of the environmental equipment.

[0019] Preferably, the identification unit determines the free ammonia concentration and the state of the free ammonia late peak magnification window, including:

[0020] The reference free ammonia ratio is determined based on the reference pH, the reference temperature, and the reference salinity, according to the ammonia dissociation equilibrium relationship in seawater.

[0021] Based on the pH, temperature, and salinity at each effective control moment, the current free ammonia ratio is determined according to the aforementioned seawater ammonia dissociation equilibrium relationship, and the current free ammonia concentration is determined based on the current total ammonia nitrogen concentration and the aforementioned current free ammonia ratio;

[0022] The effective control time that simultaneously satisfies the following conditions after the start of feeding—current dissolved oxygen higher than the reference dissolved oxygen, current dissolved carbon dioxide lower than the reference dissolved carbon dioxide, current pH higher than the reference pH, current total ammonia nitrogen concentration higher than the reference total ammonia nitrogen, current free ammonia ratio higher than the reference free ammonia ratio, and current free ammonia concentration higher than the previous effective control time—is determined as the start time of the free ammonia late peak amplification window.

[0023] Starting from the aforementioned start time, the effective control time at which any of the following occurs for the first time is defined as the end time of the free ammonia late peak amplification window: the current free ammonia concentration is not higher than the free ammonia concentration at the previous effective control time, the current total ammonia nitrogen concentration is not higher than the total ammonia nitrogen concentration at the previous effective control time, the current dissolved carbon dioxide concentration is not lower than the reference dissolved carbon dioxide concentration, the current pH concentration is not higher than the reference pH concentration, the current dissolved oxygen concentration is lower than the reference dissolved oxygen concentration, or the start time of the next feeding event.

[0024] When the start time exists, the state of the free ammonia late peak magnification window is determined to have been formed, and the environmental equipment synchronization sequence from the start time to the end time is extracted;

[0025] When the aforementioned start time does not exist, the state of the free ammonia late peak magnification window is determined to be not formed.

[0026] Preferably, the risk attribution unit determines the delayed ammonia nitrogen exposure and the decarbonization amplification exposure, including:

[0027] When the free ammonia late peak magnification window is formed, the increase in the current total ammonia nitrogen concentration relative to the baseline total ammonia nitrogen is converted according to the baseline free ammonia ratio to obtain the late ammonia nitrogen load component.

[0028] The increase in the current free ammonia ratio relative to the baseline free ammonia ratio is converted according to the current total ammonia nitrogen concentration to obtain the decarbonization amplification component;

[0029] The delayed ammonia nitrogen load component within the free ammonia delayed peak magnification window is accumulated according to the duration between adjacent effective control times to obtain the delayed ammonia nitrogen exposure amount;

[0030] The decarbonization amplification component within the free ammonia late peak amplification window is accumulated according to the duration between adjacent effective control moments to obtain the decarbonization amplification exposure amount;

[0031] When the free ammonia late peak amplification window is not formed, the late ammonia nitrogen exposure and the decarbonization amplification exposure are determined to be zero.

[0032] Preferably, the risk attribution unit allocates the decarbonization amplification exposure to the open aeration action, including:

[0033] The time from the moment when the operating intensity of the open aeration equipment changes to the moment when the operating intensity changes again or the end of the free ammonia late peak magnification window is defined as the effective period of one open aeration action.

[0034] Based on the dissolved carbon dioxide concentrations corresponding to the start and end effective control times of the aforementioned action period, determine the net decrease in dissolved carbon dioxide corresponding to the open aeration action.

[0035] An open aeration action with a net decrease in dissolved carbon dioxide greater than zero is defined as an effective open aeration action.

[0036] The decarbonization amplification exposure is allocated to each effective open aeration action according to the proportion of the net decrease in dissolved carbon dioxide corresponding to each effective open aeration action to the sum of the net decrease in dissolved carbon dioxide corresponding to all effective open aeration actions.

[0037] When the sum of the net decrease in dissolved carbon dioxide corresponding to all open aeration actions is zero, the carbon amplification exposure obtained from each open aeration action is determined to be zero.

[0038] The environmental equipment synchronization sequence, the delayed ammonia nitrogen exposure, the decarbonization amplification exposure, and the decarbonization amplification exposure obtained from the allocation of each open aeration action are combined to form a delayed risk attribution sample.

[0039] Preferably, the strategy iteration unit modifies the reward for the open-ended aeration action, including:

[0040] The current dissolved oxygen, current dissolved carbon dioxide, current pH, current total ammonia nitrogen concentration, current free ammonia concentration, number of control cycles elapsed since feeding, free ammonia late peak amplification window status, current cumulative late ammonia nitrogen exposure, current cumulative decarbonization amplification exposure, and current operating intensity of the three types of equipment are determined as status data.

[0041] The target operating intensity of the open aeration equipment, the low decarbonization oxygen supply equipment, and the ammonia nitrogen treatment circulation equipment is determined as the control action;

[0042] After the open aeration action increases dissolved oxygen, a temporary dissolved oxygen gain is generated based on the amount of dissolved oxygen increase.

[0043] After the free ammonia late peak amplification window ends, the decarbonization amplification exposure amount assigned to the open aeration action is read from the late risk attribution sample;

[0044] The average decarbonization amplification concentration is determined based on the decarbonization amplification exposure amount allocated to the open aeration action and the duration of the free ammonia late peak amplification window;

[0045] The temporary dissolved oxygen gain is dimensionless according to the dissolved oxygen safety lower limit determined based on the aquaculture species, and the average decarbonization amplification concentration is dimensionless according to the free ammonia safety boundary determined based on the aquaculture species.

[0046] The final reward for the open aeration action is obtained by subtracting the dimensionless average decarbonization amplification concentration from the dimensionless temporary dissolved oxygen gain.

[0047] A reward-corrected training sample is formed based on the state data, the control action, the final reward, and the delayed ammonia nitrogen exposure.

[0048] Preferably, the control unit and the collaborative control unit perform collaborative control, including:

[0049] The latest effective state in the synchronization sequence of the environmental equipment is input into the deep reinforcement learning control strategy to generate the target operating intensity of the open aeration equipment, the low decarbonization oxygen supply equipment and the ammonia nitrogen treatment cycle equipment;

[0050] When the free ammonia late peak magnification window is not formed, control the three types of equipment according to the target operating intensity.

[0051] When the free ammonia late peak magnification window is formed, determine the total oxygen supply capacity required to maintain the dissolved oxygen safety lower limit;

[0052] When it is necessary to increase the total oxygen supply capacity, priority should be given to increasing the operating intensity of the aforementioned low-decarbonization oxygen supply equipment;

[0053] When the low-carbon oxygen supply equipment reaches its rated maximum operating intensity, and the current dissolved oxygen is still lower than the dissolved oxygen safety lower limit, the operating intensity of the open aeration equipment is increased.

[0054] When the current free ammonia concentration is higher than the free ammonia concentration at the previous effective control time, the operating intensity of the ammonia nitrogen treatment circulation equipment is increased;

[0055] The adjusted operating intensity of the three types of equipment is implemented, water environment data and operating status are reacquired, and the reacquired data is appended to the synchronization sequence of the environmental equipment.

[0056] Preferably, the policy iteration unit updates the deep reinforcement learning control policy, including:

[0057] Based on the water environment data added to the synchronization sequence of the environmental equipment, the current free ammonia ratio and the current free ammonia concentration are re-determined, and it is determined whether the free ammonia late peak amplification window has ended;

[0058] While the free ammonia late peak amplification window has not yet ended, the late ammonia nitrogen exposure and the decarbonization amplification exposure are updated, and the control unit and the collaborative control unit continue to perform collaborative control.

[0059] When the free ammonia late peak magnification window ends, the final late ammonia nitrogen exposure and the final decarbonization magnification exposure corresponding to each open aeration action are determined. The reward-corrected training sample is formed based on the final decarbonization magnification exposure and is used as the complete training sample.

[0060] When the free ammonia late peak amplification window is not formed and the next feeding event begins, a complete training sample is formed based on the late ammonia nitrogen exposure amount and decarbonization amplification exposure amount, which are determined to be zero.

[0061] The deep reinforcement learning control policy is updated using the complete training samples, and the updated deep reinforcement learning control policy is used in the next feeding event.

[0062] Preferably, the open aeration device supplies oxygen to the aquaculture water and removes dissolved carbon dioxide through contact between the aquaculture water and air; the low decarbonization oxygen supply device is an oxygen supply device that uses pressurized pure oxygen dissolution or an oxygen supply device that uses membrane oxygen transfer; the ammonia nitrogen treatment and circulation device transports the aquaculture water to the biological filtration unit or the ammonia nitrogen adsorption unit, and transports the treated aquaculture water back to the seawater aquaculture area.

[0063] Compared with the prior art, the beneficial effects of the present invention are:

[0064] 1. This invention simultaneously acquires dissolved oxygen, dissolved carbon dioxide, pH, total ammonia nitrogen, temperature, salinity, and equipment operating status before and after feeding events. Based on the ammonia dissociation relationship in seawater, it determines the free ammonia concentration and identifies a magnified window of the late peak of free ammonia formed by open aeration removing carbon dioxide, increased pH, increased delayed total ammonia nitrogen, and increased free ammonia concentration. This avoids relying solely on the results of single tests for dissolved oxygen or total ammonia nitrogen for control. By prioritizing the use of low-carbon dioxide removal oxygen supply equipment to supplement oxygen within the magnified window and adjusting the ammonia nitrogen treatment circulation equipment according to changes in free ammonia concentration, it can maintain a safe level of dissolved oxygen in the aquaculture water while reducing excessive carbon dioxide removal and pH increases caused by continued open aeration, thus reducing the risk of delayed ammonia nitrogen load converting into free ammonia.

[0065] 2. This invention separates the increase in free ammonia concentration into a delayed ammonia nitrogen load component and a decarbonization amplification component. Based on the net decrease in dissolved carbon dioxide after equipment response correction, the decarbonization amplification exposure is allocated to the corresponding open aeration action. The resulting delayed decarbonization risk is then backfilled into the reward for the three types of equipment joint actions in the corresponding control cycle. The deep reinforcement learning control strategy can not only evaluate the immediate dissolved oxygen improvement generated by the control action, but also evaluate the subsequent additional risk of free ammonia caused by the preceding open aeration action. This avoids continuously increasing the open aeration intensity due to the immediate positive benefits masking the delayed environmental risks, and enhances the synergistic control capability and adaptability of the control strategy of open aeration equipment, low decarbonization oxygen supply equipment, and ammonia nitrogen treatment cycle equipment. Attached Figure Description

[0066] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0067] Figure 1 This is a functional block diagram of a multi-environmental factor synergistic regulation system for marine aquaculture based on deep reinforcement learning, provided in an embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Equivalent substitutions made by those skilled in the art to the device structure, data acquisition method, and computing execution carrier without departing from the technical concept of the present invention should all fall within the protection scope of the present invention.

[0069] Example 1

[0070] This embodiment provides a multi-environmental factor collaborative regulation system for marine aquaculture based on deep reinforcement learning. The system is applied to a marine aquaculture area equipped with open aeration equipment, low decarbonization oxygen supply equipment, and ammonia nitrogen treatment and circulation equipment.

[0071] The marine aquaculture area is a marine circulating aquaculture pond, a marine pond, or a marine aquaculture unit with independent water quality control conditions; this embodiment uses a marine circulating aquaculture pond as an example for illustration.

[0072] The open aeration device introduces oxygen into the aquaculture water by exposing it to air, and removes some dissolved carbon dioxide during the gas-liquid exchange process. In this embodiment, the open aeration device is a microporous aeration device. In other embodiments, the open aeration device may also be a waterwheel aeration device or a jet aeration device.

[0073] The low-carbon-decarbonation oxygen supply equipment refers to an oxygen supply equipment that removes less carbon dioxide than an open aeration equipment while providing the same actual oxygen transfer to the aquaculture water. In this embodiment, the low-carbon-decarbonation oxygen supply equipment is a pure oxygen cone oxygen supply equipment. In other embodiments, the low-carbon-decarbonation oxygen supply equipment may also be a membrane oxygen transfer equipment or a closed pressurized pure oxygen dissolution equipment.

[0074] The ammonia nitrogen treatment circulation equipment includes a circulating water pump and an ammonia nitrogen treatment unit connected to the circulating water pump. In this embodiment, the ammonia nitrogen treatment unit adopts a biological filtration unit with nitrification function. The circulating water pump transports the aquaculture water in the seawater aquaculture pond to the biological filtration unit and transports the aquaculture water treated by the biological filtration unit back to the seawater aquaculture pond. In other embodiments, the ammonia nitrogen treatment unit may also adopt an ammonia nitrogen adsorption unit.

[0075] like Figure 1 As shown, the system in this embodiment includes a synchronization unit, an identification unit, a control unit, a collaborative control unit, a risk attribution unit, and a strategy iteration unit; each unit is implemented by an industrial computer, an edge controller, or a programmable controller in conjunction with a data storage device.

[0076] The industrial computer is connected to the dissolved oxygen detection device, dissolved carbon dioxide detection device, pH detection device, total ammonia nitrogen detection device, temperature detection device and salinity detection device, respectively, and is also connected to the controllers of the open aeration equipment, low decarbonization oxygen supply equipment and ammonia nitrogen treatment circulation equipment, respectively.

[0077] Among them, the dissolved oxygen detection device outputs the dissolved oxygen concentration of the aquaculture water; the dissolved carbon dioxide detection device outputs the dissolved carbon dioxide concentration of the aquaculture water; the pH detection device outputs the pH of the aquaculture water; the total ammonia nitrogen detection device outputs the total ammonia nitrogen concentration of the aquaculture water; the temperature detection device outputs the temperature of the aquaculture water; and the salinity detection device outputs the salinity of the aquaculture water.

[0078] In this embodiment, the dissolved carbon dioxide concentration is obtained directly through a dissolved carbon dioxide detection device, without using the carbonic acid balance back-calculation method which lacks alkalinity data.

[0079] Before the system is put into seawater aquaculture control, the open aeration equipment, low decarbonization oxygen supply equipment and ammonia nitrogen treatment circulation equipment are independently calibrated.

[0080] The equipment calibration was carried out in a calibration water body without any cultured organisms. The calibration water body used seawater with the same salinity as the target marine aquaculture area, and the temperature of the calibration water body was adjusted to the normal operating temperature range of the target marine aquaculture area.

[0081] The effective volume of the calibrated water body is denoted as... The unit is cubic meters or liters, and the unit should be kept consistent in the same calculation process;

[0082] Multiple calibration operating intensities, ranging from low to high, were set for open aeration equipment and low-decarbonization oxygen supply equipment, respectively; while one oxygen supply equipment was being calibrated, other oxygen supply equipment and ammonia nitrogen treatment circulation equipment remained in a stopped state.

[0083] For the Oxygen supply equipment at calibrated operating intensity In the next calibration process, record the dissolved oxygen concentration at the start of the calibration, the dissolved oxygen concentration at the end of the calibration, the dissolved carbon dioxide concentration at the start of the calibration, the dissolved carbon dioxide concentration at the end of the calibration, and the calibration duration;

[0084] No. Oxygen supply equipment at calibrated operating intensity The actual oxygen transfer rate is determined by the following formula:

[0085]

[0086] In the formula, Indicates the first Oxygen supply equipment at calibrated operating intensity The actual oxygen transfer rate under these conditions; This indicates the dissolved oxygen concentration at the start of the calibration. This indicates the dissolved oxygen concentration at the end of the calibration. Indicates the calibration duration of the corresponding calibration process;

[0087] No. Oxygen supply equipment at calibrated operating intensity The carbon dioxide removal rate is determined by the following formula:

[0088]

[0089] In the formula, Indicates the first Oxygen supply equipment at calibrated operating intensity The carbon dioxide removal rate under these conditions; This indicates the dissolved carbon dioxide concentration at the start of the calibration. This indicates the concentration of dissolved carbon dioxide at the end of the calibration.

[0090] Based on the calibrated operating intensities and their corresponding actual oxygen transfer rates, oxygen supply capacity calibration tables for open aeration equipment and low decarbonization oxygen supply equipment are established respectively.

[0091] Based on the calibrated operating intensities and their corresponding carbon dioxide removal rates, carbon dioxide removal capacity calibration tables were established for open aeration equipment and low carbon dioxide removal oxygen supply equipment, respectively.

[0092] For equipment operating intensities located between two adjacent calibrated operating intensities, the corresponding actual oxygen transfer rate and carbon dioxide removal rate are determined by linear interpolation.

[0093] Under the same actual oxygen transfer rate, when the carbon dioxide removal rate corresponding to an oxygen supply device is less than the carbon dioxide removal rate corresponding to an open aeration device, the oxygen supply device is identified as the low carbon dioxide removal oxygen supply device described in this embodiment.

[0094] When calibrating the ammonia nitrogen treatment circulation equipment independently, set up multiple circulating water pumps to calibrate the operating intensity, record the corresponding circulating water flow rate, and establish a circulation capacity calibration table between the operating intensity and circulating water flow rate of the ammonia nitrogen treatment circulation equipment.

[0095] When calibrating the ammonia nitrogen treatment circulation equipment, the changes in dissolved oxygen concentration and dissolved carbon dioxide concentration under each calibrated operating intensity were also recorded to form a dissolved oxygen response calibration table and a dissolved carbon dioxide response calibration table for the ammonia nitrogen treatment circulation equipment, so as to subtract the water quality response caused by the ammonia nitrogen treatment circulation equipment itself from the measured water quality changes.

[0096] The oxygen supply capacity calibration table, carbon dioxide removal capacity calibration table, circulation capacity calibration table, dissolved oxygen response calibration table, and dissolved carbon dioxide response calibration table are all stored in an industrial computer.

[0097] This embodiment is implemented according to the following steps;

[0098] Step S1: Generate the environmental device synchronization sequence corresponding to the feeding event;

[0099] Specifically, the synchronization unit acquires water environment data of the seawater aquaculture pond and obtains the operating status of open aeration equipment, low decarbonization oxygen supply equipment and ammonia nitrogen treatment circulation equipment.

[0100] The water environment data includes dissolved oxygen concentration, dissolved carbon dioxide concentration, pH, total ammonia nitrogen concentration, temperature, and salinity.

[0101] The equipment operating status includes the operating intensity of open aeration equipment, the operating intensity of low decarbonization oxygen supply equipment, the operating intensity of ammonia nitrogen treatment circulation equipment, and the times when the operating intensity of each piece of equipment changes.

[0102] The operating intensity of open aeration equipment is expressed by the fan operating frequency, aeration gas flow rate, or a ratio relative to the rated maximum operating intensity; the operating intensity of low decarbonization oxygen supply equipment is expressed by the pure oxygen flow rate, pure oxygen cone circulation flow rate, or a ratio relative to the rated maximum operating intensity; the operating intensity of ammonia nitrogen treatment circulation equipment is expressed by the circulating water pump operating frequency, circulating water flow rate, or a ratio relative to the rated maximum operating intensity.

[0103] The same operating intensity is represented using the same method for all feeding events;

[0104] Before each feeding begins, the synchronization unit reads the last data sampling time that simultaneously contains dissolved oxygen concentration, dissolved carbon dioxide concentration, pH, total ammonia nitrogen concentration, temperature, and salinity, and determines this data sampling time as the reference time for the current feeding event;

[0105] The dissolved oxygen concentration at the reference time was determined as the reference dissolved oxygen. The dissolved carbon dioxide concentration at the reference time is determined as the reference dissolved carbon dioxide concentration. The pH value corresponding to the reference time is determined as the reference pH value. The total ammonia nitrogen concentration at the reference time is determined as the reference total ammonia nitrogen. The temperature corresponding to the reference time is determined as the reference temperature. The salinity at the reference time is then determined as the reference salinity. .

[0106] After feeding begins, the synchronization unit operates according to a unified control cycle. Read water environment data and equipment operating status;

[0107] Unified control cycle The sampling period is determined based on the longest sampling period among the six types of water body environmental data, or based on the period during which the total ammonia nitrogen detection device can stably output valid detection results; a unified control period is adopted. Once determined, it remains unchanged within the same feeding event;

[0108] When the sampling period of some detection devices is shorter than the unified control period When the sampling period of the detection device is equal to the unified control period, the nearest valid detection value to the end of the control period is read. At that time, the detection value corresponding to the control cycle is read directly.

[0109] The synchronization unit maps the sampling time of each water body environmental data to the corresponding unified control cycle, and maps the operating intensity of the three types of equipment and the time when the operating intensity changes to the corresponding unified control cycle.

[0110] The effective control time is defined as a unified control period that simultaneously includes dissolved oxygen concentration, dissolved carbon dioxide concentration, pH, total ammonia nitrogen concentration, temperature, and salinity.

[0111] When any of the above-mentioned water environment data is missing in a unified control period, the unified control period is marked as an invalid control time; invalid control times are not used for free ammonia ratio calculation, free ammonia concentration calculation, free ammonia late peak magnification window identification, and exposure integration;

[0112] The effective resolution of each testing device is determined according to the corresponding metrological calibration certificate or equipment technical documents;

[0113] The effective resolution of the dissolved oxygen detection device is denoted as . The effective resolution of the dissolved carbon dioxide detection device is denoted as . The effective resolution of the pH detection device is denoted as . The effective resolution of the total ammonia nitrogen detection device is denoted as . The effective resolution of the temperature detection device is denoted as The effective resolution of the salinity detection device is denoted as... ;

[0114] When comparing the magnitudes of two detected values, a valid high-low relationship is determined only if the absolute value of the difference between the two detected values ​​is not less than the effective resolution of the corresponding detection device; if the absolute value of the difference between the two detected values ​​is less than the effective resolution of the corresponding detection device, it is determined that the two detected values ​​have not changed effectively under the current detection accuracy.

[0115] When there is an invalid control moment between two adjacent valid control moments, the invalid control moment is not used as the exposure integration node, but the exposure integration uses the actual time interval between the two adjacent valid control moments.

[0116] The synchronization unit writes the dissolved oxygen concentration, dissolved carbon dioxide concentration, pH, total ammonia nitrogen concentration, temperature, salinity, operating intensity of the three types of equipment, and the number of control cycles elapsed after feeding into the data sequence corresponding to the current feeding event in chronological order, forming an environmental equipment synchronization sequence.

[0117] Step S2: Determine the free ammonia concentration and identify the magnification window for the late peak of free ammonia;

[0118] The identification unit determines the proportion of free ammonia based on the pH, temperature, and salinity at each effective control moment, according to the ammonia dissociation equilibrium relationship in seawater;

[0119] Total ammonia nitrogen includes non-ionic ammonia and ionic ammonium; the free ammonia ratio refers to the proportion of total ammonia nitrogen present in the form of non-ionic ammonia.

[0120] For the An effective control time is first determined by salinity. Determine the ionic strength of seawater :

[0121]

[0122] In the formula, Indicates the first The salinity at each effective control moment is expressed in grams per kilogram or parts per thousand, and the same numerical caliber is maintained throughout all calculations.

[0123] Based on seawater ionic strength Determine the ammonia dissociation parameters at 25 degrees Celsius. :

[0124]

[0125] The first Temperature in Celsius at each effective control moment Convert to absolute temperature :

[0126]

[0127] In the formula, The unit is Celsius. The unit is Kelvin;

[0128] No. Free ammonia ratio at each effective control time point Determined according to the following formula:

[0129]

[0130] In the formula, Indicates the first The pH level at an effective control point;

[0131] Baseline salinity Reference temperature and reference pH Substituting the same calculation relationships, the baseline free ammonia ratio is obtained. ;

[0132] No. Free ammonia concentration at each effective control time point Determined according to the following formula:

[0133]

[0134] In the formula, Indicates the first Total ammonia nitrogen concentration at each effective control time; Indicates the first The free ammonia concentration at each effective control time; both total ammonia nitrogen concentration and free ammonia concentration are measured as nitrogen mass concentration;

[0135] When the temperature or salinity exceeds the effective range of the corresponding detection device, the corresponding unified control cycle will be marked as an invalid control moment, and the free ammonia ratio calculation and free ammonia late peak magnification window judgment will not be executed in that control cycle.

[0136] The identification unit initializes the state of the free ammonia late peak magnification window at the beginning of a feeding event to "not formed";

[0137] Data is read sequentially from each effective control moment after feeding begins. When an effective control moment simultaneously meets the following conditions, that effective control moment is determined as the start time of the free ammonia late peak amplification window, and the state of the free ammonia late peak amplification window is updated from not formed to formed:

[0138] Current dissolved oxygen concentration and baseline dissolved oxygen The difference is not less than ;

[0139] Standard dissolved carbon dioxide The difference from the current dissolved carbon dioxide concentration is not less than ;

[0140] Current pH and reference pH The difference is not less than ;

[0141] Current total ammonia nitrogen concentration and baseline total ammonia nitrogen The difference is not less than ;

[0142] The current free ammonia ratio is higher than the baseline free ammonia ratio. ;

[0143] The current free ammonia concentration has increased effectively compared to the previous effective control time.

[0144] The effective resolution of the free ammonia concentration is determined by error propagation based on the effective resolutions of the total ammonia nitrogen detection device, pH detection device, temperature detection device, and salinity detection device.

[0145] Specifically, in the first At each effective control point, the total ammonia nitrogen concentration, pH, temperature, and salinity are taken as boundary values ​​within their effective resolution ranges. The maximum and minimum possible free ammonia concentrations are calculated according to the aforementioned free ammonia concentration calculation relationship. The difference between these two values ​​is determined as the [missing value]. Effective resolution of free ammonia concentration at each effective control time. ;

[0146] When the difference between the current free ammonia concentration and the free ammonia concentration at the previous effective control time is not less than At that time, it is determined that the current free ammonia concentration has effectively increased;

[0147] After the free ammonia late peak magnification window status is updated to "formed", the identification unit determines whether the free ammonia late peak magnification window has ended at each subsequent effective control moment;

[0148] Starting from the beginning of the free ammonia late peak amplification window, the effective control time will be determined as the end of the free ammonia late peak amplification window when any of the following conditions first occur:

[0149] The current free ammonia concentration has not increased effectively compared to the previous effective control time.

[0150] The current total ammonia nitrogen concentration has not increased effectively compared to the previous effective control time.

[0151] Current dissolved carbon dioxide concentration compared to baseline dissolved carbon dioxide The difference is less than Or the current dissolved carbon dioxide concentration is higher than the baseline dissolved carbon dioxide concentration. ;

[0152] Current pH and reference pH The difference is less than Or the current pH level is lower than the reference pH level. ;

[0153] Standard dissolved oxygen The difference from the current dissolved oxygen concentration is not less than .

[0154] Once the end time of the free ammonia slow peak magnification window is determined, the status of the free ammonia slow peak magnification window is updated from "formed" to "ended".

[0155] The continuous and effective control period between the start and end of the free ammonia late peak amplification window is used for subsequent calculations of late ammonia nitrogen exposure and decarbonization amplification exposure.

[0156] When there is no effective control moment in a feeding event that simultaneously satisfies all the initial conditions, the free ammonia late peak magnification window remains in the state of not being formed, and the data recording of the current feeding event ends when the next feeding event begins.

[0157] Step S3: Utilize the current deep reinforcement learning control strategy for online collaborative control;

[0158] Specifically, the control unit reads the latest valid state from the synchronization sequence of the environmental devices and inputs the latest valid state into the current deep reinforcement learning control strategy;

[0159] The current deep reinforcement learning control strategy is the control strategy that has been updated before the start of the current feeding event. The delayed risk attribution result formed in this feeding event is not used to reverse control the same free ammonia delayed peak amplification window that has ended, but is used to update the deep reinforcement learning control strategy used in the next feeding event.

[0160] Before the system is put into operation for the first time, an initial training sample set is formed using historical marine aquaculture operation data;

[0161] Historical marine aquaculture operation data should include at least the feeding start time, dissolved oxygen concentration, dissolved carbon dioxide concentration, pH, total ammonia nitrogen concentration, temperature, salinity, operating intensity of open aeration equipment, operating intensity of low decarbonization oxygen supply equipment, and operating intensity of ammonia nitrogen treatment and circulation equipment.

[0162] The historical marine aquaculture operation data are synchronized according to the unified control cycle described in step S1, and state transition samples are formed according to two adjacent effective control times.

[0163] Each historical state transition sample includes the state vector before the action is executed, the joint action vector of the three types of devices, the state vector after the action is executed, and the feeding event termination flag;

[0164] In step S2, identify the late peak amplification window of free ammonia in each historical feeding event. In step S4, determine the late ammonia nitrogen exposure and decarbonization amplification exposure. In step S5, assign the final joint reward to each historical state transition sample according to the joint reward formation method.

[0165] When the historical feeding event does not form a free ammonia late peak amplification window, the corresponding decarbonization amplification exposure is determined to be zero, but a joint reward is still formed based on the dissolved oxygen concentration and free ammonia concentration after the action is executed;

[0166] Historical state transition samples are written to the experience replay storage area, and offline training is performed.

[0167] This embodiment uses a dual-delay deep deterministic strategy gradient algorithm to establish a deep reinforcement learning control strategy to adapt to control scenarios where the operating intensity of the three types of equipment is a continuous quantity.

[0168] The dual-delay deep deterministic policy gradient algorithm includes one policy network, two value networks, one target policy network, and two target value networks;

[0169] The policy network takes a twelve-dimensional state vector as input and outputs a three-dimensional continuous action vector.

[0170] The policy network consists of an input layer, a first hidden layer, a second hidden layer, and an output layer; the first hidden layer contains 128 neurons, and the second hidden layer contains 64 neurons; both the first and second hidden layers use the linear rectified activation function; the output layer contains three neurons and uses the hyperbolic tangent activation function.

[0171] State vector Arranged in the following order:

[0172]

[0173] In the formula, Indicates the first Dissolved oxygen concentration at an effective control time; Indicates the first Dissolved carbon dioxide concentration at an effective control point; Indicates the first The pH level at an effective control point; Indicates the first Total ammonia nitrogen concentration at each effective control time; Indicates the first The concentration of free ammonia at an effective control point; Indicates the number of control cycles elapsed after feeding; This indicates the magnification window state of the late peak of free ammonia; This indicates the current cumulative delayed ammonia nitrogen exposure. This indicates the current cumulative decarbonization amplification exposure. This indicates the current operating intensity of the open-type aeration equipment; This indicates the current operating intensity of the low-carbonization oxygen supply equipment; This indicates the current operating intensity of the ammonia nitrogen treatment circulation equipment;

[0174] When the magnification window for the delayed peak of free ammonia has not yet formed, Take zero; when the free ammonia late peak magnification window has been formed, Take one; after the magnification window for the delayed peak of free ammonia is completed, Restored to zero;

[0175] Continuous state data is normalized according to the effective range of the corresponding detection device or the rated operating range of the equipment.

[0176] For continuous state data Its normalization result Determined according to the following formula:

[0177]

[0178] In the formula, and These represent the lower and upper limits of the effective range or operating range of the corresponding detection device, respectively; the normalization result is taken as zero when it is less than zero and as one when it is greater than one.

[0179] Current cumulative delayed ammonia nitrogen exposure and current cumulative decarbonization amplification exposure Normalize by dividing by the product of the free ammonia safety boundary and the duration since feeding; within the first unified control period after feeding, the duration since feeding is taken as the unified control period. ;

[0180] The three action components output by the strategy network are all located between negative one and positive one, and are respectively mapped to the target operating intensity of the open aeration equipment, the low decarbonization oxygen supply equipment, and the ammonia nitrogen treatment circulation equipment.

[0181] No. The target operating intensity of this type of equipment is determined according to the following formula:

[0182]

[0183] In the formula, The output of the policy network represents the first... Each action component; Indicates the first Target operating intensity for this type of equipment; Indicates the first The rated maximum operating intensity of this type of equipment;

[0184] The input to each value network is a fifteen-dimensional joint vector formed by concatenating a twelve-dimensional state vector and a three-dimensional action vector;

[0185] Each value network consists of a first hidden layer, a second hidden layer, and an output layer; the first hidden layer contains 128 neurons, the second hidden layer contains 64 neurons, and the output layer outputs a value estimate; the two value networks have the same structure but use independent network parameters.

[0186] The learning rate of the policy network is set to The learning rate of both value networks is set to Discount factor Set to 0.99, soft update coefficient Set it to 0.005, the training batch size to 128, and the experience replay capacity to 100,000 state transition samples.

[0187] After every two batches of value network training are completed, the policy network and each target network are updated once.

[0188] The target policy action is given Gaussian smoothed noise with a mean of zero and a standard deviation of 0.2, and the Gaussian smoothed noise is limited to between -0.5 and +0.5.

[0189] During the actual equipment control phase, no random exploration noise is added to the policy network output;

[0190] Before the system is put into operation for the first time, two hundred offline training batches are executed. When the number of state transition samples in the experience replay storage area is less than one hundred and twenty-eight, the network parameter update is temporarily suspended until the number of state transition samples in the experience replay storage area reaches the training batch size.

[0191] After completing offline training, the resulting policy network, two value networks, target policy network, and two target value networks are used as the initial deep reinforcement learning control strategy.

[0192] When the free ammonia late peak magnification window has not formed, the current cumulative late ammonia nitrogen exposure. and current cumulative decarbonization amplification exposure All are zero;

[0193] Once the start time of the free ammonia late peak amplification window is determined, the current cumulative late ammonia nitrogen exposure and the current cumulative decarbonization amplification exposure are initialized to zero.

[0194] For the At each effective control point, the increase in the current total ammonia nitrogen concentration relative to the baseline total ammonia nitrogen is converted according to the baseline free ammonia ratio to obtain the delayed ammonia nitrogen load component. :

[0195]

[0196] The increase in the current free ammonia ratio relative to the baseline free ammonia ratio is converted according to the current total ammonia nitrogen concentration to obtain the decarbonization amplification component. :

[0197]

[0198] For each new effective control point obtained, the current cumulative delayed ammonia nitrogen exposure is updated according to the following formula:

[0199]

[0200] Update the current cumulative decarbonization amplification exposure using the following formula:

[0201]

[0202] In the formula, Indicates the first There are one effective control time; ineffective control times are not integrated separately, but are not deducted from the actual time interval between two adjacent effective control times.

[0203] The current cumulative late-emission ammonia nitrogen exposure and the current cumulative decarbonization amplification exposure are only used to characterize the progress of the current free ammonia late peak amplification window and serve as status data for subsequent control cycles; the final open aeration action allocation will not be performed on the decarbonization amplification exposure before the end of the free ammonia late peak amplification window.

[0204] The control unit inputs the latest state vector into the strategy network to obtain the target operating intensity of the open aeration equipment, the low decarbonization oxygen supply equipment, and the ammonia nitrogen treatment circulation equipment;

[0205] When the free ammonia late peak magnification window has not been formed, the collaborative control unit controls the three types of equipment according to the target operating intensity output by the strategy network.

[0206] When the free ammonia late peak magnification window is formed, the collaborative control unit adjusts the target operating intensity output by the strategy network to make safety constraints.

[0207] Determine the safe lower limit of dissolved oxygen based on the aquaculture species and stage of aquaculture. ;

[0208] Based on the oxygen supply capacity calibration table and the current operating intensity of the three types of equipment, the actual oxygen transfer rate of the current open aeration equipment and low decarbonization oxygen supply equipment is determined, and the sum of the two is recorded as the current total oxygen transfer rate. ;

[0209] Based on the oxygen mass balance from the previous effective control time to the current effective control time, determine the estimated oxygen consumption rate of the current aquaculture water body. :

[0210]

[0211] Determine the total oxygen transfer rate required for the next unified control cycle. :

[0212]

[0213] When the current dissolved oxygen concentration is higher than the dissolved oxygen safety lower limit, the second term in the above formula is allowed to be negative, but the required total oxygen transfer rate shall not be less than zero;

[0214] According to the oxygen supply capacity calibration table of low decarbonization oxygen supply equipment, the operating intensity of low decarbonization oxygen supply equipment that can provide the required total oxygen transfer rate is prioritized.

[0215] When the required total oxygen transfer rate is not greater than the rated maximum oxygen transfer rate of the low decarbonization oxygen supply equipment, the additional oxygen supply demand shall be borne by the low decarbonization oxygen supply equipment, and the operating intensity of the open aeration equipment shall not be increased due to the additional oxygen supply demand.

[0216] When the required total oxygen transfer rate is greater than the rated maximum oxygen transfer rate of the low decarbonization oxygen supply equipment, the low decarbonization oxygen supply equipment is set to the rated maximum operating intensity. Based on the difference between the required total oxygen transfer rate and the rated maximum oxygen transfer rate of the low decarbonization oxygen supply equipment, and in conjunction with the oxygen supply capacity calibration table of the open aeration equipment, the supplementary operating intensity of the open aeration equipment is determined.

[0217] The collaborative control unit compares the current free ammonia concentration with the free ammonia concentration at the previous effective control time.

[0218] When the current free ammonia concentration increases effectively relative to the free ammonia concentration at the previous effective control time, increase the operating intensity of the ammonia nitrogen treatment circulation equipment to allow more aquaculture water to enter the biological filtration unit.

[0219] The operating intensity of the ammonia nitrogen treatment circulation equipment after the addition shall not exceed its rated maximum operating intensity;

[0220] When the current free ammonia concentration has not increased effectively, the operating intensity of the ammonia nitrogen treatment circulation equipment should not be increased based on the condition of increasing free ammonia concentration.

[0221] The collaborative control unit executes the adjusted operating intensity of the three types of equipment;

[0222] After the equipment has run a unified control cycle, the synchronization unit reacquires the water environment data and equipment operating status, and adds the reacquired data to the environmental equipment synchronization sequence according to the synchronization method in step S1.

[0223] The identification unit re-executes step S2 based on the supplemented data to update the free ammonia concentration and the free ammonia late peak magnification window status;

[0224] Continue with step S3 before the free ammonia late peak magnification window has ended;

[0225] When the free ammonia late peak magnification window ends, stop the priority adjustment of the oxygen supply path within the free ammonia late peak magnification window and proceed to step S4;

[0226] When the entire feeding event does not form a free ammonia late peak amplification window, the equipment control continues to be executed according to the current deep reinforcement learning control strategy until the next feeding event begins, and then the windowless reward processing process described in step S5 is entered.

[0227] Step S4: Separate the risk of free ammonia and allocate the decarbonization amplification exposure to open aeration actions;

[0228] Specifically, after the free ammonia late peak magnification window ends, the risk attribution unit reads the complete environmental equipment synchronization sequence from the start time to the end time;

[0229] No. The increase in free ammonia concentration at each effective control time relative to the baseline time satisfies:

[0230]

[0231] Among them, the delayed ammonia nitrogen loading component for:

[0232]

[0233] Decarburization amplification component for:

[0234]

[0235] The delayed ammonia nitrogen load component represents the increase in free ammonia concentration resulting from the increase in total ammonia nitrogen concentration, provided that the free ammonia ratio remains at the baseline free ammonia ratio.

[0236] The decarbonization amplification component represents the additional free ammonia concentration resulting from the increase in the current free ammonia ratio relative to the baseline free ammonia ratio under the current total ammonia nitrogen concentration conditions;

[0237] The delayed ammonia nitrogen load component is obtained by performing a trapezoidal integral on the actual time interval between adjacent effective control times. :

[0238]

[0239] The decarbonization amplification component was integraled using the same method to obtain the decarbonization amplification exposure. :

[0240]

[0241] In the formula, This indicates the effective control time number corresponding to the start time of the free ammonia late peak magnification window; This indicates the effective control time number corresponding to the end of the magnification window for the late peak of free ammonia.

[0242] The dimensions of delayed ammonia nitrogen exposure and decarbonization amplified exposure are both products of concentration and time;

[0243] The risk attribution unit will include delayed ammonia nitrogen exposure in the current feeding event and allocate decarbonization amplification exposure to open aeration actions that produce a corrected net decrease in dissolved carbon dioxide.

[0244] The open aeration action refers to a change in the operating intensity of the open aeration equipment and the duration of the change in operating intensity.

[0245] The first effective control time after the change of the operating intensity of the open aeration equipment is determined as the start effective control time of the open aeration action, and the last effective control time before the next change of the operating intensity of the open aeration equipment is determined as the end effective control time of the open aeration action.

[0246] When the next change in the operating intensity of the open aeration equipment is later than the end time of the free ammonia late peak magnification window, the end time of the free ammonia late peak magnification window will be determined as the effective control time for the end of the open aeration action.

[0247] When an open aeration action has fewer than two effective control moments within its effective period, the open aeration action should not be used to determine dissolved oxygen benefits or allocate decarbonization amplification exposure.

[0248] For the An open aeration action was used to determine the net decrease in dissolved carbon dioxide observed during its effective period. :

[0249]

[0250] In the formula, Indicates the first The dissolved carbon dioxide concentration at the effective control moment of the start of an open aeration action; Indicates the first The dissolved carbon dioxide concentration at the effective control moment after the end of an open aeration action.

[0251] Based on the carbon dioxide removal capacity calibration table of low-carbon oxygen supply equipment, and combined with the operating intensity and duration of the low-carbon oxygen supply equipment during the aforementioned operating period, the expected reduction in dissolved carbon dioxide corresponding to the low-carbon oxygen supply equipment is determined. ;

[0252] Based on the dissolved carbon dioxide response calibration table of the ammonia nitrogen treatment circulation equipment, and combined with the operating intensity and duration of the ammonia nitrogen treatment circulation equipment during the aforementioned period, the expected decrease in dissolved carbon dioxide corresponding to the ammonia nitrogen treatment circulation equipment is determined. ;

[0253] The first Prior to each open aeration operation, the most recent complete control cycle in which the operating intensity of all three types of equipment remained unchanged was defined as the background change control cycle, and the net decrease in dissolved carbon dioxide during this background change control cycle was defined as the background dissolved carbon dioxide decrease. ;

[0254] When the aforementioned background change control cycle is absent, the decrease in background dissolved carbon dioxide will be... The value is determined to be zero;

[0255] No. Corrected net decrease in dissolved carbon dioxide corresponding to each open aeration action Determined according to the following formula:

[0256]

[0257] when When it is greater than zero, the first One open aeration action was identified as an effective open aeration action;

[0258] when When the value is zero, no decarbonization amplification exposure is allocated to this open aeration action;

[0259] Using the same device response deduction method, determine the first The corrected increase in dissolved oxygen corresponding to each open aeration action;

[0260] First determine the first The observed increase in dissolved oxygen during the open aeration period is subtracted from the oxygen transfer contribution of the low-carbon oxygen supply equipment as determined by the oxygen supply capacity calibration table, the dissolved oxygen response of the ammonia nitrogen treatment cycle equipment as determined by the dissolved oxygen response calibration table, and the background dissolved oxygen change. The remaining non-negative value is then determined as the [number of values]. The corrected increase in dissolved oxygen corresponding to each open aeration action;

[0261] When at least one effective open aeration action exists, the first Decarbonization amplification exposure obtained from the allocation of effective open aeration actions Determined according to the following formula:

[0262]

[0263] In the formula, Indicates the number of effective open aeration actions;

[0264] When the sum of the net decrease in dissolved carbon dioxide corresponding to all open aeration actions is zero, proportional allocation is not performed, and the carbon amplification exposure obtained from each open aeration action is determined to be zero.

[0265] The risk attribution unit combines the environmental equipment synchronization sequence of the current feeding event, the delayed ammonia nitrogen exposure, the decarbonization amplification exposure, the duration of each effective open aeration action, and the decarbonization amplification exposure allocated to each effective open aeration action to form a delayed risk attribution sample.

[0266] Step S5: Generate reward-corrected training samples corresponding to the joint actions of the three types of devices;

[0267] The strategy iteration unit takes the operating intensity of the three types of equipment within each unified control cycle as a joint action;

[0268] No. The combined action of each control cycle Represented as:

[0269]

[0270] According to the The dissolved oxygen concentration at the end of each control cycle determines the safe dissolved oxygen component. :

[0271]

[0272] The safe dissolved oxygen component is limited to between -1 and +1; when the calculated result is less than -1, it is taken as -1, and when the calculated result is greater than +1, it is taken as +1.

[0273] According to the The free ammonia concentration at the end of each control cycle determines the immediate risk component of free ammonia. :

[0274]

[0275] In the formula, This indicates the safe boundary for free ammonia determined based on the aquaculture species and the aquaculture stage;

[0276] No. Instant joint reward for each control cycle Determined according to the following formula:

[0277]

[0278] The instant joint reward and the first The three types of equipment in each control cycle are linked in a one-to-one correspondence, so that the improvement of dissolved oxygen generated by the low decarbonization oxygen supply equipment and the reduction of the risk of free ammonia generated by the ammonia nitrogen treatment circulation equipment are both entered into the joint reward through the status after the action is executed.

[0279] After the free ammonia late peak magnification window is completed, the decarbonization magnification exposure amount obtained by allocating each effective open aeration action is determined according to step S4.

[0280] For the Each effective open aeration action is recorded as the duration of its effect. ;

[0281] No. Average decarbonization concentration corresponding to each effective open aeration action Determined according to the following formula:

[0282]

[0283] No. The delayed decarbonization risk component corresponding to each effective open aeration action. Determined according to the following formula:

[0284]

[0285] Backfill the delayed decarbonization risk component to the first... Each control cycle is covered by an effective open aeration action;

[0286] When the One effective open aeration action covers The risk of delayed decarbonization allocated to each control cycle covered by this open aeration action. for:

[0287]

[0288] The risk of delayed decarbonization is zero for control cycles that do not include effective open aeration actions.

[0289] No. The final joint reward of each control cycle Determined according to the following formula:

[0290]

[0291] Therefore, a single control cycle generates only one final joint reward corresponding to the combined actions of the three types of equipment.

[0292] The risk of delayed decarbonization is only backfilled to the control cycle that includes effective open aeration actions, while the control effect of low decarbonization oxygen supply equipment and ammonia nitrogen treatment cycle equipment is entered into the immediate joint reward through the dissolved oxygen concentration and free ammonia concentration after the action is executed.

[0293] For each control cycle, a state transition sample is generated:

[0294]

[0295] In the formula, This represents the state vector before the action is executed; This indicates the combined action of three types of equipment; Indicates the final joint award; This represents the state vector after the action is executed; This indicates the end of the feeding event;

[0296] When the current feeding event ends, the feeding event termination flag for the corresponding control cycle is set to 1; the feeding event termination flag for other control cycles is set to zero.

[0297] When the current feeding event does not form a late peak amplification window for free ammonia, the late ammonia nitrogen exposure and decarbonization amplification exposure are set to zero, and the late decarbonization risk for each control cycle is set to zero.

[0298] When the free ammonia late peak magnification window is not formed, an immediate joint reward is formed based on the dissolved oxygen concentration and free ammonia concentration at the end of each control cycle, and the immediate joint reward is used as the final joint reward for the joint action of the corresponding three types of equipment.

[0299] Step S6: Update the deep reinforcement learning control strategy and use it for the next feeding event;

[0300] Specifically, the strategy iteration unit writes the state transition samples formed in step S5 into the experience replay storage area;

[0301] After each feeding event, 128 state transition samples are randomly selected from the experience replay storage area to form a training batch;

[0302] For the next state in the training batch The target policy network generates the next target action. Gaussian smoothed noise is added to the next target action to limit the noise-added action to the allowable action range of the three types of devices.

[0303] Two target value networks output value estimates based on the next state and the next target action, respectively, and the smaller of the two value estimates is used to determine the value update target. :

[0304]

[0305] In the formula, and These represent the first target value network and the second target value network, respectively.

[0306] The parameters of the two value networks are updated based on the squared difference between the current output of the first value network and the value update target, and the squared difference between the current output of the second value network and the value update target, respectively.

[0307] Every two value network parameter updates are completed, the policy network is updated once based on the evaluation value of the policy network's output action by the first value network, thereby increasing the evaluation value output by the first value network.

[0308] After updating the policy network and the two value networks, update the target policy network and the two target value networks using a soft update method:

[0309]

[0310] In the formula, Indicates the current network parameters; This represents the network parameters of the corresponding target network; Indicates the soft update coefficient;

[0311] In this embodiment, twenty training batches of network updates are performed after each feeding event.

[0312] The updated policy network, the two value networks, the target policy network, and the two target value networks together constitute the deep reinforcement learning control strategy used in the next feeding event;

[0313] In the actual control process of the next feeding event, only the updated policy network is used to generate control actions, and the two value networks and three target networks do not directly output control commands to the device.

[0314] After the next feeding event begins, return to step S1, redetermine the baseline time and baseline environmental data for the feeding event, and perform coordinated regulation of multiple environmental factors according to steps S1 to S6.

[0315] Therefore, regardless of whether a feeding event creates a free ammonia late peak amplification window, it has a complete data synchronization, control, reward formation, and strategy update path.

[0316] The following example data is only used to illustrate the specific calculation process of free ammonia late peak magnification window identification, risk decomposition, and reward backfilling;

[0317] Let the baseline time data for a single feeding event be as follows:

[0318] The baseline dissolved oxygen is 5.8 mg / L, the baseline dissolved carbon dioxide is 10 mg / L, the baseline pH is 8.0, the baseline total ammonia nitrogen is 0.3 mg / L, the baseline temperature is 25 degrees Celsius, and the baseline salinity is 32 parts per thousand.

[0319] The unified control cycle is one hour; the effective control time data after feeding is shown in the table below:

[0320] Zero hours 5.8 10.0 8.00 0.30 25.0 32 one hour 6.4 8.8 8.15 0.33 25.5 32 Two hours 6.3 8.2 8.25 0.39 25.5 32 Three hours 6.1 8.0 8.30 0.45 26.0 32 Four hours 6.0 8.5 8.20 0.46 26.0 32

[0321] In the table, the units for dissolved oxygen concentration, dissolved carbon dioxide concentration, and total ammonia nitrogen concentration are all milligrams per liter;

[0322] Based on the aforementioned calculation relationship for the proportion of free ammonia in seawater, the baseline free ammonia proportion is approximately 0.0460;

[0323] The proportions of free ammonia at one, two, three and four hours after feeding were approximately 0.0660, 0.0817, 0.0939 and 0.0761, respectively.

[0324] The corresponding free ammonia concentrations are approximately 0.0218 mg / L, 0.0319 mg / L, 0.0423 mg / L, and 0.0350 mg / L, respectively.

[0325] The baseline free ammonia concentration is approximately 0.0138 mg / L;

[0326] One hour after feeding, the dissolved oxygen concentration was higher than the baseline dissolved oxygen, the dissolved carbon dioxide concentration was lower than the baseline dissolved carbon dioxide, the pH was higher than the baseline pH, the total ammonia nitrogen concentration was higher than the baseline total ammonia nitrogen, the free ammonia ratio was higher than the baseline free ammonia ratio, and the free ammonia concentration was higher than the free ammonia concentration at the baseline time. Therefore, one hour after feeding was determined as the starting time of the free ammonia late peak amplification window.

[0327] Four hours after feeding, the concentration of free ammonia was lower than that three hours after feeding. Therefore, four hours after feeding was determined as the end time of the free ammonia late peak magnification window.

[0328] The duration of the magnification window for the delayed peak of free ammonia is three hours.

[0329] The delayed ammonia nitrogen load component and decarbonization amplification component corresponding to each effective control time are shown in the table below:

[0330] one hour 0.00138 0.00661 Two hours 0.00414 0.01394 Three hours 0.00690 0.02157 Four hours 0.00736 0.01384

[0331] All units in the table are milligrams per liter;

[0332] Using trapezoidal integrals, the late-onset ammonia nitrogen exposure is approximately 0.0154 mg / L, and the decarbonization-amplified exposure is approximately 0.0457 mg / L.

[0333] Assuming that there are two effective open aeration actions before and within the formation of the free ammonia late peak magnification window, after deducting the response of the low carbon dioxide decarbonation oxygen supply equipment, the response of the ammonia nitrogen treatment circulation equipment, and background changes, the corrected net decrease in dissolved carbon dioxide corresponding to the two effective open aeration actions is 0.8 mg / L and 1.2 mg / L, respectively.

[0334] The decarbonization scale-up exposure obtained from the first effective open aeration action allocation is:

[0335]

[0336] The decarbonization amplification exposure obtained from the second effective open aeration action allocation is:

[0337]

[0338] The units for both results above are milligrams per hour per liter;

[0339] Assuming that the duration of the two effective open aeration actions is three hours, the corresponding average decarbonization amplification concentrations are approximately 0.0061 mg / L and 0.0091 mg / L, respectively.

[0340] Under the condition of only being used to illustrate the reward backfilling process, the lower limit of dissolved oxygen safety is set at 5 mg / L, and the safety boundary of free ammonia safety is set at 0.02 mg / L;

[0341] The delayed decarbonization risk components corresponding to the two effective open aeration actions are approximately 0.305 and 0.457, respectively.

[0342] When the first effective open aeration action covers two control cycles, the risk of delayed decarbonization allocated to each control cycle is approximately 0.1525.

[0343] When the second effective open aeration action covers three control cycles, the risk of delayed decarbonization allocated to each control cycle is approximately 0.1523.

[0344] Therefore, when the delayed risk is not backfilled, the control cycle including open aeration may only receive a positive immediate joint reward because the dissolved oxygen concentration is within a safe range; after the delayed decarbonization risk is backfilled, the final joint reward of the corresponding control cycle decreases, thereby reducing the tendency of the deep reinforcement learning control strategy to continue to increase the operating intensity of open aeration equipment under similar environmental conditions.

[0345] This embodiment forms an environmental equipment synchronization sequence by synchronously acquiring water environment data and equipment operating status before and after the feeding event; it determines the free ammonia concentration based on pH, temperature, salinity, and total ammonia nitrogen concentration, and identifies the free ammonia late peak magnification window online; within the free ammonia late peak magnification window, it determines the required total oxygen transfer rate based on oxygen mass balance, prioritizes the use of low decarbonization oxygen supply equipment to meet the increased oxygen supply demand, and increases the operating intensity of ammonia nitrogen treatment circulation equipment when the free ammonia concentration continues to rise;

[0346] After the free ammonia late peak amplification window ends, the free ammonia concentration increment is separated into late ammonia nitrogen load component and decarbonization amplification component. Based on the net decrease in dissolved carbon dioxide after equipment calibration response correction, the decarbonization amplification exposure is allocated to the corresponding preceding open aeration action.

[0347] The decarbonization amplification exposure obtained from the allocation is used to form a delayed decarbonization risk component. The delayed decarbonization risk component is then backfilled into the control cycle covered by the corresponding open aeration action, and the reward for the joint action of the three types of equipment corresponding to the control cycle is corrected.

[0348] Based on the state vector before the action is executed, the joint action of the three types of devices, the final joint reward, the state vector after the action is executed, and the feeding event termination flag, a reward correction training sample is formed. The deep reinforcement learning control strategy is updated using the reward correction training sample, and the updated deep reinforcement learning control strategy is used for the next feeding event.

[0349] Therefore, this embodiment can maintain a safe level of dissolved oxygen in mariculture water while reducing the reverse optimization of deep reinforcement learning control strategies that continuously increase the operating intensity of open aeration equipment due to immediate dissolved oxygen gains, and achieve synergistic control among open aeration equipment, low decarbonization oxygen supply equipment, and ammonia nitrogen treatment circulation equipment.

[0350] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-environmental factor synergistic regulation system for marine aquaculture based on deep reinforcement learning, characterized in that, include: The synchronization unit acquires water environment data and the operating status of open aeration equipment, low decarbonization oxygen supply equipment and ammonia nitrogen treatment circulation equipment to form an environmental equipment synchronization sequence. The identification unit determines the state of the free ammonia late peak magnification window based on the synchronization sequence of the environmental equipment. The control unit generates a target operating intensity based on the synchronization sequence of the environmental equipment and the state of the free ammonia late peak magnification window using a deep reinforcement learning control strategy. The collaborative control unit, based on the target operating intensity within the free ammonia late peak magnification window, increases the operating intensity of the low decarbonization oxygen supply equipment and adjusts the ammonia nitrogen treatment circulation equipment, and appends the controlled data to the environmental equipment synchronization sequence; After the free ammonia late peak amplification window ends, the risk attribution unit determines the late ammonia nitrogen exposure and decarbonization amplification exposure by the additional environmental equipment synchronization sequence, and allocates the decarbonization amplification exposure to the open aeration action. The strategy iteration unit adjusts the reward for the open aeration action based on the allocation result, updates the control strategy, and applies it to the next feeding event.

2. The synergistic regulation system for multiple environmental factors in marine aquaculture based on deep reinforcement learning according to claim 1, characterized in that, The synchronization unit forms the environmental device synchronization sequence, including: Dissolved oxygen, dissolved carbon dioxide, pH, total ammonia nitrogen, temperature, and salinity of the seawater aquaculture body are obtained as the water body environmental data; The operating intensity of the open aeration device, the low decarbonization oxygen supply device, and the ammonia nitrogen treatment circulation device, as well as the moment when the operating intensity changes, are obtained as the operating state. The last complete sampling time before the start of a feeding cycle is determined as the baseline time, and baseline dissolved oxygen, baseline dissolved carbon dioxide, baseline pH, baseline total ammonia nitrogen, baseline temperature, and baseline salinity are determined. The water environment data and operational status after the start of feeding are correlated over time according to a unified control cycle. The unified control cycle that simultaneously measures dissolved oxygen, dissolved carbon dioxide, pH, total ammonia nitrogen, temperature, and salinity is defined as the effective control time, and the water environment data and operating status corresponding to the effective control time are written into the synchronization sequence of the environmental equipment.

3. The synergistic regulation system for multiple environmental factors in marine aquaculture based on deep reinforcement learning according to claim 2, characterized in that, The identification unit determines the free ammonia concentration and the state of the free ammonia late peak magnification window, including: The reference free ammonia ratio is determined based on the reference pH, the reference temperature, and the reference salinity, according to the ammonia dissociation equilibrium relationship in seawater. Based on the pH, temperature, and salinity at each effective control moment, the current free ammonia ratio is determined according to the aforementioned seawater ammonia dissociation equilibrium relationship, and the current free ammonia concentration is determined based on the current total ammonia nitrogen concentration and the aforementioned current free ammonia ratio; The effective control time that simultaneously satisfies the following conditions after the start of feeding—current dissolved oxygen higher than the reference dissolved oxygen, current dissolved carbon dioxide lower than the reference dissolved carbon dioxide, current pH higher than the reference pH, current total ammonia nitrogen concentration higher than the reference total ammonia nitrogen, current free ammonia ratio higher than the reference free ammonia ratio, and current free ammonia concentration higher than the previous effective control time—is determined as the start time of the free ammonia late peak amplification window. Starting from the aforementioned start time, the effective control time at which any of the following occurs for the first time is defined as the end time of the free ammonia late peak amplification window: the current free ammonia concentration is not higher than the free ammonia concentration at the previous effective control time, the current total ammonia nitrogen concentration is not higher than the total ammonia nitrogen concentration at the previous effective control time, the current dissolved carbon dioxide concentration is not lower than the reference dissolved carbon dioxide concentration, the current pH concentration is not higher than the reference pH concentration, the current dissolved oxygen concentration is lower than the reference dissolved oxygen concentration, or the start time of the next feeding event. When the start time exists, the state of the free ammonia late peak magnification window is determined to have been formed, and the environmental equipment synchronization sequence from the start time to the end time is extracted; When the aforementioned start time does not exist, the state of the free ammonia late peak magnification window is determined to be not formed.

4. The synergistic regulation system for multiple environmental factors in marine aquaculture based on deep reinforcement learning according to claim 3, characterized in that, The risk attribution unit determines the delayed ammonia nitrogen exposure and the decarbonization amplification exposure, including: When the free ammonia late peak magnification window is formed, the increase in the current total ammonia nitrogen concentration relative to the baseline total ammonia nitrogen is converted according to the baseline free ammonia ratio to obtain the late ammonia nitrogen load component. The increase in the current free ammonia ratio relative to the baseline free ammonia ratio is converted according to the current total ammonia nitrogen concentration to obtain the decarbonization amplification component; The delayed ammonia nitrogen load component within the free ammonia delayed peak magnification window is accumulated according to the duration between adjacent effective control times to obtain the delayed ammonia nitrogen exposure amount; The decarbonization amplification component within the free ammonia late peak amplification window is accumulated according to the duration between adjacent effective control moments to obtain the decarbonization amplification exposure amount; When the free ammonia late peak amplification window is not formed, the late ammonia nitrogen exposure and the decarbonization amplification exposure are determined to be zero.

5. The synergistic regulation system for multiple environmental factors in marine aquaculture based on deep reinforcement learning according to claim 4, characterized in that, The risk attribution unit allocates the decarbonization amplification exposure to the open aeration action, including: The time from the moment when the operating intensity of the open aeration equipment changes to the moment when the operating intensity changes again or the end of the free ammonia late peak magnification window is defined as the effective period of one open aeration action. Based on the dissolved carbon dioxide concentrations corresponding to the start and end effective control times of the aforementioned action period, determine the net decrease in dissolved carbon dioxide corresponding to the open aeration action. Open aeration actions with a net decrease in dissolved carbon dioxide greater than zero are defined as effective open aeration actions. The decarbonization amplification exposure is allocated to each effective open aeration action according to the proportion of the net decrease in dissolved carbon dioxide corresponding to each effective open aeration action to the sum of the net decrease in dissolved carbon dioxide corresponding to all effective open aeration actions. When the sum of the net decrease in dissolved carbon dioxide corresponding to all open aeration actions is zero, the carbon amplification exposure obtained from each open aeration action is determined to be zero. The environmental equipment synchronization sequence, the delayed ammonia nitrogen exposure, the decarbonization amplification exposure, and the decarbonization amplification exposure obtained from the allocation of each open aeration action are combined to form a delayed risk attribution sample.

6. The synergistic regulation system for multiple environmental factors in marine aquaculture based on deep reinforcement learning according to claim 5, characterized in that, The strategy iteration unit modifies the reward for the open-ended aeration action, including: The current dissolved oxygen, current dissolved carbon dioxide, current pH, current total ammonia nitrogen concentration, current free ammonia concentration, number of control cycles elapsed since feeding, free ammonia late peak amplification window status, current cumulative late ammonia nitrogen exposure, current cumulative decarbonization amplification exposure, and current operating intensity of the three types of equipment are determined as status data. The target operating intensity of the open aeration equipment, the low decarbonization oxygen supply equipment, and the ammonia nitrogen treatment circulation equipment is determined as the control action; After the open aeration action increases dissolved oxygen, a temporary dissolved oxygen gain is generated based on the amount of dissolved oxygen increase. After the free ammonia late peak amplification window ends, the decarbonization amplification exposure amount assigned to the open aeration action is read from the late risk attribution sample; The average decarbonization amplification concentration is determined based on the decarbonization amplification exposure amount allocated to the open aeration action and the duration of the free ammonia late peak amplification window; The temporary dissolved oxygen gain is dimensionless according to the dissolved oxygen safety lower limit determined based on the aquaculture species, and the average decarbonization amplification concentration is dimensionless according to the free ammonia safety boundary determined based on the aquaculture species. The final reward for the open aeration action is obtained by subtracting the dimensionless average decarbonization amplification concentration from the dimensionless temporary dissolved oxygen gain. A reward-corrected training sample is formed based on the state data, the control action, the final reward, and the delayed ammonia nitrogen exposure.

7. A multi-environmental factor synergistic regulation system for marine aquaculture based on deep reinforcement learning according to claim 6, characterized in that, The control unit and the collaborative control unit perform collaborative control, including: The latest effective state in the synchronization sequence of the environmental equipment is input into the deep reinforcement learning control strategy to generate the target operating intensity of the open aeration equipment, the low decarbonization oxygen supply equipment and the ammonia nitrogen treatment cycle equipment; When the free ammonia late peak magnification window is not formed, control the three types of equipment according to the target operating intensity. When the free ammonia late peak magnification window is formed, determine the total oxygen supply capacity required to maintain the dissolved oxygen safety lower limit; When it is necessary to increase the total oxygen supply capacity, priority should be given to increasing the operating intensity of the low decarbonization oxygen supply equipment; When the low-carbon oxygen supply equipment reaches its rated maximum operating intensity, and the current dissolved oxygen is still lower than the dissolved oxygen safety lower limit, the operating intensity of the open aeration equipment is increased. When the current free ammonia concentration is higher than the free ammonia concentration at the previous effective control time, the operating intensity of the ammonia nitrogen treatment circulation equipment is increased; The adjusted operating intensity of the three types of equipment is implemented, water environment data and operating status are reacquired, and the reacquired data is appended to the synchronization sequence of the environmental equipment.

8. The synergistic regulation system for multiple environmental factors in marine aquaculture based on deep reinforcement learning according to claim 7, characterized in that, The policy iteration unit updates the deep reinforcement learning control policy, including: Based on the water environment data added to the synchronization sequence of the environmental equipment, the current free ammonia ratio and the current free ammonia concentration are re-determined, and it is determined whether the free ammonia late peak amplification window has ended; While the free ammonia late peak amplification window has not yet ended, the late ammonia nitrogen exposure and the decarbonization amplification exposure are updated, and the control unit and the collaborative control unit continue to perform collaborative control. When the free ammonia late peak magnification window ends, the final late ammonia nitrogen exposure and the final decarbonization magnification exposure corresponding to each open aeration action are determined. The reward-corrected training sample is formed based on the final decarbonization magnification exposure and is used as the complete training sample. When the free ammonia late peak amplification window is not formed and the next feeding event begins, a complete training sample is formed based on the late ammonia nitrogen exposure amount and decarbonization amplification exposure amount, which are determined to be zero. The deep reinforcement learning control strategy is updated using the complete training samples, and the updated deep reinforcement learning control strategy is used in the next feeding event.

9. A multi-environmental factor synergistic regulation system for marine aquaculture based on deep reinforcement learning according to claim 8, characterized in that, The open aeration equipment supplies oxygen to the aquaculture water and removes dissolved carbon dioxide through contact between the aquaculture water and air; the low decarbonization oxygen supply equipment is an oxygen supply equipment that uses pressurized pure oxygen dissolution or an oxygen supply equipment that uses membrane oxygen transfer; the ammonia nitrogen treatment and circulation equipment transports the aquaculture water to the biological filtration unit or ammonia nitrogen adsorption unit and transports the treated aquaculture water back to the seawater aquaculture area.

Images