SSA-d3qn-based salt field crystallization calculation method, system, device and medium

Abstract

The application discloses a salt field crystallization calculation method, system, device and medium based on SSA-D3QN, which comprises the following steps: initializing an SSA population and D3QN network parameters; updating the position of the SSA population and screening an optimal concentration individual; judging whether the screened individual meets the accuracy requirement until the screened individual meets the accuracy requirement; training the optimal concentration individual as an agent and storing experience into a priority experience replay buffer; extracting samples from the priority experience replay buffer to train the D3QN network; testing the overall model for several times, obtaining the minimum value and the average value in the several times of testing; and judging whether the obtained minimum value and average value meet the error requirement until the error requirement is met. The application solves the reverse calculation problem of the salt field crystallization model by means of the sparrow search algorithm SSA and the deep reinforcement learning algorithm D3QN, can meet the accuracy of the crystallization model, shortens the calculation time and improves the calculation efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of electronic information technology, and relates to a salt field crystallization calculation method, system, device and medium based on SSA-D3QN. BACKGROUND

[0002] The salt field solar salt production is a process of concentrating brine to saturation concentration and precipitating sea salt by using the method of salt field solarization. This process mainly utilizes solar radiation and wind to evaporate water, so the construction of a photovoltaic project on the salt field will inevitably affect the salt field solar salt production. The "salt and light integration" project must consider providing economic compensation for the loss of salt field production, and the amount of compensation will directly affect the economy of the project. Therefore, it is essential to quantitatively evaluate the impact of the photovoltaic project on the salt field solar salt production in such a project, so that the negotiation boundary can be reasonably controlled during the project development process. During the calculation of the crystallization model, the crystallization model needs to realize the forward and reverse calculation functions, that is, it can design various levels of salt fields, including evaporation ponds and crystallization ponds, according to the initial concentration of seawater, the concentration of each level of salt field, etc. It can also calculate the concentration of each level of brine according to the known area of each level of salt field and give the best evaporation and crystallization ratio after improvement. In short, it can be summarized as forward calculation and reverse calculation. The parameters required for forward calculation are the initial concentration of seawater, the concentration of each level of salt field, etc., and the output result is the area size of each level of evaporation and crystallization pond. The reverse calculation is the opposite, which needs to calculate the most suitable concentration of each level of salt field according to the existing area size of the evaporation and crystallization ponds and the seawater concentration.

[0003] The existing calculation method usually uses genetic algorithm or related search algorithms such as particle swarm, butterfly algorithm, artificial bee colony algorithm, firefly algorithm, and firework algorithm. In the specific implementation process, it is found that the existing genetic algorithm has problems such as long time consumption, large space occupation, large error, and even inability to converge to the required precision. The accuracy of sparrow search algorithm (SSA) can meet the requirements, but the time complexity is too high. Therefore, there is an urgent need for a method that can meet the accuracy of the reverse calculation of the crystallization model and shorten the calculation time. SUMMARY

[0004] The purpose of the present application is to solve the problems in the prior art and provide a salt field crystallization calculation method, system, device and medium based on SSA-D3QN, which uses sparrow search algorithm (SSA) and deep reinforcement learning algorithm (D3QN) to cooperatively solve the reverse calculation problem of the salt field crystallization model, can meet the accuracy of the crystallization model, and shorten the calculation time and improve the calculation efficiency.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0006] The salt field crystallization calculation method based on SSA-D3QN comprises:

[0007] Step 1: initialize the SSA population and the D3QN network parameters;

[0008] Step 2: update the SSA population position and select the optimal concentration individual;

[0009] Step 3: determine whether the selected individual meets the accuracy requirement, if yes, proceed to Step 4; if not, repeat Step 2 until the selected individual meets the accuracy requirement;

[0010] Step 4: train the optimal concentration individual as an agent and store the experience in the priority experience replay buffer;

[0011] Step 5: extract samples from the priority experience replay buffer to train the D3QN network;

[0012] Step 6: test the overall model several times to obtain the minimum value and average value in several tests;

[0013] Step 7: determine whether the obtained minimum value and average value meet the error requirement, if yes, the training is complete; if not, repeat Steps 1 to 6 until the error requirement is met.

[0014] Further improvements of the present application are:

[0015] The SSA population is an n*d matrix, where n is the population size, d is the concentration vector length, n is the number of evaporation crystallization pools minus 1, and the parameters of the SSA population are: T s is the number of iterations, S t is the discoverer alert threshold, P r is the discoverer proportion, S d is the scout proportion;

[0016] The D3QN network parameters include: T is the number of D3QN iteration rounds, γ is the decay factor, ∈ is the exploration rate, c is the target q network parameter update frequency, S is the number of SumTree leaf nodes, β is the sampling weight coefficient, m is the number of batch gradient descent samples, and N is the test iteration round number.

[0017] The SSA population position is updated and the optimal concentration individual is selected, specifically:

[0018] Each individual in the SSA population is calculated to obtain the fitness, and the population is divided into discoverers and followers, where the discoverer proportion is P r ;

[0019] updating the position of the discoverer, randomly initializing a number r2 with a value range of 0 to 1, updating the position of the discoverer according to formula (1),

[0020]

[0021] wherein Q is a random number subject to normal distribution, L is a full 1 vector with a length of d, L = [1, 1, …] 1*d ;

[0022] updating the position of the follower, the follower will monitor the discoverer with the best fitness to find food, and the position update is shown in formula (2):

[0023]

[0024] wherein, is the worst position in the t th generation population, is the best position of the producer in the t+1 th generation population, A + = A T (AA T ) -1 , A is a matrix with a size of 1*d, wherein the elements are randomly selected from {1, -1};

[0025] randomly selecting the scout in each generation, the proportion is S d , the discoverer and the follower have a probability to become the scout, updating the position of the scout, and the position update is shown in formula (3):

[0026]

[0027] wherein, f i represents the current individual fitness, f g and f w represent the optimal and worst fitness; ε is a non-zero minimum value.

[0028] Step 3 is specifically:

[0029] judging whether the Loss of the optimal individual in the t+1 th generation population meets the threshold value of reinforcement learning training, if yes, recording it as φ(S) to proceed to step 4, otherwise, repeating step 2.

[0030] training the optimal concentration individual as an agent, and storing the experience into a priority experience revisit pool PriorityReplay Buffer, specifically:

[0031] In the Q network, φ(S) is used as input, the input φ(S) is the agent itself, and the Q value output corresponding to the Q network is obtained, where the input dimension is d, the output is d / 2, and the current action A corresponding to S is selected by using the ε-greedy method;

[0032] According to the action A, the corresponding agent vector position is calculated, a step τ is added to obtain a new feature vector φ(S'), the reward R is calculated, and if the error is less than the final threshold, the is_end flag is marked as True, otherwise it is marked as False;

[0033] If the value of is_end is True, it means that the training has reached the target, and the training can be exited, and the final concentration result is φ(S'); if the value of is_end is False, the training continues;

[0034] The five-tuple {φ(S), A, R, φ(S'), is_end} is stored in the experience replay SumTree.

[0035] The reward R is calculated by using difference calculation, that is, the Loss of φ(S') and φ(S) is calculated, and the product of the difference and the coefficient δ is taken as the reward R. If the operation output by the Q network makes the Loss larger, the reward R is negative; if the operation output by the Q network makes the Loss smaller, the reward R is positive.

[0036] The D3QN network is trained by extracting samples from the priority experience replay pool Priority Replay Buffer, specifically:

[0037] It is judged whether the number of samples of the priority experience replay pool Priority Replay Buffer exceeds the size of miniBatch, if not, φ(S)=φ(S'); if yes, m samples are sampled from the priority experience replay pool Priority Replay Buffer, and the probability of sampling each sample is based on Loss function weight The current target Q value y is calculated j , and miniBatch is the data set required for random gradient descent each time;

[0038] The mean square error loss function is used The parameters w are updated by back propagation;

[0039] The error δ of all samples is recalculated j =y j -Q(φ(S j ),A j ,w), and the priority of all nodes in the experience replay pool is updated as pj = |delta j |;

[0040] If the current iteration t % c == 0, update the target Q parameter w' = w.

[0041] The salt field crystallization calculation system based on the SSA-D3QN comprises:

[0042] An initialization module is configured to initialize an SSA population and D3QN network parameters.

[0043] An updating module is configured to update the position of the SSA population and screen out an optimal concentration individual.

[0044] A first judging module is configured to judge whether the screened individual meets the accuracy requirement until the screened individual meets the accuracy requirement.

[0045] A first training module is configured to train the optimal concentration individual as an agent and store experience into a priority experience replay pool Priority Replay Buffer.

[0046] A second training module is configured to extract samples from the priority experience replay pool Priority Replay Buffer to train the D3QN network.

[0047] A testing module is configured to test the overall model several times, obtain the minimum value and average value in the several times of testing.

[0048] A second judging module is configured to judge whether the obtained minimum value and average value meet the error requirement until the error requirement is met.

[0049] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above method when executing the computer program.

[0050] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0051] Compared with the prior art, the present application has the following beneficial effects:

[0052] The present application aims at the problem of slow convergence speed of the single D3QN, adopts SSA as a pre-computation step, makes the input value error of the D3QN smaller, and accelerates the convergence speed. In view of the problem of long time consumption, experience replay is adopted, and Prioritized Replay DQN is adopted, the priority of experience storage is set, so that the algorithm converges more easily. In view of the slow algorithm convergence caused by the strong dependence relationship between the target Q value calculation and the Q network parameter update of the general DQN algorithm, DoubleDQN is adopted, the selection of the target Q value action and the calculation of the target Q value are decoupled, the convergence speed is accelerated, and the overestimation is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and other related drawings can also be obtained by those skilled in the art without creative labor on the premise of the drawings.

[0054] Figure 1 The flowchart for the crystallization module is called;

[0055] Figure 2 The flowchart for the salt field crystallization calculation method based on SSA-D3QN of the present application is shown in the figure;

[0056] Figure 3 The flowchart for the sparrow search algorithm SSA is shown in the figure;

[0057] Figure 4 The flowchart for the D3QN algorithm is shown in the figure;

[0058] Figure 5 The structure diagram of the full connection network is shown in the figure;

[0059] Figure 6 The comparison diagram of the operation results of the genetic algorithm and the SSA-D3QN is shown in the figure;

[0060] Figure 7 The structure diagram of the salt field crystallization calculation system based on SSA-D3QN of the present application is shown in the figure. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application, and obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0062] The following detailed description of embodiments of the application in the drawings provided is not intended to limit the scope of the application claimed, but merely represents selected embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the application.

[0063] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0064] In the description of embodiments of the application, it should be noted that if the terms "upper", "lower", "horizontal", "inner" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the application is used, it is only for the convenience of describing the application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application. In addition, the terms "first", "second", etc. are only used for differentiation and cannot be understood as indicating or implying relative importance.

[0065] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly inclined. For example, "horizontal" only means that its direction is relatively more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined.

[0066] In the description of embodiments of the application, it should also be noted that unless otherwise explicitly specified and limited, if the terms "arrangement", "installation", "connection", "connection" appear, they should be understood broadly, for example, they can be fixedly connected, or detachably connected, or integrally connected; can be mechanically connected, or electrically connected; can be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements. For those of ordinary skill in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0067] The application will be described in further detail below in conjunction with the drawings:

[0068] Referring to Figure 1 The application discloses a crystallization module calling flow chart, specifically: a customer calls through a desktop client, parameters are passed in during the calling process, the parameters include a large-area evaporation coefficient, a photovoltaic shading annual evaporation amount, an annual evaporation amount, an annual total precipitation amount, air humidity, production period operation days, a halogen exchange period, an evaporation area halogen retention coefficient, a design step number, a first-step evaporation pond area, a seawater initial concentration, an evaporation final concentration, a crystallization final concentration, a crystallization area dilute water discharge rate, a crystallization area step number and a salt collection period.

[0069] The crystallization module calculation engine performs calculation in response to a user call, and if it is a first call or the parameters have changed, network training is performed, and then calculation is performed after the training is completed, and the result is returned after the calculation is completed.

[0070] The calculation problem of the crystallization module calculation engine is an optimization problem, and the industry solution is usually a traditional genetic algorithm, but through experiments, the simple genetic algorithm has problems such as long time consumption, large space occupation, and large error in solving the problem, so an intelligent optimization algorithm is considered, and the sparrow search algorithm SSA is relatively novel and has outstanding performance, but SSA is easy to fall into local optimum, and through experiments, D3QN can effectively increase the calculation efficiency of SSA, and SSA can provide a good initial environment for the training of reinforcement learning.

[0071] After receiving the user call, the calculation engine receives a series of parameters, and then performs calculation according to the initial concentration and final concentration of seawater and the number of evaporation and crystallization pools. Figure 2 The application discloses a salt field crystallization calculation method based on SSA-D3QN, which comprises the following steps:

[0072] S101: initialize the SSA population and the D3QN network parameters;

[0073] Referring to Figure 3 , the SSA population is an n*d matrix, wherein n is the population size, d is the length of the concentration vector, n is the number of evaporation and crystallization pools minus 1, and the parameters of the SSA population are: T s is the number of iterations, S t is the scout alert threshold, P r is the scout proportion, S d is the scout proportion;

[0074] The D3QN network parameters comprise: T is the number of D3QN iteration rounds, γ is a decay factor, ∈ is an exploration rate, c is a target Q network parameter update frequency, S is the number of leaf nodes of a SumTree, β is a sampling weight coefficient, m is the number of sample batches of batch gradient descent, and N is the number of test iteration rounds.

[0075] S102: update the position of the SSA population, and select the optimal concentration individual;

[0076] Each individual in the SSA population is calculated to obtain the fitness, and the population is divided into scouts and followers, wherein the proportion of the scouts is P r ;

[0077] The position of the scout is updated, a number r2 in a value range of 0 to 1 is randomly initialized, and the position of the scout is updated according to formula (1).

[0078]

[0079] where Q is a random number obeying normal distribution, L is a vector of all 1 with length d, L = [1, 1, …] 1*d ;

[0080] The position of the follower is updated, and the follower monitors the best scout to find food, and the position update is shown in equation (2):

[0081]

[0082] where, is the worst position in the tth generation population, is the best position of the producer in the t+1th generation population, A + = A T (AA T ) -1 , A is a matrix with size 1*d, and the elements are randomly selected from {1, -1};

[0083] The scout is randomly selected in each generation, and the proportion is S d The scout, the discoverer and the follower have a chance to become a scout, and the position of the scout is updated, and the position update is shown in equation (3):

[0084]

[0085] where f i represents the current individual fitness, f g and f w represent the optimal and worst fitness; ε is a non-zero minimum value.

[0086] S103: Determine whether the selected individual meets the accuracy requirement, if yes, proceed to S104; if not, repeat S102 until the selected individual meets the accuracy requirement.

[0087] Determine whether the optimal individual in the t+1th generation population meets the threshold value for reinforcement learning training, if yes, record it as φ(S) and proceed to S104, otherwise, repeat S102.

[0088] Because there is a sequence between the pools, the area of the front pool is certainly greater than that of the back pool, and the concentration of the front pool is certainly less than that of the back pool, so the concentration part can be extracted into a basic vector at initialization, that is, the input of D3QN is an offset vector, which can accelerate the convergence speed during network training.

[0089] S104: Train the optimal concentration individual as an agent, and store the experience into a priority experience replay buffer.

[0090] Referring to Figure 4 In the Q network, φ(S) is used as input, and the input φ(S) is the agent itself. The Q network outputs the corresponding Q value, where the input dimension is d, and the output is d / 2. The current action A corresponding to S is selected by the ε-greedy method.

[0091] According to the action A, the corresponding agent vector position is calculated, and a step τ is added to obtain a new feature vector φ(S'), and the reward R is calculated. If the error is less than the final threshold, the is_end flag is marked as True, otherwise it is marked as False.

[0092] If the value of is_end is True, it means that the training has reached the target, and the training can be exited. The final output concentration result is φ(S'); if the value of is_end is False, the training continues.

[0093] The five-tuple {φ(S), A, R, φ(S'), is_end} is stored in the experience replay SumTree.

[0094] The difference value calculation is used to calculate the reward R, that is, the Loss of φ(S') and φ(S) is calculated respectively, and the product of the difference value and the coefficient δ is taken as the reward R. If the operation of the Q network output makes the Loss larger, the reward R is negative; if the operation of the Q network output makes the Loss smaller, the reward R is positive.

[0095] S105: Extract samples from the priority experience replay buffer Priority Replay Buffer to train the D3QN network;

[0096] Determine whether the number of samples in the priority experience replay buffer Priority Replay Buffer exceeds the size of miniBatch. If not, φ(S)=φ(S'); if yes, sample m samples from the priority experience replay buffer Priority Replay Buffer, where the probability of sampling each sample is based on Loss function weight Calculate the current target Q value y j miniBatch is the data set required for each random gradient descent;

[0097] Use the mean square error loss function Update the parameters w by backpropagation;

[0098] Recalculate all sample errors δ j = y j - Q (φ (S j ), A j , w), update the priority of all nodes in the experience replay pool as p j = | δ j |;

[0099] If the current iteration t % c == 0, update the target Q parameter w' = w.

[0100] S106: Test the overall model for several times to obtain the minimum value and the average value in the several times of testing;

[0101] Sum the results obtained in the several times of testing, and then obtain the average value of the test based on the sum result; and compare the sizes of the several times of testing results, and find the minimum test value in the results.

[0102] S107: Determine whether the obtained minimum value and the average value meet the error requirement, if yes, the training is completed; if not, repeat S101-106 until the error requirement is met.

[0103] If the accuracy does not meet the requirement, it is necessary to return to SSA for the next cycle to generate a new concentration vector individual, and then perform D3QN algorithm calculation.

[0104] The network used in the application is a full connection network, and the basic structure is as shown in Figure 5 The number of evaporation crystallization pools is 7, and the n-dimensional input vector is the concentration value. When the number of pools is large, the number of intermediate hidden layers can be increased to achieve better convergence effect.

[0105] After 1000 rounds of execution, the calculation results of the commonly used genetic algorithm and Dueling Double DQN in the industry are as shown in Figure 6 When the trained network is calculated, the convergence of SSA-D3QN has met the accuracy requirement when the calculation is performed to 400 steps, and is stable thereafter, and the fluctuation trend is not obvious. The genetic algorithm reaches the optimal value at about 900 steps, and the Loss value tends to be about 2000, which is obviously higher than that of D3QN. It has been verified that the genetic algorithm has a probability of reducing the Loss to 200 after 2000 steps, but the time cost is too high. After the model training is successful, the time complexity of the reinforcement learning algorithm is extremely low, and the time consumption is mainly concentrated in the model training.

[0106] Referring to Figure 7 , the application discloses a salt field crystallization calculation system based on SSA-D3QN, which comprises:

[0107] An initialization module is configured to initialize an SSA population and D3QN network parameters;

[0108] An updating module is configured to update the SSA population position, and screen out an optimal concentration individual;

[0109] A first judging module is configured to judge whether the screened individual meets the accuracy requirement, until the screened individual meets the accuracy requirement;

[0110] A first training module is configured to train the optimal concentration individual as an agent, and store experience into a priority experience replay pool Priority Replay Buffer;

[0111] A second training module is configured to extract samples from the priority experience replay pool Priority Replay Buffer to train the D3QN network;

[0112] A testing module is configured to test the overall model for several times, and obtain a minimum value and an average value in the several times;

[0113] A second judging module is configured to judge whether the obtained minimum value and average value meet an error requirement, until the error requirement is met.

[0114] An embodiment of the present application provides a terminal device. The terminal device of the embodiment comprises a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor implements the steps in each of the method embodiments when executing the computer program. Alternatively, the processor implements the functions of each module / unit in each of the device embodiments when executing the computer program.

[0115] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application.

[0116] The terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing devices. The terminal device can include, but is not limited to, a processor and a memory.

[0117] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc.

[0118] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the terminal device by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory.

[0119] The modules / units integrated in the terminal device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can realize the steps of the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0120] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for calculating crystallization in a salt field based on SSA-D3QN, characterized in that, The method comprises the following steps: The client calls through the desktop client, and the calling process transmits parameters, which include the large-area evaporation coefficient, the annual evaporation of photovoltaic shading, the annual evaporation, the annual total precipitation, the air humidity, the operation days in the production period, the halogen exchange period, the halogen preservation coefficient of the evaporation area, the design step number, the first-step evaporation pool area, the initial seawater concentration, the evaporation final concentration, the crystallization final concentration, the crystallization area dilute water discharge rate, the crystallization area step number and the salt collection period. The crystallization module calculation engine responds to the user call and performs calculation. If it is a first call or the parameters change, network training is performed, and then calculation is performed after the training is completed. The result is returned after the calculation is completed. The crystallization module is implemented by the following method: Step 1: initialize the SSA population and the D3QN network parameters; wherein the SSA population is an n*d matrix, wherein n is the population quantity, d is the concentration vector length, n is the number of evaporative crystallization pools minus 1, and the parameters of the SSA population are: T s is the iteration number, S t is the discoverer alert threshold, P r is the discoverer proportion, S d is the scout proportion; The D3QN network parameters comprise: T is the D3QN iteration number, γ is the attenuation factor, ε is the exploration rate, c is the target Q network parameter update frequency, S is the leaf node number of the SumTree, β is the sampling weight coefficient, m is the sample number of batch gradient descent, and N is the test iteration number. Step 2: Update the position of the SSA population, and select the optimal concentration individual; Step 3: Determine whether the selected individual meets the accuracy requirement, if yes, proceed to step 4; if not, repeat step 2 until the selected individual meets the accuracy requirement; Step 4: Train the optimal concentration individual as an agent, and store the experience in the priority experience replay pool Priority Replay Buffer; wherein the optimal concentration individual is trained as an agent, and the experience is stored in the priority experience replay pool Priority Replay Buffer, specifically: In the Q network, φ(S) is used as input, the input φ(S) is the agent itself, and the Q value output corresponding to the Q network is obtained, wherein the input dimension is d, the output is d / 2, and the current action A corresponding to S is selected by using the ∈-greedy method; According to the action A, the corresponding agent vector position is calculated, a step size τ is added, a new feature vector φ(S ′ ) is obtained, the reward R is calculated, and it is judged whether the error is less than the final threshold value. If yes, the is_end flag is marked as True, otherwise, it is marked as False. If the value of is_end is True, it means that the training has reached the target, i.e. the training can be exited, and the final output concentration result is φ(S ′ ); if the value of is_end is False, the training is continued; The five-tuple {φ(S), A, R, φ(S'), is_end} is stored in the experience replay SumTree; Step 5: Extract samples from the priority experience replay pool Priority Replay Buffer to train the D3QN network; Step 6: Test the overall model for several times to obtain the minimum value and the average value in the several tests; Step 7: Determine whether the obtained minimum value and average value meet the error requirement, if yes, the training is completed; if not, repeat steps 1 to 6 until the error requirement is met.

2. The method of claim 1, wherein the SSA-D3QN-based salt field crystallization calculation method is characterized by, The SSA population position is updated, and the optimal concentration individual is selected, specifically: The fitness of each individual in the SSA population is calculated, and the population is divided into discoverers and followers, with the discoverers accounting for P r ; The position of the discoverer is updated, a number r2 in the range of 0 to 1 is randomly initialized, and the position of the discoverer is updated according to formula (1), wherein Q is a random number subject to normal distribution, L is a full 1 vector with length d, L = [1, 1, …] 1*d ; The position of the follower is updated, and the follower monitors the best discoverer to find food, and the position update is shown in formula (2): where, is the worst position in the tth generation population, is the best position of the producers in the t+1th generation population, A + = A T (AA T ) -1 , A is a matrix of size 1*d with elements randomly chosen in {1,-1}; Each generation, a scout is randomly selected with a probability of S d Both the discoverer and the follower have a probability of becoming a scout, and the position of the scout is updated as shown in equation (3): where f i represents the current individual fitness, f g and f w represent the best and worst fitness; and ε is a non-zero small value.

3. The method of claim 1, wherein the method is a method of calculating crystallization in a salt field based on SSA-D3QN. Step 3 is specifically: Determine whether the Loss of the optimal individual in the t+1 generation population meets the threshold value of reinforcement learning training, if yes, record it as φ(S) for step 4, otherwise, repeat step 2.

4. The method of claim 3, wherein the SSA-D3QN-based salt field crystallization calculation method is characterized by, The reward R is calculated by difference calculation, that is, Loss of φ(S') and φ(S) is calculated respectively, and the product of the difference and a coefficient δ is taken as the reward R. If the operation output by the Q network makes the Loss larger, the reward R is negative; if the operation output by the Q network makes the Loss smaller, the reward R is positive.

5. The method of claim 4, wherein the SSA-D3QN-based salt field crystallization calculation method is characterized by, The D3QN network is trained by extracting samples from the priority experience replay buffer, specifically: determining whether the number of samples in the priority replay buffer exceeds the size of the miniBatch, and if not, then φ(S) = φ(S′); and if so, then sampling m samples from the priority replay buffer, where the probability of each sample being sampled is based on loss function weights computing the current target Q value y j miniBatch is the dataset needed for each stochastic gradient descent Using mean squared error loss function Update parameters w by backpropagation; Recalculate all sample errors δ j = y j - Q(φ(S j ), A j , w), update the priorities p j = |δ j | for all nodes in the experience replay pool. If the current iteration t % c == 0, the target Q parameter w' = w is updated.

6. A salt field crystallization computing system based on SSA-D3QN according to claim 1, characterized in that, It comprises: An initialization module for initializing the SSA population and the D3QN network parameters; An update module for updating the SSA population position and screening the optimal concentration individual; A first judgment module for judging whether the screened individual meets the accuracy requirement until the screened individual meets the accuracy requirement; A first training module for training the optimal concentration individual as an agent and storing the experience into the priority experience replay buffer; A second training module for training the D3QN network by extracting samples from the priority experience replay buffer; A test module for testing the overall model several times to obtain the minimum value and the average value in the several tests; A second judgment module for judging whether the obtained minimum value and average value meet the error requirement until the error requirement is met.

7. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to realize the steps of the method of any one of claims 1-5.

Images