A reinforcement learning-based method for optimizing fish feed extrusion parameters

By constructing a sample relationship matrix and a kernel regression model based on reinforcement learning, the uncertainty of parameter adjustment in fish feed extrusion production was solved, achieving efficient control of pellet bulk density, reducing the number of manual adjustments, and improving the stability and accuracy of the production process.

CN123116853APending Publication Date: 2026-11-17HUAIAN TIANSHEN CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611460802.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In the current fish feed extrusion production, there is a lack of effective modeling methods for adjusting parameters such as conditioning moisture content, screw speed, and die temperature. This results in the need for multiple rounds of trial adjustments to achieve the target pellet density, and the model prediction error is large, making it difficult to adapt to the continuity and changes in the production process.

Method used

A reinforcement learning-based approach is adopted. By constructing a sample relationship matrix, utilizing the continuous similarity between the transition samples of adjacent working conditions and the current process vector, and combining reinforcement learning to determine the hyperparameters of the kernel method regression model, the particle bulk density is predicted, and the quenching and tempering moisture content, screw speed and die temperature of the next working condition are output.

Benefits of technology

This technology reduces manual adjustments in fish feed extrusion production, improves the targeting and accuracy of pellet bulk density adjustment, reduces repetitive adjustments, and enhances the continuity and stability of the production process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123116853A_ABST
    Figure CN123116853A_ABST
Patent Text Reader

Abstract

This invention relates to the field of feed extrusion control technology, and discloses a method for optimizing fish feed extrusion parameters based on reinforcement learning. The method includes: acquiring historical records arranged by production time, the current actual operating conditions, a set of candidate operating conditions, and the target bulk density; forming transfer samples from adjacent historical records; establishing the most recent operating condition transfer direction relationship based on changes in conditioning moisture content, screw speed, and die temperature; constructing a sample relationship matrix that simultaneously includes the continuous similarity relationship of the current process vector and the most recent operating condition transfer direction relationship; determining the hyperparameters of the kernel method regression model through reinforcement learning; forming candidate transfer direction relationships for each candidate operating condition according to the same direction state rule; obtaining candidate predicted bulk densities; selecting the candidate operating condition closest to the target bulk density; and outputting the conditioning moisture content, screw speed, and die temperature for the next operating condition. This invention is used for continuous operating condition adjustment in fish feed extrusion production.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of feed extrusion control technology, and more specifically, to a method for optimizing fish feed extrusion parameters based on reinforcement learning. Background Technology

[0002] Fish feed extrusion production is typically a continuous process. After conditioning, the material enters the extruder. The conditioning moisture content, screw speed, and die temperature all affect the maturation, extrusion, and pelleting processes. Pellet bulk density is a commonly used indicator for adjusting operating conditions and controlling product quality. On the production site, process settings and pellet bulk density test results are usually kept by batch or time period. Operators adjust subsequent operating conditions based on the target bulk density.

[0003] Current production adjustments largely rely on process engineers to adjust the conditioning moisture content, screw speed, or die temperature incrementally based on the deviation between the current particle bulk density and the target bulk density. These three parameters have coupled effects on material conveying, heating, and puffing states. Furthermore, particle bulk density measurements typically correspond to already completed operating conditions. Therefore, the results of a single adjustment require confirmation through production and testing. When raw material batches, formulations, or equipment operating conditions change, existing experience cannot be directly reused, and multiple rounds of trial adjustments may be necessary to obtain acceptable operating conditions for the next stage.

[0004] To reduce manual adjustments, some production systems utilize historical records to build particle bulk density prediction models and filter subsequent operating conditions based on candidate setpoints. A common approach is to establish a correlation between input and particle bulk density based on the conditioning moisture content, screw speed, and die temperature at a given moment. For current operating conditions with similar values, this method often treats them as similar samples. However, in actual production, the continuous production processes corresponding to rising from a lower setpoint to the current operating condition, maintaining the current operating condition, or falling from a higher setpoint to the current operating condition are not the same. If the arrival direction in the historical records is not utilized, the model struggles to distinguish different adjustment processes under similar current operating conditions.

[0005] Furthermore, the hyperparameters of predictive models are often determined through manual experience or fixed-range traversal, limiting the utilization of the relationship between the adjustment process and the changes in validation errors. Some models only provide predicted pellet bulk density results, without further integrating the allowable range of fish feed extrusion technology, current actual operating conditions, and target bulk density to form a directly executable next operating condition setting. Therefore, existing methods still need to address how to use process records arranged by production time to express the continuous adjustment process, and based on this, determine the conditioning moisture content, screw speed, and die temperature of the next operating condition from candidate operating conditions. Summary of the Invention

[0006] This invention provides a method for optimizing fish feed extrusion parameters based on reinforcement learning, which solves the technical problems mentioned in the background art.

[0007] To achieve the above-mentioned objectives, the present invention provides the following technical solution.

[0008] A reinforcement learning-based method for optimizing fish feed extrusion parameters includes: Obtain historical data on fish feed extrusion, sorted by production time and including conditioning moisture content, screw speed, die temperature, and pellet bulk density; current actual operating conditions; candidate operating conditions; and target bulk density. The current actual operating conditions and each candidate operating condition include conditioning moisture content, screw speed, and die temperature. A transfer sample containing the previous working condition, the current working condition, and the corresponding particle bulk density is formed from two adjacent fish feed extrusion history records. The relationship between the most recent working condition transition direction is formed based on the changes in conditioned moisture content, screw speed, and die temperature from the previous working condition to the current working condition. Construct a sample relationship matrix, which includes a continuous similarity relationship between the current process vectors of two transferred samples to characterize the similarity of the continuous parameter space and the relationship of the most recent working condition transfer direction. The current process vector consists of the quenching and tempering moisture content, screw speed and die temperature. Hyperparameters of a kernel-based regression model for predicting particle bulk density, trained on a sample relationship matrix, were determined using reinforcement learning. Candidate transfer direction relationships from the current actual working condition to each candidate working condition are formed according to the same direction state rules as those used to form the most recent working condition transfer direction relationship, and candidate predicted bulk weights are obtained by the trained kernel method regression model. Based on the relationship between the candidate predicted bulk density and the target bulk density, the quenched moisture content, screw speed and die temperature for the next working condition are output.

[0009] Compared with the prior art, the present invention has the following substantial features and significant progress: This invention uses continuously recorded conditioning moisture content, screw speed, die temperature, and pellet bulk density from fish feed extrusion production as processing objects. It forms transfer samples from adjacent operating conditions and simultaneously retains the continuous similarity relationship between current process vectors and the most recent operating condition transfer direction relationship in the sample relationship matrix. This allows production records reaching similar operating conditions from different adjustment directions to be distinguished and utilized. Through reinforcement learning, the hyperparameters of the kernel method regression model are determined based on the change in verification error before and after hyperparameter adjustment, establishing a correspondence between the model adjustment process and the pellet bulk density prediction results. Within the preset process allowable range of fish feed extrusion, the candidate operating condition set is evaluated, and candidate predicted bulk densities are obtained according to the directional state rules consistent with the training phase. The conditioning moisture content, screw speed, and die temperature of the next operating condition are output. Therefore, it can provide a direct setting basis for continuous operating condition adjustment of the extruder, reducing repetitive adjustments caused by relying solely on manual trial adjustments and improving the targeting of pellet bulk density adjustments to approach the target bulk density. Attached Figure Description

[0010] Figure 1 This is a functional block diagram of a fish feed extrusion parameter optimization method based on reinforcement learning according to the present invention; Figure 2 This is a flowchart of a method for optimizing fish feed extrusion parameters based on reinforcement learning, according to the present invention. Detailed Implementation

[0011] The following description is provided in conjunction with the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the invention; those skilled in the art can make equivalent substitutions or combinations for the specific implementations.

[0012] The following combination Figure 1 and Figure 2 The specific embodiments of the present invention are further described below. This embodiment is used for continuous condition adjustment in fish feed extrusion production. The processing objects are the conditioning moisture content, screw speed, die temperature, and corresponding pellet bulk density, which are continuously recorded according to the production time. The method establishes a relationship between transfer samples based on the adjustment directions that have occurred between adjacent conditions, and uses this relationship together with the continuous similarity relationship between the current process vector for pellet bulk density prediction. Then, the next condition is determined from the candidate condition set based on the target bulk density. The final output is the conditioning moisture content, screw speed, and die temperature used in extrusion production.

[0013] See Figure 1 This implementation involves a fish feed extrusion data acquisition module 111, a transfer sample formation module 112, a most recent operating condition transfer direction relationship formation module 113, a sample relationship matrix construction module 114, a reinforcement learning hyperparameter determination module 115, a kernel method regression model training module 116, a candidate operating condition evaluation module 117, and a next operating condition output module 118. These modules are executed on a computing device connected to the data recording system of the fish feed extrusion production line. The computing device completes kernel method regression model training based on historical records and calls the trained kernel method regression model to evaluate candidate operating conditions during continuous production.

[0014] See Figure 2The method sequentially executes the following steps: acquiring fish feed extrusion data (S211), forming transfer samples (S212), forming the transfer direction relationship of the most recent working condition (S213), constructing a sample relationship matrix (S214), determining hyperparameters through reinforcement learning (S215), training a kernel method regression model (S216), evaluating candidate working conditions (S217), and outputting the next working condition (S218). During the training phase, a kernel method regression model is established using completed continuous working conditions and corresponding particle bulk densities. During the application phase, the current actual working condition, the set of candidate working conditions, and the target bulk density are read, and candidate transfer direction relationships are formed according to the same directional state rules as in the training phase, ensuring that the evaluation criteria for candidate working conditions remain consistent with those in the training phase.

[0015] In continuous operation of fish feed extrusion production, the conditioned material enters the extruder. The conditioned moisture content, screw speed, and die temperature collectively affect the material's conveying, heating, and extrusion state during the extrusion process. When different adjustment directions reach similar values ​​in the current operating condition, the previous operating condition has a distinguishing effect on the continuous production state of the current condition. Establishing a sample relationship solely based on the distance between current process vectors cannot represent this distinction. This implementation retains the three changes and their directional states from the previous operating condition to the current operating condition, so that the sample relationship matrix simultaneously reflects the proximity of the current process vector and the proximity of the most recent operating condition transition direction.

[0016] The fish feed extrusion data acquisition module 111 retrieves historical fish feed extrusion records according to production time. Each historical record includes at least the recording time, conditioning moisture content, screw speed, die temperature, and corresponding pellet bulk density. The recording time determines the chronological order of each historical fish feed extrusion record; the conditioning moisture content, screw speed, and die temperature are the process settings for the corresponding operating conditions; the pellet bulk density is the result obtained by pellet bulk density testing of the pellets produced under that operating condition. Batch identifiers are saved together with the recording time for verification of continuous production records.

[0017] Conditioning moisture content was recorded as a percentage by mass, screw speed as revolutions per minute, die temperature as degrees Celsius, and pellet bulk density as kilograms per cubic meter. Units were kept consistent throughout the historical extrusion records for the same group of fish feed. Pellet bulk density was measured on-site during production. The constant volume container is now complete. Stable operation is maintained under all operating conditions. Afterwards, in the following Three particulate samples were obtained at equal time intervals. The particulate samples were allowed to cool naturally to... The particles were then naturally poured into a fixed-volume container, the top edge of the container was leveled, and the mass of the particles inside the container was weighed. The particle bulk density was obtained three times from the particle mass and the container volume. The arithmetic mean of the three particle bulk densities was taken as the particle bulk density for that working condition. The sampling period was associated with the recording period of the corresponding working condition, and the obtained particle bulk density was used as the original particle bulk density monitoring label for that working condition.

[0018] No. The current process vector in the history of fish feed extrusion is represented as follows: in, For the first The current process vector in the history of fish feed extrusion. For the first Historical data on the conditioning moisture content of fish feed extrusion. For the first Screw speed in the history of fish feed extrusion. For the first The die temperature in the historical records of fish feed extrusion. The record number is determined according to the production time.

[0019] No. The original pellet bulk density monitoring label corresponding to the fish feed extrusion history is represented as follows: in, For the first The original pellet bulk density monitoring label corresponding to the history of fish feed extrusion. For the first The historical pellet bulk density corresponding to the extrusion process of fish feed. The original pellet bulk density monitoring label corresponds to the working conditions where pellet bulk density has been completed and obtained. In the application phase, the current actual working conditions and candidate working conditions are used as inputs to the kernel method regression model.

[0020] During continuous record processing, fish feed extrusion history records are arranged in ascending order of recording time. The two fish feed extrusion history records used to form the transfer samples are adjacent in production time and belong to a continuous production process. Subsequent calculations use the process settings and pellet bulk density saved in the fish feed extrusion history records.

[0021] The transfer sample formation module 112 starts from the second valid and continuous fish feed extrusion history record and combines two adjacent fish feed extrusion history records into a transfer sample. Each transferred sample is represented as: in, For the first One transferred sample, This is the current process vector for the previous operating condition. This represents the current process vector for the current operating condition. The arrows indicate the sequence of production time from the previous operating condition to the current operating condition. The original particle bulk density supervision label for each transferred sample is: This refers to the particle bulk density corresponding to the current operating condition. The previous operating condition indicates the direction of the most recent operating condition transition, while the current operating condition indicates the position of the transitioned sample in the continuous parameter space.

[0022] The transfer samples were divided into training transfer samples, validation transfer samples, and test transfer samples according to their current production time. The earliest transfer sample served as the training transfer sample, subsequent consecutive transfer samples served as validation transfer samples, and the latest transfer sample served as the test transfer sample. Each validation and test transfer sample used the preceding fish feed extrusion history record with the immediately adjacent production time to form the corresponding change. The training transfer samples were used to train the kernel method regression model and calculate the training standardized statistics set; the validation transfer samples were used to calculate the validation error; and the test transfer samples were used for independent testing after training was completed.

[0023] The current operating conditions and original particle bulk density supervision labels of the training transfer samples are used to calculate the mean moisture content, standard deviation of moisture content, mean screw speed, standard deviation of screw speed, mean die temperature, standard deviation of die temperature, mean particle bulk density, and standard deviation of particle bulk density. These eight quantities constitute the training standardized statistics set. (Arbitrary continuous quantities) The standardized representation is: in, For continuous quantities The standardization results For training transfer samples of continuous quantities The mean, For training transfer samples of continuous quantities The standard deviation of the standard deviation. The validation and test transfer samples are trained using a set of standardized statistics. The standardized current process vector consists of standardized conditioned moisture content, screw speed, and die temperature. in, For the first The standardized current process vector of each transfer sample, To standardize the moisture content of the conditioning product, The result is a standardized result of the screw speed. This is the standardized result for the die head temperature. The standardized current process vector is used to calculate continuous similarity relationships; the relationship of the most recent operating condition transition direction is determined based on the difference between the original process settings.

[0024] The module 113, which forms the relationship between the most recent operating conditions and the direction of change, calculates the changes in conditioned moisture content, screw speed, and die temperature for each transfer sample. in, For the first The change in conditioning moisture content of each transferred sample The conditioning moisture content is the value under current operating conditions. The conditioning moisture content is the moisture content of the previous operating condition; For the first The change in screw speed for each transferred sample The screw speed under the current operating conditions. The screw speed in the previous operating condition; For the first The change in the temperature of the mold head for each transferred sample The current die head temperature is... This refers to the die head temperature in the previous operating condition.

[0025] The three changes are input into the direction encoding function respectively: in, For direction encoding function, This represents any change in the input direction encoding function. When the change in quenched and tempered moisture content is negative, the quenched and tempered moisture content direction is decreasing; when the change is zero, it is maintaining; and when it is positive, it is increasing. The screw speed direction and die temperature direction are determined by the changes in screw speed and die temperature, respectively, according to the same rules.

[0026] The most recent operating condition transfer direction relationship consists of the states of quenched and tempered moisture content, screw speed, and die temperature, with each of these three states treated as an independent component. When comparing two transfer samples, the consistency of the states of quenched and tempered moisture content, screw speed, and die temperature is assessed separately. The overall consistency of the most recent operating condition transfer direction is determined by these three aspects of consistency.

[0027] To facilitate matrix operations, the deceleration direction state, the maintenance direction state, and the elevation direction state are represented in a fixed order as a matrix of length [length missing]. A single effective vector. The conditioned moisture content direction code corresponding to the conditioned moisture content direction state is denoted as... The screw speed direction code corresponding to the screw speed direction state is denoted as... The die head temperature direction code corresponding to the die head temperature direction state is denoted as The same one-bit valid vector is used for states in the same direction, while different one-bit valid vectors are used for states in different directions. Therefore, the dot product of two direction codes can directly indicate whether the states in the corresponding directions are consistent.

[0028] The orientation state is based on the increase / decrease relationship set in the original process. When the change in moisture content, screw speed, or die temperature is zero, the orientation state is maintained; zero is the criterion for maintaining the orientation state. This orientation state rule is used in both the training and application phases.

[0029] The sample relationship matrix construction module 114 first establishes three-way state consistency among the training transition samples. For the training transition samples... and training transfer samples Calculate separately: in, The values ​​for conditioning moisture content are in the same direction. and Training transfer samples and training transfer samples The direction code for the conditioned moisture content; The screw rotation speed is in the same direction. and These are the corresponding screw rotation direction codes; The die head temperature is in the same direction. and These are the corresponding die head temperature direction codes. When the corresponding direction states are consistent, the corresponding same-direction value is... When the states in the corresponding directions are inconsistent, the corresponding values ​​in the same direction are... .

[0030] Arrange the conditioning moisture content direction codes of all training transfer samples in rows to form Arrange the screw rotation direction code in rows to form Arrange the mold head temperature direction codes in rows to form Then the three matrices in the same direction can be written as follows: , and All three matrices in the same direction are Gram matrices.

[0031] make The three unidirectional matrices form the threshold values ​​for conditioned moisture content, screw speed, and die temperature, respectively: in, It is a matrix of all ones with the same dimension as the matrix in the same direction. Composed of the conditioning moisture content threshold value, Composed of screw speed and threshold value, It consists of the die head temperature threshold. The threshold values ​​for tempering moisture content, screw speed, and die head temperature are considered to be consistent when their corresponding directions are aligned. When the states in the corresponding directions are inconsistent, it is The threshold values ​​for conditioned moisture content, screw speed, and die temperature are all directly determined by the consistency of the state in the corresponding direction.

[0032] The three matrices above are multiplied by Hadamard to form a three-axis kernel gate: in, It is a matrix composed of three-axis kernel gates. This indicates element-wise multiplication. Matrix In For training transfer samples With training transfer samples The triaxial kernel gate value between them. When the states in all three directions are consistent, the triaxial kernel gate value is... When there is an inconsistency in one direction state, the triaxial kernel gate value is... When two directional states are inconsistent, the triaxial kernel gate value is... When the states in all three directions are inconsistent, the triaxial kernel gate value is... Therefore, the triaxial kernel threshold characterizes the degree of consistency between the most recent operating condition transfer directions of two transfer samples.

[0033] Beyond directional relationships, the sample relationship matrix also preserves continuous similarity relationships between the current process vectors of two transfer samples, representing the degree of similarity in continuous parameter spaces. For training transfer samples... and training transfer samples The baseline core value is calculated based on the ordinary RBF core: in, As the benchmark value, For kernel width parameter, and Training transfer samples and training transfer samples Standardized current process vectors, This represents the L2 norm. The closer the standardized current process vectors are, the larger the baseline kernel value; the greater the difference between the standardized current process vectors, the smaller the baseline kernel value.

[0034] Multiplying the baseline kernel value by the triaxial kernel gate value yields the transfer kernel value: in, For training transfer samples With training transfer samples The transition kernel values ​​between samples are used. All transition kernel values ​​form the sample relationship matrix. The kernel-based regression model employs a support vector regression model, and the sample relationship matrix serves as the pre-calculated kernel matrix for this model. Each sample relationship in this kernel matrix is ​​determined by continuous similarity relationships and a triaxial kernel gate value.

[0035] The Gram matrix formed by a regular RBF kernel is a positive semi-definite matrix, and the three matrices in the same direction are Gram matrices. Since the matrix is ​​a positive semi-definite matrix, the matrices composed of the kernel gate values ​​for conditioned moisture content, screw speed, and die temperature are all positive semi-definite matrices. According to the Schur product theorem, the Hadamard product of a positive semi-definite matrix is ​​also a positive semi-definite matrix. The matrix composed of the three-axis kernel gate values ​​and the kernel matrix formed by this matrix and the ordinary RBF kernel both satisfy the requirement for using a pre-calculated kernel matrix in a support vector regression model.

[0036] The reinforcement learning hyperparameter determination module 115 evaluates different hyperparameter states using validation transfer samples. For validation transfer samples... and training transfer samples Using the three direction codes of the validation transfer samples and the three direction codes of the training transfer samples, the same-direction values ​​of the conditioning moisture content, screw speed, and die temperature were calculated respectively: in, , and These represent the validation transfer samples. With training transfer samples Consistency across the three directional states. Calculate the three-axis kernel gate value from the verification transfer sample to the training transfer sample based on the three directional values: in, To verify the transferred sample With training transfer samples The three-axis kernel gate values ​​between. Then, a validation cross-kernel matrix is ​​formed based on the standardized current process vectors of the validation transfer samples and the training transfer samples: in, To verify the transferred sample Transfer to training samples The transfer value of the core. To verify the transferred sample Standardized current process vectors, For training transfer samples The standardized current process vector is used. The standardized particle bulk density supervision labels for the validation transfer samples are determined using the mean and standard deviation of particle bulk density from the training standardized statistics set. A support vector regression model is trained using the kernel matrix between the training transfer samples. The validation cross-kernel matrix is ​​then input into the trained support vector regression model to obtain the prediction results for each validation transfer sample at the standardized particle bulk density scale. The validation set RMSE at the standardized particle bulk density scale is used as the validation error. Validation error. Single-step reward and action value All are dimensionless quantities.

[0037] This implementation uses Q-learning to determine the penalty coefficient and kernel width parameter of the support vector regression model. The discrete candidate sets for the penalty coefficient and the discrete candidate sets for the kernel width parameter are represented as follows: in, For the candidate set of penalty coefficients, to The penalty coefficients are sorted in ascending order. The number of candidates for the penalty coefficient; For the candidate set of kernel width parameters, to The kernel width parameters are arranged in ascending order. This represents the number of candidate kernel width parameters. The standardized current process vector and standardized particle bulk density monitoring labels are both dimensionless, and the penalty coefficient and kernel width parameter are also set as dimensionless. Each set of penalty coefficients and kernel width parameters corresponds to a hyperparameter state.

[0038] No. The hyperparameter state of the step is represented as: in, For the first The hyperparameter state of the step, The index of the current penalty coefficient within the candidate set of penalty coefficients. This represents the index of the current kernel width parameter within the kernel width parameter candidate set. Hyperparameter tuning actions are arranged in a fixed order: kernel width parameter index incremented by one, penalty coefficient index incremented by one, kernel width parameter index decremented by one, penalty coefficient index decremented by one, and then maintaining the current index. The executable hyperparameter tuning actions for each hyperparameter state are those that, after execution, still satisfy... and The action consists of [the following steps]. After performing the hyperparameter adjustment action, the next hyperparameter state is obtained. .

[0039] For hyperparameter states The support vector regression model is trained according to its corresponding penalty coefficient and kernel width parameter, and the validation error is calculated. After performing hyperparameter tuning, the next hyperparameter state is... Calculate verification error The single-step reward is represented as: in, For the first Step-by-step reward To verify the error before performing hyperparameter tuning, The value represents the verification error after performing hyperparameter adjustment. A single-step reward is positive when the verification error decreases, zero when the verification error remains constant, and negative when the verification error increases. Therefore, the evaluation of the hyperparameter adjustment action directly corresponds to the change in the verification error of particle bulk density prediction.

[0040] Action value is updated according to the following formula: in, In hyperparameter state Perform hyperparameter tuning actions The value of the action, To learn step length, For single-step rewards, This is the discount factor. For the next hyperparameter state The maximum action value of each hyperparameter adjustment action. The learning step size, discount factor, and exploration probability are all dimensionless quantities. Hyperparameter adjustment actions are performed according to... -greedy rule selection To explore probabilities, random sampling was performed using an MT19937 standard 32-bit Mason swirl pseudorandom number generator, with a random seed of [missing information]. Each step begins by obtaining a 32-bit unsigned integer. This integer is divided by 2 to the power of 32 to obtain a random number within a left-closed, right-open unit interval. This random number is then compared with the exploration probability. Upon entering the exploration selection phase, another 32-bit unsigned integer is obtained, and a random number is generated in the same way. The currently executable hyperparameter adjustment actions are then equally divided and mapped to left-closed, right-open unit intervals according to a fixed order. The hyperparameter adjustment action is determined by the sub-interval containing the random number. Upon entering the action value selection phase, an action is selected from the currently executable hyperparameter adjustment actions based on its value. If multiple currently executable hyperparameter adjustment actions have the same value, the hyperparameter adjustment action that appears first in the fixed order is selected. The second 32-bit unsigned integer is consumed only when entering the exploration selection phase.

[0041] Reinforcement learning execution Rounds, each round is executed Step 1. The initial hyperparameter state of the first round is the first step of the candidate set of penalty coefficients. The first term and the candidate set of kernel width parameters Item. Each round, item number... The first step uses the same action value update process as the remaining steps, and the update item includes the maximum action value of each currently executable hyperparameter adjustment action in the next hyperparameter state. Turn number The average number of rounds in the previous round The next hyperparameter state obtained from the first step is used as the starting hyperparameter state, and the round number is used to accumulate the number of execution steps. The action value is initialized to zero at the beginning of the first round and is retained continuously between rounds. The MT19937 standard 32-bit Mason spin pseudo-random number generator starts with a random seed at the beginning of the first round. Initialize once, and use the same pseudo-random number sequence in each round. Each step sequentially completes the selection of hyperparameter adjustment actions, determination of the next hyperparameter state, verification error calculation, and action value update.

[0042] For each hyperparameter state visited during reinforcement learning, the corresponding validation error is saved. When the kernel width parameter changes, the baseline kernel value is recalculated according to the new kernel width parameter and then multiplied with the fixed three-axis kernel gate value to form the kernel matrix. When the penalty coefficient changes, the support vector regression model is retrained using the new penalty coefficient. The insensitive bandwidth is used as a preset training parameter for the support vector regression model, set as a dimensionless quantity on a standardized granularity scale, and takes the same value for all hyperparameter states.

[0043] Finish After each round of hyperparameter state visits, the penalty coefficient and kernel width parameter with the minimum validation error are selected from the set of hyperparameter states actually visited during reinforcement learning: in, The final determined penalty coefficient, For the final determined kernel width parameter, To reinforce the learning of the set of hyperparameter states actually visited, Penalty coefficient and kernel width parameter The corresponding verification error. The hyperparameter states that have actually been accessed are sorted in ascending order of verification error, and the penalty coefficient and kernel width parameter of the first position are taken.

[0044] The kernel method regression model training module 116 merges the training transfer samples and validation transfer samples to train the final support vector regression model. The merged transfer samples retain the mean moisture content, standard deviation of moisture content, mean screw speed, standard deviation of screw speed, mean die temperature, standard deviation of die temperature, mean pellet bulk density, and standard deviation of pellet bulk density determined by the training transfer samples during the reinforcement learning phase, to maintain consistency between the final kernel matrix and the standardized scale corresponding to the final determined kernel width parameter. Each transfer sample is paired according to the production time sequence of the original fish feed extrusion history and forms a corresponding directional state according to the original process settings.

[0045] Based on the merged transfer samples, the conditioned moisture content direction code, screw speed direction code, and die temperature direction code are reconstructed. Three unidirectional matrices, the conditioned moisture content kernel gate, the screw speed kernel gate, the die temperature kernel gate, and the triaxial kernel gate are calculated. The final kernel width parameter is then used. Calculate the merged baseline kernel value, multiply the baseline kernel value by the triaxial kernel gate value to form the final kernel matrix, and apply the final determined penalty coefficient. Train a support vector regression model. The support vector regression model uses standardized granularity weight supervision labels.

[0046] After training, the support vector regression model, the transition sample indices corresponding to the support vectors, the final determined penalty coefficients, the final determined kernel width parameters, the preset insensitive bandwidth, the training standardized statistics set, and the conditioned moisture content direction codes, screw speed direction codes, and die temperature direction codes corresponding to the support vectors are saved. In the application phase, the above data is used to calculate the transition kernel values ​​from candidate operating conditions to support vectors, and the standardized prediction results output by the support vector regression model are restored to the candidate predicted bulk weights.

[0047] After the final support vector regression model is trained, a test cross-kernel matrix is ​​formed based on the test transfer samples and the merged transfer samples. The test cross-kernel matrix is ​​then input into the final support vector regression model to obtain the particle bulk density prediction results of the test transfer samples, which are used to record the prediction error of the model in subsequent production periods.

[0048] The candidate operating condition evaluation module 117 reads the current actual operating condition, the set of candidate operating conditions, and the target load. The current actual operating condition is represented as follows: in, Based on the current actual working conditions, The conditioning moisture content is the value under actual working conditions. The screw speed is the actual speed under the current operating conditions. This represents the current actual die head temperature. The current actual operating conditions are taken from the process settings currently being used or recently implemented on the production line.

[0049] The candidate operating conditions are set within the preset process allowable range of the fish feed extrusion process. The preset process allowable range includes the conditioning moisture content range, screw speed range, and die temperature range. Based on the production line's predetermined adjustable gears or predetermined distances, the process settings within these three ranges are combined to obtain each candidate operating condition. The candidate operating conditions are represented as follows: in, For the first One candidate working condition For the first The conditioning moisture content for each candidate operating condition. For the first Screw speed for each candidate operating condition For the first The die head temperature for each candidate working condition This is the candidate operating condition number.

[0050] Each candidate operating condition and the current actual operating condition together form a candidate transfer sample: in, For the first The arrow indicates the candidate transfer sample, representing the adjustment from the current actual operating condition to the [number]th [stage]. The candidate operating conditions are calculated separately. The changes in quenched and tempered moisture content, screw speed, and die temperature of each candidate operating condition relative to the current actual operating condition are analyzed. The directional states of the quenched and tempered moisture content, screw speed, and die temperature are determined according to the directional coding function used during the training phase, thus forming candidate transfer directional relationships. These candidate transfer directional relationships utilize the directional state category, directional code order, and consistency judgment rules used in the most recent operating condition transfer directional relationship.

[0051] The candidate operating conditions were standardized using the mean and standard deviation of the quenched and tempered moisture content, the mean and standard deviation of the screw speed, and the mean and standard deviation of the die temperature, all saved during the final training. and the training transition samples in the final support vector regression model The three-axis core gate value is calculated based on the three direction codes of both parties: in, Candidate operating conditions With training transfer samples The three-axis core gate values ​​between , and Candidate working conditions The directional codes for conditioned moisture content, screw speed, and die temperature. , and Training transfer samples The directional codes for conditioned moisture content, screw speed, and die temperature.

[0052] The transfer kernel value from candidate working conditions to training transfer samples is represented as: in, Candidate operating conditions Transfer to training samples The transfer value of the core. For the final determined kernel width parameter, Candidate operating conditions Standardized current process vectors, For training transfer samples The standardized current process vector is obtained. For each training transition sample required by the support vector regression model, the transition kernel value is calculated sequentially to form the pre-calculated kernel vector corresponding to the candidate operating condition.

[0053] The pre-calculated kernel vectors corresponding to the candidate working conditions are input into the trained support vector regression model to obtain standardized prediction results. Then, the mean and standard deviation of particle bulk density saved during final training are used for inverse standardization to obtain the candidate predicted bulk density. Each candidate operating condition corresponds to a candidate predicted load. The candidate predicted load is associated with the candidate operating condition number and is used by the next operating condition output module 118 to select the target candidate operating condition.

[0054] The next operating condition output module 118 calculates the target deviation between each candidate predicted bulk density and the target bulk density: in, For the first The target deviation of each candidate working condition For the first Candidate predicted bulk density for each candidate working condition The target bulk density is the smallest deviation from the target. This indicates that the candidate predicted bulk density for the corresponding candidate working condition is closer to the target bulk density.

[0055] The target candidate operating condition is determined according to the following formula: in, This represents the sequence number of the target candidate operating condition. The candidate operating condition with the smallest target deviation from the candidate operating condition set is selected as the target candidate operating condition. The quenched and tempered moisture content, screw speed, and die temperature of the target candidate operating condition are then used as the quenched and tempered moisture content, screw speed, and die temperature for the next operating condition. The final output is expressed as: in, For the next working condition, The conditioning moisture content for the next operating condition. The screw speed for the next operating condition. This sets the die head temperature for the next operating condition. The output of the next operating condition can be used as a direct basis for setting subsequent fish feed extrusion conditions.

[0056] Example 1: This embodiment illustrates the data flow and numerical calculations within a complete control cycle. The numerical values ​​used are used to illustrate the implementation process of the above method. The historical data of fish feed extrusion are from continuous operation of the same continuous production line. The conditioning moisture content is expressed as a percentage by mass, the screw speed is expressed as revolutions per minute, the die temperature is expressed as degrees Celsius, and the pellet density is expressed as kilograms per cubic meter.

[0057] The conditioning moisture content range used in the long-term operation of this production line is: to The screw speed range is to The die head temperature range is to The process card for the product corresponding to this control cycle sets the preset process allowable range as the conditioned moisture content. to Screw speed to and die head temperature The candidate operating condition set is based on the conditioned moisture content. Screw speed and die head temperature The predetermined distance from the starting point is formed, which includes four candidate operating conditions.

[0058] 181 valid fish feed extrusion history records were obtained from continuous production records, forming 180 transfer samples based on production time. The first 120 transfer samples were used as training transfer samples, the next 30 as validation transfer samples, and the last 30 as test transfer samples. Each transfer sample retained the previous operating condition, the current operating condition, and the corresponding pellet bulk density. The training standardized statistics set was determined from the first 120 training transfer samples.

[0059] Taking one of the training transfer samples as an example, the conditioning moisture content of the previous working condition was... The screw speed is The die head temperature is The conditioning moisture content under current operating conditions is The screw speed is The die head temperature is The particle bulk density corresponding to the current operating condition is The change in moisture content of the transferred sample was [amount missing]. The change in screw speed is The change in die head temperature is Therefore, the state of the conditioned moisture content is decreasing, the state of the screw speed is increasing, and the state of the die temperature is increasing.

[0060] Following a fixed sequence of decreasing direction state, maintaining direction state, and increasing direction state, the conditioned moisture content direction code of this transferred sample uses the first digit as... The effective vector, the screw speed direction code and the die temperature direction code both use the third bit as A valid vector. Compare this transfer sample with another training transfer sample. If the three directional states of the other training transfer sample are also descending, ascending, and ascending in sequence, then all three directional values ​​are equal. The three-axis kernel gate value is If the only difference between the two is the direction of their conditioned moisture content, then the values ​​of their conditioned moisture contents in the same direction are... The screw speed and die temperature are in the same direction. The three-axis kernel gate value is .

[0061] The mean moisture content of the conditioning in the training transfer samples was The standard deviation of the conditioning moisture content is The average screw speed is The standard deviation of screw speed is The average temperature of the die head is The standard deviation of the die head temperature is The average particle bulk density is The standard deviation of particle bulk density is The aforementioned standardized conditioning moisture content under current operating conditions is... The screw speed is The die head temperature is The same processing was performed on each training transfer sample to obtain a standardized current process vector and a standardized particle bulk density supervision label. The validation transfer samples were processed using the training standardized statistics set.

[0062] The support vector regression model uses a pre-computed kernel matrix, and the insensitive bandwidth is set to... The candidate set of penalty coefficients is selected. The candidate set for the kernel width parameter is taken Q-learning's learning step size Discount factor is taken Explore probability The initial value of the action is zero. Reinforcement learning execution. Rounds, each round is executed Step 1, the initial hyperparameter state of the first round is the penalty coefficient. and kernel width parameter The remaining rounds are from the previous round. The next hyperparameter state obtained in the first step is used as the starting hyperparameter state. The random seed of the MT19937 standard 32-bit Mason slew pseudorandom number generator is... Each actual accessed hyperparameter state was evaluated using the same training and validation transition samples, with validation error calculated on a normalized particle bulk density scale.

[0063] Reinforcement learning starts with a penalty coefficient of The kernel width parameter is The hyperparameter state begins, and the verification error of this hyperparameter state is... The first 32-bit unsigned integer output by the MT19937 standard 32-bit Mason rot pseudo-random number generator is... The corresponding random number is Less than the probability of exploration Therefore, we proceed to the exploration selection; the second 32-bit unsigned integer output is... The corresponding random number is The value falls into the third sub-interval of the five currently executable hyperparameter tuning intervals, therefore the hyperparameter tuning action of decrementing the kernel width parameter index by one is executed. The kernel width parameter becomes... Then, the verification error of the next hyperparameter state is The single-step reward is The updated corresponding action value is The second step uses the next 32-bit unsigned integer. The corresponding random number is The hyperparameter is adjusted by incrementing the kernel width parameter index by one according to the action value, and the kernel width parameter is returned. The hyperparameter state, the single-step reward is The third step is to use the next 32-bit unsigned integer. The corresponding random number is The hyperparameter is adjusted by adding one to the kernel width parameter index according to the action value, and the penalty coefficient is... The kernel width parameter is The hyperparameter state; the verification error of this hyperparameter state is The single-step reward is Subsequent steps continue to use the same pseudo-random number sequence to select hyperparameter tuning actions.

[0064] Reinforcement learning completed After the round, the set of hyperparameter states actually visited includes One hyperparameter state: the penalty coefficient is... And the kernel width parameters are respectively , , and The four hyperparameter states have a penalty coefficient of . And the kernel width parameters are respectively , , and The four hyperparameter states, and the penalty coefficient are And the kernel width parameters are respectively , and The three hyperparameter states. To replicate the verification environment of this embodiment, the verification errors for the twenty hyperparameter states are listed together. The kernel width parameter is according to... , , , , When arranged in order, the penalty coefficient is The five verification errors are as follows: , , , , The penalty coefficient is The five verification errors are as follows: , , , , The penalty coefficient is The five verification errors are as follows: , , , , The penalty coefficient is The five verification errors are as follows: , , , , In the actually accessed hyperparameter states, the penalty coefficient is... The kernel width parameter is The hyperparameter state has the minimum verification error Therefore, the final determined penalty coefficient for The final determined kernel width parameter for The training and validation transfer samples are merged, and the standardized statistics set used in training is reused to form the final kernel matrix based on the finally determined kernel width parameter. The final support vector regression model is then trained using the finally determined penalty coefficient.

[0065] The final support vector regression model contains three support vectors. The standardized current process vector of the first support vector is... The states of the conditioned moisture content direction, the screw speed direction, and the die temperature direction are, respectively, decreasing, increasing, and increasing, with a dual coefficient of . The normalized current process vector of the second support vector is... The three directional states are, in order, the hold direction state, the rise direction state, and the rise direction state, with dual coefficients of... The standardized current process vector of the third support vector is: The three directional states are, in order, decreasing, maintaining, and increasing, with dual coefficients of... The intercept of the support vector regression model is... The dual coefficients and intercepts mentioned above act on the standardized particle bulk density scale, and the order of the support vectors also serves as the element order of the pre-computed kernel vectors.

[0066] At the start of a control cycle, the current actual operating conditions are read. The current actual operating condition's conditioning moisture content is... The screw speed is The die head temperature is The target bulk density is Based on the preset process allowable range and predetermined deviation length of this control cycle, four candidate operating conditions are formed, and the candidate predicted bulk density is calculated for each of the four candidate operating conditions.

[0067] The conditioning moisture content of the first candidate operating condition is The screw speed is The die head temperature is The corresponding three directional states are, in order, decreasing, increasing, and increasing. The conditioning moisture content of the second candidate operating condition is... The screw speed is The die head temperature is The corresponding three directional states are, in order, maintaining the directional state, increasing the directional state, and increasing the directional state. The conditioning moisture content of the third candidate working condition is... The screw speed is The die head temperature is The corresponding three directional states are, in order, decreasing, maintaining, and increasing. The conditioning moisture content of the fourth candidate working condition is... The screw speed is The die head temperature is The three corresponding directional states are, in order, maintaining the directional state, maintaining the directional state, and rising the directional state.

[0068] For the first candidate operating condition, its standardized current process vector is obtained using the trained standardized statistics set. The squared distance between the first candidate operating condition and the first support vector on the standardized current process vector is: The three directions are consistent, and the baseline kernel value and the transfer kernel value are both... Following the same calculation process, the pre-calculated kernel vectors corresponding to the first candidate working condition are as follows: The pre-calculated kernel vectors corresponding to the second candidate working condition are as follows: The pre-calculated kernel vectors corresponding to the third candidate working condition are as follows: The pre-calculated kernel vectors corresponding to the fourth candidate working condition are as follows: .

[0069] The four pre-computed kernel vectors are sequentially input into the final support vector regression model. The support vector regression model, following the order of the three pre-computed kernel vector elements, multiplies each element by its corresponding dual coefficient, sums the results, and then adds the intercept, yielding four standardized prediction results, in the following order: , , and The average particle bulk density was used. and particle bulk density standard deviation After destandardization, the candidate predicted bulk density for the first candidate load case is: The candidate predicted bulk density for the second candidate operating condition is The candidate predicted bulk density for the third candidate working condition is The candidate predicted bulk density for the fourth candidate working condition is .

[0070] Target bulk density is The target deviations for the four candidate operating conditions are as follows: , , and The first candidate operating condition has the smallest target deviation in the complete set of candidate operating conditions. Therefore, the first candidate operating condition is selected as the target candidate operating condition, and the conditioning moisture content of the next operating condition is output. The screw speed is The die head temperature is Thus, one control cycle is completed.

[0071] Example 3; This embodiment uses five historical records of fish feed extrusion arranged by production time to illustrate the complete recalculation process from the original records to the support vector regression model and the output of the next working condition.

[0072] The first historical record of fish feed extrusion shows the following parameters in order: conditioning moisture content, screw speed, die temperature, and pellet density. , , and The second item is as follows: , , and The third item is as follows: , , and Article 4 is as follows: , , and Article 5 is as follows: , , and .

[0073] The first training transfer sample was formed from the first and second historical records of fish feed extrusion. The second training transfer sample was formed from the second and third historical records. The validation transfer sample was formed from the third and fourth historical records. The test transfer sample was formed from the fourth and fifth historical records. The conditioning moisture content, screw speed, and die temperature of the two training transfer samples were all in an increasing direction; the validation and test transfer samples were all in a decreasing direction. The original particle bulk density monitoring label for each transfer sample was taken from the particle bulk density corresponding to its current operating condition.

[0074] A set of standardized training statistics was calculated using the current operating conditions and original particle bulk density supervision labels of two training transfer samples. The standard deviation was calculated using the number of training transfer samples as the denominator. The mean and standard deviation of the conditioned moisture content were respectively... and The mean screw speed and the standard deviation of screw speed are respectively and The mean and standard deviation of the die head temperature are respectively and The mean and standard deviation of particle bulk density are respectively and Following the aforementioned standardization method, the standardized current process vector of the first training transfer sample is: Standardized particle bulk density monitoring label is The standardized current process vector of the second training transfer sample is: Standardized particle bulk density monitoring label is The standardized current process vector for verifying the transfer sample is: Standardized particle bulk density monitoring label is The standardized current process vector for the test transfer sample is: Standardized particle bulk density monitoring label is .

[0075] Take the penalty coefficient as The kernel width parameter is Insensitive bandwidth is The three-directional states of the two training transfer samples are identical, and the three-axis kernel gate value between them is... The squared distance between two standardized current process vectors is The corresponding transfer kernel value is The transfer kernel value between each training transfer sample and itself is The kernel matrix composed of the four transition kernel values ​​is input into the support vector regression model to obtain two support vectors, whose standardized current process vectors are as follows: and The dual coefficients are as follows: and The intercept is Substituting the dual coefficients into the kernel matrix, the standardized prediction results for the two training transition samples are as follows: and The deviations from the two standardized particle bulk density monitoring labels are both insensitive bandwidths. .

[0076] The validation transfer sample is inconsistent with the three-directional states of the two training transfer samples, and both three-axis kernel gate values ​​are [missing information]. The pre-computed kernel vector for verifying the transfer sample to two training transfer samples is: After inputting into the support vector regression model, standardized prediction results are obtained. The verification error is The pre-computed kernel vectors for transferring test samples to two training samples are: Standardized prediction results were obtained. The predicted particle bulk density after destandardization is The difference in particle bulk density between the sample and the test transfer sample is Therefore, the standardized statistical set, orientation state, kernel matrix, support vector regression model, validation error, and test error can be recalculated sequentially from the five fish feed extrusion history records.

[0077] Using the operating conditions from the fifth historical record of fish feed extrusion as the current actual operating conditions, the target bulk density is... The preset process allowable range for this control cycle fixes the screw speed at a certain value. The die head temperature is fixed at The conditioning moisture content was determined to be... or This results in two candidate operating conditions. The first candidate operating condition has the following parameters: quenched and tempered moisture content, screw speed, and die temperature, respectively. , and The second candidate operating conditions are as follows: , and .

[0078] The first candidate working condition has three directional states that are decreasing relative to the current actual working condition, and the corresponding pre-calculated kernel vector is... The standardized prediction result is Candidate predicted bulk density is The target deviation is The second candidate operating condition has an increasing moisture content direction, while the screw speed and die temperature directions are both decreasing. The corresponding pre-calculated kernel vector is... The standardized prediction result is Candidate predicted bulk density is The target deviation is The second candidate operating condition has a smaller target deviation, so it is selected as the target candidate operating condition, and the conditioning moisture content for the next operating condition is output as follows: The screw speed is The die head temperature is .

[0079] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A fish feed puffing parameter tuning method based on reinforcement learning, characterized in that, include: Obtain historical data on fish feed extrusion, sorted by production time and including conditioning moisture content, screw speed, die temperature, and pellet bulk density; current actual operating conditions; candidate operating conditions; and target bulk density. The current actual operating conditions and each candidate operating condition include conditioning moisture content, screw speed, and die temperature. A transfer sample containing the previous working condition, the current working condition, and the corresponding particle bulk density is formed from two adjacent fish feed extrusion history records. The relationship between the most recent working condition transition direction is formed based on the changes in conditioned moisture content, screw speed, and die temperature from the previous working condition to the current working condition. Construct a sample relationship matrix, which includes a continuous similarity relationship between the current process vectors of two transferred samples to characterize the similarity of the continuous parameter space and the relationship of the most recent working condition transfer direction. The current process vector consists of the quenching and tempering moisture content, screw speed and die temperature. Hyperparameters of a kernel-based regression model for predicting particle bulk density, trained on a sample relationship matrix, were determined using reinforcement learning. Candidate transfer direction relationships from the current actual working condition to each candidate working condition are formed according to the same direction state rules as those used to form the most recent working condition transfer direction relationship, and candidate predicted bulk weights are obtained by the trained kernel method regression model. Based on the relationship between the candidate predicted bulk density and the target bulk density, the quenched moisture content, screw speed and die temperature for the next working condition are output.

2. The fish feed puffing parameter optimization method based on reinforcement learning according to claim 1, characterized in that, The most recent working condition transfer direction relationship consists of the conditioned moisture content direction state, the screw speed direction state, and the die head temperature direction state. The conditioned moisture content direction state, the screw speed direction state, and the die head temperature direction state are all one of the following: a decreasing direction state, a maintaining direction state, or an increasing direction state.

3. The method for optimizing fish feed extrusion parameters based on reinforcement learning according to claim 2, characterized in that, When the change in the conditioned moisture content, the change in the screw speed, and the change in the die temperature are negative, they correspond to a decreasing direction; when they are zero, they correspond to a maintaining direction; and when they are positive, they correspond to an increasing direction.

4. The fish feed puffing parameter optimization method based on reinforcement learning according to claim 2, characterized in that, When constructing the sample relationship matrix, the consistency of the conditioning moisture content direction, the consistency of the screw speed direction, and the consistency of the die temperature direction of the two transferred samples are determined respectively, and the three consistency terms form a triaxial core gate value to characterize the comprehensive consistency of the most recent working condition transfer direction of the two transferred samples.

5. The fish feed extrusion parameter tuning method based on reinforcement learning according to claim 4, characterized in that, The kernel regression model is a support vector regression model, the sample relationship matrix is ​​the kernel matrix of the support vector regression model, the triaxial kernel gate participates in forming the sample relationship in the kernel matrix, and together with the continuous similarity relationship between the current process vector, the sample relationship is determined.

6. The fish feed puffing parameter optimization method based on reinforcement learning according to claim 1, characterized in that, When determining the hyperparameters of the kernel method regression model through reinforcement learning, the hyperparameter adjustment action is selected based on the change in verification error before and after the hyperparameter adjustment action, and the next hyperparameter state is determined based on the hyperparameter adjustment action.

7. The fish feed extrusion parameter optimization method based on reinforcement learning according to claim 1, characterized in that, The set of candidate operating conditions is formed within the preset process allowable range of the fish feed extrusion process, which includes the conditioning moisture content range, screw speed range, and die temperature range.

8. The method for optimizing fish feed extrusion parameters based on reinforcement learning according to claim 1, characterized in that, selecting a candidate operating condition closest to the target bulk density as a target candidate operating condition from the candidate operating condition set, and setting the conditioning moisture content, screw rotation speed and die temperature of the target candidate operating condition as the conditioning moisture content, screw rotation speed and die temperature of the next operating condition.