Intelligent control method and device of flow field simulation system, storage medium and computer equipment

Abstract

The application relates to the technical field of ocean engineering experiments, and specifically provides an intelligent control method and device of a flow field simulation system, a storage medium and computer equipment. The application establishes a control relationship between a flow field state and a flow generating component (an electric valve and a water pump) through a DQN algorithm, realizes automation and intelligentization of a flow field direction adjustment process. The DQN algorithm uses a Q value neural network to construct a Q value table, so that flow field simulation becomes more accurate and simple in direction adjustment and flow rate adjustment. On the basis of improving self-adaptability, the flow field simulation system has self-updating function and intelligent decision-making ability, greatly improves valve adjustment efficiency and experiment efficiency, reduces technical popularization difficulty, and provides a more accurate and safer flow field simulation environment for ocean engineering experiments.

Description

Technical Field

[0001] This application relates to the field of marine engineering experimental technology, and in particular to an intelligent control method, device, storage medium and computer equipment for a flow field simulation system. Background Technology

[0002] In marine engineering model testing, to accurately verify the effectiveness of the design, it is usually necessary to conduct model verification in a laboratory simulating the marine environment. Among these steps, ocean flow field simulation is a crucial one. During flow field simulation, the flow field simulation system drives the water body to circulate and form a flow to create a flow field within a flow-generating pool. The flow direction is generally from the outlet at one end of the flow-generating pool to the outlet at the other end. Due to the limitations imposed by the outlet arrangement, the flow field direction is singular, which greatly restricts the experimental testing capabilities.

[0003] To simulate ocean flow fields in different directions, the marine engineering laboratory is equipped with a flow field simulation system with multiple valves controlling the outlet. Although the system still uses a water pump to generate flow, it can simulate multi-directional flow fields by adjusting the opening and closing degrees of different valves, based on the controllability of the outlet valves, thus meeting the simulation requirements of experimental model tests for multi-directional flow fields.

[0004] However, current valve control requires experimenters to rely on experience to adjust the opening and closing of each valve to regulate the flow direction. This process is heavily dependent on operator experience, resulting in low operational efficiency and making it difficult to standardize operational skills due to the transmission of experience, significantly increasing the difficulty of widespread adoption. Summary of the Invention

[0005] The purpose of this application is to at least solve one of the above-mentioned technical defects, especially the technical defect that the opening and closing adjustment of valves in the prior art relies heavily on the experience of operators, which not only results in low operating efficiency, but also makes it difficult to standardize operating skills due to the inheritance of experience, which significantly increases the difficulty of promotion.

[0006] In a first aspect, embodiments of this application provide an intelligent control method for a flow field simulation system. The flow field simulation system includes: a flow-generating pool, a water pump device, multiple electric valves installed at the outlet of the flow-generating pool, and multiple flow meters installed in the experimental testing area of ​​the flow-generating pool; the method includes:

[0007] The opening and closing angles of the plurality of electric valves are set to initial angles, and the rotational speed of the water pump device is set to initial rotational speed;

[0008] A state space is constructed using the actual flow field data collected by the multiple flow meters, and an action space is constructed using the rotational speed of the water pump and the opening and closing angles of the multiple electric valves. The rotational speed of the water pump and the opening and closing angles of the multiple electric valves are adjusted based on a deep Q-network algorithm until a stopping condition is met. The stopping condition is that the difference between the actual flow field in the experimental test area and the predetermined target flow field is less than a preset threshold.

[0009] In some embodiments, adjusting the rotational speed of the water pump and the opening / closing angle of the plurality of electric valves based on the deep Q-network algorithm includes:

[0010] The actual flow field data at the current moment is collected using the multiple flow meters;

[0011] Determine whether the stopping condition is met based on the actual flow field data.

[0012] If not satisfied, then construct the current state space s based on the actual flow field data at the current moment. t and the current state space s t The data is input into a Q-value neural network to obtain the output data of the Q-value neural network; wherein, the Q-value neural network is a neural network trained based on the empirical replay mechanism of the deep Q-network algorithm;

[0013] Based on the output data and the preset action selection strategy, the current action space a is determined. t ;

[0014] According to the current action space a t Adjust the rotational speed of the water pump device and the opening and closing angles of the multiple electric valves, and after a preset time interval, perform the step of collecting the actual flow field data at the current moment through the multiple flow meters.

[0015] In some embodiments, the preset duration is the sum of a preset fixed delay duration and a preset flow field stabilization duration.

[0016] In some embodiments, the reward function of the deep Q-network algorithm includes a task reward, a constraint reward, and a smoothing reward; wherein, the task reward... for:

[0017]

[0018] In the formula, As the first weighting coefficient, This is the second weighting coefficient. Let x be the flow velocity value collected by the x-th flow meter. The velocity value of the target flow field. Let x be the flow field vector angle collected by the x-th current meter. Let x be the flow field vector angle of the target flow field, and let x take values ​​in the range of [1, M], where M is the number of flow meters;

[0019] The constraint reward for:

[0020]

[0021] In the formula, The third weighting coefficient, It is the fourth weighting coefficient. For conditional judgment, All electric valves located on the first side of the water-generating pool are closed. All electric valves located on the second side of the water-generating pool are closed. The rotational speed of the water pump device;

[0022] The smooth reward for:

[0023]

[0024] In the formula, This is the fifth weighting coefficient. It is the sixth weighting coefficient. Let y be the difference in opening and closing angles of the y-th electric valve at adjacent times, where y ranges from [1, N] and N is the number of electric valves; The difference in rotational speed of the water pump device at adjacent moments.

[0025] In some embodiments, the first weighting coefficient, the second weighting coefficient, the third weighting coefficient, the fourth weighting coefficient, and the fifth weighting coefficient are set according to a preset task priority and / or the difference between the actual flow field and the target flow field.

[0026] In some embodiments, the step of setting the weighting coefficients includes:

[0027] If the difference between the velocity value of the actual flow field and the velocity value of the target flow field is greater than a preset velocity threshold, and the difference between the flow field vector angle of the actual flow field and the flow field vector angle of the target flow field is less than a preset vector angle threshold, then the first weighting coefficient is increased.

[0028] If the difference between the velocity value of the actual flow field and the velocity value of the target flow field is less than a preset velocity threshold, and the difference between the flow field vector angle of the actual flow field and the flow field vector angle of the target flow field is greater than a preset vector angle threshold, then the second weighting coefficient is increased.

[0029] If the rotational speed of the water pump device is higher than the preset rotational speed threshold, the fourth weighting coefficient is increased.

[0030] In some embodiments, the state space may also include water level information of the water-generating pool.

[0031] Secondly, embodiments of this application provide an intelligent control device for a flow field simulation system. The flow field simulation system includes: a flow-generating pool, a water pump device, multiple electric valves installed at the outlet of the flow-generating pool, and multiple flow meters installed in the experimental testing area of ​​the flow-generating pool; the device includes:

[0032] An initialization module is used to set the opening and closing angles of the plurality of electric valves to initial angles and the rotational speed of the water pump device to initial rotational speed.

[0033] The intelligent control module is used to construct a state space based on the actual flow field data collected by the multiple flow meters, and to construct an action space based on the rotational speed of the water pump device and the opening and closing angles of the multiple electric valves. The module adjusts the rotational speed of the water pump device and the opening and closing angles of the multiple electric valves based on a deep Q-network algorithm until a stopping condition is met. The stopping condition is that the difference between the actual flow field in the experimental test area and the predetermined target flow field is less than a preset threshold.

[0034] Thirdly, embodiments of this application provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the intelligent control method for the flow field simulation system described in any of the above embodiments.

[0035] Fourthly, embodiments of this application provide a computer device, which includes: one or more processors, and a memory;

[0036] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the intelligent control method for the flow field simulation system described in any of the above embodiments.

[0037] As can be seen from the above technical solutions, the embodiments of this application have the following technical effects:

[0038] This application establishes the control relationship between the flow field state and the flow-generating components (electric valves and water pumps) using the DQN (Deep Q-Network) algorithm, realizing the automation and intelligence of the flow field direction adjustment process. The DQN algorithm uses a Q-value neural network to construct a Q-value table, making flow field simulation more accurate and convenient in terms of direction and velocity adjustment. While improving adaptability, it also enables the flow field simulation system to have self-updating capabilities and intelligent decision-making abilities, significantly improving valve adjustment efficiency and experimental efficiency, and reducing the difficulty of technology promotion. This provides a more accurate and safer flow field simulation environment for marine engineering experiments. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 Here is a system architecture diagram of the flow field simulation system in some embodiments;

[0041] Figure 2 This is a flowchart illustrating the intelligent control method of the flow field simulation system in some embodiments;

[0042] Figure 3 This is a structural block diagram of the intelligent control device of the flow field simulation system in some embodiments;

[0043] Figure 4 This is a diagram of the internal structure of a computer device in some embodiments. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] In some embodiments, this application provides an intelligent control method for a flow field simulation system. This method automatically adjusts the opening and closing angles of electric valves and the rotational speed of water pumps in the flow field simulation system using the DQN algorithm, thereby achieving automated and intelligent flow field regulation. This significantly reduces the involvement of experimental personnel in the flow field regulation process, thereby improving valve regulation efficiency and ultimately enhancing the experimental efficiency of marine engineering. Furthermore, the solution of this application allows flow field regulation to be achieved by running a flow-generating control program on a computer device such as a personal laptop PC, and provides an experimental environment for the target flow field in a flow-generating pool. This operation is simple, convenient, has low learning costs, and a low learning threshold, greatly reducing the difficulty of promoting flow-generating technology. Therefore, this application provides a more accurate and safer flow field simulation environment for marine engineering experiments.

[0046] It should be noted that the intelligent control method provided in this application can be applied to a flow field simulation system equipped with electric valves and water pumps. This system can be used to simulate flow fields in any direction. Specifically, the flow field simulation system may include a flow-generating pool, a water pump device, multiple electric valves installed at the outlet of the flow-generating pool, and multiple flow meters installed in the experimental testing area of ​​the flow-generating pool. The electric valves can adjust their opening and closing angles according to instructions issued by the computer equipment, and the water pump device can adjust its pump speed according to instructions issued by the computer equipment.

[0047] It should be noted that the flow field simulation system can be implemented with any architecture, and the electric valves and flow meters can be arranged in any way. This application does not impose any specific restrictions on this.

[0048] In some embodiments, Figure 1 A large-scale flow field simulation system in any direction is provided. The system includes: a flow pool, valve control cabinet A, valve control cabinet B, a switch, a PC, multiple electric water pumps and water pump devices, and the water pump devices may include water pumps and frequency converters.

[0049] The PC serves as the main control terminal of the flow field simulation system and can be connected to a switch. The switch connects to valve control cabinet A, valve control cabinet B, and a frequency converter. The frequency converter connects to the water pumps. Valve control cabinet A can connect to the electric water pumps located on the first side of the flow-generating pool, and valve control cabinet B can connect to the electric water pumps located on the second side of the flow-generating pool. The first and second sides can be arranged opposite each other. Each electric valve can be equipped with a high-precision optical encoder to record the current opening and closing angle of the electric valve.

[0050] In this embodiment, a switch is used to connect the PC to the valve control cabinet and the frequency converter. This allows the PC to send operating commands to the valve control cabinet and the frequency converter via the switch, thereby controlling the opening and closing angles of each electric valve and the speed and direction of the water pump, thus regulating the flow field in the water tank.

[0051] It should be noted that in this embodiment, the switch can interconnect devices through wired and / or wireless connections, and this application does not impose specific limitations on this. In some examples, the switch can be an Ethernet switch, and various devices can communicate with each other via Ethernet.

[0052] Multiple flowmeters can be installed in the experimental testing area of ​​the flow tank to measure the actual flow field at different locations within the testing area. The experimental testing area refers to the region within the flow tank used to place the model to be tested.

[0053] It is understood that multiple flowmeters can be configured in any way, and any type of flowmeter can be used; this application does not impose any specific restrictions in this regard. In some examples, multiple flowmeters can be arranged in a matrix within the actual test area of ​​the flowing water tank to uniformly test the actual flow field at different locations, so as to simulate the target flow field at different locations in the experimental test area. In other examples, acoustic Doppler current meters (ADVs) can be used as flowmeters.

[0054] Based on the flow field simulation system provided in the above embodiments, the intelligent control method of the flow field simulation system provided in this application will be described below.

[0055] In some embodiments, this application provides an intelligent control method for a flow field simulation system, which is used to automatically and intelligently adjust the opening and closing angle of the electric valve and the rotation speed of the water pump device in the flow field simulation system, so as to automatically adjust the flow field in the water-generating pool.

[0056] The following embodiments illustrate the application of this method to a computer device. By running a flow control program (i.e., an agent), the computer device executes the steps provided in the following embodiments, thereby using the DQN algorithm in reinforcement learning to learn and train the control system from samples, and achieving regulation based on the DQN algorithm. It is understood that the computer device described in this application can be a device with data acquisition, data processing, and command issuance functions, and can be, but is not limited to, a PC, desktop computer, laptop computer, computer, server, etc.

[0057] like Figure 2 As shown, the method provided in this application may include the following steps:

[0058] S210: Set the opening and closing angles of multiple electric valves to the initial angles, and set the rotational speed of the water pump device to the initial rotational speed;

[0059] S220: The state space is constructed using actual flow field data collected by multiple flow meters, and the action space is constructed using the rotational speed of the water pump and the opening and closing angles of multiple electric valves. The rotational speed of the water pump and the opening and closing angles of multiple electric valves are adjusted based on the deep Q-network algorithm until the stopping condition is met. The stopping condition is that the difference between the actual flow field in the experimental test area and the predetermined target flow field is less than a preset threshold.

[0060] Specifically, in the intelligent control process, the device initialization procedure is executed first. During this procedure, the opening and closing angles of each electric valve are set to initial angles, and the pump speed is set to initial speed. It can be understood that the specific values ​​of the initial angle and initial speed can be determined based on the actual situation. In some examples, the initial angle can be fully open, and the initial speed can be zero. That is, in the initialization procedure, each electric valve is adjusted to a fully open state, and the pump speed is set to zero to avoid disrupting the initial flow field of the water tank.

[0061] Furthermore, to simplify the control process, this application can set the direction of the water pump device according to the direction of the target flow field, so that the direction of the water pump device is consistent with the flow field direction of the target flow field.

[0062] After completing the device initialization process, this application can adjust the pump speed and the opening and closing angles of each electric valve based on the Deep Q-Network (DQN) algorithm until the difference between the actual flow rate and the target flow field in the experimental test area of ​​the flow tank is less than a preset threshold. The target flow rate refers to the flow field that the experimenters need to achieve in the experimental test area.

[0063] The DQN algorithm is a fusion of deep learning and reinforcement learning. Its core principle is to approximate the Q-value function using a deep neural network (i.e., a Q-value neural network). In application, the input data to the deep neural network is the current state space, and the output data is the Q-value for each action. The Q-value represents the expected cumulative reward for performing that action in a given state. Based on the output data of the deep neural network, the action space that yields the maximum reward in the current state space can be determined, thus identifying the actions to be performed.

[0064] This demonstrates that the DQN algorithm involves both a state space and an action space. This application sets the state space to the actual flow field data collected by multiple flowmeters, and the action space to the rotational speed of the water pump and the opening and closing angles of multiple electric valves. For example, when setting up Q-Learning reinforcement learning, this application sets the state space of the agent as follows: ,in This refers to the actual flow field data collected by the x-th flow meter, where x ranges from 1, 2, ..., m, and m is the number of flow meters. The action space can be an N+1 dimensional vector, specifically... Among them, D y Let be the opening / closing angle of the y-th electric valve, and N be the number of electric valves in the flow field simulation system. Valve codes 1~S represent the electric valves located on the first side of the flow-generating pool, and S~N represent the electric valves located on the second side of the flow-generating pool. P RPM The value is the rotational speed of the water pump device, ranging from 0 to MaxRPM, where MaxRPM is the maximum rotational speed of the water pump device.

[0065] In some examples, considering the flow field is vector data, therefore, each It can be represented in spherical coordinates as , For the magnitude of the flow rate, The polar angle and azimuth angle of the flow field vector are given.

[0066] In other examples, the opening and closing angles of electric valves theoretically exist in a continuous numerical space from 0 degrees to 90 degrees. Optimizing within this continuous numerical space would significantly increase the computational load, leading to a lengthy learning and optimization process and reduced experimental efficiency. To shorten the learning and optimization time and further improve experimental efficiency, this application uses a partitioning method to discretize the opening and closing angles of the electric valves into K levels, thereby reducing the computational load. It is understood that the specific value of K can be determined based on the actual situation. For example, when K=11, the partitioning method can be used to discretize the continuous angle space of 0~90 degrees into 0~10, where 0 represents fully closed, 10 represents fully open, and 1~9 represent different degrees of opening and closing angles.

[0067] During the intelligent adjustment process, actual flow field data at different locations in the experimental test area at the current time step can be collected using a flow meter, and the current state space can be constructed accordingly. Using the DQN algorithm, this application can determine the action to be performed based on the actual flow field data, i.e., determine the opening and closing angles of each electric valve and the rotational speed of the water pump, and adjust accordingly. This application can repeatedly adjust at different time steps until the difference between the actual flow field in the experimental test area and the predetermined target flow field is less than a preset threshold, thereby providing a test environment for the target flow field in the flow-generating pool.

[0068] In the above embodiments, the control relationship between the flow field state and the flow-generating components (electric valves and water pumps) is established through the DQN (Deep Q-Network) algorithm, realizing the automation and intelligence of the flow field direction adjustment process. The DQN algorithm uses a Q-value neural network to construct a Q-value table, making the flow field simulation more accurate and convenient in terms of direction and velocity adjustment. While improving adaptability, it also enables the flow field simulation system to have self-updating capabilities and intelligent decision-making abilities, significantly improving valve adjustment efficiency and experimental efficiency, and reducing the difficulty of technology promotion. This provides a more accurate and safer flow field simulation environment for marine engineering experiments.

[0069] In some embodiments, the deep Q-network algorithm includes a reward function that guides the agent to learn the optimal behavior strategy, enabling the agent to adjust the opening and closing angle of the electric valve and the rotation speed of the water pump in a direction closer to the target flow field.

[0070] In this embodiment, the reward function of the DQN algorithm may include task reward, constraint reward, and smoothing reward. The reward function is defined as follows. The task reward is used to make the actual flow field in the experimental test area approximate the target flow field. The constraint reward is used to avoid ineffective operation of the flow field simulation system and ensure the safety of the intelligent control process. The smoothing reward is used to reduce the frequent start-stop / abrupt changes of water pumps / electric valves and ensure the stability of the intelligent control process.

[0071] Specifically, task rewards for:

[0072]

[0073] In the formula, As the first weighting coefficient, This is the second weighting coefficient. Let x be the flow velocity value collected by the x-th flow meter. Let be the velocity value of the target flow field. Let x be the flow field vector angle collected by the x-th current meter. Let x be the flow field vector angle of the target flow field, and let x take values ​​in the range of [1, M], where M is the number of flow meters.

[0074] Understandable. This represents the difference between the velocity value collected by the x-th flow meter and the velocity value of the target flow field. This represents the difference between the flow field vector angle collected by the x-th velocity meter and the flow field vector angle of the target flow field. Therefore, the task reward... The difference between the actual flow field and the target flow field can be measured from two aspects: flow velocity and flow field vector angle.

[0075] Constraints and rewards for:

[0076]

[0077] In the formula, The third weighting coefficient, It is the fourth weighting coefficient. For conditional judgment, All electric valves located on the first side of the water-generating pool are closed. All electric valves located on the second side of the water-generating pool are closed. The rotational speed of the water pump unit.

[0078] The physical meaning of constraint rewards refers to the conditions that the flow field simulation system needs to meet in terms of physical hardware and fluid control. When all electric valves are fully closed and / or the pump speed is too high, the flow field simulation system is prone to equipment damage. In this situation, constraint rewards can be used to penalize and guide the agent to choose safer actions for adjustment.

[0079] To avoid blindly searching for problems, this application also includes a smoothing reward. Specifically:

[0080]

[0081] In the formula, This is the fifth weighting coefficient. It is the sixth weighting coefficient. Let y be the difference in opening and closing angles of the y-th electric valve at adjacent times, where y ranges from [1, N] and N is the number of electric valves; This represents the difference in rotational speed of the water pump unit at adjacent moments.

[0082] When the opening and closing angle of an electric valve changes drastically between adjacent moments, and / or the rotational speed of a water pump changes drastically between adjacent moments, a smooth reward system can be used to penalize such abrupt changes, guiding the agent to choose action spaces with smoother transitions. This avoids frequent start-ups and shutdowns of the equipment, extending its service life.

[0083] In the reward function, ~ These are all weighting coefficients and require parameter tuning. In some embodiments, the first, second, third, fourth, and fifth weighting coefficients are set according to at least one of the following two cases:

[0084] (1) Preset task priority;

[0085] (2) Set the difference between the actual flow field and the target flow field.

[0086] Specifically, if adjustments are made based on task priority, the task priorities obtained by the experimenters can be acquired, and weight coefficients can be set accordingly. Higher task priorities can have larger weight coefficients. For example, if the task priorities are: velocity difference > flow direction difference > stability > safety, then the weight coefficients can be set as: first weight coefficient > second weight coefficient > fifth weight coefficient = sixth weight coefficient > third weight coefficient = fourth weight coefficient.

[0087] When setting parameters based on the difference between the actual flow field and the target flow field, this application allows for adjustments and settings according to actual test conditions. This application can adjust various weight parameters based on the differences between the actual and target flow fields in aspects such as flow velocity and flow field vector angle, enabling the agent to make adjustments with emphasis based on the current actual flow field data, thereby further improving experimental efficiency.

[0088] In some embodiments, the step of setting the weighting coefficients includes:

[0089] If the difference between the actual flow velocity value and the target flow velocity value is greater than the preset flow velocity threshold, and the difference between the actual flow field vector angle and the target flow field vector angle is less than the preset vector angle threshold, then the first weighting coefficient is increased.

[0090] If the difference between the actual flow velocity value and the target flow velocity value is less than the preset flow velocity threshold, and the difference between the actual flow field vector angle and the target flow field vector angle is greater than the preset vector angle threshold, then the second weighting coefficient is increased.

[0091] If the speed of the water pump device is higher than the preset speed threshold, the fourth weighting coefficient will be increased.

[0092] Specifically, if the actual flow field and the target flow field in the experimental test area at the current time step have a large difference in flow velocity, and the flow field direction already meets the set requirements, then the first weighting coefficient can be increased to make the influence of flow velocity adjustment greater, so that the agent can focus more on adjusting the flow velocity value.

[0093] Similarly, if the actual flow field in the experimental test area at the current time step already meets the set requirements in terms of flow velocity magnitude, but the difference in flow field direction is significant, a second weighting coefficient can be added to increase the influence of directional adjustment. Furthermore, in intelligent control processes, if the pump speed is found to be too high, a fourth weighting coefficient can be added to increase the impact of pump speed on the results.

[0094] This embodiment adjusts the corresponding weighting coefficients based on the actual flow field data, allowing the agent to focus on adjusting the opening / closing angle / rotation speed, thereby further improving experimental efficiency.

[0095] In some embodiments, adjusting the rotational speed of the water pump and the opening / closing angle of multiple electric valves based on a deep Q-network algorithm includes:

[0096] The actual flow field data at the current moment is collected using multiple flow meters;

[0097] Determine whether the stopping conditions are met based on actual flow field data;

[0098] If not satisfied, then construct the current state space s based on the actual flow field data at the current moment. t and the current state space s t The data is input into a Q-value neural network to obtain the output data of the Q-value neural network; the Q-value neural network is a neural network trained based on the empirical replay mechanism of the deep Q-network algorithm.

[0099] Based on the output data and the preset action selection strategy, determine the current action space a. t ;

[0100] Based on the current action space a t Adjust the speed of the water pump and the opening and closing angles of multiple electric valves, and after a preset time interval, perform the step of collecting the actual flow field data at the current moment through multiple flow meters.

[0101] Specifically, during the iterative training of DQN, an experience pool can be constructed, and the Q-value neural network can be trained based on the experience playback mechanism. In this process, the network parameters of the Q-value neural network can first be initialized by setting random values. The agent can perform different actions, such as adjusting the water pump to different speeds or adjusting the opening and closing angles of the electric valve to different angles, and collect experience data. Each set of collected flow field experience data can include the state space at time t, the action space at time t, the reward value, and the state space at time t+1. After collecting the flow field experience data, the data can be stored in the experience pool.

[0102] When the amount of data in the experience pool is greater than or equal to a set threshold (e.g., the set threshold batch_size can be 32), a batch of flow field experience data is randomly sampled from the experience pool as training data. This training data is then used to train the Q-value neural network to adjust its network parameters. When action adjustment is needed, the agent can use the output data of the Q-value neural network and a preset action selection strategy to select an action, and then select an action based on the selected current action space a. tAdjust the speed of the water pump and the opening and closing angle of the electric valve.

[0103] For example, a preset action selection strategy could be: Greedy strategy or Boltzmann strategy. For example, the preset action selection strategy could be... Greedy strategy. Therefore, in the early stages of training a Q-value neural network, the current action space 'a' is determined primarily by random sampling. t Accumulate experience data. In the later stages of training, determine the action corresponding to the maximum Q value based on the output data of the Q-value neural network, and use this as the current action space a. t This allows them to quickly approach the target. A greedy strategy can balance exploration and exploitation simultaneously, avoiding getting trapped in local optima.

[0104] The agent can repeatedly execute the steps of random sampling from the experience pool, training the Q-value neural network, and adjusting the flow field until the difference between the actual flow field and the target flow field is less than a preset threshold, thus achieving the target flow field simulation. In some examples, the stopping conditions may include a velocity stopping condition and a flow direction stopping condition. The stopping condition is considered satisfied only when both the velocity stopping condition and the flow direction stopping condition are met simultaneously. The velocity stopping condition can be that the velocity values ​​collected by a flowmeter at any location all meet the specified conditions. The flow field direction stopping condition can be that the flow field vector angles collected by the current meter at any location all satisfy the following condition. ,in and Set thresholds for users.

[0105] Since flow field control is a system with a large time delay, the response of the flow field state after control adjustment is relatively slow. Therefore, after adjusting the speed of the water pump and the opening and closing angles of multiple electric valves, the flow field data acquisition step can be executed again after a preset time interval to ensure the validity of the actual flow field data and improve the accuracy and reliability of intelligent control. After executing the flow field data acquisition step again, this application can determine whether the stopping condition is met based on the latest acquired actual flow field data. If not, it will readjust through a Q-value neural network until the stopping condition is met.

[0106] It should be noted that the Q-value neural network of this application can be a neural network of any structure. For example, the Q-value neural network can adopt a multilayer perceptron structure with ReLU activation function.

[0107] In some embodiments, the preset duration may be a preset fixed delay duration Δt. fix and the preset flow field steady-state duration Δt evl The sum. If the system at t ctl If the rotation speed / opening angle is adjusted continuously, then the flow velocity acquisition time t cap =t ctl+△t fix +△t evl Among them, the fixed delay duration Δt fix The steady-state duration Δt can be set based on experience. evl To achieve a fixed delay duration Δt fix The time required to acquire stable flow field data is then determined. The method for determining the stability of the flow field data is as follows: The flow field data curve acquired by each current meter is smoothed and filtered, and the velocity gradient curve corresponding to each current meter is calculated based on the filtered flow field data curve. The flow field data is considered stable when the velocity gradient curves of all current meters are stable. This application can determine Δt based on the aforementioned determination method and historical data. evl The value of .

[0108] In this embodiment, when the fixed delay duration Δt fix When the settings are unreasonable, during the period of △t fix A further delay is needed to obtain reasonable actual flow field data. Therefore, this application sets the flow field stationary time Δt. evl This ensures the validity of the collected actual flow field data, further improves the accuracy of intelligent control, and increases experimental efficiency.

[0109] In some embodiments, in addition to the actual flow field data collected by each current meter, the state space also includes the water level information of the flow-generating pool. Specifically, even if the target flow field is the same, the control data will differ depending on the water level. When the state space does not include water level information, after changing the water level of the flow-generating pool, the above-mentioned steps of constructing the empirical pool and training the network need to be repeated to obtain the control parameters under the new water level. Although this can reduce the computational load, the network training needs to be re-executed every time the water level is changed, resulting in slightly slower experimental efficiency.

[0110] To further improve experimental efficiency, this application can add water level information to the state space of the agent to participate in the calculation, so that the agent can combine water level information, the opening and closing angle of the electric valve and the rotation speed of the water pump device to perform Q-value neural network training and select actions.

[0111] In some embodiments, without replacing the electric valves and water pumps, this application can accumulate control parameters for flow generation at different water depths through continuous experiments. In this way, when encountering the same situation again, no further training is required; the accumulated empirical parameters can be directly used for flow generation control, thereby greatly improving experimental efficiency.

[0112] The intelligent control device of the flow field simulation system provided in the embodiments of this application is described below. The intelligent control device of the flow field simulation system described below and the intelligent control method of the flow field simulation system described above can be referred to in correspondence with each other.

[0113] In some embodiments, this application provides an intelligent control device 300 for a flow field simulation system, the flow field simulation system comprising: a flow-generating pool, a water pump device, multiple electric valves installed at the outlet of the flow-generating pool, and multiple flow rate meters installed in the experimental testing area of ​​the flow-generating pool. Figure 3 As shown, the device 300 includes:

[0114] The initialization module 310 is used to set the opening and closing angles of the plurality of electric valves to initial angles and the rotational speed of the water pump device to initial rotational speed.

[0115] The intelligent control module 320 is used to construct a state space based on the actual flow field data collected by the multiple flow meters, construct an action space based on the rotational speed of the water pump device and the opening and closing angles of the multiple electric valves, and adjust the rotational speed of the water pump device and the opening and closing angles of the multiple electric valves based on the deep Q-network algorithm until a stopping condition is met; wherein, the stopping condition is that the difference between the actual flow field of the experimental test area and the predetermined target flow field is less than a preset threshold.

[0116] In some embodiments, the intelligent control module 320 of this application includes:

[0117] The flow field acquisition unit is used to acquire the actual flow field data at the current moment through the multiple flow meters;

[0118] A stop determination unit is used to determine whether the stop condition is met based on the actual flow field data.

[0119] The Q-value neural network prediction unit is used to construct the current state space s based on the actual flow field data at the current moment if the condition is not met. t and the current state space s t The data is input into a Q-value neural network to obtain the output data of the Q-value neural network; wherein, the Q-value neural network is a neural network trained based on the empirical replay mechanism of the deep Q-network algorithm;

[0120] The action determination unit is used to determine the current action space a based on the output data and the preset action selection strategy. t ;

[0121] Action execution unit, used to perform actions based on the current action space a t Adjust the rotational speed of the water pump device and the opening and closing angles of the multiple electric valves, and after a preset time interval, perform the step of collecting the actual flow field data at the current moment through the multiple flow meters.

[0122] In some embodiments, the preset duration is the sum of a preset fixed delay duration and a preset flow field stabilization duration.

[0123] In some embodiments, the reward function of the deep Q-network algorithm includes a task reward, a constraint reward, and a smoothing reward; wherein, the task reward... for:

[0124]

[0125] In the formula, As the first weighting coefficient, This is the second weighting coefficient. Let x be the flow velocity value collected by the x-th flow meter. The velocity value of the target flow field. Let x be the flow field vector angle collected by the x-th current meter. Let x be the flow field vector angle of the target flow field, and let x take values ​​in the range of [1, M], where M is the number of flow meters;

[0126] The constraint reward for:

[0127]

[0128] In the formula, The third weighting coefficient, It is the fourth weighting coefficient. For conditional judgment, All electric valves located on the first side of the water-generating pool are closed. All electric valves located on the second side of the water-generating pool are closed. The rotational speed of the water pump device;

[0129] The smooth reward for:

[0130]

[0131] In the formula, This is the fifth weighting coefficient. It is the sixth weighting coefficient. Let y be the difference in opening and closing angles of the y-th electric valve at adjacent times, where y ranges from [1, N] and N is the number of electric valves; The difference in rotational speed of the water pump device at adjacent moments.

[0132] In some embodiments, the first weighting coefficient, the second weighting coefficient, the third weighting coefficient, the fourth weighting coefficient, and the fifth weighting coefficient are set according to a preset task priority and / or the difference between the actual flow field and the target flow field.

[0133] In some embodiments, the device 300 of this application further includes:

[0134] The first coefficient adjustment module is used to increase the first weighting coefficient if the difference between the velocity value of the actual flow field and the velocity value of the target flow field is greater than a preset velocity threshold, and the difference between the flow field vector angle of the actual flow field and the flow field vector angle of the target flow field is less than a preset vector angle threshold.

[0135] The second coefficient adjustment module is used to increase the second weighting coefficient if the difference between the velocity value of the actual flow field and the velocity value of the target flow field is less than a preset velocity threshold, and the difference between the flow field vector angle of the actual flow field and the flow field vector angle of the target flow field is greater than a preset vector angle threshold.

[0136] The third coefficient adjustment module is used to increase the fourth weighting coefficient if the rotational speed of the water pump device is higher than the preset rotational speed threshold.

[0137] In some embodiments, the state space may also include water level information of the water-generating pool.

[0138] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the intelligent control method of the flow field simulation system in any embodiment.

[0139] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the intelligent control method of the flow field simulation system in any embodiment.

[0140] Indicatively, Figure 4 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. In one example, the computer device can be a server. (Refer to...) Figure 4 The computer device 900 includes a processing component 902, which further includes one or more processors, and memory resources represented by memory 901 for storing instructions, such as application programs, that can be executed by the processing component 902. The application programs stored in memory 901 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 902 is configured to execute instructions to perform the steps of the methods described in any of the above embodiments.

[0141] The computer device 900 may also include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate on an operating system stored in memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0142] Those skilled in the art will understand that the internal structure of the computer device shown in this application is merely a block diagram of a portion of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, "a," "an," "the," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. "Multiple" refers to at least two, such as 2, 3, 5, or 8, etc. "And / or" includes any and all combinations of the related listed items.

[0144] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0145] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent control method of a flow field simulation system, characterized by, The flow field simulation system comprises a flow generating pool, a water pump device, a plurality of electric valves arranged at an outflow port of the flow generating pool, and a plurality of flow meters arranged at an experimental test area of the flow generating pool; the method comprises: setting opening and closing angles of the plurality of electric valves to initial angles, and setting a rotating speed of the water pump device to an initial rotating speed; constructing a state space based on actual flow field data collected by the plurality of flow meters, constructing an action space based on the rotating speed of the water pump device and the opening and closing angles of the plurality of electric valves, adjusting the rotating speed of the water pump device and the opening and closing angles of the plurality of electric valves based on a deep Q network algorithm until a stop condition is met; wherein the stop condition is that a difference between the actual flow field of the experimental test area and a predetermined target flow field is less than a preset threshold, and a reward function of the deep Q network algorithm comprises a task reward, a constraint reward and a smoothing reward; Wherein the task reward is: ； In the formula, is a first weight coefficient, is a second weight coefficient, is a flow velocity value collected by an xth flow velocity meter, is a flow velocity value of the target flow field, is a flow field vector angle collected by an xth flow velocity meter, is a flow field vector angle of the target flow field, and x is in a range of [1, M], where M is the number of flow velocity meters. The constraint reward Is: ； wherein, is a third weight coefficient, is a fourth weight coefficient, is a condition determination, are electric valves disposed on the first side of the flow-creating basin are all closed, are electric valves disposed on the second side of the flow-creating basin are all closed, is a rotation speed of the water pump device; The smooth reward Is: ； In the formula, is a fifth weight coefficient, is a sixth weight coefficient, is a difference between the opening and closing angles of the yth electric valve at adjacent time points, y is in the range of [1, N], and N is the number of electric valves; is a difference between the rotational speeds of the water pump device at adjacent time points.

2. The method of claim 1, wherein, the adjusting of the rotating speed of the water pump device and the opening and closing angles of the plurality of electric valves based on the deep Q network algorithm comprises: collecting actual flow field data at a current time by the plurality of flow meters; determining whether the stop condition is met according to the actual flow field data; If not satisfied, a current state space s is constructed according to actual flow field data at the current time t , and the current state space s t is input into a Q value neural network to obtain output data of the Q value neural network; wherein the Q value neural network is a neural network trained based on an experience replay mechanism of the deep Q network algorithm. According to the output data and a preset action selection strategy, a current action space a is determined t ; According to the current action space a t Adjust the rotation speed of the water pump device and the opening angle of the plurality of electric valves, and execute the step of collecting the actual flow field data at the current time by the plurality of flow meters after a preset interval duration.

3. The method of claim 2, wherein, the preset time length is a sum of a preset fixed delay time length and a preset flow field stable time length.

4. The method of claim 1, wherein, The first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient and the fifth weight coefficient are set according to a preset task priority and / or a difference between the actual flow field and the target flow field.

5. The method of claim 4, wherein, The setting step of the weight coefficient comprises: if a difference between a flow rate value of the actual flow field and a flow rate value of the target flow field is greater than a preset flow rate threshold, and a difference between a flow field vector angle of the actual flow field and a flow field vector angle of the target flow field is less than a preset vector angle threshold, then the first weight coefficient is increased; if the difference between the flow rate value of the actual flow field and the flow rate value of the target flow field is less than the preset flow rate threshold, and the difference between the flow field vector angle of the actual flow field and the flow field vector angle of the target flow field is greater than the preset vector angle threshold, then the second weight coefficient is increased; if the rotating speed of the water pump device is higher than a preset rotating speed threshold, then the fourth weight coefficient is increased.

6. The method according to any one of claims 1 to 3, characterized in that, The state space further comprises water level information of the flow generating pool.

7. An intelligent control device for a flow field simulation system, characterized by, The flow field simulation system comprises a flow generating pool, a water pump device, a plurality of electric valves arranged at an outflow port of the flow generating pool, and a plurality of flow meters arranged at an experimental test area of the flow generating pool; the device comprises: an initialization module configured to set opening and closing angles of the plurality of electric valves to initial angles, and set a rotating speed of the water pump device to an initial rotating speed; an adjusting module configured to adjust the rotating speed of the water pump device and the opening and closing angles of the plurality of electric valves based on a deep Q network algorithm until a stop condition is met; The intelligent control module is configured to construct a state space based on actual flow field data collected by the plurality of flowmeters, construct an action space based on a rotating speed of the water pump device and opening and closing angles of the plurality of electric valves, and adjust the rotating speed of the water pump device and the opening and closing angles of the plurality of electric valves based on a deep Q network algorithm until a stop condition is met; the stop condition is that a difference between the actual flow field of the experimental test region and a predetermined target flow field is less than a preset threshold, and a reward function of the deep Q network algorithm includes a task reward, a constraint reward, and a smoothing reward. Wherein the task reward is: ； wherein, is a first weight coefficient, is a second weight coefficient, is a flow velocity value collected by an xth flow velocity meter, is a flow velocity value of the target flow field, is a flow field vector angle collected by an xth flow velocity meter, is a flow field vector angle of the target flow field, x is in the range of [1, M], and M is the number of flow velocity meters. The constraint reward Is: ； wherein, is a third weight coefficient, is a fourth weight coefficient, is a condition determination, is that the electric valves arranged on the first side of the flow-creating pool are all closed, is that the electric valves arranged on the second side of the flow-creating pool are all closed, is the rotating speed of the water pump device; The smooth reward Is: ； In the formula, is a fifth weight coefficient, is a sixth weight coefficient, is a difference between the opening and closing angles of the yth electric valve at adjacent time points, y is in the range of [1, N], and N is the number of electric valves; is a difference in the rotational speed of the water pump device at adjacent time points.

8. A storage medium, characterized by The storage medium has computer readable instructions stored therein, and the computer readable instructions, when executed by one or more processors, cause the one or more processors to perform the steps of the intelligent control method of the flow field simulation system according to any one of claims 1 to 6.

9. A computer device, comprising: Comprise: One or more processors, and a memory; The memory has computer readable instructions stored therein, and the computer readable instructions, when executed by the one or more processors, perform the steps of the intelligent control method of the flow field simulation system according to any one of claims 1 to 6.

Images