A hybrid output control method and device based on emission optimization

Abstract

The application discloses a hybrid power output control method and device based on emission optimization, and belongs to the field of ship power system control. The method is as follows: predicting the propelling power demand of the next time step according to the propelling power data of the historical time step; determining the first combustion state of the main engine of the current time step according to the first exhaust component data, and combining the first load state of the engine to obtain a multi-modal feature vector; obtaining the first main engine output power decision of the optimal first predicted Q value based on the multi-modal feature vector and a deep neural network model; wherein the deep neural network is obtained by using a deep reinforcement learning algorithm and training through a reward function with the target of minimizing exhaust emission; and controlling the hybrid power output system to perform power distribution operation according to the propelling power demand and the first main engine output power decision. Through the application, the problem that too much exhaust emission is caused by unreasonable power distribution of the hybrid power system can be solved.

Description

Technical Field

[0001] This application relates to the field of marine power system control, and in particular to a hybrid power output control method and device based on emission optimization. Background Technology

[0002] Because traditional power systems that rely on internal combustion engines have high emissions and high energy consumption, hybrid propulsion systems, with their advantages of lower energy consumption, are gradually becoming the primary choice for ship propulsion systems.

[0003] However, existing hybrid power output control methods typically prioritize power demand or energy efficiency as optimization objectives, neglecting the real-time changes in exhaust emissions. This leads to problems such as severe emission fluctuations or excessive emissions under certain operating conditions due to improper power distribution. Furthermore, the combustion state of the main engine is affected by various factors such as load changes and sea state fluctuations, making it difficult for traditional control strategies to simultaneously ensure power response and emission control.

[0004] Therefore, how to ensure power response while avoiding excessive exhaust emissions due to unreasonable power distribution in hybrid systems has become a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a hybrid power output control method and device based on emission optimization, which can solve the problem of excessive exhaust emissions caused by unreasonable power distribution control of hybrid power systems in the prior art.

[0006] One embodiment of this application provides a hybrid power output control method based on emission optimization, comprising:

[0007] The propulsion power data of the ship at several historical time steps are collected, and the propulsion power requirement for the next time step is obtained based on all the propulsion power data using a pre-trained long short-term memory network model.

[0008] Collect the first exhaust gas component data of the ship's exhaust gas at the current time step, and determine the first combustion state of the main engine at the current time step based on the first exhaust gas component data and a pre-trained support vector machine.

[0009] The first load state of the main engine at the current time step is collected, and the first combustion state and the first load state are weighted and fused through the attention mechanism module to obtain a multimodal feature vector;

[0010] A pre-trained deep neural network model is obtained, and based on the deep neural network model and the multimodal feature vector, the first predicted Q value for the output power decision of each main engine in a preset discrete action space is obtained; wherein, the deep neural network is obtained by constructing a reward function with the goal of minimizing exhaust emissions, and training it using a deep reinforcement learning algorithm based on the reward function;

[0011] The first main engine output power decision is obtained to obtain the optimal first predicted Q value, and the hybrid power output system is controlled to perform power distribution operation based on the propulsion power demand and the first main engine output power decision.

[0012] Compared to existing technologies, the above embodiments have the following advantages: Since the composition of a ship's exhaust gas is related to the combustion and load states of the main engine, determining the current combustion state of the main engine using a support vector machine first establishes the actual operating state of the main engine, providing accurate data input for the subsequent deep neural network model. Simultaneously, since the power allocation decision at the current time step affects the load and combustion states of the main engine in the next time step, and these states determine the composition of the emitted exhaust gas, exhaust emissions can be optimized by improving the power allocation decision. This application employs a deep reinforcement learning algorithm, training a deep neural network model with a reward function aimed at minimizing exhaust emissions. The deep neural network model evaluates the predicted Q-value of each main engine output power decision, ensuring that the final first main engine output power decision minimizes exhaust emissions. Finally, by predicting the ship's propulsion power demand in advance and combining this demand with the first main engine output power decision, the energy storage output power decision can be determined, ensuring that the final power allocation operation meets the ship's power response requirements in the next time step.

[0013] Further, determining the first combustion state of the main engine at the current time step based on the first exhaust gas composition data using a pre-trained support vector machine includes:

[0014] Based on the first load state, a corresponding first support vector machine is selected; wherein, the first support vector machine is obtained by training based on the second exhaust gas component data of the ship's exhaust gas when the main engine is in different combustion states under the first load state;

[0015] Based on the first support vector machine and according to the first exhaust gas composition data, the first combustion state of the main engine under the first load state at the current time step is determined.

[0016] Compared with the prior art, the above embodiments have the following beneficial effects: Since different load states and combustion states are two independent factors affecting the composition of the final exhaust gas emissions, the corresponding support vector machine model is first selected based on the actual observable first load state of the main engine, thereby removing the interference of the load state on the combustion state judgment process; furthermore, when the load state of the main engine is fixed, the composition of the exhaust gas under different combustion states is significantly different, so the support vector machine can accurately reflect the current combustion state of the main engine, providing a reliable input basis for subsequent power allocation optimization.

[0017] Furthermore, the first load state of the main engine at the current time step is collected, and the first combustion state and the first load state are weighted and fused through an attention mechanism module to obtain a multimodal feature vector, including:

[0018] The first combustion state is embedded and encoded through a preset embedding layer to obtain a first vector;

[0019] By using a pre-defined feedforward neural network, the first load state is linearly transformed and activated to obtain a second vector with the same dimension as the first vector.

[0020] Based on the correlation between the first combustion state and the first load state and the first exhaust gas component data, determine the first weights corresponding to the first vector and the second vector respectively.

[0021] The first vector and the second vector are multiplied by their respective first weights and then added together to obtain the multimodal feature vector.

[0022] Compared to existing technologies, the above embodiments have the following advantages: Since the first combustion state is discrete data, it requires embedding encoding to transform it from a single integer into a dense representation capable of computation. Simultaneously, since the load state is real, collectable equipment operating data, it is continuous data. Therefore, through linear transformation and activation, the multidimensional continuous signal is compressed or expanded to the same dimension as the embedded vector, ensuring the feasibility of subsequent weighted fusion. Furthermore, since the load state and combustion state are two key factors affecting the final exhaust emissions, feature extraction is performed on the first combustion state and the first load state respectively to obtain a first vector and a second vector. Then, an attention mechanism is used to weightedly fuse the first vector and the second vector to obtain multimodal data, more realistically reflecting the current operating state of the main engine, thereby improving the input quality and learning ability of the deep neural network.

[0023] Further, the step of controlling the hybrid power output system to perform power distribution operations based on the propulsion power demand and the output power decision of the first main engine includes:

[0024] The output power of the main engine is controlled based on the output power decision of the first main engine, and the output power of the energy storage is controlled based on the difference between the propulsion power demand and the output power decision of the first main engine.

[0025] After the ship performs the power distribution operation, the third exhaust gas component data of the ship's exhaust gas at the next time step is collected, and the experience pool is updated based on the difference between the first exhaust gas component data and the third exhaust gas component data.

[0026] Based on the updated experience pool, the parameters of the deep neural network model are fine-tuned using a supervised learning method.

[0027] Compared with the prior art, the above embodiments have the following beneficial effects: After knowing the output power decision of the first main engine, the propulsion power gap of the current propulsion power demand can be determined based on the output power decision of the first main engine. This gap is the output power that needs to be allocated to the energy storage, thereby ensuring that the power output of the main engine and the energy storage can meet the power response requirements of the ship in the next time step; furthermore, when the hybrid power system executes the corresponding output power decision, the operating state of the main engine will change, and the exhaust gas emission composition data will also change accordingly. The third exhaust gas composition data obtained at this time is the real sample data. The experience pool is updated according to the real sample data, and the neural network parameters are continuously fine-tuned in combination with supervised learning to enhance the adaptability of the deep neural network model to changes in the real environment, while improving the robustness and generalization ability of the deep neural network model under actual working conditions.

[0028] Further, updating the experience pool based on the difference between the first exhaust gas component data and the third exhaust gas component data includes:

[0029] Based on the first load state and the first combustion state, the second weights corresponding to various exhaust gas components are determined by the entropy weight method to obtain the weight parameter combination; wherein, one weight parameter combination corresponds to a combination of load state and combustion state.

[0030] For each exhaust gas component, calculate the first difference between its value in the first exhaust gas component data and its value in the third exhaust gas component data, and then sum the first difference corresponding to each exhaust gas component and the first product of the second weight to obtain a weighted sum.

[0031] The weighted sum is used as the actual Q value, and the first load state and the first combustion state are used as input data. Training samples are constructed based on the actual Q value, the input data, and the first main engine output power, and the training samples are added to the experience pool.

[0032] Compared with the prior art, the above embodiments have the following beneficial effects: Since the load state and combustion state are two factors that simultaneously affect the components of the final exhaust gas emission, when a certain load state and combustion state are fixed, the exhaust gas component data will have a fixed proportion of a certain exhaust gas component. Therefore, the entropy weight method is used to assign component weights based on the exhaust gas samples under the load and combustion states to obtain the second weight. The second weight reflects the difference in the impact of each emission on the total emission under different load and combustion states. The actual Q value is determined according to the second weight, and the actual Q value is used as the supervision signal in the subsequent deep neural network fine-tuning process, thereby improving the adaptability of the deep neural network to changes in the real environment.

[0033] Further, the step of determining the second weights corresponding to various exhaust gas components using the entropy weight method based on the first load state and the first combustion state includes:

[0034] Acquire several fourth exhaust gas component data when the main engine is in the first load state and the first combustion state. For each exhaust gas component, add up the corresponding values ​​in each of the fourth exhaust gas component data to obtain the first cumulative sum corresponding to each exhaust gas component.

[0035] Calculate the ratio of the value of each exhaust gas component in each of the fourth exhaust gas component data items to the corresponding first summation, and obtain the second product of each ratio and the logarithm of the ratio;

[0036] After negating each of the second products, for each exhaust gas component, sum all the corresponding second products to obtain the second summation for each exhaust gas component;

[0037] For each exhaust gas component, calculate the second difference between the corresponding second cumulative sum and the preset threshold, and use the normalized second difference as the second weight of the corresponding exhaust gas component.

[0038] Compared with the prior art, the above embodiments have the following beneficial effects: For each exhaust gas component, the ratio of its value in the fourth exhaust gas component data to the corresponding first cumulative sum is calculated to determine its probability of occurrence. The ratio and the corresponding logarithm are multiplied, the negative is taken, and then the sum is accumulated to obtain the second cumulative sum. This assesses the dispersion and uncertainty of each exhaust gas component in the sample. The larger the second cumulative sum, the smaller the fluctuation of the corresponding exhaust gas component under the same load and combustion conditions, which means that the exhaust gas component is less correlated with the load and combustion conditions. Conversely, the smaller the second cumulative sum, the greater the correlation between the exhaust gas component and the load and combustion conditions. Therefore, in order to ensure the rationality of the weights, the second difference between the second cumulative sum and the preset threshold is taken, so that the second difference is positively correlated with the above-mentioned correlation, thereby improving the accuracy of the subsequent actual Q value estimation.

[0039] Furthermore, the step of fine-tuning the parameters of the deep neural network model using a supervised learning method based on the updated experience pool includes:

[0040] For each training sample in the experience pool, based on its corresponding input data and the deep neural network model, the second predicted Q value for each main engine output power decision is obtained, and based on each second predicted Q value, the second main engine output power decision corresponding to the optimal second predicted Q value is determined.

[0041] Select a first training sample from all the training samples whose second main engine output power decision is consistent with the first main engine output power decision;

[0042] For each of the first training samples, calculate the third difference between the corresponding actual Q value and the optimal second predicted Q value;

[0043] The sum of squares and mean of the third difference is used as the loss value, and the parameters of the deep neural network model are adjusted through backpropagation based on the loss value.

[0044] Compared with the prior art, the above embodiments have the following beneficial effects: Since the deep neural network model is in a state of continuous iteration, the first main engine output power decision recorded in the training samples in the experience pool may be the result output by the deep neural network model before multiple adjustments. The second main engine output power decision output by the latest deep neural network model may have changed. At this time, the actual Q value corresponding to the second main engine output power decision is also different from that recorded in the experience pool. Therefore, it is necessary to screen the samples in the experience pool to avoid adjustment errors. By continuously learning and optimizing the deep neural network model through actual collected real sample data, the adaptability of the deep neural network to changes in the real environment is improved.

[0045] Furthermore, the step of obtaining the propulsion power requirement for the next time step using the pre-trained Long Short-Term Memory network model based on all the propulsion power data includes:

[0046] Collect the first sea state data of the aforementioned historical time steps and the second sea state data of the next time step, and concatenate the first sea state data with the propulsion power data according to the time steps to obtain a two-dimensional time series feature tensor, and concatenate the second sea state data to the end of the two-dimensional time series feature tensor;

[0047] The propulsion power trend change features are extracted from the two-dimensional temporal feature tensor using a one-dimensional convolutional neural network.

[0048] The propulsion power trend change characteristics are input into the long short-term memory network model to obtain the propulsion power requirement for the next time step.

[0049] Compared with existing technologies, the above embodiments have the following beneficial effects: by constructing a time series tensor using sea state data and propulsion power data, the propulsion trend of the ship under the influence of external factors is more comprehensively reflected. Furthermore, by extracting trend features through one-dimensional convolution and predicting time series information using a short-time memory network model, the accuracy of propulsion power prediction and dynamic response capability are enhanced. This provides forward-looking input for subsequent power scheduling, enabling early response to the differences in power demand caused by different sea state changes, and ensuring that the final power allocation operation can meet the power response requirements of the ship in the next time step.

[0050] Another embodiment of this application also provides a hybrid power output control device based on emission optimization, including: a propulsion power demand prediction module, a combustion state classification module, a feature fusion module, a Q value prediction module, and a power distribution module;

[0051] The propulsion power demand prediction module is used to collect propulsion power data of the ship at several historical time steps, and obtain the propulsion power demand for the next time step based on all the propulsion power data through a pre-trained long short-term memory network model.

[0052] The combustion state classification module is used to collect the first exhaust gas component data of the ship's exhaust gas at the current time step, and determine the first combustion state of the main engine at the current time step through a pre-trained support vector machine based on the first exhaust gas component data.

[0053] The feature fusion module is used to collect the first load state of the main engine at the current time step, and through the attention mechanism module, weightedly fuse the first combustion state and the first load state to obtain a multimodal feature vector.

[0054] The Q-value prediction module is used to acquire a pre-trained deep neural network model and, based on the deep neural network model and the multimodal feature vector, acquire the first predicted Q-value for the output power decision of each main engine in a preset discrete action space; wherein, the deep neural network is acquired by constructing a reward function with the goal of minimizing exhaust emissions and training it using a deep reinforcement learning algorithm based on the reward function.

[0055] The power distribution module is used to obtain the first main engine output power decision with the optimal first predicted Q value, and control the hybrid power output system to perform power distribution operation according to the propulsion power demand and the first main engine output power decision.

[0056] Further, the combustion state classification module includes: a support vector machine selection unit and a support vector machine invocation unit; the combustion state classification module is used to determine the first combustion state of the main engine at the current time step based on the first exhaust gas composition data using a pre-trained support vector machine, including:

[0057] The support vector machine selection unit is used to select a corresponding first support vector machine according to the first load state; wherein, the first support vector machine is obtained by training based on the second exhaust gas composition data of the ship's exhaust gas when the main engine is in different combustion states under the first load state.

[0058] The support vector machine invocation unit is used to determine the first combustion state of the main engine when it is in the first load state at the current time step, based on the first support vector machine and the first exhaust gas composition data. Attached Figure Description

[0059] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0060] Figure 1 This is a schematic flowchart of a hybrid power output control method based on emission optimization provided in some embodiments of this application;

[0061] Figure 2 This is a schematic diagram of a hybrid power output control device based on emission optimization provided in some embodiments of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0064] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0065] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0066] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0067] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0068] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0069] Traditional internal combustion engine-based power systems suffer from high emissions and energy consumption, while hybrid propulsion systems, with their lower energy consumption, are increasingly becoming the preferred choice for ship propulsion. Existing hybrid power output control methods typically prioritize power demand or energy efficiency, neglecting the real-time changes in exhaust emissions. This leads to significant emission fluctuations or excessive emissions under certain operating conditions due to improper power distribution. Furthermore, the combustion state of the main engine is affected by various factors such as load variations and sea state fluctuations, making it difficult for traditional control strategies to simultaneously ensure power response and emission control.

[0070] Example 1

[0071] Please refer to Figure 1 To address the problem in existing technologies of how to ensure power response while avoiding excessive exhaust emissions due to unreasonable power distribution in hybrid power systems, this application provides a hybrid power output control method based on emission optimization, including steps S101 to S105, specifically:

[0072] S101: Collect propulsion power data for several historical time steps of the ship, and obtain the propulsion power requirement for the next time step based on all the propulsion power data using a pre-trained Long Short-Term Memory network model.

[0073] Furthermore, in some embodiments of this application, the step of obtaining the propulsion power requirement for the next time step based on all the propulsion power data using a pre-trained Long Short-Term Memory network model includes:

[0074] Collect the first sea state data of the aforementioned historical time steps and the second sea state data of the next time step, and concatenate the first sea state data with the propulsion power data according to the time steps to obtain a two-dimensional time series feature tensor, and concatenate the second sea state data to the end of the two-dimensional time series feature tensor;

[0075] The propulsion power trend change features are extracted from the two-dimensional temporal feature tensor using a one-dimensional convolutional neural network.

[0076] The propulsion power trend change characteristics are input into the long short-term memory network model to obtain the propulsion power requirement for the next time step.

[0077] Preferably, in some embodiments of this application, the propulsion power data of the plurality of historical time steps includes at least: the total output power of the main engine and energy storage at the current time and the plurality of historical time steps; the sea state data includes at least: the sea wind speed vector, wave height and sea wave current vector at the corresponding time step. It should be noted that the first sea state data is the actual sea state data collected, and the second sea state data is the predicted sea state data obtained through the meteorological platform.

[0078] Preferably, in some embodiments of this application, the step of concatenating the first sea state data and the propulsion power data according to time steps to obtain a two-dimensional time-series feature tensor, and concatenating the second sea state data to the end of the two-dimensional time-series feature tensor, includes: assuming the shape of the first sea state data and the second sea state data is (T, D1), where T is the time step and D1 is the feature dimension of the sea state data, such as the wind speed vector, wave height, and wave current vector; assuming the shape of the propulsion power data is (T, D2), where D2 is the feature dimension of the propulsion power data, such as the sum of the output power of the main engine and the energy storage; then, after concatenating the first sea state data and the propulsion power data according to time steps, a two-dimensional time-series feature tensor with the shape (T, D1+D2) is obtained; further, the second sea state data is concatenated to the end of the two-dimensional time-series feature tensor to obtain a two-dimensional time-series feature tensor with the shape (T+1, D1+D2), wherein placeholders are used to replace the propulsion power data corresponding to the second sea state data.

[0079] Preferably, in some embodiments of this application, the step of extracting propulsion power trend change features from the two-dimensional temporal feature tensor through a one-dimensional convolutional neural network includes: performing sliding extraction in the time dimension, setting the convolutional kernel size to (k, D1+D2), where k is the width of the sliding window, which can be set according to actual conditions, and this application does not limit this value.

[0080] Preferably, in some embodiments of this application, the step of inputting the propulsion power trend change characteristics into the long short-term memory network model to obtain the propulsion power demand for the next time step includes: wherein the long short-term memory network model is used to capture the long-term dependence and dynamic pattern of propulsion power in the time series; after the propulsion power trend change characteristics are input into the long short-term memory network model, after forward propagation, the long short-term memory network model outputs the propulsion power data corresponding to the second sea state data, that is, the propulsion power demand value corresponding to the next time step.

[0081] As can be seen from the above embodiments, this application constructs a time series tensor by using sea state data and propulsion power data, which more comprehensively reflects the propulsion trend of the ship under the influence of external factors. Furthermore, it extracts trend features through one-dimensional convolution and predicts time series information using a short-time memory network model, thereby enhancing the accuracy and dynamic response capability of propulsion power prediction. This provides forward-looking input for subsequent power scheduling, enabling early response to the differences in power demand caused by different sea state changes, and ensuring that the final power allocation operation can meet the power response requirements of the ship in the next time step.

[0082] S102: Collect the first exhaust gas component data of the ship's exhaust gas at the current time step, and determine the first combustion state of the main engine at the current time step using the pre-trained support vector machine based on the first exhaust gas component data.

[0083] Furthermore, in some embodiments of this application, determining the first combustion state of the main engine at the current time step based on the first exhaust gas composition data using a pre-trained support vector machine includes:

[0084] Based on the first load state, a corresponding first support vector machine is selected; wherein, the first support vector machine is obtained by training based on the second exhaust gas component data of the ship's exhaust gas when the main engine is in different combustion states under the first load state;

[0085] Based on the first support vector machine and according to the first exhaust gas composition data, the first combustion state of the main engine under the first load state at the current time step is determined.

[0086] Preferably, in some embodiments of this application, the combustion states include: a complete combustion state, in which the fuel is fully oxidized, and the exhaust emissions show low levels of carbon oxides and hydrocarbons, moderate levels of nitrogen oxides, and low smoke opacity; an incomplete combustion state, in which the concentrations of carbon oxides and hydrocarbons increase, possibly accompanied by an increase in smoke opacity, and nitrogen oxides are relatively low; a pre-ignition / early combustion state, in which combustion occurs earlier, peak pressure occurs earlier, which may lead to knocking or engine knocking, and nitrogen oxides and smoke opacity increase simultaneously; a cold combustion state, in which combustion is incomplete, and the exhaust gas temperature is low and the amount of hydrocarbons in the exhaust gas increases; and a delayed combustion state, in which combustion is delayed and the amount of hydrocarbons in the exhaust gas increases.

[0087] Preferably, in some embodiments of this application, the load state is the set of current operating condition parameters of the main engine, including but not limited to data collected by multiple sensors such as main engine output power, speed, torque, fuel flow, intake pressure, and exhaust temperature. Further, when selecting the corresponding first support vector machine based on the first load state, classification intervals for different load states are first set (e.g., defining the intervals of each operating condition parameter when the main engine is under low, medium, and high loads). Based on the actual operating condition parameters in the first load state, combined with the above classification intervals, the classification of the current first load state of the main engine is determined, and the corresponding support vector machine is further determined based on this classification. When training the support vector machine, it is ensured that the second exhaust gas composition data used for the current training is the exhaust gas composition data generated when the main engine is under the corresponding first load state.

[0088] Preferably, in some embodiments of this application, the exhaust gas composition data includes, but is not limited to, the content data of carbon oxides, hydrocarbons, nitrogen oxides, and smoke opacity.

[0089] As can be seen from the above embodiments, since different load states and combustion states are two independent factors affecting the final exhaust gas composition, the corresponding support vector machine model is first selected based on the actual observable first load state of the main engine, thereby removing the interference of load state on the combustion state judgment process. Furthermore, when the load state of the main engine is fixed, the composition of exhaust gas under different combustion states is significantly different. Therefore, the support vector machine can accurately reflect the current combustion state of the main engine, providing a reliable input basis for subsequent power allocation optimization.

[0090] S103: Collect the first load state of the main engine at the current time step, and obtain a multimodal feature vector by weighted fusion of the first combustion state and the first load state through the attention mechanism module.

[0091] Furthermore, in some embodiments of this application, the step of collecting the first load state of the main engine at the current time step and obtaining a multimodal feature vector by weighted fusing the first combustion state and the first load state through an attention mechanism module includes:

[0092] The first combustion state is embedded and encoded through a preset embedding layer to obtain a first vector;

[0093] By using a pre-defined feedforward neural network, the first load state is linearly transformed and activated to obtain a second vector with the same dimension as the first vector.

[0094] Based on the correlation between the first combustion state and the first load state and the first exhaust gas component data, determine the first weights corresponding to the first vector and the second vector respectively.

[0095] The first vector and the second vector are multiplied by their respective first weights and then added together to obtain the multimodal feature vector.

[0096] Preferably, in some embodiments of this application, the step of embedding and encoding the first combustion state through a preset embedding layer to obtain a first vector includes: taking the first combustion state identified by the support vector machine at the current time step as a discrete label (e.g., for a fully combusted state, the corresponding discrete label is defined as the number "2"); inputting the discrete label into the preset embedding layer to obtain a first vector of a preset dimension.

[0097] Preferably, in some embodiments of this application, the step of performing linear transformation and activation operations on the first load state through a preset feedforward neural network to obtain a second vector with the same dimension as the first vector includes: inputting the collected first load state into a preset feedforward neural network, which includes a fully connected layer and an activation layer, wherein the activation layer uses the ReLU activation function and outputs a second vector with the same dimension as the first vector.

[0098] Preferably, in some embodiments of this application, determining the first weights corresponding to the first vector and the second vector based on the correlation between the first combustion state and the first load state and the first exhaust gas component data includes: constructing a fully connected attention network and connecting the fully connected attention network to a subsequent deep neural network; training the fully connected attention network while training the deep neural network, so that the fully connected attention network can automatically learn how to allocate the first weights, thereby improving the accuracy of the subsequent input data to the deep neural network. Further, the step of multiplying the first vector and the second vector with their respective first weights and then summing them to obtain the multimodal feature vector is achieved by constructing a fully connected layer. The input of this fully connected layer includes the first weights, the first vector, and the second vector output by the fully connected attention network, thus achieving weighted calculation.

[0099] As can be seen from the above embodiments, since the first combustion state is discrete data, it needs to be embedded and encoded to transform it from a single integer into a dense representation that can participate in calculations. Meanwhile, since the load state is real, collectable equipment operating data, it is continuous data. Therefore, through linear transformation and activation, the multidimensional continuous signal is compressed or expanded to the same dimension as the embedded vector, ensuring the feasibility of subsequent weighted fusion. Furthermore, since the load state and combustion state are two key factors affecting the final exhaust emissions, feature extraction is performed on the first combustion state and the first load state respectively to obtain a first vector and a second vector. An attention mechanism is then used to weightedly fuse the first vector and the second vector to obtain multimodal data, more realistically reflecting the current operating state of the main engine, thereby improving the input quality and learning ability of the deep neural network.

[0100] S104: Obtain the pre-trained deep neural network model, and based on the deep neural network model and the multimodal feature vector, obtain the first predicted Q value for the output power decision of each main engine in the preset discrete action space; wherein, the deep neural network is obtained by constructing a reward function with the goal of minimizing exhaust emissions, and training it using a deep reinforcement learning algorithm based on the reward function.

[0101] Preferably, in some embodiments of this application, the deep neural network is a multi-layer perceptron (MLP) structure, the input is the multimodal feature vector fused at the current time step (obtained by fusing combustion state and load state through an attention mechanism), and the output is a set of Q-value vectors, representing the action prediction value corresponding to the power output decision of each main engine in the discrete action space.

[0102] Preferably, in some embodiments of this application, the reward function is obtained by assigning corresponding weights to each exhaust gas component and calculating them in a weighted manner, combined with a simulation model. The simulation model takes as input data the current load state and combustion state of the main engine, as well as the main engine output power decision being executed, and outputs exhaust gas component data. This application does not limit the modeling process or method of this simulation model.

[0103] Preferably, in some embodiments of this application, the step of calculating the reward value for an input sample using a reward function includes: inputting the load state, combustion state, and the executed main engine output power decision from the input sample into a simulation model to obtain exhaust gas composition data; determining the second weight corresponding to each exhaust gas component in the exhaust gas composition data using the entropy weight method; determining the maximum and minimum values ​​of each exhaust gas component under different load and combustion states, and using these maximum and minimum values ​​to normalize the various exhaust gas components in the exhaust gas composition data; and finally calculating the weighted sum based on the second weight and the result of the normalization process to obtain the reward value. For example, given the load state F, combustion state B, and main engine output power decision P of a certain sample, exhaust gas composition data [carbon oxides: X1, hydrocarbons: X2, nitrogen oxides: X3, smoke opacity: X4] can be obtained through a simulation model. Next, the maximum and minimum values ​​of various exhaust gas components are obtained when the main engine is under load state F and combustion state B. Assuming that the maximum value for carbon oxides is max and the minimum value is min, and the actual content of carbon oxides is X1, the normalized value is determined by (max-X1) / (max-min). After obtaining the normalized value of each exhaust gas component, the normalized value is multiplied by the corresponding second weight and then added to obtain the reward value.

[0104] S105: Obtain the first main engine output power decision with the optimal first predicted Q value, and control the hybrid power output system to perform power distribution operation based on the propulsion power demand and the first main engine output power decision.

[0105] Furthermore, in some embodiments of this application, controlling the hybrid power output system to perform power distribution operations based on the propulsion power demand and the output power decision of the first main engine includes:

[0106] The output power of the main engine is controlled based on the output power decision of the first main engine, and the output power of the energy storage is controlled based on the difference between the propulsion power demand and the output power decision of the first main engine.

[0107] After the ship performs the power distribution operation, the third exhaust gas component data of the ship's exhaust gas at the next time step is collected, and the experience pool is updated based on the difference between the first exhaust gas component data and the third exhaust gas component data.

[0108] Based on the updated experience pool, the parameters of the deep neural network model are fine-tuned using a supervised learning method.

[0109] Preferably, in some embodiments of this application, the step of controlling the output power of the main engine based on the output power decision of the first main engine, and controlling the output power of the energy storage based on the difference between the propulsion power demand and the output power decision of the first main engine, includes: assuming that the output power decision of the first main engine is P1 and the propulsion power demand is P2, then the output power that the energy storage needs to be configured to be P2-P1.

[0110] Preferably, in some embodiments of this application, the data structure of the sample data in the experience pool is [input data: [load state, combustion state], label data: [actual Q value, first main engine output power decision]]. This application does not limit the setting of the initial experience pool; it can be set to empty, or it can be constructed using data from the training dataset used when training a deep neural network.

[0111] As can be seen from the above embodiments, once the output power decision of the first main engine is known, the propulsion power gap of the current propulsion power demand can be determined based on the first main engine output power decision. This gap is the output power that needs to be allocated to the energy storage, thereby ensuring that the power output of the main engine and the energy storage can meet the power response requirements of the ship in the next time step. Furthermore, when the hybrid power system executes the corresponding output power decision, the operating state of the main engine will change, and the exhaust gas emission composition data will also change accordingly. The third exhaust gas composition data obtained at this time is the real sample data. The experience pool is updated based on the real sample data, and the neural network parameters are continuously fine-tuned in combination with supervised learning to enhance the adaptability of the deep neural network model to changes in the real environment, while improving the robustness and generalization ability of the deep neural network model under actual working conditions.

[0112] Furthermore, in some embodiments of this application, updating the experience pool based on the difference between the first exhaust gas component data and the third exhaust gas component data includes:

[0113] Based on the first load state and the first combustion state, the second weights corresponding to various exhaust gas components are determined by the entropy weight method to obtain the weight parameter combination; wherein, one weight parameter combination corresponds to a combination of load state and combustion state.

[0114] For each exhaust gas component, calculate the first difference between its value in the first exhaust gas component data and its value in the third exhaust gas component data, and then sum the first difference corresponding to each exhaust gas component and the first product of the second weight to obtain a weighted sum.

[0115] The weighted sum is used as the actual Q value, and the first load state and the first combustion state are used as input data. Training samples are constructed based on the actual Q value, the input data, and the first main engine output power, and the training samples are added to the experience pool.

[0116] Preferably, in some embodiments of this application, the step of calculating the reward value for an input sample using a reward function can further be as follows: inputting the load state, combustion state, and the executed main engine output power decision from the input sample into the simulation model to obtain exhaust gas composition data; determining the second weights corresponding to various exhaust gas components in the exhaust gas composition data using the entropy weight method; for each exhaust gas component, calculating the first difference between its value in the first exhaust gas composition data and its value in the third exhaust gas composition data, accumulating the first product of the first difference corresponding to each exhaust gas component and the second weight to obtain a weighted sum, and using the weighted sum as the reward value.

[0117] As can be seen from the above embodiments, since load state and combustion state are two factors that simultaneously affect the components of the final exhaust gas emissions, when a certain load state and combustion state are fixed, the exhaust gas component data will have a fixed proportion of a certain exhaust gas component. Therefore, the entropy weight method is used to assign weights to the components based on the exhaust gas samples under load and combustion states to obtain a second weight. The second weight reflects the difference in the impact of each emission on the total emissions under different load and combustion states, and the actual Q value is determined according to the second weight. This actual Q value is used as a supervision signal in the subsequent deep neural network fine-tuning process, thereby improving the deep neural network's ability to adapt to changes in the real environment.

[0118] Furthermore, in some embodiments of this application, the step of determining the second weights corresponding to various exhaust gas components using the entropy weight method based on the first load state and the first combustion state includes:

[0119] Acquire several fourth exhaust gas component data when the main engine is in the first load state and the first combustion state. For each exhaust gas component, add up the corresponding values ​​in each of the fourth exhaust gas component data to obtain the first cumulative sum corresponding to each exhaust gas component.

[0120] Calculate the ratio of the value of each exhaust gas component in each of the fourth exhaust gas component data items to the corresponding first summation, and obtain the second product of each ratio and the logarithm of the ratio;

[0121] After negating each of the second products, for each exhaust gas component, sum all the corresponding second products to obtain the second summation for each exhaust gas component;

[0122] For each exhaust gas component, calculate the second difference between the corresponding second cumulative sum and the preset threshold, and use the normalized second difference as the second weight of the corresponding exhaust gas component.

[0123] For example, the step of determining the second weights corresponding to various exhaust gas components based on the first load state and the first combustion state using the entropy weight method includes: summing the values ​​of each exhaust gas component in several fourth exhaust gas component data sets. Assuming the first cumulative sum for each exhaust gas component is [carbon oxides: Sum1_1, hydrocarbons: Sum1_2, nitrogen oxides: Sum1_3, smoke opacity: Sum1_4], then dividing the value of each exhaust gas component in each fourth exhaust gas component data set [carbon oxides: X1, hydrocarbons: X2, nitrogen oxides: X3, smoke opacity: X4] by the aforementioned first cumulative sum to obtain the ratio data corresponding to each fourth exhaust gas component data set [carbon oxides: X1 / Sum1_1, hydrocarbons: X2 / Sum1_2, nitrogen oxides: X3 / Sum1_3, smoke opacity: X4 / Sum1_2]. [Sum1_4], then multiply the ratio and its corresponding logarithm for each exhaust component data to obtain the second product [carbon oxides: IE1, hydrocarbons: IE2, nitrogen oxides: IE3, smoke opacity: IE4]. Here, the logarithm can be base-2. Since the ratio is less than 1, taking the logarithm will result in a negative value, so it needs to be negativeed. After negativeing, for each exhaust component, sum the second product corresponding to each fourth exhaust component data to obtain the second sum [carbon oxides: Sum2_1, hydrocarbons: Sum2_2, nitrogen oxides: Sum2_3, smoke opacity: Sum2_4]. Then, set a preset threshold based on the number of fourth exhaust component data involved in the current calculation (e.g., assuming the above ratios are taken with base-2 logarithms, the number of fourth exhaust component data is...). The preset threshold is After subtracting the second sum corresponding to each exhaust gas component from the preset threshold, normalization is performed to obtain the second weight of each exhaust gas component (assuming the preset threshold is 1). Then the second weight of the above carbon oxides is: ( -Sum2_1) / Sum2, where Sum2=Sum2_1+Sum2_2+Sum2_3+Sum2_4).

[0124] As can be seen from the above embodiments, for each exhaust gas component, this application determines its probability of occurrence by calculating the ratio of its value in the fourth exhaust gas component data to the corresponding first cumulative sum. Furthermore, the ratio is multiplied by its corresponding logarithm, the negative value is taken, and the sum is accumulated to obtain the second cumulative sum. This assesses the dispersion and uncertainty of each exhaust gas component's distribution in the sample. The larger the second cumulative sum, the smaller the fluctuation of the corresponding exhaust gas component under the same load and combustion conditions, meaning the weaker the correlation between the exhaust gas component and the load and combustion conditions. Conversely, a smaller second cumulative sum indicates a stronger correlation between the exhaust gas component and the load and combustion conditions. Therefore, to ensure the reasonableness of the weights, the second difference between the second cumulative sum and a preset threshold is taken, making the second difference positively correlated with the aforementioned correlation, thus improving the accuracy of subsequent actual Q-value estimation.

[0125] Furthermore, in some embodiments of this application, the step of fine-tuning the parameters of the deep neural network model using a supervised learning method based on the updated experience pool includes:

[0126] For each training sample in the experience pool, based on its corresponding input data and the deep neural network model, the second predicted Q value for each main engine output power decision is obtained, and based on each second predicted Q value, the second main engine output power decision corresponding to the optimal second predicted Q value is determined.

[0127] Select a first training sample from all the training samples whose second main engine output power decision is consistent with the first main engine output power decision;

[0128] For each of the first training samples, calculate the third difference between the corresponding actual Q value and the optimal second predicted Q value;

[0129] The sum of squares and mean of the third difference is used as the loss value, and the parameters of the deep neural network model are adjusted through backpropagation based on the loss value.

[0130] Preferably, in some embodiments of this application, the step of selecting a first training sample from all the training samples whose second main engine output power decision is consistent with the first main engine output power decision includes: for each training sample in the experience pool [input data: [load state, combustion state], label data: [actual Q value, first main engine output power decision]], firstly, inputting the multimodal feature vector corresponding to the input data into the deep neural network, obtaining the second main engine output power decision with the optimal second predicted Q value output by the current deep neural network, if the second main engine output power decision is different from the first main engine output power decision, it indicates that this training sample is no longer suitable for fine-tuning the deep neural network, and selecting all samples whose second main engine output power decisions are the same as the first main engine output power decisions for subsequent fine-tuning of the deep neural network to avoid adjustment errors.

[0131] The purpose of using a deep neural network model in this application is to enable the deep neural network model to accurately predict the quality of the output power decision for each main engine in the decision space. The predicted Q value output by the deep neural network is the data used to evaluate the quality of the main engine output power decision. Therefore, the third difference between the actual Q value and the optimal second predicted Q value is used as the loss value. The parameters of the deep neural network model are adjusted through backpropagation using this loss value to improve the accuracy of the deep neural network model.

[0132] As can be seen from the above embodiments, since the deep neural network model is in a state of continuous iteration, the first main engine output power decision recorded in the training samples in the experience pool may be the result output by the deep neural network model before multiple adjustments. The second main engine output power decision output by the latest deep neural network model may have changed. At this time, the actual Q value corresponding to the second main engine output power decision is also different from that recorded in the experience pool. Therefore, it is necessary to screen the samples in the experience pool to avoid adjustment errors. By using real sample data collected in practice, the deep neural network model can be continuously learned and optimized online to improve the adaptability of the deep neural network to changes in the real environment.

[0133] In summary, compared with the prior art, the hybrid power output control method based on emission optimization provided in this application has the following beneficial effects:

[0134] Since the composition of a ship's exhaust gas is related to the combustion and load states of the main engine, a support vector machine (SVM) is used to determine the current combustion state of the main engine, thus establishing its actual operating state and providing accurate data input for the subsequent deep neural network model. Simultaneously, because the power allocation decision at the current time step affects the load and combustion states of the main engine in the next time step, which in turn determine the composition of the emitted exhaust gas, exhaust emissions can be optimized by improving the power allocation decision. This application employs a deep reinforcement learning algorithm, training a deep neural network model with a reward function aimed at minimizing exhaust emissions. The deep neural network model evaluates the predicted Q-value of each main engine output power decision, ensuring that the final first main engine output power decision minimizes exhaust emissions. Finally, by predicting the ship's propulsion power demand in advance and combining this demand with the first main engine output power decision, the energy storage output power decision can be determined, ensuring that the final power allocation operation meets the ship's power response requirements in the next time step.

[0135] Example 2

[0136] like Figure 2 As shown, based on the above-mentioned method embodiments, an embodiment of this application provides a hybrid power output control device based on emission optimization, including: a propulsion power demand prediction module 201, a combustion state classification module 202, a feature fusion module 203, a Q-value prediction module 204, and a power distribution module 205.

[0137] Further, in some embodiments of this application, the propulsion power demand prediction module 201 is used to collect propulsion power data of the ship at several historical time steps, and obtain the propulsion power demand for the next time step based on all the propulsion power data using a pre-trained long short-term memory network model; the combustion state classification module 202 is used to collect the first exhaust gas component data of the ship's exhaust gas at the current time step, and determine the first combustion state of the main engine at the current time step using a pre-trained support vector machine based on the first exhaust gas component data; the feature fusion module 203 is used to collect the first load state of the main engine at the current time step, and weightedly fuse the first combustion state and the load state using an attention mechanism module. The system describes a first load state and obtains a multimodal feature vector. The Q-value prediction module 204 is used to obtain a pre-trained deep neural network model and, based on the deep neural network model and the multimodal feature vector, obtain a first predicted Q-value for the output power decision of each main engine within a preset discrete action space. The deep neural network is obtained by constructing a reward function with the goal of minimizing exhaust emissions and training it using a deep reinforcement learning algorithm. The power allocation module 205 is used to obtain the first main engine output power decision with the optimal first predicted Q-value and, based on the propulsion power requirement and the first main engine output power decision, control the hybrid power output system to perform a power allocation operation.

[0138] Further, in some embodiments of this application, the combustion state classification module 202 includes: a support vector machine selection unit and a support vector machine invocation unit; the combustion state classification module 202 is used to determine the first combustion state of the main engine at the current time step based on the first exhaust gas composition data using a pre-trained support vector machine, including: the support vector machine selection unit is used to select a corresponding first support vector machine based on the first load state; wherein, the first support vector machine is trained and obtained based on the second exhaust gas composition data of the ship's exhaust gas when the main engine is in different combustion states under the first load state; the support vector machine invocation unit is used to determine the first combustion state of the main engine when it is in the first load state at the current time step based on the first support vector machine and the first exhaust gas composition data.

[0139] Further, in some embodiments of this application, the feature fusion module 203 includes: a first vector extraction unit, a second vector extraction unit, a first weight determination unit, and a feature fusion unit; the feature fusion module 203 is used to collect the first load state of the main engine at the current time step, and to obtain a multimodal feature vector by weighted fusion of the first combustion state and the first load state through an attention mechanism module, including: the first vector extraction unit is used to embed and encode the first combustion state through a preset embedding layer to obtain a first vector; the second vector extraction unit is used to perform linear transformation and activation operations on the first load state through a preset feedforward network to obtain a second vector with the same dimension as the first vector; the first weight determination unit is used to determine the first weights corresponding to the first vector and the second vector respectively based on the correlation between the first combustion state and the first load state and the first exhaust gas component data; the feature fusion unit multiplies the first vector and the second vector with their respective first weights and then adds them to obtain the multimodal feature vector.

[0140] Further, in some embodiments of this application, the power allocation module 205 includes: an output power control unit, an experience pool update unit, and a parameter fine-tuning unit; the power allocation module 205 is used to control the hybrid power output system to perform a power allocation operation based on the propulsion power demand and the first main engine output power decision, including: the output power control unit is used to control the output power of the main engine based on the first main engine output power decision, and to control the output power of the energy storage based on the difference between the propulsion power demand and the first main engine output power decision; the experience pool update unit is used to collect the third exhaust gas component data of the ship's exhaust gas at the next time step after the ship performs the power allocation operation, and update the experience pool based on the difference between the first exhaust gas component data and the third exhaust gas component data; the parameter fine-tuning unit is used to fine-tune the parameters of the deep neural network model through a supervised learning method based on the updated experience pool.

[0141] Further, in some embodiments of this application, the experience pool update unit is used to update the experience pool based on the difference between the first exhaust gas component data and the third exhaust gas component data, including: determining the second weights corresponding to various exhaust gas components using the entropy weight method based on the first load state and the first combustion state, and obtaining a weight parameter combination; wherein, a weight parameter combination corresponds to a combination of a load state and a combustion state; for each exhaust gas component, calculating the first difference between its value in the first exhaust gas component data and its value in the third exhaust gas component data, and accumulating the first product of the first difference corresponding to each exhaust gas component and the second weight to obtain a weighted sum; using the weighted sum as the actual Q value, using the first load state and the first combustion state as input data, constructing training samples based on the actual Q value, the input data, and the first main engine output power decision, and adding the training samples to the experience pool.

[0142] Further, in some embodiments of this application, the step of determining the second weights corresponding to various exhaust gas components based on the first load state and the first combustion state using the entropy weight method includes: acquiring several fourth exhaust gas component data when the main engine is in the first load state and the first combustion state; for each exhaust gas component, adding its corresponding value in each of the fourth exhaust gas component data to obtain a first cumulative sum corresponding to each exhaust gas component; calculating the ratio of the value of each exhaust gas component in each of the fourth exhaust gas component data to the corresponding first cumulative sum, and obtaining a second product of each ratio and the logarithm of the ratio; after negativening each second product, for each exhaust gas component, accumulating all its corresponding second products to obtain a second cumulative sum corresponding to each exhaust gas component; for each exhaust gas component, calculating a second difference between its corresponding second cumulative sum and a preset threshold, and using the normalized second difference as the second weight corresponding to the exhaust gas component.

[0143] Further, in some embodiments of this application, the parameter fine-tuning unit is used to fine-tune the parameters of the deep neural network model using a supervised learning method based on the updated experience pool, including: for each training sample in the experience pool, obtaining a second predicted Q value for each main engine output power decision based on its corresponding input data and the deep neural network model, and determining a second main engine output power decision corresponding to the optimal second predicted Q value based on each second predicted Q value; selecting a first training sample from all the training samples whose second main engine output power decision is consistent with the first main engine output power decision; for each first training sample, calculating a third difference between its corresponding actual Q value and the optimal second predicted Q value; using the sum of squares and mean of the third difference as a loss value, and adjusting the parameters of the deep neural network model through backpropagation based on the loss value.

[0144] Further, in some embodiments of this application, the propulsion power demand prediction module 201 includes: a two-dimensional time-series feature tensor acquisition unit, a propulsion power trend change feature extraction unit, and a propulsion power demand prediction unit; the propulsion power demand prediction module 201 is used to obtain the propulsion power demand for the next time step based on all the propulsion power data using a pre-trained long short-term memory network model, including: the two-dimensional time-series feature tensor acquisition unit is used to collect first sea state data for the several historical time steps and second sea state data for the next time step, and concatenate the first sea state data and the propulsion power data according to the time steps to obtain a two-dimensional time-series feature tensor, and concatenate the second sea state data to the end of the two-dimensional time-series feature tensor; the propulsion power trend change feature extraction unit is used to extract propulsion power trend change features from the two-dimensional time-series feature tensor using a one-dimensional convolutional neural network; the propulsion power demand prediction unit is used to input the propulsion power trend change features into the long short-term memory network model to obtain the propulsion power demand for the next time step.

[0145] It is understood that the above-described device embodiments correspond to the method embodiments of this application, and can implement the emission-optimized hybrid power output control method provided by any of the above-described method embodiments of this application.

[0146] In summary, compared with the prior art, the hybrid power output control device based on emission optimization provided in this application has the following beneficial effects:

[0147] Since the composition of a ship's exhaust gas is related to the combustion and load states of the main engine, a support vector machine (SVM) is used to determine the current combustion state of the main engine, thus establishing its actual operating state and providing accurate data input for the subsequent deep neural network model. Simultaneously, because the power allocation decision at the current time step affects the load and combustion states of the main engine in the next time step, which in turn determine the composition of the emitted exhaust gas, exhaust emissions can be optimized by improving the power allocation decision. This application employs a deep reinforcement learning algorithm, training a deep neural network model with a reward function aimed at minimizing exhaust emissions. The deep neural network model evaluates the predicted Q-value of each main engine output power decision, ensuring that the final first main engine output power decision minimizes exhaust emissions. Finally, by predicting the ship's propulsion power demand in advance and combining this demand with the first main engine output power decision, the energy storage output power decision can be determined, ensuring that the final power allocation operation meets the ship's power response requirements in the next time step.

[0148] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0149] Based on the above embodiments of the emission-optimized hybrid power output control method, another embodiment of this application provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the emission-optimized hybrid power output control method of any embodiment of this application.

[0150] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more module units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0151] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0152] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0153] Based on the above-described method embodiments, another embodiment of this application provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the emission-optimized hybrid power output control method described in any of the above-described method embodiments of this application.

[0154] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

Claims

1. A hybrid power output control method based on emission optimization, characterized in that, include: The propulsion power data of the ship at several historical time steps are collected, and the propulsion power requirement for the next time step is obtained based on all the propulsion power data using a pre-trained long short-term memory network model. Collect the first exhaust gas component data of the ship's exhaust gas at the current time step, and determine the first combustion state of the main engine at the current time step based on the first exhaust gas component data and a pre-trained support vector machine. The first load state of the main engine at the current time step is collected, and the first combustion state and the first load state are weighted and fused through the attention mechanism module to obtain a multimodal feature vector; A pre-trained deep neural network model is obtained, and based on the deep neural network model and the multimodal feature vector, the first predicted Q value for the output power decision of each main engine in a preset discrete action space is obtained; wherein, the deep neural network is obtained by constructing a reward function with the goal of minimizing exhaust emissions, and training it using a deep reinforcement learning algorithm based on the reward function; The first main engine output power decision is obtained to obtain the optimal first predicted Q value, and the hybrid power output system is controlled to perform power distribution operation based on the propulsion power demand and the first main engine output power decision.

2. The hybrid power output control method based on emission optimization as described in claim 1, characterized in that, The step of determining the first combustion state of the main engine at the current time step based on the first exhaust gas composition data using a pre-trained support vector machine includes: Based on the first load state, a corresponding first support vector machine is selected; wherein, the first support vector machine is obtained by training based on the second exhaust gas component data of the ship's exhaust gas when the main engine is in different combustion states under the first load state; Based on the first support vector machine and according to the first exhaust gas composition data, the first combustion state of the main engine under the first load state at the current time step is determined.

3. The hybrid power output control method based on emission optimization as described in claim 1, characterized in that, The process involves acquiring the first load state of the main engine at the current time step, and then using an attention mechanism module to weightedly fuse the first combustion state and the first load state to obtain a multimodal feature vector, including: The first combustion state is embedded and encoded through a preset embedding layer to obtain a first vector; By using a pre-defined feedforward neural network, the first load state is linearly transformed and activated to obtain a second vector with the same dimension as the first vector. Based on the correlation between the first combustion state and the first load state and the first exhaust gas component data, determine the first weights corresponding to the first vector and the second vector respectively. The first vector and the second vector are multiplied by their respective first weights and then added together to obtain the multimodal feature vector.

4. The hybrid power output control method based on emission optimization as described in claim 1, characterized in that, The step of controlling the hybrid power output system to perform power distribution operations based on the propulsion power demand and the output power of the first main engine includes: The output power of the main engine is controlled based on the output power decision of the first main engine, and the output power of the energy storage is controlled based on the difference between the propulsion power demand and the output power decision of the first main engine. After the ship performs the power distribution operation, the third exhaust gas component data of the ship's exhaust gas at the next time step is collected, and the experience pool is updated based on the difference between the first exhaust gas component data and the third exhaust gas component data. Based on the updated experience pool, the parameters of the deep neural network model are fine-tuned using a supervised learning method.

5. The hybrid power output control method based on emission optimization as described in claim 4, characterized in that, The step of updating the experience pool based on the difference between the first exhaust gas component data and the third exhaust gas component data includes: Based on the first load state and the first combustion state, the second weights corresponding to various exhaust gas components are determined by the entropy weight method to obtain the weight parameter combination; wherein, one weight parameter combination corresponds to a combination of load state and combustion state. For each exhaust gas component, calculate the first difference between its value in the first exhaust gas component data and its value in the third exhaust gas component data, and then sum the first difference corresponding to each exhaust gas component and the first product of the second weight to obtain a weighted sum. The weighted sum is used as the actual Q value, and the first load state and the first combustion state are used as input data. Training samples are constructed based on the actual Q value, the input data, and the first main engine output power, and the training samples are added to the experience pool.

6. The hybrid power output control method based on emission optimization as described in claim 5, characterized in that, The step of determining the second weights corresponding to various exhaust gas components based on the first load state and the first combustion state using the entropy weight method includes: Acquire several fourth exhaust gas component data when the main engine is in the first load state and the first combustion state. For each exhaust gas component, add up the corresponding values ​​in each of the fourth exhaust gas component data to obtain the first cumulative sum corresponding to each exhaust gas component. Calculate the ratio of the value of each exhaust gas component in each of the fourth exhaust gas component data items to the corresponding first summation, and obtain the second product of each ratio and the logarithm of the ratio; After negating each of the second products, for each exhaust gas component, sum all the corresponding second products to obtain the second summation for each exhaust gas component; For each exhaust gas component, calculate the second difference between the corresponding second cumulative sum and the preset threshold, and use the normalized second difference as the second weight of the corresponding exhaust gas component.

7. The hybrid power output control method based on emission optimization as described in claim 5, characterized in that, The step of fine-tuning the parameters of the deep neural network model using a supervised learning method based on the updated experience pool includes: For each training sample in the experience pool, based on its corresponding input data and the deep neural network model, the second predicted Q value for each main engine output power decision is obtained, and based on each second predicted Q value, the second main engine output power decision corresponding to the optimal second predicted Q value is determined. Select a first training sample from all the training samples whose second main engine output power decision is consistent with the first main engine output power decision; For each of the first training samples, calculate the third difference between the corresponding actual Q value and the optimal second predicted Q value; The sum of squares and mean of the third difference is used as the loss value, and the parameters of the deep neural network model are adjusted through backpropagation based on the loss value.

8. The hybrid power output control method based on emission optimization as described in claim 1, characterized in that, The method of obtaining the propulsion power requirement for the next time step using the pre-trained Long Short-Term Memory network model based on all the propulsion power data includes: Collect the first sea state data of the aforementioned historical time steps and the second sea state data of the next time step, and concatenate the first sea state data with the propulsion power data according to the time steps to obtain a two-dimensional time series feature tensor, and concatenate the second sea state data to the end of the two-dimensional time series feature tensor; The propulsion power trend change features are extracted from the two-dimensional temporal feature tensor using a one-dimensional convolutional neural network. The propulsion power trend change characteristics are input into the long short-term memory network model to obtain the propulsion power requirement for the next time step.

9. A hybrid power output control device based on emission optimization, characterized in that, include: The hybrid power output control device based on emission optimization is characterized by comprising: a propulsion power demand prediction module, a combustion state classification module, a feature fusion module, a Q-value prediction module, and a power distribution module. The propulsion power demand prediction module is used to collect propulsion power data of the ship at several historical time steps, and obtain the propulsion power demand for the next time step based on all the propulsion power data through a pre-trained long short-term memory network model. The combustion state classification module is used to collect the first exhaust gas component data of the ship's exhaust gas at the current time step, and determine the first combustion state of the main engine at the current time step through a pre-trained support vector machine based on the first exhaust gas component data. The feature fusion module is used to collect the first load state of the main engine at the current time step, and through the attention mechanism module, weightedly fuse the first combustion state and the first load state to obtain a multimodal feature vector. The Q-value prediction module is used to acquire a pre-trained deep neural network model and, based on the deep neural network model and the multimodal feature vector, acquire the first predicted Q-value for the output power decision of each main engine in a preset discrete action space; wherein, the deep neural network is acquired by constructing a reward function with the goal of minimizing exhaust emissions and training it using a deep reinforcement learning algorithm based on the reward function. The power distribution module is used to obtain the first main engine output power decision with the optimal first predicted Q value, and control the hybrid power output system to perform power distribution operation according to the propulsion power demand and the first main engine output power decision.

10. A hybrid power output control device based on emission optimization as described in claim 9, characterized in that, The combustion state classification module includes: a support vector machine selection unit and a support vector machine invocation unit; the combustion state classification module is used to determine the first combustion state of the main engine at the current time step based on the first exhaust gas composition data and through a pre-trained support vector machine, including: The support vector machine selection unit is used to select a corresponding first support vector machine according to the first load state; wherein, the first support vector machine is obtained by training based on the second exhaust gas composition data of the ship's exhaust gas when the main engine is in different combustion states under the first load state. The support vector machine invocation unit is used to determine the first combustion state of the main engine when it is in the first load state at the current time step, based on the first support vector machine and the first exhaust gas composition data.

Images