Maintenance method and device for fan components, program product and storage medium

Abstract

This application discloses a maintenance method, apparatus, program product, and storage medium for wind turbine components. The method includes: inputting the state information of the wind turbine component into a maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions, wherein the state information indicates the environmental information of the wind turbine component; determining mask values ​​corresponding to the multiple reference actions based on the state information, and updating the multiple reference probabilities based on the mask values, wherein the mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions; normalizing the updated reference probabilities to obtain target probabilities corresponding to the multiple reference actions, and determining target actions based on the target probabilities. This application solves the technical problem of high wind turbine operation and maintenance costs provided by related technologies.

Description

Technical Field

[0001] This application relates to the field of wind power generation, and more specifically, to a method and apparatus for maintaining wind turbine components, a program product, and a storage medium. Background Technology

[0002] Wind turbines are prone to component wear due to natural and mechanical loads in both onshore and offshore environments, such as bearings, gears, and blades. In some wind farms, due to remote locations, complex environments, and a limited number of resident maintenance personnel, it is necessary to assess the condition of the turbines to determine whether maintenance is required.

[0003] In related technologies, a signal can be sent after a unit malfunction to indicate that the unit needs maintenance. However, this method requires the unit to malfunction first, which can easily lead to prolonged downtime and economic losses. Alternatively, preventative maintenance or time-based maintenance can be performed periodically, but this method can easily lead to over-maintenance of the unit.

[0004] In other words, the condition-based maintenance methods for wind turbine components provided by related technologies require wind turbine components to be damaged or inspected in advance, resulting in cost losses during the wind turbine maintenance process, thus leading to the problem of high existing wind turbine operation and maintenance costs.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides a method and apparatus for maintaining wind turbine components, a program product and a storage medium, to at least solve the technical problem of high wind turbine operation and maintenance costs provided by related technologies.

[0007] According to one aspect of the embodiments of this application, a maintenance method for a wind turbine component is provided. The method includes: inputting state information of the wind turbine component into a maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions, wherein the state information is used to indicate the environmental information of the wind turbine component; determining mask values ​​corresponding to the multiple reference actions based on the state information, and updating the multiple reference probabilities based on the mask values, wherein the mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively; normalizing the updated reference probabilities to obtain target probabilities corresponding to the multiple reference actions, and determining target actions based on the target probabilities.

[0008] Optionally, the above-mentioned determination of mask values ​​corresponding to multiple reference actions based on state information, and probability update of multiple reference probabilities based on mask values, includes: when a reference action is determined to be a disrecommended action based on state information, updating the reference probability corresponding to the reference action determined to be a disrecommended action to a first probability; when a reference action is determined to be an unavailable action based on state information, updating the reference probability corresponding to the reference action determined to be an unavailable action to a second probability, wherein the first probability is greater than the second probability.

[0009] Optionally, the training process of the above maintenance action prediction model includes: determining the number of training environments for the maintenance action prediction model; initializing the parameters of the training environments; and training the maintenance action prediction model sequentially in multiple training environments.

[0010] Optionally, the above-mentioned training of the maintenance action prediction model in multiple training environments sequentially includes: the training process in each training environment includes: determining the state space of the current training environment; in the i-th round of training, performing the following operations: determining the i-th action and the i-th reward parameter based on the i-th state information and the policy function, wherein the state information is determined by the state space, and i is an integer greater than 0; updating the policy function based on the i-th action and the i-th reward parameter to obtain the (i+1)-th policy function; calculating the accuracy of the maintenance action prediction model, and if the accuracy of the maintenance action prediction model is greater than a preset threshold, determining the (i+1)-th policy function as the trained policy function.

[0011] Optionally, the above-mentioned updating of the policy function based on the i-th action and the i-th reward parameter to obtain the (i+1)-th policy function includes: determining the i-th value difference, where the value difference is the difference between the expected reward obtained by performing the i-th action under the i-th state information and the expected reward of the i-th policy function under the i-th state information; determining the i-th new-old policy probability ratio, where the i-th new-old policy probability ratio is the probability of the i-th policy function performing the i-th action under the i-th state information and the probability of the (i-1)-th policy function performing the i-th action under the i-th state information; trimming the product of the i-th value difference and the i-th new-old policy probability ratio to obtain a loss function, and updating the parameters of the i-th policy function based on the loss function to obtain the (i+1)-th policy function.

[0012] Optionally, the above-mentioned determination of the i-th action and the i-th reward parameter based on the i-th state information and the policy function includes: determining the mask values ​​corresponding to multiple reference actions based on the i-th state information, and updating the probabilities corresponding to multiple reference actions based on the mask values; normalizing the probabilities corresponding to the updated reference actions to obtain the probabilities corresponding to multiple reference actions, and determining the i-th action based on the probabilities.

[0013] Optionally, the above-mentioned training of the maintenance action prediction model in multiple training environments includes: setting up three training environments when the wind turbine component is a wind turbine blade, wherein the state space and reward function of the training environments are different; sorting the training environments in order of difficulty level, and training the maintenance action prediction model in the three training environments in sequence to obtain the trained maintenance action prediction model.

[0014] Optionally, when the wind turbine component is a wind turbine blade, three training environments are set up, including at least one of the following: In the case of a simple difficulty level training environment, the state space is where the wind speed changes within the first and second intervals, and the blade state changes linearly, with the reward function being the difference between the first reward and the first penalty; In the case of a medium difficulty level training environment, the state space is where the wind speed changes within the first, second, and third intervals, and the blade state changes linearly, with the reward function being the difference between the first reward and the first and second penalties; In the case of a complex difficulty level training environment, the state space is where the wind speed changes within the first, second, third, and fourth intervals, and the blade state changes non-linearly, with the reward function being the difference between the first reward and the first and second penalties.

[0015] According to another aspect of the embodiments of this application, a maintenance device for wind turbine components is also provided. The device includes: a probability prediction unit, used to input the state information of the wind turbine components into a maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions, wherein the state information is used to indicate the environmental information of the wind turbine components; an action masking unit, used to determine mask values ​​corresponding to multiple reference actions based on the state information, and to update the multiple reference probabilities based on the mask values, wherein the mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions respectively; and an action determination unit, used to normalize the updated reference probabilities to obtain target probabilities corresponding to multiple reference actions, and to determine target actions based on the target probabilities.

[0016] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the maintenance method of the above-described wind turbine components when running.

[0017] According to another aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program / instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program / instructions from the computer-readable storage medium, and executes the computer program / instructions, causing the computer device to perform the maintenance method for the fan components described above.

[0018] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the maintenance method of the aforementioned wind turbine components.

[0019] In this embodiment, the state information of wind turbine components is input into a maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions. The state information indicates the environmental information of the wind turbine components. Mask values ​​corresponding to multiple reference actions are determined based on the state information, and the probabilities of multiple reference actions are updated based on these mask values. The mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively. The updated reference probabilities are normalized to obtain target probabilities corresponding to multiple reference actions, and target actions are determined based on these target probabilities. The probability of each action can be determined based on the state information of the wind turbine components, and the execution probability of the reference actions can be determined based on the state information. The probability of issuing maintenance instructions in unsafe environments is eliminated by using action masks, ensuring that the obtained actions conform to actual operation and maintenance work arrangements and safety regulations. Furthermore, the probabilities of each reference action are recalculated, providing maintenance actions more adaptable to complex environments, improving maintenance efficiency, reducing maintenance costs and risks, and solving the technical problem of high wind turbine operation and maintenance costs provided by related technologies. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a schematic diagram of the hardware environment for an optional maintenance method for a wind turbine component according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of an optional maintenance method for a wind turbine component according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of an optional maintenance method for a wind turbine component according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of another optional maintenance method for a wind turbine component according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of an optional maintenance device for a fan component according to an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] According to one aspect of the embodiments of this application, a method for maintaining wind turbine components is provided. As an optional implementation, the above-described method for maintaining wind turbine components can be applied to, but is not limited to, [examples of such methods]. Figure 1 The hardware environment shown represents a maintenance system for wind turbine components. This maintenance system may include, but is not limited to, terminal device 102, network 110, server 112, and database 114. Terminal device 102 runs a target client (e.g., ...). Figure 1 As shown, taking a target client as an example of a client monitoring the status of a wind turbine unit. The aforementioned terminal device 102 includes a display 108, a processor 106, and a memory 104. The display 108 can be used to display the status of the wind turbine unit and the surrounding environmental data, and also provides a human-machine interface to receive human-machine interaction operations via touch on different controls. The processor is used to generate interaction instructions in response to the aforementioned human-machine interaction operations and send the interaction instructions to the server to obtain environmental information and the status information of the wind turbine components. The memory is used to store the probability of the reference actions corresponding to the wind turbine components.

[0030] In addition, server 112 includes processing engine 116, which is used to perform store or read operations on database 114. Specifically, processing engine 116 reads the status information of wind turbine components from database 114.

[0031] Assumption Figure 1 The terminal device 102 runs a client for monitoring the status of wind turbine components. The specific process in this embodiment is as follows: In step S102, in response to a touch operation on the client, the terminal device 102 obtains the status information of the wind turbine components and the surrounding environment from the server 112 via the network 110. Then, steps S104-S108 are executed, inputting the status information of the wind turbine components into the maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions. The mask values ​​corresponding to the multiple reference actions are determined based on the status information to update the reference probabilities. Finally, the reference probabilities are normalized to obtain the target probability to determine the target action.

[0032] Optionally, in this embodiment, the terminal device 102 can be a terminal device configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client can be a data acquisition client, a visual monitoring client, or other client that supports providing wind turbine maintenance action prediction tasks. The network may include, but is not limited to, wired networks and wireless networks. The wired network includes: local area network (LAN), metropolitan area network (MAN), and wide area network (WAN). The wireless network includes: Bluetooth, Wi-Fi, and other networks that enable wireless communication. The server may be a single server, a server cluster consisting of multiple servers, or a cloud server. The above is merely an example, and no limitation is made in this embodiment.

[0033] Optionally, in this embodiment, the above-mentioned maintenance method for wind turbine components can be applied, but is not limited to, onshore wind farms, such as in high-altitude areas where wind speeds may be high and the surrounding environment is complex, making it difficult to deploy a large number of on-site maintenance personnel for continuous maintenance. Therefore, using this maintenance method, the status information of the wind turbine components (such as bearing information and surrounding environmental information) can be obtained in each prediction cycle, and then the reference probability of a reference action can be obtained based on the status information. The reference action is then masked based on the status information. For example, in high-altitude areas where wind speeds may be high, if the wind speed exceeds a certain range, the maintenance action is set to an unavailable action to avoid personnel safety issues. If the wind speed is normal, a suitable target action is determined based on the actual scenario, and maintenance suggestions are given. This reduces maintenance costs and improves maintenance efficiency while ensuring personnel safety.

[0034] Optionally, in this embodiment, the maintenance method for the wind turbine components described above can be applied, but is not limited to, offshore wind farms, such as wind turbine units deployed on the sea surface. Their blades are susceptible to wear and failure due to the influence of sea waves and corrosive salt water. However, the large area of ​​the sea surface makes it impossible to allocate a sufficient number of maintenance personnel to each wind farm. Therefore, using the aforementioned maintenance method for wind turbine components, environmental information and blade status information of the wind turbine unit at sea can be obtained in each prediction cycle, and maintenance actions can be predicted to provide suggestions. When the offshore wind turbine is in a critical stage of operation, reference actions can be masked as unrecommended actions, and target probabilities can be calculated to provide target actions. By combining the wind turbine's operating status with reasonable action predictions, the flexibility of maintenance is improved.

[0035] The above application scenarios are merely examples. Any scenario related to the maintenance of wind turbine components can utilize the solution of this application. This embodiment does not impose any limitations on these scenarios.

[0036] In this embodiment, the state information of wind turbine components is input into a maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions. The state information indicates the environmental information of the wind turbine components. Mask values ​​corresponding to multiple reference actions are determined based on the state information, and the probabilities of multiple reference actions are updated based on these mask values. The mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively. The updated reference probabilities are normalized to obtain target probabilities corresponding to multiple reference actions, and target actions are determined based on these target probabilities. The probability of each action can be determined based on the state information of the wind turbine components, and the execution probability of the reference actions can be determined based on the state information. The probability of issuing maintenance instructions in unsafe environments is eliminated by using action masks, ensuring that the obtained actions conform to actual operation and maintenance work arrangements and safety regulations. Furthermore, the probabilities of each reference action are recalculated, providing maintenance actions more adaptable to complex environments, improving maintenance efficiency, reducing maintenance costs and risks, and solving the technical problem of high wind turbine operation and maintenance costs provided by related technologies.

[0037] As an optional implementation method, such as Figure 2 As shown, the maintenance method for the above-mentioned wind turbine components includes the following steps:

[0038] S202, input the status information of the wind turbine components into the maintenance action prediction model to obtain the reference probabilities corresponding to multiple reference actions. The status information is used to indicate the environmental information of the wind turbine components.

[0039] It should be noted that during the operation of the wind turbine unit, due to its special working environment and mechanical stress, some components will experience wear. The components of the wind turbine can include: bearings, gears and gearboxes, blades, bolts, slip rings and carbon brushes, and electrical system.

[0040] Bearings in wind turbines include main shaft bearings, yaw bearings, pitch bearings, and generator bearings. They play a supporting role and reduce friction in wind turbines. Bearing wear primarily manifests as fatigue wear, abrasive wear, and oxidative wear. The gearbox is a critical component in wind turbines, containing various gears. Gearboxes have a relatively high failure rate, and common wear problems include fatigue wear, adhesive wear, and surface wear. Blades are a key non-metallic component, typically made of composite materials. Wear on the blade leading edge is severe; due to prolonged exposure to harsh weather, they are subjected to impact and abrasion, leading to cracking or even peeling of the protective coating. Metal components such as bolts used in wind turbines, due to corrosion or inadequate protection, can cause the wind turbine to malfunction, and in severe cases, even collapse. Carbon brush wear on the generator slip ring is categorized into mechanical and electrical types. When the carbon brush contacts the slip ring, factors such as spring pressure and material elastic deformation create pressure at the direct contact point, resulting in significant relative sliding, which can scratch the slip ring surface and increase wear. The robustness and reliability of electrical connections are crucial for the stable operation of wind turbines. Wear, corrosion, or loosening of cables and connections require special attention, especially in high-temperature areas, to prevent cable aging.

[0041] It should be noted that the maintenance action prediction model can infer target actions based on data obtained from sensors. It can make decisions and execute actions based on the perceived environment.

[0042] In a possible implementation, the maintenance action prediction model can use state information and preset reference actions to obtain output results. The output results can be suggested actions, which can be one or more of the reference actions.

[0043] It should be noted that the state information can be the model's state space. The parameters to be acquired can be predetermined, and data corresponding to the current environmental parameters can be obtained. Parameters such as time, blade age, system state, cumulative fatigue damage, critical threshold, recent wind speed, the number and severity of extreme weather events encountered during service life, and the number of imperfect maintenance operations performed can be used as the state space. Data corresponding to parameters in the state space above the current moment can be obtained.

[0044] In a possible implementation, the reference actions are predetermined, and all reference actions are defined as an action space. Since the objective of this application is to determine the maintenance actions for wind turbine components, the reference actions are discrete and can be "replace all", "no maintenance", "perform A repair", etc. The setting of the reference actions can be set according to the actual situation.

[0045] It should be noted that the maintenance action prediction model can be a reference probability for each reference action. It can output the probability distribution of each reference action. Based on the probability distribution, the target action to be output can be selected according to the preset strategy. This will be explained further later and will not be repeated here.

[0046] S204, determine the mask values ​​corresponding to multiple reference actions based on the status information, and update the probability of multiple reference probabilities based on the mask values. The mask values ​​include the mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively.

[0047] It should be noted that after obtaining the probability distribution of the action space, action constraints can be applied to the reference actions. In a given application environment, some actions may be illegal or infeasible. Impossible or unusable actions can be filtered out by masking the reference actions.

[0048] In an optional implementation, masking the reference actions needs to be done according to the actual environment, that is, actual environmental feedback is required, and it can be determined whether each reference action is executable based on the status information.

[0049] In an optional implementation, the reference action can be masked into three categories: the reference action can be usable (its mask value can be 2), the reference action can be unrecommended (its mask value can be 1), and the reference action can be unusable (its mask value can be 0).

[0050] It should be noted that the three types of reference actions can be determined based on state information. For example, the mask type of the reference action can be determined comprehensively based on environmental conditions, system status, and the degree of blade fatigue damage. Alternatively, the mask type of the reference action can be determined based on the current parameters of the state space mentioned above. As mentioned earlier, in some areas (such as the plateau areas mentioned above), wind speeds may exceed the safe operating environment regulations (e.g., 12 m / s). Under such wind speeds, safe operation may not be guaranteed, and maintenance actions may be set as unavailable (e.g., determining reference actions such as performing A-level repair, B-level repair, and C-level repair as unavailable), while not performing maintenance can be determined as available. Or, at certain times, the wind turbine is in a critical operating phase and cannot be immediately stopped for maintenance. In such cases, maintenance actions can be set as not recommended (e.g., determining reference actions such as performing A-level repair, B-level repair, and C-level repair as not recommended). In the next forecast cycle, the wind turbine may have stopped operating at a critical stage, and the maintenance action can be determined as available.

[0051] In an optional implementation, after the model is determined, different probabilities can be set for different types of actions. For example, the probability of an available action can be set to one probability, the probability of a discouraged action can be set to one probability, and the probability of an unavailable action can be set to one probability. The probabilities of actions of the same type can be set to the same value.

[0052] S206, normalize the updated reference probabilities to obtain the target probabilities corresponding to multiple reference actions, and determine the target action based on the target probabilities.

[0053] It's important to note that after adjusting the probabilities of discouraged and unavailable actions, the probabilities of all actions need to be renormalized to ensure they sum to 1. This can be achieved by dividing the probability of each action by the sum of the probabilities of all actions.

[0054] In an optional implementation, after normalization, the target probability corresponding to the reference action can be obtained, and after determining the target probability, the recommended target action can be determined. For example, a recommendation can be made using a trained policy function.

[0055] It should be noted that the strategy for recommending the target action can be a deterministic strategy, which determines the reference action with the highest probability as the target action. It can also be a stochastic strategy, which randomly selects a reference action based on its target probability. Alternatively, a parameter ε can be preset, selecting the optimal reference action with a probability of 1-ε, and randomly selecting a reference action based on the target probability with a probability of ε.

[0056] In this embodiment, the state information of wind turbine components is input into a maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions. The state information indicates the environmental information of the wind turbine components. Mask values ​​corresponding to multiple reference actions are determined based on the state information, and the probabilities of multiple reference actions are updated based on these mask values. The mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively. The updated reference probabilities are normalized to obtain target probabilities corresponding to multiple reference actions, and target actions are determined based on these target probabilities. The probability of each action can be determined based on the state information of the wind turbine components, and the execution probability of the reference actions can be determined based on the state information. The probability of issuing maintenance instructions in unsafe environments is eliminated by using action masks, ensuring that the obtained actions conform to actual operation and maintenance work arrangements and safety regulations. Furthermore, the probabilities of each reference action are recalculated, providing maintenance actions more adaptable to complex environments, improving maintenance efficiency, reducing maintenance costs and risks, and solving the technical problem of high wind turbine operation and maintenance costs provided by related technologies.

[0057] In an optional implementation, the above-mentioned determination of mask values ​​corresponding to multiple reference actions based on state information and probability update of multiple reference probabilities based on mask values ​​includes: when a reference action is determined to be a deprecated action based on state information, updating the reference probability corresponding to the deprecated action to a first probability; when a reference action is determined to be an unusable action based on state information, updating the reference probability corresponding to the unusable action to a second probability, wherein the first probability is greater than the second probability.

[0058] It should be noted that the first and second probabilities can be set according to the actual situation. The first probability can be set relatively small, and the second probability can be set very small. For example, the first probability can be set to 1e-2, and the second probability to 1e-10. Available actions can remain unchanged in their probabilities, retaining their original probabilities.

[0059] In an optional implementation, it can be seen that after the above probability update, the probability corresponding to all reference actions will no longer be equal to 1, and they can be renormalized by dividing the probability of each action by the sum of the probabilities of all actions.

[0060] In an optional implementation, the training process of the maintenance action prediction model includes: determining the number of training environments for the maintenance action prediction model; initializing the parameters of the training environments; and training the maintenance action prediction model sequentially in multiple training environments.

[0061] It should be noted that during the model training process, learning can proceed gradually from simple to complex to help the model generalize better and reduce overfitting during training.

[0062] In an optional implementation, the number of training environments can be determined first, and the model can be randomly initialized at the start of training. Training can then begin in the simplest environment, allowing the model to learn tasks within that simple context. After training in the simple environment, the model can be moved to the next difficulty level for further training, and this process can be repeated in various difficulty environments until training in all environments is complete. The training method is explained in detail below using wind turbine blades as an example.

[0063] In an optional implementation, the above-mentioned training of the maintenance action prediction model in multiple training environments includes: when the wind turbine component is a wind turbine blade, setting up three training environments, wherein the state space and reward function of the training environments are different; sorting the training environments in order of difficulty level, and training the maintenance action prediction model in the three training environments in sequence to obtain a trained maintenance action prediction model.

[0064] In an optional implementation, when the wind turbine component is a wind turbine blade, three training environments are set, including at least one of the following: In the case of a simple difficulty level training environment, the state space is where the wind speed changes within a first interval and a second interval, and the blade state changes linearly, with the reward function being the difference between a first reward and a first penalty; In the case of a medium difficulty level training environment, the state space is where the wind speed changes within a first interval, a second interval, and a third interval, and the blade state changes linearly, with the reward function being the difference between a first reward and a first penalty and a second penalty; In the case of a complex difficulty level training environment, the state space is where the wind speed changes within a first interval, a second interval, a third interval, and a fourth interval, and the blade state changes non-linearly, with the reward function being the difference between a first reward and a first penalty and a second penalty.

[0065] It should be noted that the three environments mentioned above are just examples. In the actual training process, you can set them yourself as needed, and you can also set the environment settings yourself.

[0066] In optional implementations, wind speed is a crucial factor for the blades; therefore, different environments can be set based on environmental information. For example, in a simple environment, the wind speed can be relatively stable, remaining within a first range (e.g., 8±1 m / s) for most of the time (e.g., 70%) and a second range (e.g., 8±2.5 m / s) for a small portion of the time (e.g., 30%). Only one blade can be damaged at a time, and the blade's trajectory can be a smooth downward slope. For the training process, the reward function is important. For wind turbine component repair, maximizing economic benefits can be the objective, and the reward function can be the economic result derived from the relationship between power generation revenue and maintenance costs. In a simple environment, the difference between power generation revenue and maintenance costs can be used as the reward function.

[0067] In an optional implementation, under moderate conditions, the wind speed can be set to fluctuate occasionally, ensuring that the wind speed remains in the first range (e.g., 8±1 m / s) for most of the time (e.g., 70%), in the second range (e.g., 8±2.5 m / s) for a small portion of the time (e.g., 20%), and in the third range (e.g., 10%) for a very small portion of the time (e.g., 10%). Two or fewer blades can be damaged to the same degree, and the change in blade condition should be a smooth, gradual decline. Maintenance costs, electricity prices, and related safety requirements can remain constant. The reward function can be the difference between power generation revenue and maintenance costs and safety penalties.

[0068] In an optional implementation, under complex conditions, the wind speed can be set to frequently fluctuate. This allows the wind speed to remain in the first range (e.g., 8±1 m / s) for most of the time (e.g., 70%), in the second range (e.g., 8±2.5 m / s) for a small portion of the time (e.g., 15%), in the third range (e.g., 8±4 m / s) for a very small portion of the time (e.g., 4%), and occasionally (1%) change to the fourth range (e.g., 8±6 m / s). Furthermore, two or fewer blades can be damaged to the same degree, and the change in blade condition is not a smooth decline but may involve abrupt changes, the probability of which can be set (e.g., 10%). Maintenance costs and electricity prices can vary, while relevant safety requirements can remain constant. The reward function can be the difference between power generation revenue and maintenance costs and safety penalties.

[0069] Figure 3 This is a schematic diagram of an optional maintenance method for a wind turbine component according to an embodiment of this application; as shown. Figure 3As shown, learning can proceed sequentially from simple to complex environments. The model can only move to the next difficulty level environment after achieving the training objective in the current environment. The model's action space remains consistent across all environments. After determining the state space of each environment, in step S302, training can begin in the simple environment. In each iteration, step S304 is executed to determine if the learning objective has been achieved. If not, training continues to the next round; if the learning objective is achieved, training can proceed to the next environment. In step S306, training can begin in the medium environment. In each round, step S308 is executed to determine if the learning objective has been achieved. If the learning objective has been achieved, training proceeds to the complex environment. In step S310, training is performed in the complex environment, and step S312 is executed to determine if the learning objective has been achieved. If the learning objective has been achieved, the model training is considered complete.

[0070] It should be noted that to determine whether the learning objective has been achieved, the accuracy of the model can be obtained by comparing the results of expert judgments based on experience in the same environment. Once the accuracy reaches a preset threshold or after a certain number of training rounds, the learning objective is considered to have been achieved.

[0071] The above-described embodiments of this application present a training process for a predictive model of wind turbine blade maintenance actions under three environments. First, the number of environments is determined, then the difficulty level of each environment is determined, and the state space of the environment is changed according to the difficulty level. After setup, learning can proceed sequentially from simple to complex environments. By setting different training environments, the training efficiency of the model is improved.

[0072] It should be noted that the above only illustrates the general framework of model training, that is, training from simple environments first and then gradually moving to complex environments. The following text will explain how to train in each environment.

[0073] In an optional implementation, the above-mentioned training of the maintenance action prediction model in multiple training environments sequentially includes: the training process in each training environment includes: determining the state space of the current training environment; in the i-th round of training, performing the following operations: determining the i-th action and the i-th reward parameter based on the i-th state information and the policy function, wherein the state information is determined by the state space, and i is an integer greater than 0; updating the policy function based on the i-th action and the i-th reward parameter to obtain the (i+1)-th policy function; calculating the accuracy of the maintenance action prediction model, and if the accuracy of the maintenance action prediction model is greater than a preset threshold, determining the (i+1)-th policy function as the trained policy function.

[0074] In an optional implementation, the above-mentioned updating of the policy function based on the i-th action and the i-th reward parameter to obtain the (i+1)-th policy function includes: determining the i-th value difference, wherein the value difference is the difference between the expected reward obtained by performing the i-th action under the i-th state information and the expected reward of the i-th policy function under the i-th state information; determining the i-th new-old policy probability ratio, wherein the i-th new-old policy probability ratio is the probability of the i-th policy function performing the i-th action under the i-th state information and the probability of the (i-1)-th policy function performing the i-th action under the i-th state information; trimming the product of the i-th value difference and the i-th new-old policy probability ratio to obtain a loss function, and updating the parameters of the i-th policy function based on the loss function to obtain the (i+1)-th policy function.

[0075] In an optional implementation, the above-mentioned determination of the i-th action and the i-th reward parameter based on the i-th state information and the policy function includes: determining the mask values ​​corresponding to multiple reference actions based on the i-th state information, and updating the probabilities corresponding to multiple reference actions based on the mask values; normalizing the probabilities corresponding to the updated reference actions to obtain the probabilities corresponding to multiple reference actions, and determining the i-th action based on the probabilities.

[0076] It should be noted that training the model can involve training the parameters of its policy network. The policy network can accept the state of the environment as input and output a probability distribution that indicates the probability of performing each possible action in that state. The state of the environment can be obtained by randomly changing state information in the state space.

[0077] It should be noted that during the training of the above model, gradient updates can be performed using clipping, as shown in the following formula:

[0078]

[0079] Where θ is the strategy parameter, L CLIP (θ) represents the loss function using CLIP, which will be used to update θ. This represents the expected value based on experience. `min` represents the minimum of the two results. t (θ) represents the ratio of the old and new policies at time step t. The advantage function is specifically the reward at time step t and the state s. t The difference in value estimates. clip(r) t (θ), 1-∈, 1+∈) is a clipping operation that guarantees r t The value of (θ) is restricted to the range of 1-∈ and 1+∈. ∈ is the CLIP threshold, a preset constant, such as 0.15.

[0080]

[0081] Where, π θ (a t |s t ) is the current policy in state s t Take action a t The probability, Is the old strategy in state s? t Take action a t The probability of.

[0082] It's important to note that the advantage function is used to estimate the merit of an action relative to the average level, i.e., the relative value of an action. The advantage function can be determined by the difference between the action value function Q and the state value function V. Q can represent the expected reward obtained by taking action a in state s and following the current policy, taking into account the immediate reward after the action and the value of subsequent states. V can represent the expected reward obtained by following the current policy in state s. It is an average value that considers the expected reward of all possible actions starting from state s.

[0083] It should be noted that the expected value can be calculated, which can represent the average value of a certain function on the dataset generated by the policy, that is, the expected value is calculated for all sampled trajectories at time step t.

[0084] It should be noted that the old policy can be the policy function corresponding to the previous policy parameters, or the policy function corresponding to the policy parameters a preset number of steps ago. Pruning can limit the probability ratio of the new and old policies to a preset range. The pruning operation prevents excessive changes in policy updates, which helps maintain the stability of the training process and avoids performance degradation caused by excessively large single-step updates.

[0085] Figure 4 This is a schematic diagram of another optional maintenance method for a wind turbine component according to an embodiment of this application; as shown. Figure 4 As shown, the training process of the model is within the dashed box, including steps S402-S410. After the model training is completed, the model can be used in step S412. Step S412 occurs after step S410, but there are no other timing restrictions. The model can be called when needed.

[0086] Step S402, setting learning objectives. After identifying the specific component on the wind turbine that needs training, a learning objective can be set. This objective can be to maximize economic benefits (including power generation revenue and reduced maintenance costs).

[0087] Step S404, Training Environment Design. In the environment design, the environment can be determined according to the components that need to be maintained. Generally, the difficulty level can be determined first, then the adjustment parameters for each difficulty environment can be determined, and finally the difficulty adjustment strategy can be determined.

[0088] Step S406: Masking rules are determined. Reference actions can be categorized into multiple types, such as three types: usable, not recommended, and unusable. The mask value of the reference action at the current moment can be determined based on the status information. The reference actions are then masked, and the probability value corresponding to each mask value is determined.

[0089] Step S408: Setting up the training environment. Taking a blade as an example, during training, parameters such as time, blade age, system state, cumulative fatigue damage, critical threshold, recent wind speed, number and severity of extreme weather events encountered during service life, and number of imperfect maintenance operations performed can be used as parameters in the state space. Different operations performed can be used as parameters in the action space. Maintenance costs (divided into preparation costs, activity costs, operating costs, etc.) and electricity prices can be used as positive parameters in the reward function, while failure to meet safety requirements can be used as negative parameters in the reward function. The model parameters and environmental parameters can be initialized.

[0090] Step S410, Model Training. Training can proceed sequentially from simple to complex environments. After achieving the target in each environment, training moves to the next environment until training is complete. Achieving the target can be achieved when the model's accuracy reaches a preset threshold.

[0091] Step S412: Obtain status information and output target action. When component maintenance and inspection are required, environmental data can be obtained as status information, and then the target action can be derived through a model.

[0092] In this application, the status information of wind turbine components is input into a maintenance action prediction model to obtain reference probabilities corresponding to multiple reference actions. The status information indicates the environmental information of the wind turbine components. Mask values ​​corresponding to multiple reference actions are determined based on the status information, and the multiple reference probabilities are updated based on these mask values. The mask values ​​include mask values ​​corresponding to available actions, discouraged actions, and unavailable actions, respectively. The updated reference probabilities are normalized to obtain target probabilities corresponding to multiple reference actions, and target actions are determined based on these target probabilities. Corresponding maintenance prediction models can be trained on each component of the wind turbine for component monitoring. Furthermore, the above process considers maintenance time and economic costs, and can guide maintenance personnel to carry the correct tools that meet the maintenance requirements based on recommendations for different maintenance levels, saving maintenance time. It also considers the safety needs in actual operation and maintenance scenarios, reducing risks and improving maintenance efficiency through masking.

[0093] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0094] According to another aspect of the embodiments of this application, a maintenance device for wind turbine components is also provided for implementing the above-described maintenance method for wind turbine components. For example... Figure 5 As shown, the device includes:

[0095] The probability prediction unit 502 is used to input the status information of the wind turbine components into the maintenance action prediction model to obtain the reference probabilities corresponding to multiple reference actions. The status information is used to indicate the environmental information of the wind turbine components.

[0096] Action masking unit 504 is used to determine the mask values ​​corresponding to multiple reference actions based on the state information, and to update the probability of multiple reference probabilities based on the mask values. The mask values ​​include mask values ​​corresponding to available actions, unrecommended actions, and unavailable actions, respectively.

[0097] The action determination unit 506 is used to normalize the updated reference probabilities to obtain the target probabilities corresponding to multiple reference actions, and to determine the target action based on the target probabilities.

[0098] Optionally, the above-mentioned action masking unit 504 includes: a first action masking module, used to update the reference probability corresponding to the reference action that is determined to be a non-recommended action to a first probability when the reference action is determined to be a non-recommended action based on the state information; and a second action masking module, used to update the reference probability corresponding to the reference action that is determined to be an unavailable action to a second probability when the reference action is determined to be an unavailable action based on the state information, wherein the first probability is greater than the second probability.

[0099] Optionally, the above apparatus further includes: a model training unit, used to determine the number of training environments for the maintenance action prediction model; initialize the parameters of the training environments; and train the maintenance action prediction model sequentially in multiple training environments.

[0100] Optionally, the above-mentioned training of the maintenance action prediction model in multiple training environments sequentially includes: the training process in each training environment includes: determining the state space of the current training environment; in the i-th round of training, performing the following operations: determining the i-th action and the i-th reward parameter based on the i-th state information and the policy function, wherein the state information is determined by the state space, and i is an integer greater than 0; updating the policy function based on the i-th action and the i-th reward parameter to obtain the (i+1)-th policy function; calculating the accuracy of the maintenance action prediction model, and if the accuracy of the maintenance action prediction model is greater than a preset threshold, determining the (i+1)-th policy function as the trained policy function.

[0101] Optionally, the above-mentioned updating of the policy function based on the i-th action and the i-th reward parameter to obtain the (i+1)-th policy function includes: determining the i-th value difference, where the value difference is the difference between the expected reward obtained by performing the i-th action under the i-th state information and the expected reward of the i-th policy function under the i-th state information; determining the i-th new-old policy probability ratio, where the i-th new-old policy probability ratio is the probability of the i-th policy function performing the i-th action under the i-th state information and the probability of the (i-1)-th policy function performing the i-th action under the i-th state information; trimming the product of the i-th value difference and the i-th new-old policy probability ratio to obtain a loss function, and updating the parameters of the i-th policy function based on the loss function to obtain the (i+1)-th policy function.

[0102] Optionally, the above-mentioned determination of the i-th action and the i-th reward parameter based on the i-th state information and the policy function includes: determining the mask values ​​corresponding to multiple reference actions based on the i-th state information, and updating the probabilities corresponding to multiple reference actions based on the mask values; normalizing the probabilities corresponding to the updated reference actions to obtain the probabilities corresponding to multiple reference actions, and determining the i-th action based on the probabilities.

[0103] Optionally, the above-mentioned training of the maintenance action prediction model in multiple training environments includes: setting up three training environments when the wind turbine component is a wind turbine blade, wherein the state space and reward function of the training environments are different; sorting the training environments in order of difficulty level, and training the maintenance action prediction model in the three training environments in sequence to obtain the trained maintenance action prediction model.

[0104] Optionally, when the wind turbine component is a wind turbine blade, three training environments are set up, including at least one of the following: In the case of a simple difficulty level training environment, the state space is where the wind speed changes within the first and second intervals, and the blade state changes linearly, with the reward function being the difference between the first reward and the first penalty; In the case of a medium difficulty level training environment, the state space is where the wind speed changes within the first, second, and third intervals, and the blade state changes linearly, with the reward function being the difference between the first reward and the first and second penalties; In the case of a complex difficulty level training environment, the state space is where the wind speed changes within the first, second, third, and fourth intervals, and the blade state changes non-linearly, with the reward function being the difference between the first reward and the first and second penalties.

[0105] Optionally, in this embodiment, the implementation of each of the above-mentioned unit modules can be referred to the above-mentioned method embodiments, which will not be repeated here.

[0106] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described maintenance method for wind turbine components is also provided. This electronic device may be... Figure 6 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments via the computer program.

[0107] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0108] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0109] S1. Input the status information of the wind turbine components into the maintenance action prediction model to obtain the reference probabilities corresponding to multiple reference actions. The status information is used to indicate the environmental information of the wind turbine components.

[0110] S2, determine the mask values ​​corresponding to multiple reference actions based on the state information, and update the probability of multiple reference probabilities based on the mask values. The mask values ​​include the mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively.

[0111] S3, normalize the updated reference probabilities to obtain the target probabilities corresponding to multiple reference actions, and determine the target action based on the target probabilities.

[0112] Alternatively, as those skilled in the art will understand, Figure 6The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 6 This does not limit the structure of the aforementioned electronic devices or electronic equipment. For example, electronic devices or electronic equipment may also include components that are more... Figure 6 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 6 The different configurations shown.

[0113] The memory 602 can be used to store software programs and modules, such as the program instructions / modules corresponding to the wind turbine component maintenance method and apparatus in this embodiment. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, thereby realizing the aforementioned wind turbine component maintenance method. The memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include memory remotely located relative to the processor 604, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 602 may be used, but is not limited to, for storing wind turbine data. As an example, such as... Figure 6 As shown, the memory 602 may include, but is not limited to, the probability prediction unit 502, the action mask unit 504, and the action determination unit 506 in the maintenance device for the wind turbine components. Furthermore, it may include, but is not limited to, other module units in the maintenance device for the wind turbine components, which will not be elaborated upon in this example.

[0114] Optionally, the transmission device 606 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 606 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 606 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0115] In addition, the aforementioned electronic device also includes: a display 608 for displaying fan operating data; and a connection bus 610 for connecting the various module components in the aforementioned electronic device.

[0116] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0117] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0118] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the aforementioned maintenance method for the fan components.

[0119] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0120] S1. Input the status information of the wind turbine components into the maintenance action prediction model to obtain the reference probabilities corresponding to multiple reference actions. The status information is used to indicate the environmental information of the wind turbine components.

[0121] S2, determine the mask values ​​corresponding to multiple reference actions based on the state information, and update the probability of multiple reference probabilities based on the mask values. The mask values ​​include the mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively.

[0122] S3, normalize the updated reference probabilities to obtain the target probabilities corresponding to multiple reference actions, and determine the target action based on the target probabilities.

[0123] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0124] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0125] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0128] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0130] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for maintaining a fan component, characterized in that, The method includes: The status information of the wind turbine components is input into the maintenance action prediction model to obtain the reference probabilities corresponding to multiple reference actions. The status information is used to indicate the environmental information of the wind turbine components, and the reference actions are used to indicate the maintenance actions for the wind turbine components. The mask values ​​corresponding to the plurality of reference actions are determined based on the state information, and the plurality of reference probabilities are updated based on the mask values. The mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively. The updated reference probabilities are normalized to obtain the target probabilities corresponding to the multiple reference actions, and the target action is determined based on the target probabilities. The training process of the maintenance action prediction model includes: determining the number of training environments for the maintenance action prediction model; initializing the parameters of the training environments; and training the maintenance action prediction model sequentially in multiple training environments. The training process in each training environment includes: determining the state space of the current training environment; In the i-th round of training, the following operations are performed: determine the i-th action and the i-th reward parameter based on the i-th state information and the policy function, wherein the state information is determined by the state space, and i is an integer greater than 0; Determine the i-th value difference, where the value difference is the difference between the expected return obtained by performing the i-th action under the i-th state information and the expected return of the i-th policy function under the i-th state information; determine the i-th new-old policy probability ratio, where the i-th new-old policy probability ratio is the probability of the i-th policy function performing the i-th action under the i-th state information and the probability of the (i-1)-th policy function performing the i-th action under the i-th state information; trim the product of the i-th value difference and the i-th new-old policy probability ratio to obtain a loss function, and update the parameters of the i-th policy function according to the loss function to obtain the (i+1)-th policy function; Calculate the accuracy of the maintenance action prediction model. If the accuracy of the maintenance action prediction model is greater than a preset threshold, determine the (i+1)th policy function as the policy function that has been trained.

2. The method according to claim 1, characterized in that, The step of determining the mask values ​​corresponding to the plurality of reference actions based on the state information, and updating the plurality of reference probabilities based on the mask values, includes: If the reference action is determined to be a non-recommended action based on the state information, the reference probability corresponding to the non-recommended action is updated to the first probability; If the reference action is determined to be an unavailable action based on the state information, the reference probability corresponding to the reference action of the unavailable action is updated to a second probability, wherein the first probability is greater than the second probability.

3. The method according to claim 1, characterized in that, The step of determining the i-th action and the i-th reward parameter based on the i-th state information and the policy function includes: The mask value corresponding to the plurality of reference actions is determined based on the i-th state information, and the probability corresponding to the plurality of reference actions is updated based on the mask value; The probabilities corresponding to the updated reference actions are normalized to obtain the probabilities corresponding to the multiple reference actions, and the i-th action is determined based on the probabilities.

4. The method according to any one of claims 1 to 3, characterized in that, The step of sequentially training the maintenance action prediction model in multiple training environments includes: When the wind turbine component is a wind turbine blade, three training environments are set up, wherein the state space and reward function of the training environments are different; The training environments are ordered according to their difficulty level, and the maintenance action prediction model is trained sequentially in the three training environments to obtain the trained maintenance action prediction model.

5. The method according to claim 4, characterized in that, When the wind turbine component is a wind turbine blade, three training environments are set up, including at least one of the following: When the training environment is at a simple difficulty level, the state space is such that the wind speed changes within a first interval and a second interval, and the blade state changes linearly. The reward function is the difference between a first reward and a first penalty. When the training environment is at a medium difficulty level, the state space is defined by wind speed changes in the first, second, and third intervals, and the blade state changes linearly. The reward function is the difference between the first reward and the first and second penalties. When the training environment is at a complex difficulty level, the state space consists of wind speed changes in the first, second, third, and fourth intervals, and the blade state changes nonlinearly. The reward function is the difference between the first reward and the first and second penalties.

6. A maintenance device for a fan component, characterized in that, The device includes: The probability prediction unit is used to input the status information of the wind turbine components into the maintenance action prediction model to obtain the reference probabilities corresponding to multiple reference actions. The status information is used to indicate the environmental information of the wind turbine components, and the reference actions are used to indicate the maintenance actions for the wind turbine components. An action masking unit is used to determine the mask values ​​corresponding to the plurality of reference actions based on the state information, and to update the probability of the plurality of reference probabilities based on the mask values, wherein the mask values ​​include mask values ​​corresponding to available actions, not recommended actions, and unavailable actions, respectively. An action determination unit is used to normalize the updated reference probabilities to obtain the target probabilities corresponding to the plurality of reference actions, and to determine the target action based on the target probabilities. The device further includes: a model training unit, used to determine the number of training environments for the maintenance action prediction model; initialize the parameters of the training environments; and train the maintenance action prediction model sequentially in multiple training environments; The training process in each training environment includes: determining the state space of the current training environment; In the i-th round of training, the following operations are performed: determine the i-th action and the i-th reward parameter based on the i-th state information and the policy function, wherein the state information is determined by the state space, and i is an integer greater than 0; Determine the i-th value difference, where the value difference is the difference between the expected return obtained by performing the i-th action under the i-th state information and the expected return of the i-th policy function under the i-th state information; determine the i-th new-old policy probability ratio, where the i-th new-old policy probability ratio is the probability of the i-th policy function performing the i-th action under the i-th state information and the probability of the (i-1)-th policy function performing the i-th action under the i-th state information; trim the product of the i-th value difference and the i-th new-old policy probability ratio to obtain a loss function, and update the parameters of the i-th policy function according to the loss function to obtain the (i+1)-th policy function; Calculate the accuracy of the maintenance action prediction model. If the accuracy of the maintenance action prediction model is greater than a preset threshold, determine the (i+1)th policy function as the policy function that has been trained.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 5.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1 to 5.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

Images