Equipment predictive maintenance method based on deep reinforcement learning

Abstract

The present invention provides a device predictive maintenance method based on deep reinforcement learning, which relates to the technical field of equipment monitoring. The present invention obtains the replacement reason and timestamp of each component, forms a work data set based on historical work data and forms a task set in which each component participates, constructs a fault probability judgment model based on the task set, and sets a reward formula to obtain a replacement guidance model, obtains the replacement score of each component based on the installation time of the currently equipped components, obtains the depreciation coefficient based on the next task, obtains the age coefficient through the life of the equipment, forms a comprehensive replacement coefficient for each component and constructs a correction threshold to obtain the replacement status of each component. The present invention judges the crane's execution capability for future tasks based on the synergy of different depreciation conditions between components, thereby achieving accurate predictive maintenance of crane components.

Description

Technical Field

[0001] The present invention relates to the field of equipment monitoring technology, and specifically to an equipment predictive maintenance method based on deep reinforcement learning. Background Art

[0002] With the advancement of intelligent manufacturing technology, equipment maintenance and management are becoming increasingly important in industrial production. This is especially true for heavy-duty lifting equipment, which operates in a high-risk, high-intensity environment. This places high demands on equipment safety, and timely maintenance and servicing are crucial. However, the maintenance process is complex, lengthy, and challenging. Furthermore, due to the high intensity of the work, different components experience significant wear and tear at a uniform rate, making components very expensive. To ensure both optimal utilization and operational safety, a method is needed to accurately assess the performance of each component. This method ensures that the equipment meets operational requirements while maintaining safety, enabling predictive maintenance based on the task at hand. Deep reinforcement learning (DRL) extracts high-dimensional features from raw data, offering high accuracy for equipment performance analysis. Therefore, combining DRL with the performance analysis of individual components is crucial. The hoisting motor, a core component in a crane, reflects the overall working conditions of the connected wire rope, pulley, brake pad, and motor bearings. Deep reinforcement learning focuses on analyzing these components, demonstrating its importance in analyzing the status of these components.

[0003] However, the current maintenance method for cranes usually adopts a fixed-cycle maintenance method. This method cannot accurately judge the life status of each component. Relying solely on empirical methods cannot make full use of the components, resulting in wasted component life. It also cannot construct specific quantitative data to judge the crane's mission capability, and cannot provide effective guidance on the crane's mission execution capability and component replacement.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a device predictive maintenance method based on deep reinforcement learning to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The equipment predictive maintenance method based on deep reinforcement learning includes the following steps:

[0008] Step 1: Obtain the replacement reasons for crane parts and add timestamps. Obtain the crane's historical operating data, which includes the operating power, operating duration, vibration curve, voltage curve, and timestamp of each task. Analyze the vibration and voltage curves of each task to obtain the vibration and voltage variation coefficients, and form a working data set for each task. Analyze the parts replacement and working data sets based on the timestamps to obtain the set of tasks each part participated in from installation and use to replacement.

[0009] Step 2: Obtain the replacement cost of each component and the failure cost of each task, and construct a reward formula based on the replacement action. Input the task set of each component into the deep Q network model for training, and output the reward value of the replacement action of each component in different tasks to obtain the replacement guidance model;

[0010] Step 3: Obtain the installation time of the components currently used by the crane, as well as the operating power, operating time, vibration curve, and voltage curve of each component's tasks. This is used to form a task set and input it into the replacement guidance model. The reward value associated with each component's replacement action is then obtained to form a replacement score for each component.

[0011] Step 4: Based on the working power and working hours required by the hoisting motor for the next mission, the crane's depreciation coefficient is calculated by combining the obtained rated power of the hoisting motor. The depreciation coefficient of each component is calculated by combining the replacement times and theoretical replacement times of each component with the crane's depreciation coefficient.

[0012] Step 5: Obtain the crane's lifespan at rated power and construct an age coefficient based on the crane's historical operating data. Use the age coefficient and the depreciation coefficient of each component to correct the replacement score of each component to obtain a comprehensive replacement score. At the same time, use the comprehensive replacement score to correct the set component score threshold to obtain a correction threshold. Based on the relationship between the comprehensive replacement score and the correction threshold, determine whether the crane should replace components before the next mission.

[0013] Furthermore, the reasons for replacement of crane parts are obtained and timestamps are added. The parts include wire ropes, pulleys, brake pads and motor bearings. At the same time, the life of the replaced parts is evaluated to obtain the remaining life. The remaining life is obtained through expert evaluation. The reasons for replacement include preventive replacement and fault replacement. The reasons for parts replacement are obtained through maintenance logs. The historical working data of the crane is obtained. The working data includes the working power, working time, vibration curve and voltage curve of each task and the timestamp. Each task is the process of a single cargo lifting. The vibration curve and voltage curve are obtained by monitoring the crane's hoisting motor, which are the vibration curve and voltage curve of a single lifting of the hoisting motor. The working power and working time are the operating power and operating time of the hoisting motor during a single task. The horizontal axis of the vibration curve is the start-up time and the vertical axis is the amplitude. The horizontal axis of the voltage curve is the start-up time and the vertical axis is the voltage. The vibration curve and voltage curve of each task are recorded separately and correspond one to one.

[0014] The amplitude curve and voltage curve of each task are analyzed separately to obtain the vibration variation coefficient and voltage variation coefficient of each task. The logic is as follows:

[0015] Get the mean and standard deviation of each vibration curve and voltage curve respectively, and get the vibration coefficient of variation and voltage coefficient of variation according to the following formula:

[0016]

[0017]

[0018] in, is the vibration variation coefficient, is the mean value of the vibration curve, is the standard deviation of the vibration curve, is the voltage variation coefficient, is the mean value of the voltage curve, is the standard deviation of the voltage curve.

[0019] Furthermore, the lifting tasks are sorted according to the timestamps, and the working power, working time, vibration coefficient of variation, and voltage coefficient of variation of each task are summarized to form a single task work data set. The formula is as follows:

[0020]

[0021] in, Indicates the The working dataset for each task, is the working power, For working hours, is the vibration variation coefficient, is the voltage variation coefficient, Retrieve the number for the working dataset, , , is the total number of tasks;

[0022] All replaced parts are numbered, and the part number, replacement reason, and remaining life are mapped to the work data based on the timestamp. The task set that each part participates in during each period from installation and use to replacement is obtained from the work data set. The formula is as follows:

[0023]

[0024] in, Indicates the Part No. The set of tasks involved from installation and use to replacement, Indicates the The working dataset for each task, Retrieve the number for the working dataset, , , is the total number of tasks, For the The number of the first working data set involved from installation to replacement, For replacement reasons, For the remaining life of the replaced parts, , , Retrieve variable for number of parts replacements, , , Indicates the Total number of parts replaced;

[0025] Among them, when the reason for replacement is preventive replacement of this task, , when the replacement reason is the fault replacement in this task, , the last working dataset is a related dataset, which indicates the replacement of parts in this task link. When the last working data set is completed When replacing this part, When the last working data set Replacement of the component during the process.

[0026] Furthermore, the tag action is set according to the following formula:

[0027]

[0028] in, Indicates action, Indicates no replacement. Indicates replacement;

[0029] The replacement cost of each component is obtained separately. The replacement cost includes the cost of the replaced component, labor cost, and time cost. The time cost is expressed as a monetary amount and is determined by the crane's organization. The failure cost of each component's task is obtained. The failure cost is the property loss caused by task failure and is calibrated by the crane's organization. The task set of each component is input into the deep Q network model for training, and the reward formula is set according to the following formula:

[0030]

[0031] in, is the reward value, For action, For replacement cost, Working dataset for a single task The failure cost, is the failure probability.

[0032] Furthermore, the logic for obtaining the failure probability is as follows:

[0033] Obtain the rated life of each replacement part. The rated life of the replacement part is obtained from the manufacturer. Based on the task set in which each component participates, obtain the working power and working time of the task experienced by the component before replacement. Determine the failure probability based on the remaining life of the replaced component. The failure probability is based on the formula:

[0034]

[0035] in, is the failure probability, For the The working power of the sub-task, For the The working time of the task, is the rated life of the component, Retrieve variable for task number, , , To calculate the failure probability of this task, the number of tasks that this component has experienced before, and is the coefficient, obtained through the fault probability judgment model;

[0036] The task set that each component participates in from installation to replacement is used to obtain the working power and working time of each component in each task as the training set. The linear regression model is input for training and the fitting formula is set. The formula is as follows:

[0037]

[0038] in, For the The working power of the sub-task, For the The working time of the task, The remaining life of the replacement part. is the rated life of this replacement part, Retrieve the ID for the task, , , The total number of tasks performed for this replacement part;

[0039] By fitting the formula, the failure probability judgment model of the component is obtained, and the coefficients corresponding to each component are obtained in this way. and ;

[0040] The output is the reward value for the two actions of replacing and not replacing parts. The deep reinforcement learning model of each part is obtained. Furthermore, the installation time of the parts used by the current crane, as well as the working power, working time, vibration curve and voltage curve of the tasks experienced by each part are obtained. A task set is formed and input into the replacement guidance model. The reward value for each part for replacement and non-replacement is obtained respectively to form a replacement score. The formula is as follows:

[0041]

[0042] in, For the Replacement rating for each component, is the reward value for the change action, is the reward value of not changing the action, Retrieve variables for part numbers, , ;

[0043] Get the replacement score for each component separately.

[0044] Furthermore, the next task to be performed is obtained, and based on the working power and working time required by the hoisting motor for the next task, the rated power of the hoisting motor is obtained, and the depreciation coefficient of the crane is constructed according to the following formula:

[0045]

[0046] in, is the depreciation factor of the crane, is the coefficient, The working power of the next task, is the rated power, for working hours;

[0047] Obtain the number of replacements for each component, which is obtained from the maintenance log. Obtain the theoretical number of replacements for each component, which is obtained from the vendor. Build a depreciation coefficient for each component. The depreciation coefficient for each component is based on the following formula:

[0048]

[0049] in, Indicates the The depreciation factor of each component, Indicates the Number of parts replacements, Indicates the Theoretical replacement times of each component, is the depreciation factor of the crane, Retrieve variables for part numbers, , .

[0050] Furthermore, the life of the crane at rated power is obtained. The life is obtained from the crane manufacturer, and the equipment life coefficient is formed based on the crane's historical operating data. The formula is as follows:

[0051]

[0052] in, is the age coefficient, Indicates the number of historical work data The working power of the sub-task, Indicates the number of historical work data The working time of the task, is the rated power, The lifespan at rated power is , , The total number of all tasks in the historical work data.

[0053] Furthermore, the replacement score of each component is corrected according to the age factor and the depreciation factor of each component to obtain a comprehensive replacement score. The formula is as follows:

[0054]

[0055] in, Indicates the The comprehensive replacement score of each component, is the age coefficient, Indicates the The depreciation factor of each component, For the Replacement rating for each component, Retrieve variables for part numbers, , ;

[0056] Set a component score threshold and obtain a modified threshold based on the comprehensive replacement score of all components. The modified threshold is based on the following formula:

[0057]

[0058] in, To correct the threshold, is the scoring threshold, Indicates the The comprehensive replacement score of each component, Retrieve variables for part numbers, , ;

[0059] When the comprehensive replacement score is greater than the correction threshold, the component will be replaced immediately;

[0060] When the comprehensive replacement score is less than the correction threshold, it means that the component can continue to be used.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] The present invention constructs a replacement guidance model based on historical parts replacement records and work data, obtains the replacement score of each part through the replacement guidance model according to the task set of the current part, and then constructs the depreciation coefficient and age coefficient to obtain a comprehensive replacement score, and judges the replacement status of the part based on the comprehensive replacement score and the correction threshold. The present invention constructs the replacement score through historical tasks, quantifies the usage status of the parts, provides data support for judging the working capacity of the parts, constructs the comprehensive replacement score through the next task, and forms a correction threshold according to the comprehensive replacement score of each part to judge whether the part should be replaced, and judges the crane's execution capability for future tasks according to the synergistic effect of different depreciation conditions between parts, thereby realizing accurate predictive maintenance of crane parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of the overall method of the present invention. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments.

[0065] It should be noted that, unless otherwise defined, the technical or scientific terms used in the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0066] Example:

[0067] See also Figure 1 , the present invention provides a technical solution:

[0068] The equipment predictive maintenance method based on deep reinforcement learning includes the following steps:

[0069] Step 1: Obtain the replacement reasons for crane parts and add timestamps. Obtain the crane's historical operating data, which includes the operating power, operating duration, vibration curve, voltage curve, and timestamp of each task. Analyze the vibration and voltage curves of each task to obtain the vibration and voltage variation coefficients, and form a working data set for each task. Analyze the parts replacement and working data sets based on the timestamps to obtain the set of tasks each part participated in from installation and use to replacement.

[0070] The step 1 includes the following:

[0071] Step 101: Obtain the replacement reasons of crane parts and add timestamps. The parts include wire ropes, pulleys, brake pads, and motor bearings. At the same time, perform life assessment on the replaced parts to obtain the remaining life. The remaining life is obtained through expert evaluation. The replacement reasons include preventive replacement and fault replacement. The parts replacement reasons are obtained through maintenance logs. Obtain historical working data of the crane. The working data includes working power, working time, vibration curve, voltage curve, and timestamp for each task. Each task is a single cargo lifting process. The vibration curve and voltage curve are obtained by monitoring the crane's hoisting motor and are the vibration curve and voltage curve of a single lifting of the hoisting motor. The working power and working time are the operating power and operating time of the hoisting motor during a single task. The horizontal axis of the vibration curve is the startup time and the vertical axis is the amplitude. The horizontal axis of the voltage curve is the startup time and the vertical axis is the voltage. The vibration curve and voltage curve of each task are recorded separately and correspond one to one.

[0072] The amplitude curve and voltage curve of each task are analyzed separately to obtain the vibration variation coefficient and voltage variation coefficient of each task. The logic is as follows:

[0073] Get the mean and standard deviation of each vibration curve and voltage curve respectively, and get the vibration coefficient of variation and voltage coefficient of variation according to the following formula:

[0074]

[0075]

[0076] in, is the vibration variation coefficient, is the mean value of the vibration curve, is the standard deviation of the vibration curve, is the voltage variation coefficient, is the mean value of the voltage curve, is the standard deviation of the voltage curve.

[0077] and Refers to the relative fluctuations of vibration signals and voltage signals, which reflect the degree of discreteness of these two physical quantities during the lifting task of the equipment. and Respectively represent the mean of the vibration curve and voltage curve, that is, the average value of the overall operation of the equipment during a single lifting process, and and The standard deviation of the vibration and voltage data reflects the magnitude of data fluctuation and uncertainty. By calculating the ratio of the two, we can determine the relative degree of fluctuation in the vibration and voltage signals throughout operation, assessing the stability of the equipment and potential failure risks. When the equipment is operating smoothly, the mean is high, the standard deviation is low, and the coefficient of variation increases, indicating consistent equipment status. Conversely, if significant fluctuations occur during operation, the standard deviation is high and the coefficient of variation decreases, indicating possible anomalies or potential failures. Analyzing the coefficient of variation helps to promptly identify potential abnormal fluctuations within the equipment by monitoring these ratios, providing a quantitative reference for preventive maintenance.

[0078] Initial collection of key information such as the replacement reasons, timestamps, and remaining life of each crane component is obtained. At the same time, statistical processing of physical signals such as vibration and voltage collected during equipment operation is performed, providing a rich and accurate raw data foundation for subsequent data mining. By distinguishing between preventive replacement and fault replacement, the inherent connection between component service life and failure mode under different conditions is further revealed, which is conducive to the subsequent extraction of data features and pattern learning. At the same time, by adding a timestamp to each task to form a continuous time series data, the evolution trend of the equipment health status can be tracked more accurately, and cross-validation in the time dimension is provided for the construction of subsequent task sets, ensuring the traceability and continuity of the data. This not only improves the problems of discrete and insufficient samples in traditional empirical maintenance data, but also provides training samples with intrinsic physical meaning for deep reinforcement learning models, forming a complete information chain from raw monitoring data to evaluation decisions.

[0079] Step 102: Sort the lifting tasks according to the timestamps, and summarize the working power, working time, vibration coefficient of variation, and voltage coefficient of variation of each task to form a single task work data set. The formula is as follows:

[0080]

[0081] in, Indicates the The working dataset for each task, is the working power, For working hours, is the vibration variation coefficient, is the voltage variation coefficient, Retrieve the number for the working dataset, , , is the total number of tasks;

[0082] Indicates the power of the hoisting motor during the mission, Indicates the working time of the task, that is, the actual duration of the lifting; It represents the coefficient of variation of vibration, which reflects the vibration fluctuation of the motor during the crane mission; is the voltage variation coefficient, which is used to quantify the amplitude of motor voltage fluctuation. and Represent the load intensity and continuous working time of the task, respectively, and and As a tool to reveal the inherent fluctuations of signals during equipment operation, especially the sensitivity of equipment status to changes in load or working conditions, the construction of a single task working data set not only integrates the key working condition data of each task, but also provides a stable input data foundation for subsequent model learning.

[0083] All replaced parts are numbered, and the part number, replacement reason, and remaining life are mapped to the work data based on the timestamp. The task set that each part participates in during each period from installation and use to replacement is obtained from the work data set. The formula is as follows:

[0084]

[0085] in, Indicates the Part No. The set of tasks involved from installation and use to replacement, Indicates the The working dataset for each task, Retrieve the number for the working dataset, , , is the total number of tasks, For the The number of the first working data set involved from installation to replacement, For replacement reasons, For the remaining life of the replaced parts, , , Retrieve variable for number of parts replacements, , , Indicates the Total number of parts replaced;

[0086] The task set that each component participates in from installation to replacement describes the data set of all tasks that a single component (such as wire rope, pulley, brake pad or motor bearing) participates in from installation to replacement. to Indicates the consecutive tasks that the component participates in within a replacement cycle (arranged in chronological order); It reflects the replacement reason corresponding to the replacement (for example, preventive replacement is assigned a value of 1, and failure replacement is assigned a value of 2), reflecting the category of reasons that led to the replacement of parts; The remaining life of a component, recorded at the time of replacement, is typically determined by on-site expert assessment and serves as a quantitative indicator of component health. Records between missions help summarize the loads borne by a component and the effects of fluctuations such as vibration and voltage experienced during multiple missions, thereby reflecting its wear and degradation trends. The marking not only makes it possible to distinguish different replacement types, but also provides a basis for subsequent risk assessment; This indicates the extent to which the component may be close to failure due to degradation during the mission. When the component experiences a large mission load and the data fluctuates significantly, it may cause the remaining life to This approach not only links the operating condition data of each task with the operating history of a specific component, but also integrates replacement reasons and remaining life information to build a complete data chain capable of tracking component performance throughout its entire lifecycle. This provides a scientific basis for subsequent deep reinforcement learning models to generate accurate maintenance decisions and strategy adjustments.

[0087] Among them, when the reason for replacement is preventive replacement of this task, , when the replacement reason is the fault replacement in this task, , the last working dataset is a related dataset, which indicates the replacement of parts in this task link. When the last working data set is completed When replacing this part, When the last working data set Replacement of the component during the process.

[0088] By sorting the data of each task according to the time series and uniformly organizing the working power, working time, vibration coefficient of variation and voltage coefficient of variation, a standardized single-task working data set is constructed. Effective fusion between different data sources is achieved to ensure that the working condition data of each task can have a consistent expression format in the model, so that the model can capture the changes in equipment status based on the subtle differences of a single task while grasping the overall trend. By mapping the tasks in which parts participate in the complete time period from installation to replacement in the data set, a relationship map directly corresponding to parts and historical tasks is formed. This data association mechanism greatly improves the inherent logic and utilization efficiency of the data, and provides a more spatiotemporal and temporal continuity training basis for the deep reinforcement learning model, thereby improving the model's accuracy and predictive ability, and laying a solid foundation for data connection and effective feedback in subsequent steps.

[0089] Step 2: Obtain the replacement cost of each component and the failure cost of each task, and construct a reward formula based on the replacement action. Input the task set of each component into the deep Q network model for training, and output the reward value of the replacement action of each component in different tasks to obtain the replacement guidance model;

[0090] The step 2 includes the following:

[0091] Step 201: Set the label action according to the following formula:

[0092]

[0093] in, Indicates action, Indicates no replacement. Indicates replacement;

[0094] The replacement cost of each component is obtained separately. The replacement cost includes the cost of the replaced component, labor cost, and time cost. The time cost is expressed as a monetary amount and is determined by the crane's organization. The failure cost of each component's task is obtained. The failure cost is the property loss caused by task failure and is calibrated by the crane's organization. The task set of each component is input into the deep Q network model for training, and the reward formula is set according to the following formula:

[0095]

[0096] in, is the reward value, For action, For replacement cost, Working dataset for a single task The failure cost, is the failure probability.

[0097] By combining maintenance decisions with costs and failure risks, a comprehensive incentive signal is constructed for deep reinforcement learning models to evaluate and optimize during training. is a label action variable, with values ​​of 0 or 1, corresponding to "do not replace" and "replace" respectively; It represents the replacement cost of parts, reflecting the economic cost of immediate replacement; ), the model needs to consider the current task The risk cost of possible equipment failure is borne by and The product of Reflects the failure cost corresponding to the current task data, Denotes the failure probability, which indicates the possibility of failure if the component is not replaced. Reflects the total cost incurred under a specific decision: If you choose to replace ( ), then the penalty term is the direct replacement cost; and if you choose not to replace ( ) must bear the risk of future failures. This formula establishes a balancing act, allowing decision-makers or models to intuitively weigh the relationship between direct economic expenditure and potential risk when evaluating replacement decisions, while also reflecting the cost risks associated with component status uncertainty in actual operating environments. This balances economic rationality and risk sensitivity in preventive maintenance decisions.

[0098] Clearly labeled actions and related reward formulas combine actual replacement costs with task failure losses to construct a comprehensive indicator system for risk and maintenance behavior assessment. This essentially transforms the equipment maintenance decision-making problem into a numerical optimization problem, enabling the deep Q network to weigh losses when faced with positive and negative incentives. By introducing realistic replacement and failure costs, the system can automatically learn under what circumstances replacing parts can minimize overall risk and economic burden. This not only makes the model learning process more realistic and targeted, but also forms a dynamic feedback mechanism that allows the model to adaptively adjust its decision-making strategy, providing a solid theoretical foundation for accurate judgment of failure probability and model robustness in subsequent steps, while further promoting the dual improvement of the economic benefits and safety of equipment maintenance.

[0099] As a preferred embodiment, a deep Q network model is set to include an input layer, a hidden layer and an output layer. The input layer converts the task set of each component into a feature vector. The hidden layer adopts a multi-layer perceptron with 3 layers and 256 neurons in each layer. The output layer outputs the Q value of each action, and the Bellman equation is used to update the Q value. The learning rate is set to 0.001 and the number of training rounds is 1000.

[0100] Step 202: The logic for obtaining the failure probability is as follows:

[0101] Obtain the rated life of each replacement part. The rated life of the replacement part is obtained from the manufacturer. Based on the task set in which each component participates, obtain the working power and working time of the task experienced by the component before replacement. Determine the failure probability based on the remaining life of the replaced component. The failure probability is based on the formula:

[0102]

[0103] in, is the failure probability, For the The working power of the sub-task, For the The working time of the task, is the rated life of the component, Retrieve variable for task number, , , To calculate the failure probability of this task, the number of tasks that this component has experienced before, and is the coefficient, obtained through the fault probability judgment model;

[0104] It represents the probability of failure due to the accumulated workload of all tasks before the component is replaced; Indicates the Working power in the sub-task, Indicates the working time of the task, and the two are exponentially and Respectively reflects the importance of power and duration in causing component fatigue and wear; It is the rated life of the component, which is used as a measure of the cumulative use compared to the design life. It measures the cumulative load of the component after several missions relative to its original service life. , thereby giving a failure probability value that reflects the remaining safety margin.

[0105] 、 and When the failure probability increases will gradually increase, revealing the trend of increased wear and potential failure of components. and When it is larger, The value of rises rapidly, The value of increases, indicating that the closer the component is to its design life, the higher the risk of failure; on the contrary, if the task load is relatively mild, the cumulative value is low, Lower, which indicates that the component is in a relatively healthy condition.

[0106] The task set that each component participates in from installation to replacement is used to obtain the working power and working time of each component in each task as the training set. The linear regression model is input for training and the fitting formula is set. The formula is as follows:

[0107]

[0108] in, For the The working power of the sub-task, For the The working time of the task, The remaining life of the replacement part. is the rated life of this replacement part, Retrieve the ID for the task, , , The total number of tasks performed for this replacement part;

[0109] and Corresponding to each task data, Indicates the remaining life of the replacement part, and This means adding a margin to the rated life to balance the measurement errors and environmental fluctuations in reality. This fitting process optimizes the relationship between historical mission data and remaining life data. and The value of , so that the failure probability model can more accurately reflect the fatigue accumulation and failure risk of components under actual working conditions.

[0110] By fitting the formula, the failure probability judgment model of the component is obtained, and the coefficients corresponding to each component are obtained in this way. and ;

[0111] The output is the reward value for the two actions of replacing and not replacing parts. The deep reinforcement learning model of each part is obtained and calibrated as the replacement guidance model.

[0112] The probability of component failure is determined, and actual component load data (operating power and operating hours) and remaining lifespan are used to predict equipment failure risk. A linear regression model is employed to achieve a relatively concise and intuitive failure quantification method. This step in data mining transitions from raw physical quantities to probabilistic assessments. It transforms the degradation trend of equipment status into a measurable and predictable risk indicator, which is then used as a dynamic variable in the reward function. This approach not only enables the model to promptly reflect the actual equipment condition during decision-making, but also uses the regression model to smooth the prediction results and make the error more manageable, significantly improving the accuracy of fault warnings in actual operation. This allows the entire deep reinforcement learning system to consider potential future economic losses while continuously revising the prediction model based on data, effectively avoiding the risk of bias caused by subjective judgment in traditional maintenance.

[0113] Step 3: Obtain the installation time of the components currently used by the crane, as well as the operating power, operating time, vibration curve, and voltage curve of each component's tasks. This is used to form a task set and input it into the replacement guidance model. The reward value associated with each component's replacement action is then obtained to form a replacement score for each component.

[0114] The step 3 includes the following:

[0115] The installation time of the components used in the current crane, as well as the working power, working time, vibration curve, and voltage curve of each component in the tasks they have experienced are obtained to form a task set and input into the replacement guidance model. The reward value for each component for replacement and non-replacement is obtained to form a replacement score based on the following formula:

[0116]

[0117] in, For the Replacement rating for each component, is the reward value for the change action, is the reward value of not changing the action, Retrieve variables for part numbers, , ;

[0118] Quantifying the necessity of replacing a component in its current state means comparing the reward values ​​of the two decisions of "replace" and "do not replace", with the resulting score being between 0 and 1. The higher the value, the more obvious the economic benefit or risk reduction effect of the replacement action; and if A larger value means that giving up replacement may result in a higher risk of failure or financial loss. and Respectively reflects the results of direct replacement and continued use in a real working environment, The cost of replacing parts and the cost of potential failures avoided due to timely replacement are combined. This implies the risks and losses that may be faced if the parts are not replaced. Replacement of parts due to deterioration can significantly reduce risks and economic losses. will be higher, and Although it is negative, its absolute value is small, then Close to 1, it means that the system strongly recommends taking replacement measures; on the contrary, if the risk of not replacing is relatively low ( The absolute value of is higher, while relatively low), then The value will be low, indicating that the component can still be used.

[0119] Get the replacement score for each component separately.

[0120] Current equipment status data is combined with installation time and then fed into a replacement guidance model to obtain real-time reward values ​​for replacement and non-replacement, which are then used to form a specific component replacement score. This achieves closed-loop control of data collection and model prediction. On the one hand, real-time data feedback enables timely updates of maintenance decisions based on the actual working status of the current equipment. On the other hand, by transforming traditional static maintenance decisions into data-driven dynamic predictions, the flexibility and responsiveness of the decision-making process are improved. The replacement score not only reflects the current wear condition of the component but also integrates two key factors: historical mission data and equipment degradation trends. This provides a more accurate and scientific initial value for downstream score adjustments (such as the application of depreciation and age factors). This significantly enhances the refined management level of maintenance decisions and establishes a solid information exchange channel with the previous steps, ensuring efficient coordination and improved robustness of the overall system operation.

[0121] Step 4: Based on the working power and working hours required by the hoisting motor for the next mission, the crane's depreciation coefficient is calculated by combining the obtained rated power of the hoisting motor. The depreciation coefficient of each component is calculated by combining the replacement times and theoretical replacement times of each component with the crane's depreciation coefficient.

[0122] The step 4 includes the following:

[0123] Get the next task to be performed, and according to the working power and working time required by the hoisting motor for the next task, get the rated power of the hoisting motor and construct the depreciation coefficient of the crane. The formula is as follows:

[0124]

[0125] in, is the depreciation factor of the crane, is the coefficient, The working power of the next task, is the rated power, for working hours;

[0126] Describes the overall wear and performance degradation of the crane due to the deviation between the actual working power and the rated power and the duration of the task when the crane performs the next task. Greater than or equal to rated power When using the formula ; and when Less than When . It is used to reflect the degree of depreciation of the crane and its components due to long-term actual load and working hours. If the working power of the crane is higher than the rated value during the task, it means that the load on the equipment is large. After a long period of high-power operation, its wear will increase. The value of the exponential function increases rapidly, making The result shows that the equipment wear is accelerating. When the working power is lower than the rated value, the equipment is less worn due to the lighter load. The negative exponent is used in the formula and the , so that Growth depreciation coefficient Compared to 1, it decreases slightly, thus retaining a certain depreciation effect. Through the nonlinear characteristics of the exponential function, the relative difference between actual power and rated power and the cumulative effect of mission duration on equipment wear are combined to achieve a quantitative expression of the equipment's depreciation status, providing an intuitive and scientific numerical basis for the subsequent calculation of component depreciation coefficients and maintenance decisions.

[0127] Obtain the number of replacements for each component, which is obtained from the maintenance log. Obtain the theoretical number of replacements for each component, which is obtained from the vendor. Build a depreciation coefficient for each component. The depreciation coefficient for each component is based on the following formula:

[0128]

[0129] in, Indicates the The depreciation factor of each component, Indicates the Number of parts replacements, Indicates the Theoretical replacement times of each component, is the depreciation factor of the crane, Retrieve variables for part numbers, , .

[0130] Combine the depreciation coefficient of the crane and the number of replacements of this component to reflect the depreciation coefficient of this component. Due to the multiple replacements of components and the wear of surrounding matching parts, the overall effect of each replacement component is not as good as the overall effect of the previous replacement. Therefore, it is necessary to quantify the depreciation coefficient of each component. The depreciation coefficient of each component is a coefficient formed by the number of replacements based on the depreciation coefficient of the crane. The first item of the coefficient is 1, which means that the depreciation coefficient of each component should be based on the depreciation effect of the crane as a whole. It indicates the ratio of actual replacement times to theoretical replacement times. The closer the actual replacement times are to the theoretical replacement times, the more times the component has been replaced. The larger the value is, the more serious the damage effect of this component is compared with the damage effect of the crane.

[0131] By constructing a crane depreciation coefficient, the actual task requirements are associated with the equipment's rated parameters and historical replacement times, forming a numerical indicator that reflects the equipment's wear condition. By organically combining future task load requirements with the equipment's historical wear information, subsequent revisions to the comprehensive component scores are more consistent with the equipment's actual operating environment. By comparing the actual number of component replacements with the theoretical number of replacements, the life consumption of the crane under actual working conditions can be more accurately reflected, making the depreciation coefficient more valuable in providing future task load predictions and risk assessments. The depreciation coefficient acts as a bridge in the system, not only establishing an intuitive numerical mapping between task data and equipment maintenance thresholds, but also providing a dynamic adjustment mechanism for overall maintenance decisions, enabling equipment maintenance plans to be intelligently adjusted according to actual workloads, thereby effectively avoiding resource waste or safety hazards caused by inaccurate predictions.

[0132] Step 5: Obtain the crane's lifespan at rated power and construct an age coefficient based on the crane's historical operating data. Use the age coefficient and the depreciation coefficient of each component to correct the replacement score of each component to obtain a comprehensive replacement score. At the same time, use the comprehensive replacement score to correct the set component score threshold to obtain a correction threshold. Based on the relationship between the comprehensive replacement score and the correction threshold, determine whether the crane should replace components before the next mission.

[0133] The step 5 includes the following contents:

[0134] Step 501: Obtain the life of the crane at rated power. The life is obtained from the crane manufacturer. The equipment life coefficient is calculated based on the crane's historical operating data. The formula is as follows:

[0135]

[0136] in, is the age coefficient, Indicates the number of historical work data The working power of the sub-task, Indicates the number of historical work data The working time of the task, is the rated power, The lifespan at rated power is , , The total number of all tasks in the historical work data.

[0137] It reflects the relative relationship between the cumulative workload borne by the crane during actual operation and the design life of the equipment at rated power. It indicates the consumption ratio or "depreciation degree" of the equipment relative to its design life after the equipment has been used for historical tasks, which truly reflects the aging or remaining life of the equipment under actual use conditions. and working hours They represent the power level output by the crane in a single operation and the duration of the operation. It can be regarded as the load or energy consumed by the equipment during the operation. It reflects the operating capacity and service life that the equipment is expected to support under the design state. The product of the two is a standardized reference value used to judge whether the equipment has exceeded its predetermined load level. When the power output or operating time experienced by the crane in historical operations increases, the total load increases, causing the numerator value to rise, thus It will increase, indicating that the equipment is actually bearing a higher load and is aging more seriously. It is a fixed rated index and plays a role of standardization, comparing the actual cumulative load with the design standard. When the actual operating load is closer to or exceeds the design rated value, The larger the value, the more likely the device is approaching or exceeding its design life. The lower it is, the more the equipment is still operating within the rated range, which is beneficial to extending its service life.

[0138] By introducing the life information and historical load data of the crane at rated power and calculating the age coefficient, the overall quantification of the aging status of the equipment is achieved. By making full use of the accumulated working data of the equipment and comprehensively analyzing the working power and duration in historical tasks, the attenuation process and life loss of the equipment during operation are converted into a controllable indicator. The age coefficient is not only an important parameter for measuring the overall health and decline of the equipment, but also provides a global perspective for the subsequent comprehensive score correction of components. Equipment aging has a profound impact on the actual performance of individual components. Its introduction ensures that maintenance decisions are not based solely on the local status of the components, but radiate to the working life of the entire crane. This evaluation model that combines macro and micro perspectives helps to form a comprehensive, objective and dynamic equipment maintenance management mechanism, thereby greatly improving the reliability and economy of the overall system.

[0139] Step 502: Correct the replacement score of each component based on the age factor and the depreciation factor of each component to obtain a comprehensive replacement score. The formula is as follows:

[0140]

[0141] in, Indicates the The comprehensive replacement score of each component, is the age coefficient, Indicates the The depreciation factor of each component, For the Replacement rating for each component, Retrieve variables for part numbers, , ;

[0142] Reflects the actual replacement needs of the component under the current equipment operating conditions and provides a quantitative basis for maintenance decisions. It is used to express the ratio between the accumulated working load of the crane and the design life of the equipment under rated conditions, which is regarded as the degree of aging and life consumption of the entire equipment; The first The depreciation coefficient of each component, which combines the actual replacement times of the component with the theoretical replacement times, and then combines it with the depreciation index of the crane to reflect the performance loss of the component due to frequent use or wear; It is the original replacement score obtained from the deep reinforcement learning model, which represents the system's preliminary evaluation of the reward value of the component replacement action based on historical work data and real-time monitoring information. It reflects the cumulative effect of the whole machine workload. When the equipment is overloaded for a long time, Larger, it will amplify the risk of component replacement; The health status information of the parts is interactively adjusted. Parts that are frequently replaced or severely worn have a greater depreciation coefficient, which will directly improve the overall score. As a score obtained directly from the deep reinforcement learning model, it comprehensively reflects multiple factors such as individual task data, failure probability, reward value, etc.

[0143] Set a component score threshold and obtain a modified threshold based on the comprehensive replacement score of all components. The modified threshold is based on the following formula:

[0144]

[0145] in, To correct the threshold, is the scoring threshold, Indicates the The comprehensive replacement score of each component, Retrieve variables for part numbers, , ;

[0146] To correct the threshold, a dynamic critical value is set, which is directly linked to the health status of each component, thereby guiding whether to replace it immediately. It is the preset scoring threshold, which represents the judgment boundary of the safety and risk of each component of the equipment in the default state. It is the quantitative result of the comprehensive replacement score of a single component, which cumulatively reflects the overall use and aging level of the equipment. The average value is normalized, and the formula uses an exponential function to feed back the health of the entire system to the threshold. As it increases, the exponential part becomes smaller, i.e. Below 1, then will be better than A small score means that the corrected critical value is lowered, and the system will be more strict in judging the replacement action, which is easy to promote early replacement. This reflects that when the overall wear or aging of the equipment is more serious, even if the score drops slightly, maintenance will be triggered in advance, thereby ensuring operational safety. On the contrary, when When it is lower, the equipment is in good condition and the normalized term is smaller. Close to 1, Close , thus maintaining the original judgment expectation. Overall, this formula is Threshold The index correction forms an adaptive judgment standard, which can dynamically adjust the maintenance strategy according to the current actual operating status of the equipment components, thereby more accurately guiding the predictive maintenance of the equipment.

[0147] When the comprehensive replacement score is greater than the correction threshold, the component will be replaced immediately;

[0148] When the comprehensive replacement score is less than the correction threshold, it means that the component can continue to be used.

[0149] The component scores are corrected to form a final comprehensive replacement score, and then the decision on whether to replace the component is made based on the preset score threshold. This achieves the organic integration of multi-indicator and multi-level information, and uniformly converts multiple dimensional data such as historical data, equipment load, economic cost, and aging trend into a clear decision output. It not only eliminates the possible limitations of a single indicator, but also gives the system adaptive adjustment characteristics, making maintenance decisions more in line with actual operating conditions and economic benefits. Through the revised score threshold decision, hidden dangers can be discovered in time and intervened in advance to prevent sudden failures, significantly improving the reliability of the overall maintenance system. At the same time, the synergistic effect between the various components in the comprehensive score also provides a system complementary correction mechanism, making the entire predictive maintenance method more real-time and flexible in actual application, and better supporting smart equipment management and economic maintenance.

[0150] By comparing the comprehensive score with the dynamic correction threshold, a closed decision loop from data analysis to maintenance execution is finally achieved. The comprehensive score integrates the replacement score, depreciation factor and age factor, which not only incorporates the real-time model recommendation priority, but also corrects the impact of task load and long-term aging, avoiding the one-sidedness of a single indicator. The correction threshold is calculated by the dynamic mean of the scores of all components, and the judgment criteria are adaptively adjusted. For example, when the overall score is high, the threshold is lowered, which solves the problem of fixed threshold failure under different equipment or working conditions. Ultimately, the clear comparison of the score and the threshold provides clear instructions to technicians, reduces subjective judgment errors, and continuously optimizes the model input data through closed-loop feedback. Step 6 integrates the outputs of all the previous steps, ensures the flexibility and scalability of the maintenance strategy through dynamic logic, and realizes the full process automation of predictive maintenance from data collection to decision execution.

[0151] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0152] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution.

[0153] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment as needed.

[0154] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. The equipment predictive maintenance method based on deep reinforcement learning is characterized by: The specific steps include: Step 1: Obtain the replacement reasons for crane parts and add timestamps. Obtain the crane's historical operating data, which includes the operating power, operating duration, vibration curve, voltage curve, and timestamp of each task. Analyze the vibration and voltage curves of each task to obtain the vibration and voltage variation coefficients, and form a working data set for each task. Analyze the parts replacement and working data sets based on the timestamps to obtain the set of tasks each part participated in from installation and use to replacement. Step 2: Obtain the replacement cost of each component and the failure cost of each task, and construct a reward formula based on the replacement action. Input the task set of each component into the deep Q network model for training, and output the reward value of the replacement action of each component in different tasks to obtain the replacement guidance model; Step 3: Obtain the installation time of the components currently used by the crane, as well as the operating power, operating time, vibration curve, and voltage curve of each component's tasks. This is used to form a task set and input it into the replacement guidance model. The reward value associated with each component's replacement action is then obtained to form a replacement score for each component. Step 4: Based on the working power and working hours required by the hoisting motor for the next mission, the crane's depreciation coefficient is calculated by combining the obtained rated power of the hoisting motor. The depreciation coefficient of each component is calculated by combining the replacement times and theoretical replacement times of each component with the crane's depreciation coefficient. Step 5: Obtain the crane's lifespan at rated power and construct an age coefficient based on the crane's historical operating data. Use the age coefficient and the depreciation coefficient of each component to correct the replacement score of each component to obtain a comprehensive replacement score. At the same time, use the comprehensive replacement score to correct the set component score threshold to obtain a correction threshold. Based on the relationship between the comprehensive replacement score and the correction threshold, determine whether the crane should replace components before the next mission.

2. The equipment predictive maintenance method based on deep reinforcement learning according to claim 1, characterized in that: Obtain the replacement reasons of crane parts and add timestamps. The parts include wire ropes, pulleys, brake pads, and motor bearings. At the same time, perform life assessment on the replaced parts to obtain the remaining life. The remaining life is obtained through expert evaluation. The replacement reasons include preventive replacement and fault replacement. The parts replacement reasons are obtained through maintenance logs. Obtain the historical working data of the crane. The working data includes the working power, working time, vibration curve and voltage curve of each task and timestamp. Each task is the process of a single cargo lifting. The vibration curve and voltage curve are obtained by monitoring the crane's hoisting motor and are the vibration curve and voltage curve of a single lifting of the hoisting motor. The working power and working time are the operating power and operating time of the hoisting motor during a single task. The horizontal axis of the vibration curve is the startup time and the vertical axis is the amplitude. The horizontal axis of the voltage curve is the startup time and the vertical axis is the voltage. The vibration curve and voltage curve of each task are recorded separately and correspond one to one. The amplitude curve and voltage curve of each task are analyzed separately to obtain the vibration variation coefficient and voltage variation coefficient of each task. The logic is as follows: Get the mean and standard deviation of each vibration curve and voltage curve respectively, and get the vibration coefficient of variation and voltage coefficient of variation according to the following formula: in, is the vibration variation coefficient, is the mean value of the vibration curve, is the standard deviation of the vibration curve, is the voltage variation coefficient, is the mean value of the voltage curve, is the standard deviation of the voltage curve.

3. The equipment predictive maintenance method based on deep reinforcement learning according to claim 2, characterized in that: The lifting tasks are sorted according to the timestamps, and the working power, working time, vibration coefficient of variation and voltage coefficient of variation of each task are summarized to form a single task work data set. The formula is as follows: in, Indicates the The working dataset for each task, is the working power, For working hours, is the vibration variation coefficient, is the voltage variation coefficient, Retrieve the number for the working dataset, , , is the total number of tasks; All replaced parts are numbered, and the part number, replacement reason, and remaining life are mapped to the work data based on the timestamp. The task set that each part participates in during each period from installation and use to replacement is obtained from the work data set. The formula is as follows: in, Indicates the Part No. The set of tasks involved from installation and use to replacement, Indicates the The working dataset for each task, Retrieve the number for the working dataset, , , is the total number of tasks, For the The number of the first working data set involved from installation to replacement, For replacement reasons, For the remaining life of the replaced parts, , , Retrieve variable for number of parts replacements, , , Indicates the Total number of parts replaced; Among them, when the reason for replacement is preventive replacement of this task, , when the replacement reason is the fault replacement in this task, , the last working dataset is a related dataset, which indicates the replacement of parts in this task link. When the last working data set is completed When replacing this part, When the last working data set Replacement of the component during the process.

4. The equipment predictive maintenance method based on deep reinforcement learning according to claim 3 is characterized by: Set the label action based on the following formula: in, Indicates action, Indicates no replacement. Indicates replacement; The replacement cost of each component is obtained separately. The replacement cost includes the cost of the replaced component, labor cost, and time cost. The time cost is expressed as a monetary amount and is determined by the crane's organization. The failure cost of each component's task is obtained. The failure cost is the property loss caused by task failure and is calibrated by the crane's organization. The task set of each component is input into the deep Q network model for training, and the reward formula is set according to the following formula: in, is the reward value, For action, For replacement cost, Working dataset for a single task The failure cost, is the failure probability.

5. The equipment predictive maintenance method based on deep reinforcement learning according to claim 4 is characterized in that: The logic for obtaining the failure probability is as follows: Obtain the rated life of each replacement part. The rated life of the replacement part is obtained from the manufacturer. Based on the task set in which each component participates, obtain the working power and working time of the task experienced by the component before replacement. Determine the failure probability based on the remaining life of the replaced component. The failure probability is based on the formula: in, is the failure probability, For the The working power of the sub-task, For the The working time of the task, is the rated life of the component, Retrieve variable for task number, , , To calculate the failure probability of this task, the number of tasks that this component has experienced before, and is the coefficient, obtained through the fault probability judgment model; The task set that each component participates in from installation to replacement is used to obtain the working power and working time of each component in each task as the training set. The linear regression model is input for training and the fitting formula is set. The formula is as follows: in, For the The working power of the sub-task, For the The working time of the task, The remaining life of the replacement part. is the rated life of this replacement part, Retrieve the ID for the task, , , The total number of tasks performed for this replacement part; By fitting the formula, the failure probability judgment model of the component is obtained, and the coefficients corresponding to each component are obtained in this way. and ; The output is the reward value for the two actions of replacing and not replacing parts. The deep reinforcement learning model of each part is obtained and calibrated as the replacement guidance model.

6. The equipment predictive maintenance method based on deep reinforcement learning according to claim 5, characterized in that: The installation time of the components used in the current crane is obtained, as well as the working power, working time, vibration curve, and voltage curve of each component in the task it has experienced. This is used to form a task set and input it into the replacement guidance model. The reward value for each component for replacement and non-replacement is obtained to form a replacement score based on the following formula: in, For the Replacement rating for each component, is the reward value for the change action, is the reward value of not changing the action, Retrieve variables for part numbers, , ; Get the replacement score for each component separately.

7. The equipment predictive maintenance method based on deep reinforcement learning according to claim 6, characterized in that: Get the next task to be performed, and according to the working power and working time required by the hoisting motor for the next task, get the rated power of the hoisting motor and construct the depreciation coefficient of the crane. The formula is as follows: in, is the depreciation factor of the crane, is the coefficient, The working power of the next task, is the rated power, for working hours; Obtain the number of replacements for each component, which is obtained from the maintenance log. Obtain the theoretical number of replacements for each component, which is obtained from the vendor. Build a depreciation coefficient for each component. The depreciation coefficient for each component is based on the following formula: in, Indicates the The depreciation factor of each component, Indicates the Number of parts replacements, Indicates the Theoretical replacement times of each component, is the depreciation factor of the crane, Retrieve variables for part numbers, , .

8. The equipment predictive maintenance method based on deep reinforcement learning according to claim 7, characterized in that: Obtain the life of the crane at rated power. The life is obtained from the crane manufacturer. The equipment life coefficient is calculated based on the crane's historical operating data. The formula is as follows: in, is the age coefficient, Indicates the number of historical work data The working power of the sub-task, Indicates the number of historical work data The working time of the task, is the rated power, The lifespan at rated power is , , The total number of all tasks in the historical work data.

9. The equipment predictive maintenance method based on deep reinforcement learning according to claim 8, characterized in that: The replacement score of each component is adjusted based on the age factor and the depreciation factor of each component to obtain the comprehensive replacement score. The formula is as follows: in, Indicates the The comprehensive replacement score of each component, is the age coefficient, Indicates the The depreciation factor of each component, For the Replacement rating for each component, Retrieve variables for part numbers, , ; Set a component score threshold and obtain a modified threshold based on the comprehensive replacement score of all components. The modified threshold is based on the following formula: in, To correct the threshold, is the scoring threshold, Indicates the The comprehensive replacement score of each component, Retrieve variables for part numbers, , ; When the comprehensive replacement score is greater than the correction threshold, the component will be replaced immediately; When the comprehensive replacement score is less than the correction threshold, it means that the component can continue to be used.

Images