Collaborative optimization method and device for high-speed rail wheel pair maintenance and spare part management

By constructing a degradation and repair behavior model of high-speed rail wheel pairs and a replacement and spare parts management behavior model, combining MDP and CMDP models, the maintenance and spare parts management of high-speed rail wheel pairs are optimized, solving the high maintenance cost and application limitations in the existing technology. The problem of this has achieved efficient and safe wheel-to-wheel repair and spare parts management.

CN118916995APending Publication Date: 2024-11-08TSINGHUA UNIVERSITY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411086618.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing high-speed rail wheel pair maintenance strategy does not fully consider the impact of the internal wheels on spare parts inventory and the structural correlation between wheel pairs, resulting in high maintenance costs, low efficiency of solution algorithms, great application limitations, and excessive maintenance and potential safety risks.

Method used

By constructing the degradation and startup behavior model of high-speed rail wheel pairs, the reboot gain is determined, and based on this, the replacement and spare parts management behavior model is constructed, the MDP five-tuple model and the CMDP model are established, and the actuator-evaluator strategy gradient architecture and Constraint strategy optimization algorithm to realize dynamic inventory control and maintenance strategy optimization.

Benefits of technology

It significantly improves the system operation efficiency, reduces downtime caused by failures or out of stock, reduces inventory costs, avoids resource waste, and improves the ability to deal with sudden failures, ensuring the safe and reliable operation of high-speed trains.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916995A_ABST
    Figure CN118916995A_ABST
Patent Text Reader

Abstract

The invention relates to a collaborative optimization method and device for high-speed rail wheel set maintenance and spare part management, and the method comprises the steps: constructing a degradation and turning repair behavior model of a high-speed rail wheel set, and determining the re-turning repair gain of a (t + 1) th decision node, so as to obtain a mathematical expression of the wheel diameter and the rim thickness after the turning repair of the (t + 1) th decision node; constructing a replacement and spare part management behavior model of the high-speed rail wheel set to determine a dynamic inventory control strategy; and constructing an MDP quintuple model based on the dynamic inventory control strategy, establishing a CMDP model in combination with the security constraint set, and solving the CMDP model by using a preset actuator-evaluator strategy gradient architecture and a constraint strategy optimization algorithm to obtain a collaborative optimization result. Therefore, the problems that an existing high-speed rail wheel set maintenance strategy does not fully consider the influence of the spare part stock of the wheel set in the motor train station and the structural relevance between the wheel sets, the maintenance cost is high, and the efficiency of a wheel set maintenance optimization model solving algorithm is low are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of maintenance and spare parts management of complex systems, and in particular to a collaborative optimization method and device for high-speed railway wheelset maintenance and spare parts management. Background Art

[0002] Wheel sets are a key component of high-speed trains. They are in direct contact with the track and have the important task of supporting and propelling the train. A high-speed train consists of 8 to 16 carriages, each of which is supported by four wheel sets, each of which includes two wheels. These wheels are the main contact interface with the railway tracks, and long-term interaction causes the wheels to gradually wear out or even fail. Wheel maintenance must follow strict geometry and tolerance standards to ensure operational safety and efficiency. The diameters of wheels on the same axle, in the same carriage, and even in the entire high-speed train must comply with specific tolerance limits. This requirement makes precise turning or replacement of wheels a necessary measure to ensure train safety and functionality.

[0003] Over time, the components in the wheelset system will gradually degrade. Railway companies use data collected regularly by sensors to carry out inspection and maintenance activities and make maintenance and inventory decisions dynamically. Once a wheel is in poor condition, it will be turned or replaced accordingly to ensure its full functionality. If the wheel diameter does not meet the standard, the wheelset must be replaced, which requires the EMU to reserve a sufficient number of spare parts for timely replacement; however, excessive reserves of spare parts will lead to increased inventory costs, while insufficient spare parts may lead to high out-of-stock costs; balancing the number of spare parts is the key to reducing total costs. The goal of coordinated optimization of wheelset maintenance and spare parts management is to determine the best strategy for turning and replacement by checking the wheelset diameter and rim thickness, while reducing the joint cost of system maintenance and spare parts management while ensuring the reliability requirements of high-speed trains.

[0004] At present, China's high-speed train maintenance mode is mainly periodic maintenance, and the wheelset is reprocessed every 200,000 kilometers. This periodic maintenance strategy may lead to excessive use and waste of resources. Therefore, by adopting the emerging deep reinforcement learning technology, through interactive learning with the environment and adapting to changes, it is possible to optimize the maintenance strategy, realize dynamic adjustment of the maintenance strategy, minimize costs and achieve reliability assurance. The existing high-speed rail wheelset maintenance strategy that realizes dynamic, efficient and low-cost still has the following challenges:

[0005] 1. The railway wheelset maintenance strategy is more of a fixed periodic maintenance strategy, rather than a dynamic adjustment based on the equipment status. Over-maintenance is more serious, resulting in a large amount of cost and resource waste.

[0006] 2. The existing high-speed rail wheelset maintenance strategy is not linked to system reliability, and the structural correlation between wheelsets is insufficiently considered, which may bring potential risks to the safe operation of high-speed rail.

[0007] 3. The formulation of the wheelset maintenance strategy does not fully consider the impact of the wheelset spare parts inventory in the EMU depot. When formulating the wheelset maintenance strategy, considering the comprehensive cost of wheelset maintenance and spare parts management can further achieve an overall reduction in vehicle operation and maintenance costs.

[0008] 4. The current wheelset maintenance optimization model solving algorithm has the problems of low solving efficiency, limited solving ability, and small scale of problems that can be solved. It has great application limitations for EMU depots that manage multiple high-speed trains.

[0009] In summary, the existing high-speed rail wheelset maintenance strategy does not fully consider the impact of wheelset spare parts inventory in the EMU depot and the structural correlation between wheelsets, and over-maintenance is relatively serious. The efficiency of the solution algorithm is low and the application limitations are large, which need to be solved urgently. Summary of the invention

[0010] The present application provides a collaborative optimization method and device for high-speed rail wheelset maintenance and spare parts management to solve the problems that the existing high-speed rail wheelset maintenance strategy does not fully consider the impact of wheelset spare parts inventory in the EMU depot and the structural correlation between wheelsets, and the maintenance cost is high, and the efficiency of the wheelset maintenance optimization model solution algorithm is low.

[0011] The first aspect of the present application provides a collaborative optimization method for high-speed railway wheelset maintenance and spare parts management, comprising the following steps: constructing a degradation and turning behavior model for high-speed railway wheelsets, and determining the re-turning gain of the (t+1)th decision node according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and rim thickness of the wheel after turning at the (t+1)th decision node through the re-turning gain, wherein t is a positive integer; constructing a replacement and spare parts management behavior model for the high-speed railway wheelset based on the degradation and turning behavior model and the mathematical expression of the wheel diameter and rim thickness, so as to determine a dynamic inventory control strategy according to the replacement and spare parts management behavior model; constructing an MDP five-tuple model based on the dynamic inventory control strategy, and establishing a CMDP model according to the MDP five-tuple model and a preset safety constraint set, and solving the CMDP model using a preset actuator-evaluator policy gradient architecture and a constraint strategy optimization algorithm, so as to obtain a collaborative optimization result of high-speed railway wheelset maintenance and spare parts management that meets the preset optimization requirements.

[0012] Optionally, in one embodiment of the present application, the re-turning gain of the (t+1)th decision node is determined according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node through the re-turning gain, including: determining the lower limit of the wheel diameter and the lower limit of the rim thickness of the high-speed rail wheel; obtaining the actual wheel diameter and actual rim thickness of the high-speed rail wheel at the tth decision node; and calculating the wheel diameter and actual rim thickness of the (t+1)th decision node based on the actual wheel diameter and the actual rim thickness. The wheel diameter and rim thickness of the wheel are determined, and whether the high-speed rail wheel meets the preset fault risk condition is determined according to the lower limit of the wheel diameter, the lower limit of the rim thickness, the wheel diameter and rim thickness of the (t+1)th decision node; if the high-speed rail wheel meets the preset fault risk condition, the re-turning gain of the (t+1)th decision node is calculated through the degradation and turning behavior model; according to the re-turning gain and a preset nonlinear regression function, a mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node is obtained.

[0013] Optionally, in one embodiment of the present application, the dynamic inventory control strategy is determined according to the replacement and spare parts management behavior model, including: determining the inventory level of the high-speed rail wheel at the t-th decision node and the number of replacement wheels; calculating the spare parts inventory level of the (t+1)-th decision node based on the replacement and spare parts management behavior model, a preset piecewise linear function, the inventory level of the t-th decision node and the number of replacement wheels; judging whether the high-speed rail wheel meets a preset out-of-stock condition according to the spare parts inventory level of the (t+1)-th decision node, and determining the number of spare parts ordered at the t-th decision node if the high-speed rail wheel meets the preset out-of-stock condition, so as to update the spare parts inventory level of the (t+1)-th decision node using the ordered spare parts quantity.

[0014] Optionally, in one embodiment of the present application, constructing an MDP quintuple model based on the dynamic inventory control strategy includes: calculating the wheel radial quantity and rim thickness vector of the t-th decision node, and determining the state space of the MDP quintuple model based on the inventory level of the t-th decision node, the wheel radial quantity and the rim thickness vector; determining the wheel maintenance behavior and spare parts ordering quantity behavior of the high-speed rail wheel at the t-th decision node, so as to establish the behavior space of the MDP quintuple model according to the wheel maintenance behavior and the spare parts ordering quantity behavior; determining the transfer model of the MDP quintuple model according to the state space and the behavior space; calculating the wheel repair cost, replacement cost, unit stock-out cost, inventory cost and ordering startup cost of the high-speed rail wheel, so as to determine the reward function of the MDP quintuple model through the wheel repair cost, the replacement cost, the unit stock-out cost, the inventory cost and the ordering startup cost; constructing the MDP quintuple model according to the state space, the behavior space, the transfer model and the reward function.

[0015] Optionally, in one embodiment of the present application, the CMDP model is established according to the MDP quintuple model and the preset safety constraint set, and the CMDP model is solved using the preset actuator-evaluator policy gradient architecture and constraint policy optimization algorithm to obtain the collaborative optimization result of high-speed rail wheelset maintenance and spare parts management that meets the preset optimization requirements, including: based on the state space, the behavior space and the reward function, the safety constraint set is determined to construct the CMDP model according to the MDP quintuple model and the safety constraint set; wheel diameter difference detection is performed on adjacent high-speed rail wheels in the same carriage of the target high-speed rail train to obtain wheel diameter difference results, and additional safety constraints are established based on the wheel diameter difference results; based on the additional safety constraints, an expected discounted return is obtained, and the reliability constraints of the CMDP model are determined based on the expected discounted return and the preset safety threshold of the target high-speed rail train; the CMDP model is parameterized through the actuator-evaluator policy gradient architecture, and based on the parameterized CMDP model, and in combination with the preset trust region policy optimization algorithm, the constraint policy update rule and the constraint policy optimization algorithm, the collaborative optimization result under the reliability constraint is calculated.

[0016] The second aspect of the present application provides a collaborative optimization device for high-speed railway wheelset maintenance and spare parts management, including: a first construction module, used to construct a degradation and turning behavior model of the high-speed railway wheelset, and determine the re-turning gain of the (t+1)th decision node according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node through the re-turning gain, wherein t is a positive integer; a second construction module, used to construct a degradation and turning behavior model, the wheel diameter and rim thickness based on the degradation and turning behavior model, the wheel diameter and rim thickness A mathematical expression is used to construct a replacement and spare parts management behavior model of the high-speed railway wheelset, so as to determine a dynamic inventory control strategy according to the replacement and spare parts management behavior model; a solving module is used to construct an MDP five-tuple model based on the dynamic inventory control strategy, and to establish a CMDP model according to the MDP five-tuple model and a preset safety constraint set, and to solve the CMDP model using a preset actuator-evaluator policy gradient architecture and a constraint strategy optimization algorithm, so as to obtain a collaborative optimization result of high-speed railway wheelset maintenance and spare parts management that meets the preset optimization requirements.

[0017] Optionally, in one embodiment of the present application, the first construction module includes: a first determination unit, used to determine the lower limit of the wheel diameter and the lower limit of the rim thickness of the high-speed rail wheel; a first acquisition unit, used to obtain the actual wheel diameter and the actual rim thickness of the high-speed rail wheel at the tth decision node; a first judgment unit, used to calculate the wheel diameter and the rim thickness of the (t+1)th decision node based on the actual wheel diameter and the actual rim thickness, and judge whether the high-speed rail wheel meets the preset fault risk condition according to the lower limit of the wheel diameter, the lower limit of the rim thickness, the wheel diameter and the rim thickness of the (t+1)th decision node; a first calculation unit, used to calculate the re-turning gain of the (t+1)th decision node through the degradation and turning behavior model if the high-speed rail wheel meets the preset fault risk condition; a second calculation unit, used to obtain the mathematical expression of the wheel diameter and the rim thickness after turning at the (t+1)th decision node according to the re-turning gain and a preset nonlinear regression function.

[0018] Optionally, in one embodiment of the present application, the second construction module includes: a second determination unit, used to determine the inventory level and the number of replacement wheels of the high-speed rail wheel at the tth decision node; a third calculation unit, used to calculate the spare parts inventory level of the (t+1)th decision node based on the replacement and spare parts management behavior model, a preset piecewise linear function, the inventory level and the number of replacement wheels of the tth decision node; a second judgment unit, used to judge whether the high-speed rail wheel meets the preset out-of-stock condition according to the spare parts inventory level of the (t+1)th decision node, and if the high-speed rail wheel meets the preset out-of-stock condition, determine the number of spare parts ordered at the tth decision node, so as to update the spare parts inventory level of the (t+1)th decision node using the ordered spare parts quantity.

[0019] Optionally, in one embodiment of the present application, the solution module includes: a fourth calculation unit, used to calculate the wheel radial quantity and the rim thickness vector of the t-th decision node, and determine the state space of the MDP five-tuple model based on the inventory level of the t-th decision node, the wheel radial quantity and the rim thickness vector; an establishment unit, used to determine the wheel maintenance behavior and spare parts ordering quantity behavior of the high-speed rail wheel at the t-th decision node, so as to establish the behavior space of the MDP five-tuple model according to the wheel maintenance behavior and the spare parts ordering quantity behavior; a third determination unit, used to determine the state space of the MDP five-tuple model based on the inventory level of the t-th decision node, the wheel radial quantity and the rim thickness vector ... A determining unit is used to determine the transfer model of the MDP quintuple model according to the state space and the behavior space; a fifth computing unit is used to calculate the wheel repair cost, replacement cost, unit stock-out cost, inventory cost and ordering startup cost of the high-speed rail wheel, so as to determine the reward function of the MDP quintuple model through the wheel repair cost, the replacement cost, the unit stock-out cost, the inventory cost and the ordering startup cost; a first modeling unit is used to construct the MDP quintuple model according to the state space, the behavior space, the transfer model and the reward function.

[0020] Optionally, in one embodiment of the present application, the solution module also includes: a second modeling unit, used to determine the safety constraint set based on the state space, the behavior space and the reward function, so as to construct the CMDP model according to the MDP quintuple model and the safety constraint set; a detection unit, used to perform wheel diameter difference detection on adjacent high-speed rail wheels in the same carriage of the target high-speed rail train to obtain wheel diameter difference results, and establish additional safety constraints based on the wheel diameter difference results; a second acquisition unit, used to obtain the expected discounted return based on the additional safety constraints, and determine the reliability constraints of the CMDP model based on the expected discounted return and the preset safety threshold of the target high-speed rail train; a parameterization unit, used to parameterize the CMDP model through the actuator-evaluator policy gradient architecture, and based on the parameterized CMDP model, and in combination with the preset trust region policy optimization algorithm, the constraint policy update rule and the constraint policy optimization algorithm, calculate the collaborative optimization result under the reliability constraint.

[0021] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the collaborative optimization method for high-speed rail wheelset maintenance and spare parts management as described in the above embodiment.

[0022] The fourth aspect embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, it implements the above-mentioned collaborative optimization method of high-speed railway wheelset maintenance and spare parts management.

[0023] The fifth aspect of the present application provides a computer program product, including a computer program, which is executed to implement the above-mentioned collaborative optimization method for high-speed railway wheelset maintenance and spare parts management.

[0024] Therefore, the embodiments of the present application have the following beneficial effects:

[0025] The embodiments of the present application can construct a degradation and turning behavior model of a high-speed railway wheelset, and determine the re-turning gain of the (t+1)th decision node according to the degradation and turning behavior model, so as to obtain the mathematical expression of the wheel diameter and rim thickness of the wheel after turning at the (t+1)th decision node through the re-turning gain, wherein t is a positive integer; based on the degradation and turning behavior model, the mathematical expression of the wheel diameter and rim thickness, construct a replacement and spare parts management behavior model of a high-speed railway wheelset, so as to determine the dynamic inventory control strategy according to the replacement and spare parts management behavior model; based on the dynamic inventory control strategy, construct an MDP five-tuple model, and establish a CMDP model according to the MDP five-tuple model and a preset safety constraint set, and use the preset actuator-evaluator policy gradient architecture and constraint strategy optimization algorithm to solve the CMDP model, so as to obtain the collaborative optimization results of high-speed railway wheelset maintenance and spare parts management that meet the preset optimization requirements. This application can significantly improve the operating efficiency of the system, reduce downtime caused by failures or out-of-stock, and ensure that the system can continue to operate stably; in addition, this application helps to reduce inventory costs and avoid waste of resources through reasonable collaborative optimization strategies, while improving the ability to deal with sudden failures. This solves the problems that the existing high-speed rail wheelset maintenance strategy does not fully consider the impact of wheelset spare parts inventory in the EMU depot and the structural correlation between wheelsets, and the maintenance cost is high, and the efficiency of the wheelset maintenance optimization model solution algorithm is low.

[0026] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A flowchart of a collaborative optimization method for high-speed railway wheelset maintenance and spare parts management provided according to an embodiment of the present application;

[0029] Figure 2 A schematic diagram of wheel maintenance and spare parts management of a motor vehicle depot provided for one embodiment of the present application;

[0030] Figure 3 A schematic diagram of the execution logic of a constraint strategy optimization algorithm provided for one embodiment of the present application;

[0031] Figure 4 A schematic diagram of the execution logic of a collaborative optimization method for high-speed railway wheelset maintenance and spare parts management provided for one embodiment of the present application;

[0032] Figure 5A schematic diagram of an iteration curve of different costs provided for an embodiment of the present application;

[0033] Figure 6 A schematic diagram of the results of different costs varying with average degradation rate and inventory capacity provided for one embodiment of the present application;

[0034] Figure 7 A schematic diagram of the results of different costs as a function of unit repair and replacement costs provided for one embodiment of the present application;

[0035] Figure 8 This is an example diagram of a collaborative optimization device for high-speed railway wheelset maintenance and spare parts management according to an embodiment of the present application;

[0036] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0037] Among them, 10-a collaborative optimization device for high-speed railway wheelset maintenance and spare parts management; 100-a first building module, 200-a second building module, 300-a solution module; 901-a memory, 902-a processor, 903-a communication interface. DETAILED DESCRIPTION

[0038] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0039] The following describes the collaborative optimization method and device for high-speed railway wheelset maintenance and spare parts management in an embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology, the present application provides a collaborative optimization method for high-speed railway wheelset maintenance and spare parts management. In the method, a degradation and turning behavior model of the high-speed railway wheelset is constructed, and the re-turning gain of the (t+1)th decision node is determined according to the degradation and turning behavior model, so as to obtain the mathematical expression of the wheel diameter and rim thickness of the (t+1)th decision node after turning through the re-turning gain, wherein t is a positive integer; based on the degradation and turning behavior model, the mathematical expression of the wheel diameter and rim thickness, a replacement and spare parts management behavior model of the high-speed railway wheelset is constructed, so as to determine the dynamic inventory control strategy according to the replacement and spare parts management behavior model; based on the dynamic inventory control strategy, an MDP five-tuple model is constructed, and a CMDP model is established according to the MDP five-tuple model and a preset safety constraint set, and the preset actuator-evaluator policy gradient architecture and constraint strategy optimization algorithm are used to solve the CMDP model, so as to obtain the collaborative optimization results of high-speed railway wheelset maintenance and spare parts management that meet the preset optimization requirements. This application can significantly improve the operating efficiency of the system, reduce downtime caused by failures or out-of-stock, and ensure that the system can continue to operate stably; in addition, this application helps to reduce inventory costs and avoid waste of resources through reasonable collaborative optimization strategies, while improving the ability to deal with sudden failures. This solves the problems that the existing high-speed rail wheelset maintenance strategy does not fully consider the impact of wheelset spare parts inventory in the EMU depot and the structural correlation between wheelsets, and the maintenance cost is high, and the efficiency of the wheelset maintenance optimization model solution algorithm is low.

[0040] Specifically, Figure 1 A flow chart of a collaborative optimization method for high-speed rail wheelset maintenance and spare parts management provided in an embodiment of the present application.

[0041] like Figure 1 As shown, the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management includes the following steps:

[0042] In step S101, a degradation and turning behavior model of the high-speed rail wheelset is constructed, and the re-turning gain of the (t+1)th decision node is determined based on the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node through the re-turning gain, where t is a positive integer.

[0043] The embodiments of the present application can first mathematically model the degradation and turning behaviors respectively, and on this basis define the re-turning gain of the (t+1)th decision node, thereby obtaining the calculation expression of the wheel diameter and rim thickness after turning at the (t+1)th decision moment.

[0044] Optionally, in one embodiment of the present application, the re-turning gain of the (t+1)th decision node is determined according to the degradation and turning behavior model, so as to obtain the mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node through the re-turning gain, including: determining the lower limit of the wheel diameter and the lower limit of the rim thickness of the high-speed rail wheel; obtaining the actual wheel diameter and actual rim thickness of the high-speed rail wheel at the tth decision node; calculating the wheel diameter and actual rim thickness of the (t+1)th decision node based on the actual wheel diameter and the actual rim thickness. The wheel diameter and rim thickness of the point are determined, and whether the high-speed rail wheel meets the preset fault risk conditions is judged according to the lower limit of the wheel diameter, the lower limit of the rim thickness, the wheel diameter and rim thickness of the (t+1)th decision node; if the high-speed rail wheel meets the preset fault risk conditions, the re-turning gain of the (t+1)th decision node is calculated through the degradation and turning behavior model; according to the re-turning gain and the preset nonlinear regression function, the mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node is obtained.

[0045] It should be noted that the embodiments of the present application focus on two key dimensions, wheel diameter and rim thickness, to capture the degradation and turning and repair process of railway wheels. These two dimensions are the most critical factors in wheel failure modes. The turning and repair strategy is implemented by evaluating the condition of the wheel diameter, while the replacement strategy is implemented based on the condition of the rim thickness.

[0046] The embodiment of the present application can firstly and Represent the initial wheel diameter and wheel rim thickness of a new and non-degraded wheel, respectively. X and Y They represent the lower limits of wheel diameter and wheel rim thickness respectively, and all new wheels preset in the embodiment of the present application have consistent structural dimensions and the same degradation mode. For a specific wheel k located in the jth carriage of the i-th high-speed train, its wheel diameter and wheel rim thickness at the t-th inspection are respectively and Indicates that the vector and They represent the wheel diameter and rim thickness vector at time ι respectively.

[0047] During the operation of high-speed trains, wheel degradation is manifested as a simultaneous decrease in wheel diameter and wheel rim thickness, such as Figure 2 As shown; in particular, when the degradation rate of the rim thickness exceeds the degradation rate of the wheel diameter, if there is no intervention, these two dimensions will continue to shrink. From the tth to the (t+1)th decision moment, the wheel diameter and rim thickness of the wheel k on the jth carriage of the i-th high-speed train change as follows:

[0048]

[0049] When the monitoring data of wheel degradation indicates that there is a possibility of failure of the high-speed rail wheel, the embodiment of the present application can perform a turning operation to restore the wheel rim thickness, but this operation requires sacrificing the wheel diameter as a price; the unit turning amount of the wheel rim thickness can be represented by δ, and the actual turning amount at time t is quantified as mδ,

[0050] Therefore, the decision of wheel turning needs to be carefully evaluated to balance the wheel diameter loss and the wheel rim thickness gain. The degree of wheel rim thickness recovery is related to the degree of wheel diameter reduction. A more aggressive recovery strategy will increase the frequency of replacement intervention and its related costs. Although minimizing wheel turning can reduce the reduction of wheel diameter, it may lead to accelerated wheel rim wear. Existing research has proposed several wheel turning models to define the wheel turning benefits. The model aims to clarify the proportional relationship between wheel diameter cost and wheel rim recovery. If the turning level is m, then the gain of re-turning at the (t+1)th decision node is for:

[0051]

[0052] in, and They represent the wheel diameters before and after turning at the (t+1)th decision time, respectively, and and Indicate the rim thickness before and after turning at the same decision moment.

[0053] Based on the existing fitted regression model, the gain Usually the front wheel rim thickness is repaired by turning and decreases with the increase of wheel rim thickness gain mδ. Through the nonlinear regression function The mathematical expression is as follows:

[0054]

[0055] in, represents the random ratio between the fitting error and the analytical model value, and α, β1, β2, μ1, μ2 are constant coefficients, which depend on the physical characteristics of the wheel.

[0056] Afterwards, the embodiment of the present application can be obtained by combining equation Definition and nonlinear regression function Estimate the wheel diameter after boring at the (t+1)th decision time The calculation expression is as follows:

[0057]

[0058]

[0059] Correspondingly, the mathematical expression of the wheel rim thickness after turning at the (t+1)th decision time is:

[0060]

[0061] Therefore, the embodiments of the present application mathematically model the degradation and re-turning behaviors and define the re-turning gains of the corresponding decision nodes, thereby obtaining the calculation formulas for the wheel diameter and rim thickness after re-turning, providing reliable theoretical support for the construction of subsequent replacement and spare parts management behavior models and CMDP models.

[0062] In step S102, based on the degradation and turning behavior model, the mathematical expression of the wheel diameter and the rim thickness, a replacement and spare parts management behavior model of the high-speed railway wheelset is constructed to determine the dynamic inventory control strategy according to the replacement and spare parts management behavior model.

[0063] Furthermore, the embodiments of the present application also need to construct a replacement and spare parts management behavior model for high-speed rail wheelsets based on the degradation and turning behavior model, the mathematical expression of wheel diameter and rim thickness, so as to When it is too low, it is replaced and when replacement occurs, the inventory level is reduced with the inspection cycle.

[0064] Therefore, the embodiments of the present application develop a more flexible dynamic inventory control strategy by utilizing replacement demand monitoring data based on wheel degradation status.

[0065] Optionally, in one embodiment of the present application, a dynamic inventory control strategy is determined based on the replacement and spare parts management behavior model, including: determining the inventory level of high-speed rail wheels at the tth decision node and the number of replacement wheels; calculating the spare parts inventory level at the (t+1)th decision node based on the replacement and spare parts management behavior model, a preset piecewise linear function, the inventory level at the tth decision node and the number of replacement wheels; judging whether the high-speed rail wheels meet a preset out-of-stock condition based on the spare parts inventory level at the (t+1)th decision node, and determining the number of spare parts ordered at the tth decision node if the high-speed rail wheels meet the preset out-of-stock condition, so as to update the spare parts inventory level at the (t+1)th decision node using the ordered number of spare parts.

[0066] In the actual implementation process, the embodiment of the present application can be based on the wheel diameter Determine whether to replace the wheel if the wheel diameter is below the safety threshold X , it indicates that the health of the wheel is unreliable and the probability of failure is high, and the wheel needs to be replaced.

[0067] It should be noted that if the wheel diameter difference between the wheels does not meet the safety constraints determined below, corresponding replacement measures will also be considered. This replacement consumes spare parts and reduces the inventory level of the maintenance facility. Various inventory control models have been established. Since inspection and maintenance activities are carried out regularly, when replacement occurs, the inventory level will decrease with the inspection cycle. Therefore, the embodiment of the present application also adopts a periodic strategy for spare parts inventory management.

[0068] The present application embodiment can utilize z t Denote the inventory level at time t, using n t represents the number of wheels replaced, the capacity of the maintenance facility is W, and as the uncertain replacement quantity consumes spare parts, the spare parts inventory level at the next decision time t+1 is:

[0069]

[0070] in,(.) + is a piecewise linear function, the term (z t -n t ) + is equivalent to:

[0071]

[0072] Thus, once a risk of out-of-stock is observed, embodiments of the present application can replenish inventory by sending a specific number of spare parts orders to the supply system.

[0073] In the embodiment of the present application, the number of spare parts ordered at the tth decision time can be q t After the order is sent to the supplier, the maintenance facility receives the spare parts after a fixed lead time. The embodiment of the present application may assume that the lead time is less than the inspection interval of the replacement and spare parts management behavior model in the embodiment of the present application. In this case, the spare parts order quantity at the t-th decision time can arrive at the maintenance facility before the t+1-th decision time. When the order is placed at the t-th decision time, the spare parts inventory level at the t+1-th decision time is:

[0074]

[0075] It should be noted that in the traditional inventory control model, orders are placed based on a fixed safety stock level or a fixed review interval. The predetermined fixed value provides a static strategy for inventory control, while the replacement and spare parts management behavior model of the embodiment of the present application can utilize the replacement demand monitoring data based on the wheel degradation status to develop a more flexible dynamic inventory control strategy.

[0076] Therefore, the embodiment of the present application takes the EMU depot as the core and combines the specific structural characteristics of the high-speed rail wheelset to establish a model that takes into account both the wheelset turning and repair and the wheelset spare parts management, thereby effectively ensuring the construction of the MDP quintuple model, etc.

[0077] In step S103, based on the dynamic inventory control strategy, an MDP quintuple model is constructed, and a CMDP (Constrained Markov Decision Process) model is established according to the MDP quintuple model and the preset safety constraint set, and the CMDP model is solved using the preset actuator-evaluator policy gradient architecture and constraint strategy optimization algorithm to obtain the collaborative optimization results of high-speed rail wheelset maintenance and spare parts management that meet the preset optimization requirements.

[0078] Furthermore, the embodiments of the present application can construct an MDP five-tuple model, and on the basis of the MDP five-tuple model, a corresponding CMDP model can be constructed in combination with safety constraints to handle a large number of problems subject to safety constraints or reliability constraints in maintenance plans.

[0079] Afterwards, the embodiments of the present application can be based on the actuator-evaluator policy gradient architecture and design a policy update scheme for secure reinforcement learning to introduce reliability constraints, thereby solving the CMDP model.

[0080] Therefore, the embodiment of the present application takes maintenance facilities as the core of the problem and combines the specific structural characteristics of high-speed rail wheelsets to establish a model that takes into account both wheelset turning and repair and wheelset spare parts management; at the same time, a collaborative optimization problem of complex system maintenance and spare parts management is established, and the reliability constraints involved are fully considered, and the CMDP method is used to characterize the problem; in addition, the embodiment of the present application uses a novel constraint strategy optimization algorithm to solve the problem. During the solution process, corresponding iterative strategies are formulated for situations that meet the constraints and violate the constraints, thereby achieving the guarantee of reliability constraints, and providing new ideas for the formulation of railway wheelset health management and operation and maintenance measures.

[0081] Optionally, in one embodiment of the present application, an MDP quintuple model is constructed based on a dynamic inventory control strategy, including: calculating the wheel radial quantity and rim thickness vector of the t-th decision node, and determining the state space of the MDP quintuple model based on the inventory level, wheel radial quantity and rim thickness vector of the t-th decision node; determining the wheel maintenance behavior and spare parts ordering quantity behavior of the high-speed rail wheel at the t-th decision node, so as to establish the behavior space of the MDP quintuple model according to the wheel maintenance behavior and spare parts ordering quantity behavior; determining the transition model of the MDP quintuple model according to the state space and the behavior space; calculating the wheel repair cost, replacement cost, unit out-of-stock cost, inventory cost and ordering startup cost of the high-speed rail wheel, so as to determine the reward function of the MDP quintuple model through the wheel repair cost, replacement cost, unit out-of-stock cost, inventory cost and ordering startup cost; and constructing the MDP quintuple model according to the state space, behavior space, transition model and reward function.

[0082] In the specific implementation process, the MDP in the embodiment of the present application is defined by a quintuple (S, A, P, R, γ), where S is the state set (i.e., state space), A is the action set (i.e., action space), p: S×A×S→A is the transition probability function (i.e., transition model), R: S×A×S→R is the reward function, γ is the discount factor, π is used to balance the importance of immediate rewards and future rewards. The goal of this embodiment of the present application is to select a strategy that can maximize the discounted total return J(π) within an infinite time range.

[0083] The following is a detailed description of the state space, action space, transition model and reward function in the five-tuple of the embodiment of the present application.

[0084] 1. State space S: through s t ∈S means at the decision time The state observed by the condition monitor. In an infinite time range, the embodiment of the present application may consider the overall current state of the maintenance facility including multiple high-speed trains and inventory, which is represented by s t =(x t ,y t ,z t ), where x t The vector representing the wheel diameter, y t Vector representing the rim thickness, z t is the inventory level of spare parts at the tth decision moment, and the mathematical expression of the state space is:

[0085]

[0086] 2. Behavior space A: The manager of the maintenance facility observes the system at each decision moment and makes decisions regarding wheel maintenance and spare parts inventory control. Behavior a at decision moment lt ∈A, which can be expressed as a t =(w t ,q t ),in represents the wheel maintenance behavior, q t represents the spare parts order quantity behavior. The maintenance behavior of wheel i on carriage j of high-speed train i as follows:

[0087]

[0088] Among them, "0" means "no action", and "m" means "wheel turning and wheel diameter reduction". ”, “M+1” means “wheel change”.

[0089] The joint behavior space for maintenance and spare parts ordering is:

[0090]

[0091] 3. Transfer model: The state transfer between two decision moments in the embodiment of the present application can be divided into two steps: t →a t →s t+1 From the maintenance perspective, the post-decision state of the wheel due to the maintenance behavior starts from the original state at the decision time t, and then after a time interval of τ, due to the degradation process, the health state changes, when the behavior is to take no action When the wheel condition is based on the degradation process, it will only follow the normal degradation process. When the wheel is repaired, the wheel health status is restored according to the mode of considering the repair. In the case of and In terms of spare parts inventory, the inventory level changes. It should be noted that the number of replacement wheels n t Available through all wheel replacement behaviors:

[0092]

[0093] in, Is an indicator function to indicate whether the wheel is replaced

[0094] 4. Reward function R: The reward function of the maintenance facility collects immediate rewards after performing maintenance and spare parts actions. Immediate reward r t (s t ,a t ,s t+1 ) is calculated as the sum of maintenance cost and spare parts management cost:

[0095]

[0096] Among them, c p If the behavior The wheel repair cost at the time of [M] = 1, parameter c r For if behavior The replacement cost at that time, the parameters b, h and o represent the unit shortage cost, inventory cost and order start-up cost respectively; the embodiment of the present application has an unlimited discount expected total return And give a uniform cost c for various imperfect turning behaviors p , rather than different levels of repair costs. However, due to the interaction between wheel diameter and rim thickness, this does not lead to a random selection of repair behavior.

[0097] Optionally, in one embodiment of the present application, a CMDP model is established according to the MDP quintuple model and a preset safety constraint set, and the CMDP model is solved using a preset actuator-evaluator policy gradient architecture and a constraint policy optimization algorithm to obtain a collaborative optimization result of high-speed rail wheelset maintenance and spare parts management that meets the preset optimization requirements, including: determining a safety constraint set based on the state space, behavior space, and reward function to construct a CMDP model according to the MDP quintuple model and the safety constraint set; performing wheel diameter difference detection on adjacent high-speed rail wheels in the same carriage of a target high-speed rail train to obtain a wheel diameter difference result, and establishing additional safety constraints based on the wheel diameter difference result; based on the additional safety constraints, obtaining an expected discounted return, and determining the reliability constraint of the CMDP model based on the expected discounted return and a preset safety threshold of the target high-speed rail train; parameterizing the CMDP model through an actuator-evaluator policy gradient architecture, and calculating the collaborative optimization result under reliability constraints based on the parameterized CMDP model, and in combination with a preset trust region policy optimization algorithm, a constraint policy update rule, and a constraint policy optimization algorithm.

[0098] Those skilled in the art should understand that during the operation of a high-speed train, the wheel diameters of adjacent wheels should be sufficiently consistent to avoid unreliable problems. If the wheel diameters of wheels on the same carriage differ greatly, it will hinder the safety of the high-speed train and affect the comfort of passengers.

[0099] Therefore, an embodiment of the present application may consider a security constraint based on the MDP five-tuple model to construct a corresponding CMDP model, which is composed of a six-tuple (S, A, P, R, C, γ), where C = {C: S × A × S → R} is a security constraint set.

[0100] In addition, the embodiment of the present application may also define additional security constraints p t (st ,a t ,s t+1 )as follows:

[0101]

[0102] Among them, represents the difference in wheel diameter between the two wheels. If p t (s t ,a t ,s t+1 ) is smaller, it means that the safety level of high-speed train i is higher, and ∈ represents the safety threshold of high-speed train. Denotes the expected discounted return of strategy π, then under the condition of expected safety constraints, the sum of the expected maintenance and spare parts costs is minimized as follows:

[0103]

[0104] It should be noted that the above CMDP modeling method is widely applicable and can be used to deal with a large number of problems subject to safety constraints or reliability constraints in maintenance planning.

[0105] In traditional reinforcement learning, solving unconstrained MDP problems typically involves the use of approximate value iteration or policy iteration, as well as model-free methods such as Monte Carlo methods or temporal difference learning. However, when there are reliability constraints in the model, the algorithm needs to be adjusted to balance exploration and reliability requirements, because excessive exploration may lead to the emergence of unreliable behavior. In an embodiment of the present application, the constraints are explicitly modeled as a multi-device system, and the difference in compensation states of devices within a single subsystem must be controlled within a certain range. This trade-off between the objective function and the constraints increases the complexity of the problem, so the embodiments of the present application need to design a more complex algorithm to meet the requirements of both algorithm performance and reliability.

[0106] In order to achieve a balance between algorithm performance and reliability in complex models, the embodiments of the present application can adaptively use a deep reinforcement learning algorithm based on constraint policy optimization on maintenance optimization problems with reliability constraints. In the constraint policy optimization algorithm, the intelligent agent can learn the optimal strategy without violating the reliability constraints, thereby minimizing the expected composite maintenance strategy cost.

[0107] First, the embodiment of the present application can illustrate the policy optimization framework used to solve the CMDP model. In the embodiment of the present application, the policy can be updated according to the executor-evaluator policy gradient architecture and the policy gradient method can be used to iteratively obtain the optimal strategy. In this architecture, the executor is responsible for generating random strategies, and the evaluator updates the value function. In order to overcome the "curse of dimensionality" problem faced by the CMDP state space and behavior space, the executor and the evaluator are set as two parallel deep neural networks. In the executor network, the embodiment of the present application can parameterize the strategy with θ, and record the strategy as π θ , the corresponding strategy space is denoted as Π θ , similarly, the objective function is parameterized as J θ (π), the constraints are parameterized as

[0108] At the same time, in the evaluator network, the embodiment of the present application can use φ to parameterize the value function. In order to facilitate those skilled in the art to understand the algorithm derivation process, the embodiment of the present application uses π and π θ , J(π) and J θ (π), J C (π) and No distinction is made.

[0109] The application of the constrained policy optimization framework based on deep neural networks requires the input of the state data obtained during the CMDP simulation process in the input layer, which is represented by s t =(x t ,y t ,z t ), respectively representing the compensation state vectors, functional state variables, and inventory levels of maintenance facilities of all multi-device systems at time ι. According to the settings of the actuator-evaluator network, the policy network determines the probability of taking each action in a given state, while the value network evaluates the value of a given state.

[0110] In order to solve the problem of updating the model after adding constraints to the MDP, the embodiment of the present application first considers a common deep reinforcement learning algorithm with constraints, namely the trust region strategy optimization algorithm. In order to maximize each state s t The state value of a reinforcement learning solution is the policy gradient algorithm, which completes the iterative update of the parameter θ based on gradient information; however, the policy gradient algorithm has the problem of selecting the iterative step size. If the increment is inappropriate, the updated parameters may represent a worse strategy.

[0111] The trust region policy optimization algorithm can use KL divergence (Kullback–Leibler divergence) to calculate the distance between the current policy and the old policy at each update, and maintain the "trust region" of the policy based on KL divergence by using constraints at each update to ensure the stability and convergence of the training. Overall, the trust region policy optimization algorithm maximizes the cumulative expected reward function under the KL divergence distance constraint, and its specific update rules are as follows:

[0112]

[0113] Among them, π l is the strategy for the first iteration, D KL Represents the distance between the two strategies based on KL divergence, and the feasible domain of the trust region strategy optimization algorithm in the embodiment of the present application can be defined as Π θ .

[0114] Furthermore, the embodiment of the present application can consider the limitation of the difference in compensation state of devices in the same subsystem through the CMDP model. Under the CMDP model, the policy update rule is restricted to the parameterized policy space Π θ and the set of policies that satisfy the security constraints Π C ={π|J C (π)≤∈}. Therefore, the optimization problem needs to be solved in the intersection of the two c ∩Π θ That is to say, based on the trust region strategy optimization algorithm, that is, formula (1), maximizing the cumulative reward not only needs to consider the limitation of KL divergence, but also the limitation of actual reliability. Therefore, the update rules of the embodiment of the present application are as follows:

[0115]

[0116] It is understandable that due to the constraints in the above formula, it is difficult to obtain gradient updates through sampling. In the traditional processing of constrained MDP problems, the constraints can be added to the objective function as penalty terms using Lagrange multipliers. In order to achieve the goal of minimizing constraint violations and maximizing cumulative rewards, the embodiments of the present application need to determine a suitable Lagrange multiplier value to ensure the balance between the constraints and the reward function. Too small Lagrange multiplier values ​​may lead to constraint violations caused by inappropriate penalties. On the contrary, too large Lagrange multiplier values ​​may cause the optimization to be overly inclined to improve the constraints, reduce the degree of maintenance cost reduction, and the optimality of the solution is difficult to ensure.

[0117] In the actual implementation process, the adjustment of the Lagrange multiplier term requires a lot of work to complete, which brings great difficulties to the algorithm tuning. Inspired by the trust region policy optimization algorithm, the embodiment of the present application can adopt a similar idea to design a policy update solution for secure reinforcement learning to solve the above problems.

[0118] First, the embodiment of the present application can determine the equivalent form of the problem under the trust region strategy optimization, by defining V π (s) is the value function of taking strategy π in state s, given V π (s), the present application further defines A π (s,a) is the advantage function of taking action a in state s, and its calculation expression is as follows:

[0119] A π (s,a)=r(s,a,s′)+γV π (s′)-V π (s) (3)

[0120] According to the trust region policy optimization algorithm, the equivalent form of the policy optimization problem with KL divergence constraints is as follows:

[0121]

[0122] Among them, And the step size is limited to d>0.

[0123] gather It is called a trust region. For a policy optimization problem with constraints, the embodiment of the present application can simultaneously apply the trust region policy optimization algorithm to the maintenance cost discount expectation objective function and the reliability discount expectation constraint function, so as to obtain the update rules of the constrained policy.

[0124] As a theoretical basis commonly used in reinforcement learning to ensure the increase of policy performance, the performance difference theorem has proved that when conditions are met, a single-step update will lead to an improvement in policy performance. This theorem takes into account the error caused by estimation and limits the range of the error.

[0125] The following embodiment of the present application gives the form of the performance difference theorem under KL divergence as Lemma 1. Lemma 1 lays the foundation for the derivation of performance in the convergence process of subsequent objective functions and reliability constraints by giving upper and lower bounds on the effect of policy updates.

[0126] Lemma 1 (Performance Difference Theorem under KL Divergence): For any function f:S→R and any policies π′ and π, we define:

[0127] δ f (s,a,s′)=r(s,a,s′)+γf(s′)-f(s),

[0128]

[0129] Among them, d π is the discounted future state distribution, define And the following boundaries hold:

[0130]

[0131] On this basis, the embodiment of the present application uses the lower bound and upper bound in Lemma 1 to give the changes of the discounted cost objective function and reliability constraints of the two strategies during the iteration process, as described below:

[0132] Corollary 1 (Objective function performance difference); For any two strategies π′ and π, given The following boundaries hold:

[0133]

[0134] Based on Corollary 1, when π′ is in the neighborhood of π, the term It can be used as an approximate estimate of J(π′)-J(π). Corresponding to the objective function, we define is the value function of the reliability constraint using strategy π in state s, given The present application further defines is the reliability advantage function of taking action a in state s, and its calculation expression is as follows:

[0135]

[0136] The following conclusions can also be obtained from the embodiments of the present application with respect to the reliability constraint conditions:

[0137] Corollary 2 (Reliability Function Performance Difference): For any two strategies π′, π and reliability function C, given The following boundaries hold:

[0138]

[0139] By using Corollary 2, the embodiment of the present application can obtain the relationship between the reliability functions of the previous and next iterations, and use the lower bound to form an estimate of the result of the next iteration. In addition, the embodiment of the present application can let π l and π l+1 Substitute π and π′ in Equation (5) of Corollary 1 and Equation (7) of Corollary 2, respectively, to obtain the following constrained strategy update rule:

[0140]

[0141] Compared with the trust region policy optimization algorithm, the constraint policy optimization introduces the limitation of reliability constraints on its basis; in addition, in addition to considering the lower bound of policy improvement as in the trust region policy optimization, the constraint policy optimization also needs to consider the upper bound of violation of reliability constraints. Based on this, the update rule in the above formula maximizes the expected benefit under the condition of satisfying the two sets of constraints of reliability and KL divergence, that is, minimizing the total cost of the expected discounted composite maintenance strategy. The first set of reliability constraints ensures that the strategy satisfies the J C (π l ) defines the reliability requirements for multi-device systems, and the other set ensures that the deviation between the new strategy and the current strategy during the iteration process does not exceed the original strategy, thus ensuring the stability of the algorithm.

[0142] Since the algorithm is implemented based on an actuator-evaluator architecture composed of a deep neural network, the parameterization strategy is implemented through a highly nonlinear function composed of a large number of hyperparameters of the neural network. l ,θ l+1 Represent the policy network parameters at the lth and l+1th iterations, respectively. In the face of the difficulty in solving large-scale parameterized strategies faced by the optimization problem of composite maintenance strategies for multiple multi-equipment systems, it is not realistic to directly solve equation (1). Therefore, the embodiment of the present application can propose a corresponding approximate estimation equivalent form that is easier to solve for this problem through Theorem 1, so as to improve the solution efficiency and reduce the solution cost.

[0143] Theorem 1 (approximate equivalent form): By Perform a first-order Taylor series expansion and KL divergence By performing a second-order Taylor series expansion, equation (1) can be approximated as a second-order Taylor series expansion, which can be approximately expressed as follows:

[0144]

[0145] From the above formula, we can see that according to the properties of KL divergence, its Hessian matrix is ​​semi-positive definite. Therefore, compared with formula (1), the approximate strategy update method proposed in Theorem 1 is essentially a quadratic convex optimization problem, the global optimality is guaranteed, and the difficulty of solving is greatly simplified. For the convenience of expression, the following definition can be given in the embodiment of the present application:

[0146]

[0147]

[0148]

[0149] As a result, the optimization problem (9) can be expressed in the following form:

[0150]

[0151] Since the strong duality holds true under the original problem, through its Lagrangian function, the embodiment of the present application can express its equivalent dual problem in the form of:

[0152] Proposition 1 (Dual problem of approximate estimation update rule): The dual problem of formula (10) is:

[0153]

[0154] And the optimal solution to the original problem is:

[0155]

[0156] Where (λ * ,v * ) is the optimal solution to the dual problem.

[0157] Equation (11) in Proposition 1 gives a feasible update rule under reliability constraints. However, due to the potential inaccuracy of the approximation, the update rule (11) may not satisfy the actual reliability constraints. In order to solve this problem, the embodiments of the present application may implement the following update rules in practice:

[0158]

[0159] The step size a is dynamically adjusted to meet the constraints through the backtracking line search method.

[0160] In addition, the embodiments of the present application also need to consider the situation where the update rules are not feasible due to sampling errors or suboptimal updates. To improve this problem, the embodiments of the present application adopt the following strategy to restore the update rules so that the strategy can quickly recover to the feasible domain composed of reliability constraints and KL divergence constraints:

[0161]

[0162] Constrained strategy optimization algorithms such as Figure 3 As shown, the algorithm first initializes the actuator network parameters θ 0 , evaluator network parameters φ 0 , the maximum number of iterations K, the number of stored trajectories D, the trajectory length T, and the KL divergence threshold d; secondly, the algorithm enters the main loop and starts to train the two networks. In each iteration, the algorithm uses the current actuator network parameters θ l , according to the description from line 2 to line 11, the generated trajectories are stored in set D, as Figure 4As shown, as the training data for this iteration; afterwards, using these collected training data, the embodiment of the present application can obtain the latest round of actuator network parameters θ according to the description of solving the optimization problem in lines 12 to 18. l+1 , if the optimization problem (10) is feasible, then the embodiment of the present application can directly solve (11) to obtain θ * , otherwise, the parameters are shrunk back to the feasible domain according to the update rule described in equation (13), and the parameters are adjusted using the backtracking line search method to complete θ l+1 Finally, the evaluator network parameters are updated by mean square error regression. When the algorithm ends, the embodiment of the present application will obtain the final trained actuator network parameters and evaluator network parameter values, thereby obtaining the optimal solution for composite maintenance strategy optimization under reliability constraints.

[0163] In summary, the embodiment of the present application provides an imperfect daily maintenance model of a complex system based on MDP modeling, and adds reliability constraints on the basis of MDP to model it as a CMDP model; in addition, the daily maintenance of complex systems is often also affected by spare parts management. At the same time, the consumption rate of spare parts is also closely related to the failure of the equipment. The definition of the collaborative optimization problem of complex system maintenance and spare parts management involves how to efficiently arrange maintenance activities and manage spare parts inventory to ensure high reliability and low operating costs of the system. The core of this problem lies in how to balance the timeliness of maintenance resources and the adequacy of spare parts supply, while taking into account the optimization of cost-effectiveness and system reliability. By collaboratively optimizing maintenance plans and spare parts management, the operating efficiency of the system can be significantly improved, the downtime caused by failures or out-of-stocks can be reduced, and the system can be ensured to operate continuously and stably; in addition, reasonable collaborative optimization strategies can help reduce inventory costs, avoid waste of resources, and improve the ability to cope with sudden failures.

[0164] The following is an explanation and introduction of the execution logic of the collaborative optimization method for high-speed rail wheelset maintenance and spare parts management of the present application through a specific embodiment and in combination with the accompanying drawings.

[0165] In a specific embodiment of the present application, the above collaborative optimization problem and CPO algorithm can be experimented with in combination with data from a CRH380AL high-speed train. The experiments are all run on a computer equipped with an AMD Ryzen 5 56000X 6-core processor and an NVIDIA GeForce RTX 3070 graphics card. All algorithm codes are implemented using Pytorch in a Python 3.6 environment.

[0166] Specifically, the system consists of three high-speed trains, each with eight carriages and eight wheels in each carriage. The initial wheel diameter and rim thickness of the new wheels are 792 mm and 34 mm, respectively. During normal operation, the wheel diameter degradation follows a truncated normal distribution. The rim thickness degradation also follows a normal distribution The unit time is 30 days; turning involves machining the wheel diameter in increments of δ = 0.01, and the cost per operation is 0.4×10 3 $; wheel replacement requires a visit to the repair station, 60 wheels are in stock; initial replacement order cost is $18×10 3 USD, out of stock cost per wheel 1×10 3 USD, and the storage cost is 0.05×10 per unit 3 Dollar.

[0167] 1. Algorithm iteration changes:

[0168] Figure 5 Iteration curve diagram of different costs. Figure 5 As shown, it shows the curve of the main cost reduction part during the iteration process of the training process. The embodiment of the present application summarizes the overall cumulative cost and the corresponding two parts: maintenance cost and spare parts cost. The results show that the total cost gradually increases with iteration.

[0169] It should be noted that at the beginning of the training process, the reduction in spare parts costs dominated the reduction in total costs, mainly because spare parts costs were much higher than maintenance costs. In addition, the dimension of ordering behavior was lower than that of maintenance behavior, and learning was easier. After the spare parts cost was reduced to a level comparable to the maintenance cost, the maintenance cost began to play a leading role in the total cost and began to decrease rapidly. In this experiment, the total cost under the optimal strategy during the entire planning period was -89.7×10 3 USD, the maintenance cost is -78.4×10 3 USD, spare parts cost is 11.3×10 3 Dollar.

[0170] In addition, when other parameters remain unchanged, the embodiment of the present application also counts the changes in total cost, maintenance cost and spare parts cost under different numbers of vehicles. Through experiments, it can be seen that when the number of vehicles changes, the total cost, maintenance cost and spare parts cost will all change; however, when the inventory of maintenance facilities is constant, the number of vehicles is increased, that is, the consumption of the overall inventory is increased, so the change in spare parts cost is much greater than the change in maintenance cost; in terms of low standard deviation, under different numbers of vehicles, the standard deviation of each data is at a low level, indicating that the strategy in the embodiment of the present application is relatively stable under parameter changes, reflecting the robustness of the algorithm.

[0171] Table 1

[0172]

[0173] 2. Sensitivity analysis

[0174] The embodiment of the present application can also analyze the impact of the model and algorithm on the optimal solution objective function, namely the total composite maintenance cost, maintenance cost and spare parts cost, by replacing different parameter values.

[0175] First, the embodiment of the present application can analyze the interactive relationship between the maintenance cost and the spare parts cost by changing the size of the mean value of the compensation state degradation distribution and the size of the inventory capacity. x The values ​​of various costs are varied from 0.06 mm to 0.30 mm in intervals of 0.06 mm, and the spare parts inventory capacity is varied from 20 to 50 in intervals of 10. Figure 6 As shown by Figure 6 It can be seen that with the increase of the mean value of the compensation state degradation distribution, the maintenance cost continues to increase; in addition, with the continuous increase of capacity, the total cost gradually decreases. When the capacity increases to a certain value, the total cost no longer decreases. The reason is that with the increase of capacity, the risk of shortage of spare parts gradually decreases, so the cost of spare parts continues to decrease.

[0176] Secondly, the embodiment of the present application can also analyze the interactive relationship between the maintenance cost and the spare parts cost by changing the unit maintenance cost and the replacement cost. p From 0.2×10 3 USD to 1.0×10 3 The U.S. dollar is 0.2×10 3 The interval of US dollars keeps changing, making the spare parts inventory capacity change from 20 to 50 in intervals of 10. The values ​​of total cost, maintenance cost and spare parts cost under different parameters are as follows Figure 7 As shown by Figure 7 It can be seen that with the increase of unit imperfect maintenance cost, both maintenance cost and spare parts cost show an increasing trend. With the increase of unit imperfect maintenance cost, the increase of total maintenance cost is a very natural phenomenon. At the same time, the increase of maintenance cost will make components more inclined to choose replacement rather than imperfect maintenance behavior, and replacement will bring more spare parts consumption. Therefore, with the increase of maintenance cost, the total spare parts cost also increases accordingly. In addition, with the increase of unit replacement cost, the spare parts cost has not shown obvious changes, but the maintenance cost has increased significantly. Components are more inclined to choose imperfect maintenance behavior to improve the status of components rather than replacement behavior, so spare parts consumption is reduced and spare parts cost is reduced.

[0177] It can be understood that the embodiment of the present application first takes the EMU depot as the core, and combines the specific structural characteristics of the high-speed rail wheelset to establish a model that considers both wheelset turning and repair and wheelset spare parts management; secondly, the collaborative optimization problem of high-speed rail wheelset maintenance and spare parts management is modeled and analyzed, and the reliability constraints involved in the problem are fully considered. The overall problem is modeled as a CMDP model; finally, a novel constraint strategy optimization algorithm is used to solve the problem. During the solution process, corresponding iterative strategies are formulated for the situations of compliance with constraints and violation of constraints, respectively, to achieve the guarantee of reliability constraints, so as to generate a dynamic maintenance planning plan for high-speed rail wheelsets, ensure the continuous stable operation and efficient operation and maintenance of high-speed rail, thereby ensuring the safe and reliable operation of high-speed rail trains under cost minimization, and the algorithm is applied to actual railway cases to further verify the feasibility and effectiveness of the method.

[0178] According to the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management proposed in the embodiment of the present application, a degradation and turning behavior model of the high-speed railway wheelset is constructed, and the re-turning gain of the (t+1)th decision node is determined according to the degradation and turning behavior model, so as to obtain the mathematical expression of the wheel diameter and rim thickness of the wheel after the (t+1)th decision node is turned through the re-turning gain, wherein t is a positive integer; based on the degradation and turning behavior model, the mathematical expression of the wheel diameter and rim thickness, a replacement and spare parts management behavior model of the high-speed railway wheelset is constructed, so as to determine the dynamic inventory control strategy according to the replacement and spare parts management behavior model; based on the dynamic inventory control strategy, an MDP five-tuple model is constructed, and a CMDP model is established according to the MDP five-tuple model and a preset safety constraint set, and the preset actuator-evaluator policy gradient architecture and constraint strategy optimization algorithm are used to solve the CMDP model, so as to obtain the collaborative optimization result of high-speed railway wheelset maintenance and spare parts management that meets the preset optimization requirements. This application can significantly improve the operating efficiency of the system, reduce downtime caused by failures or out-of-stocks, and ensure that the system can continue to operate stably; in addition, this application helps to reduce inventory costs and avoid waste of resources through reasonable collaborative optimization strategies, while improving the ability to deal with sudden failures.

[0179] Secondly, the collaborative optimization device for high-speed railway wheelset maintenance and spare parts management proposed in accordance with the embodiment of the present application is described with reference to the accompanying drawings.

[0180] Figure 8 It is a block diagram of a collaborative optimization device for high-speed railway wheelset maintenance and spare parts management according to an embodiment of the present application.

[0181] like Figure 8 As shown, the collaborative optimization device 10 for high-speed railway wheelset maintenance and spare parts management includes: a first building module 100, a second building module 200 and a solution module 300.

[0182] Among them, the first construction module 100 is used to construct a degradation and turning behavior model of the high-speed rail wheelset, and determine the re-turning gain of the (t+1)th decision node according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node through the re-turning gain, wherein t is a positive integer.

[0183] The second construction module 200 is used to construct a replacement and spare parts management behavior model for high-speed rail wheelsets based on the degradation and turning behavior model, the mathematical expression of the wheel diameter and the rim thickness, so as to determine a dynamic inventory control strategy according to the replacement and spare parts management behavior model.

[0184] The solving module 300 is used to construct an MDP five-tuple model based on a dynamic inventory control strategy, and to establish a CMDP model according to the MDP five-tuple model and a preset safety constraint set, and to solve the CMDP model using a preset actuator-evaluator policy gradient architecture and a constraint strategy optimization algorithm to obtain a collaborative optimization result of high-speed rail wheelset maintenance and spare parts management that meets preset optimization requirements.

[0185] Optionally, in one embodiment of the present application, the first building module 100 includes: a first determining unit, a first acquiring unit, a first judging unit, a first calculating unit and a second calculating unit.

[0186] Among them, the first determination unit is used to determine the lower limit of the wheel diameter and the lower limit of the wheel rim thickness of the high-speed railway wheel.

[0187] The first acquisition unit is used to obtain the actual wheel diameter and actual wheel rim thickness of the high-speed railway wheel at the tth decision node.

[0188] The first judgment unit is used to calculate the wheel diameter and rim thickness of the (t+1)th decision node based on the actual wheel diameter and the actual rim thickness, and judge whether the high-speed rail wheel meets the preset fault risk conditions according to the lower limit of the wheel diameter, the lower limit of the rim thickness, the wheel diameter and the rim thickness of the (t+1)th decision node.

[0189] The first calculation unit is used to calculate the re-turning gain of the (t+1)th decision node through the degradation and turning behavior model if the high-speed railway wheel meets the preset fault risk condition.

[0190] The second calculation unit is used to obtain the mathematical expression of the wheel diameter and the wheel rim thickness after the (t+1)th decision node turning according to the re-turning gain and the preset nonlinear regression function.

[0191] Optionally, in one embodiment of the present application, the second building module 200 includes: a second determining unit, a third calculating unit and a second judging unit.

[0192] Among them, the second determination unit is used to determine the inventory level of high-speed rail wheels and the number of replacement wheels at the tth decision node.

[0193] The third calculation unit is used to calculate the spare parts inventory level of the (t+1)th decision node based on the replacement and spare parts management behavior model, the preset piecewise linear function, the inventory level of the tth decision node and the number of replaced wheels.

[0194] The second judgment unit is used to judge whether the high-speed rail wheel meets the preset out-of-stock condition according to the spare parts inventory level of the (t+1)th decision node, and when the high-speed rail wheel meets the preset out-of-stock condition, determine the quantity of spare parts ordered at the tth decision node, so as to update the spare parts inventory level of the (t+1)th decision node with the ordered quantity of spare parts.

[0195] Optionally, in one embodiment of the present application, the solution module 300 includes: a fourth calculation unit, a building unit, a third determination unit, a fifth calculation unit and a first modeling unit.

[0196] Among them, the fourth calculation unit is used to calculate the wheel radial quantity and rim thickness vector of the t-th decision node, and determine the state space of the MDP quintuple model based on the inventory level, wheel radial quantity and rim thickness vector of the t-th decision node.

[0197] A unit is established to determine the wheel maintenance behavior and spare parts ordering quantity behavior of the high-speed rail wheel at the t-th decision node, so as to establish the behavior space of the MDP quintuple model according to the wheel maintenance behavior and spare parts ordering quantity behavior.

[0198] The third determining unit is used to determine the transfer model of the MDP quintuple model according to the state space and the behavior space.

[0199] The fifth calculation unit is used to calculate the wheel repair cost, replacement cost, unit stock-out cost, inventory cost and order startup cost of the high-speed rail wheels, so as to determine the reward function of the MDP quintuple model through the wheel repair cost, replacement cost, unit stock-out cost, inventory cost and order startup cost.

[0200] The first modeling unit is used to construct an MDP five-tuple model according to the state space, the behavior space, the transition model and the reward function.

[0201] Optionally, in one embodiment of the present application, the solution module 300 further includes: a second modeling unit, a detection unit, a second acquisition unit and a parameterization unit.

[0202] The second modeling unit is used to determine the safety constraint set based on the state space, the behavior space and the reward function, so as to construct the CMDP model according to the MDP quintuple model and the safety constraint set.

[0203] The detection unit is used to perform wheel diameter difference detection on adjacent high-speed rail wheels in the same carriage of the target high-speed rail train to obtain wheel diameter difference results and establish additional safety constraints based on the wheel diameter difference results.

[0204] The second acquisition unit is used to acquire the expected discounted return based on the additional safety constraint, and determine the reliability constraint of the CMDP model according to the expected discounted return and the preset safety threshold of the target high-speed rail train.

[0205] The parameterization unit is used to parameterize the CMDP model through the actuator-evaluator policy gradient architecture, and based on the parameterized CMDP model, and in combination with the preset trust region policy optimization algorithm, the constrained policy update rule and the constrained policy optimization algorithm, calculate the collaborative optimization result under reliability constraints.

[0206] It should be noted that the aforementioned explanation of the embodiment of the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management is also applicable to the collaborative optimization device for high-speed railway wheelset maintenance and spare parts management of this embodiment, and will not be repeated here.

[0207] The collaborative optimization device for high-speed rail wheel set maintenance and spare parts management proposed in the embodiment of the present application includes a first construction module for constructing a degradation and turning behavior model of the high-speed rail wheel set, and determining the re-turning gain of the (t+1)th decision node according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and rim thickness after turning at the (t+1)th decision node through the re-turning gain, wherein t is a positive integer; a second construction module for constructing a mathematical expression of the wheel diameter and rim thickness based on the degradation and turning behavior model, the wheel diameter and rim thickness Expression, construct the replacement and spare parts management behavior model of high-speed railway wheelsets, so as to determine the dynamic inventory control strategy according to the replacement and spare parts management behavior model; the solution module is used to construct the MDP five-tuple model based on the dynamic inventory control strategy, and establish the CMDP model according to the MDP five-tuple model and the preset safety constraint set, and solve the CMDP model using the preset actuator-evaluator policy gradient architecture and constraint strategy optimization algorithm to obtain the collaborative optimization results of high-speed railway wheelset maintenance and spare parts management that meet the preset optimization requirements. This application can significantly improve the operating efficiency of the system, reduce the downtime caused by failures or out-of-stock, and ensure that the system can continue to operate stably; in addition, this application helps to reduce inventory costs and avoid waste of resources through reasonable collaborative optimization strategies, while improving the ability to deal with sudden failures.

[0208] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0209] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0210] When the processor 902 executes the program, the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management provided in the above embodiment is implemented.

[0211] Furthermore, the electronic device further comprises:

[0212] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0213] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0214] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0215] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0216] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0217] The processor 902 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0218] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management as described above is implemented.

[0219] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned collaborative optimization method for high-speed railway wheelset maintenance and spare parts management.

[0220] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0221] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0222] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0223] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0224] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0225] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0226] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0227] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A collaborative optimization method for high-speed railway wheelset maintenance and spare parts management, characterized in that: The following steps are involved: A degradation and turning behavior model of a high-speed railway wheelset is constructed, and a re-turning gain of the (t+1)th decision node is determined according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and wheel rim thickness after turning at the (t+1)th decision node through the re-turning gain, wherein t is a positive integer; Based on the degradation and turning behavior model and the mathematical expressions of the wheel diameter and wheel rim thickness, a replacement and spare parts management behavior model of the high-speed railway wheelset is constructed to determine a dynamic inventory control strategy according to the replacement and spare parts management behavior model; Based on the dynamic inventory control strategy, an MDP quintuple model is constructed, and a CMDP model is established according to the MDP quintuple model and a preset safety constraint set. The CMDP model is solved using a preset actuator-evaluator policy gradient architecture and a constraint strategy optimization algorithm to obtain a collaborative optimization result of high-speed railway wheelset maintenance and spare parts management that meets the preset optimization requirements.

2. The method according to claim 1, characterized in that The re-turning gain of the (t+1)th decision node is determined according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and rim thickness of the wheel after turning at the (t+1)th decision node through the re-turning gain, including: Determining the lower limit of the wheel diameter and the lower limit of the wheel rim thickness of the high-speed rail wheel; Obtaining the actual wheel diameter and actual wheel rim thickness of the high-speed rail wheel at the t-th decision node; Based on the actual wheel diameter and the actual wheel rim thickness, the wheel diameter and wheel rim thickness of the (t+1)th decision node are calculated, and according to the lower limit of the wheel diameter, the lower limit of the wheel rim thickness, the wheel diameter and wheel rim thickness of the (t+1)th decision node, whether the high-speed rail wheel meets the preset fault risk condition is judged; If the high-speed railway wheel meets the preset fault risk condition, the re-turning gain of the (t+1)th decision node is calculated by using the degradation and turning behavior model; The mathematical expressions of the wheel diameter and wheel rim thickness after the wheel is re-repaired at the (t+1)th decision node are obtained according to the re-repairing gain and a preset nonlinear regression function.

3. The method according to claim 2, characterized in that Determining the dynamic inventory control strategy according to the replacement and spare parts management behavior model includes: Determine the inventory level of the high-speed rail wheels and the number of replacement wheels at the t-th decision node; Calculate the spare parts inventory level at the (t+1)th decision node based on the replacement and spare parts management behavior model, the preset piecewise linear function, the inventory level at the tth decision node and the number of wheels to be replaced; Whether the high-speed rail wheel meets the preset out-of-stock condition is determined according to the spare parts inventory level of the (t+1)th decision node, and when the high-speed rail wheel meets the preset out-of-stock condition, the quantity of spare parts ordered at the tth decision node is determined, so as to update the spare parts inventory level of the (t+1)th decision node using the ordered spare parts quantity.

4. The method according to claim 3, characterized in that The method of constructing an MDP five-tuple model based on the dynamic inventory control strategy includes: Calculating the wheel radial quantity and the wheel rim thickness vector of the t-th decision node, and determining the state space of the MDP quintuple model based on the inventory level of the t-th decision node, the wheel radial quantity and the wheel rim thickness vector; Determine the wheel maintenance behavior and spare parts order quantity behavior of the high-speed rail wheel at the t-th decision node, so as to establish the behavior space of the MDP quintuple model according to the wheel maintenance behavior and the spare parts order quantity behavior; Determine a transition model of the MDP quintuple model according to the state space and the behavior space; Calculating the wheel repair cost, replacement cost, unit stock-out cost, inventory cost and ordering start-up cost of the high-speed rail wheel, so as to determine the reward function of the MDP quintuple model through the wheel repair cost, the replacement cost, the unit stock-out cost, the inventory cost and the ordering start-up cost; The MDP five-tuple model is constructed according to the state space, the behavior space, the transition model and the reward function.

5. The method according to claim 4, characterized in that The CMDP model is established according to the MDP quintuple model and the preset safety constraint set, and the CMDP model is solved by using the preset actuator-evaluator policy gradient architecture and constraint strategy optimization algorithm to obtain the collaborative optimization result of high-speed railway wheelset maintenance and spare parts management that meets the preset optimization requirements, including: Based on the state space, the behavior space and the reward function, determine the safety constraint set to construct the CMDP model according to the MDP quintuple model and the safety constraint set; Perform wheel diameter difference detection on adjacent high-speed rail wheels of the same carriage of the target high-speed rail train to obtain a wheel diameter difference result, and establish additional safety constraints according to the wheel diameter difference result; Based on the additional safety constraint, an expected discounted return is obtained, and a reliability constraint of the CMDP model is determined according to the expected discounted return and a preset safety threshold of the target high-speed train; The CMDP model is parameterized through the actuator-evaluator policy gradient architecture, and based on the parameterized CMDP model, and in combination with a preset trust region policy optimization algorithm, a constrained policy update rule and a constrained policy optimization algorithm, a collaborative optimization result under the reliability constraint is calculated.

6. A collaborative optimization device for high-speed railway wheelset maintenance and spare parts management, characterized in that: include: The first construction module is used to construct a degradation and turning behavior model of a high-speed railway wheelset, and determine the re-turning gain of the (t+1)th decision node according to the degradation and turning behavior model, so as to obtain a mathematical expression of the wheel diameter and wheel rim thickness after turning at the (t+1)th decision node through the re-turning gain, wherein t is a positive integer; The second construction module is used to construct a replacement and spare parts management behavior model of the high-speed railway wheelset based on the degradation and turning behavior model and the mathematical expression of the wheel diameter and rim thickness, so as to determine a dynamic inventory control strategy according to the replacement and spare parts management behavior model; A solving module is used to construct an MDP quintuple model based on the dynamic inventory control strategy, and to establish a CMDP model according to the MDP quintuple model and a preset safety constraint set, and to solve the CMDP model using a preset actuator-evaluator policy gradient architecture and a constraint strategy optimization algorithm to obtain a collaborative optimization result of high-speed rail wheelset maintenance and spare parts management that meets preset optimization requirements.

7. The device according to claim 6, characterized in that The first building block comprises: A first determining unit is used to determine the lower limit of the wheel diameter and the lower limit of the wheel rim thickness of the high-speed railway wheel; The first acquisition unit is used to obtain the actual wheel diameter and actual wheel rim thickness of the high-speed railway wheel at the t-th decision node; a first judgment unit, configured to calculate the wheel diameter and rim thickness of the (t+1)th decision node based on the actual wheel diameter and the actual rim thickness, and judge whether the high-speed rail wheel meets a preset fault risk condition according to the wheel diameter lower limit, the rim thickness lower limit, the wheel diameter and rim thickness of the (t+1)th decision node; A first calculation unit is used to calculate the re-turning gain of the (t+1)th decision node through the degradation and turning behavior model if the high-speed railway wheel meets the preset fault risk condition; The second calculation unit is used to obtain the mathematical expression of the wheel diameter and rim thickness of the wheel after the (t+1)th decision node turning according to the re-turning gain and a preset nonlinear regression function.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the collaborative optimization method for high-speed railway wheelset maintenance and spare parts management as described in any one of claims 1-5.

Citation Information

Cited By

  • Power distribution network maintenance plan setting method and system based on multi-target reinforcement learning

    CN119831565A