Aircraft carrier ship surface unmanned equipment intelligent scheduling method considering security equipment degradation

Abstract

The invention discloses an aircraft carrier ship surface unmanned equipment intelligent scheduling method considering security equipment degradation. The method comprises the steps of collecting related information of ship-based aircraft deck operation unmanned scheduling, wherein the related information comprises deck layout, position coordinates of working equipment matched with a ship-based aircraft, the number of ship surface unmanned equipment and the operation process of the ship-based aircraft; assumed conditions for guaranteeing degradation of the equipment and constraint conditions of the unmanned ship surface equipment are set, and a mathematical model of the unmanned ship surface equipment is established; based on a Markov decision process model, under a Markov decision framework, an epsilon-greedy exploration strategy in Q learning is improved, and exploration efficiency and strategy adaptability are enhanced; and selecting the state and the action by adopting the finally iterated Q value table to obtain an optimal scheduling sequence of unmanned scheduling of the shipboard aircraft deck operation considering the degradation of the support equipment.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aviation support systems and operation scheduling, and in particular to an intelligent scheduling method for unmanned equipment on a deck of an aircraft carrier taking into account the degradation of support equipment. Background Art

[0002] Aircraft carriers play a vital role in modern naval warfare. Their powerful unmanned equipment groups and perfect combat systems enable them to be quickly deployed and perform various combat missions far away from land, which not only improves the navy's long-range combat capability, but also provides solid support for the struggle for sea control. With the continuous development of intelligent technology, future aircraft carriers will further integrate intelligent unmanned systems to cope with complex and changing battlefield environments. Since the sustained combat capability of aircraft carriers depends on the frequency of dispatching unmanned equipment on the deck, and the main factor affecting the frequency of dispatching is the efficiency of the dispatching of unmanned equipment on the deck, the intelligent unmanned dispatching of unmanned equipment on the deck has become an important direction for the future development of the navy, that is, aircraft carriers can dispatch unmanned equipment on the deck without direct operation by personnel, thereby reducing personnel risks, enhancing operational flexibility, and improving operational efficiency.

[0003] The scheduling of deck operations of unmanned equipment on the ship is a complex optimization problem. It is necessary to allocate support resources to the unmanned equipment on the ship safely and efficiently according to the deployment plan and support requirements of the unmanned equipment on the ship, minimize the time cost of support operations, and increase the dispatch rate of unmanned equipment on the ship. In recent years, domestic and foreign scholars have conducted a lot of research on this problem. Commonly used methods include mathematical programming methods, queuing theory methods, modeling and simulation methods, heuristic algorithms, reinforcement learning methods, etc. As a typical reinforcement learning method, the Q learning algorithm does not need to know the transition probability or reward function of the environment in advance. It only needs to interact with the environment and learn how to make the best actions to achieve the maximum cumulative reward, thereby obtaining the optimal strategy. However, the balance between exploration and utilization and the convergence speed are still problems that the algorithm needs to solve.

[0004] For the intelligent unmanned scheduling of deck operations of unmanned equipment on the ship, the above method does not take into account the degradation of the support equipment due to continuous operation during the actual scheduling process, which may cause the support equipment to fail during the mission. In order to extend the service life and reduce the degree of degradation, when the degradation state of the support equipment reaches a certain threshold, the maintenance or replacement of the support equipment can be arranged. However, frequent maintenance may lead to an increase in the total time cost. Therefore, reasonably balancing the relationship between the scheduling time of unmanned equipment on the ship and the degree of degradation of the support equipment can minimize the failure rate of the support equipment, thereby improving the efficiency of the scheduling task. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention discloses an intelligent scheduling method for unmanned equipment on the deck of an aircraft carrier taking into account the degradation of support equipment, which specifically includes the following steps:

[0006] Collect relevant information about the unmanned dispatch of aircraft deck operations, including deck layout, location coordinates of equipment working with aircraft, number of unmanned equipment on the deck, and operation flow of aircraft;

[0007] Set the assumptions of degradation of support equipment and constraints of unmanned equipment on the ship deck and establish a mathematical model of unmanned equipment on the ship deck;

[0008] Reconstructing the mathematical model of the ship deck unmanned equipment into a Markov decision process model;

[0009] Based on the Markov decision process model and in the Markov decision framework, the ε-greedy exploration strategy in Q-learning is improved to enhance the exploration efficiency and strategy adaptability.

[0010] The Q-value table of the final iteration is used to select states and actions, and the optimal scheduling sequence for unmanned scheduling of aircraft deck operations considering the degradation of support equipment is obtained.

[0011] Furthermore, the assumptions for the degradation of the support equipment are as follows: the total amount of various consumable resources stored on the deck is sufficient; any support task cannot be interrupted after it begins; the support equipment is a one-stop support location and its initial state is brand new; it enters a preventive or maintenance state when performing support operations and does not affect the current support operations. The degradation state of the support equipment is included in the constraint range, and the corresponding maintenance activities are considered to ensure the normal operation of the support equipment.

[0012] Furthermore, the degradation phenomenon of the operating wear and tear and sea breeze erosion experienced by the support equipment over time is described, which has the characteristics of monotonicity and non-negative increments. The support equipment is divided into three states according to the degradation state value of the support equipment:

[0013] When the status value falls within the normal range, D pt ∈[0,D P ) When the unmanned equipment on the ship is in normal condition, the support time is normal;

[0014] When the state value falls within the prevention threshold range, D pt ∈[D P ,D M ), the support time of unmanned equipment on the ship becomes longer, which is a preventive state;

[0015] When the status value exceeds the fault threshold, D pt ∈[D M ,+∞), the unmanned equipment on the ship cannot perform support operations and is in a fault state;

[0016] To prevent the support equipment from entering a fault state, determine whether to perform maintenance and the type of maintenance based on the current degradation state of the support equipment. When the support equipment is normal, no maintenance is required and the next support operation can be carried out directly. When the support equipment is in a preventive state, preventive maintenance is carried out and the degradation state is randomly rolled back. When the support equipment is in a fault state, it needs to be replaced after the fault and the degradation state is reset to zero.

[0017] Furthermore, the objective function of the optimal scheduling sequence for unmanned scheduling of aircraft carrier deck operations is:

[0018] F=minC max (1)

[0019] Among them, F means that the objective function is to minimize the maximum completion time, C max Indicates the maximum completion time of all support operations for a wave of unmanned equipment on the ship;

[0020] The constraints of the unmanned equipment on the ship are:

[0021]

[0022]

[0023]

[0024]

[0025] y ijj' +y ij'j =1 (6)

[0026]

[0027]

[0028]

[0029] 0<D P <D M , t P <t M (10)

[0030] where e ij represents the completion time of the ship's unmanned equipment support, (i, j) represents the support task j of the ship's unmanned equipment i, J = {(i, j)|i∈I,j∈J i} represents the set of shipboard unmanned equipment cluster support tasks, I = {1, 2, ..., n} represents the set of shipboard unmanned equipment, J i ={1,2,…,J i} represents the support task set of the ship's unmanned equipment i;

[0031] Formula (3) indicates that the end time of the guarantee task includes its duration and maintenance time, and the maintenance needs to wait until the current guarantee task is completed, where s ij d ij Respectively represent the start time and duration of (i, j). If the preventive maintenance of equipment p is completed after (i, j), then z Pijp =1, otherwise z Pijp = 0, if the guarantee equipment p completes (i, j) and then performs post-failure maintenance, then z Mijp =1, otherwise z Mijp =0,t P Indicates the duration of preventive maintenance of equipment, t M represents the maintenance duration after the failure of the guarantee equipment, P = {1, 2, ..., 18} represents the guarantee equipment set;

[0032] Formula (4) indicates that all the preceding guarantee tasks must be completed before the guarantee task starts. ij represents the set of immediate preceding guarantee tasks of (i, j);

[0033] Formulas (5)-(6) indicate that the order of the guarantee tasks in the exchangeable guarantee task set can be exchanged, but not simultaneously, where Y ij represents the exchangeable support task set of (i, j). If the unmanned equipment i on the ship is providing support at the support equipment p, then x ijp =1, otherwise x ijp = 0, if (i,j) is before (i,j') then y ijj' =1, otherwise y ijj' =0;

[0034] Formula (7) indicates that the same equipment can only support one unmanned equipment on the ship at the same time, Q = {1, 2, ..., m} represents the equipment set;

[0035] Formula (8) indicates that at the same time, each support task of each ship-surface unmanned equipment can only be supported by one device;

[0036] Formula (9) indicates that the number of devices used at the same time must not exceed the total number of devices;

[0037] Formula (10) represents the relationship between preventive maintenance and post-failure maintenance, D P Denotes the preventive maintenance threshold of the equipment, D M Indicates the maintenance threshold after a guaranteed device failure.

[0038] Furthermore, based on the Markov decision process model, the ε-greedy exploration strategy in Q learning is improved under the Markov decision framework:

[0039] Assume that there is only one intelligent agent in the intelligent scheduling method of unmanned equipment on the aircraft carrier deck, and select the scheduling center as the intelligent agent. The global status of the deck includes the equipment number, equipment usage status, ship deck unmanned equipment number, ship deck unmanned equipment usage status, the current or pending support tasks of the ship deck unmanned equipment, and the execution time of the support tasks;

[0040] The action directly performed by the ship's surface unmanned equipment is to move a certain ship's surface unmanned equipment to a certain position, which specifically includes the ship's surface unmanned equipment number and equipment number;

[0041] The feedback obtained by the interaction between the ship's unmanned equipment and the deck status is the reward function, and the system availability is defined as

[0042] R pt =1-D pt (11)

[0043] The reward function takes formula (1) as the target, considers the moving distance reward and the guarantee equipment availability reward, and designs the reward function as follows:

[0044] r=λR l +c(1-λ)R u (12)

[0045] Among them, R r is the moving distance reward, R u To ensure the available rewards for the device, λ is the weight of the two, and considering the different units and data levels of the two, parameter c is introduced to adjust the order of magnitude;

[0046] Considering the impact of learning times and equipment degradation on availability, the ε-greedy exploration strategy is improved. For learning times, exploration should be the main method when the number of learning times is small. As the number of learning times increases, more accumulated experience should be used. For equipment availability, the equipment with the largest value in the Q value table is selected. When the availability of the equipment is low, the occurrence of equipment failure is controlled. The next action should be randomly selected with a larger probability of ε. Otherwise, the action with the largest value in the Q value table should be selected with a larger probability of 1-ε. Therefore, the exploration strategy is designed as

[0047] ε=ae -(bM+cRpt) (13)

[0048] Among them, a, b, c are appropriate parameters, and M is the number of training times.

[0049] Further, when improving the -greedy exploration strategy in Q learning, the specific steps include:

[0050] S321: Initialize the learning factor α, discount factor γ and Q value table;

[0051] S322: Determine the current state s;

[0052] S323: Select action a according to the Q value table, and determine the exploration rate ε using the improved ε-greedy exploration strategy;

[0053] S324: Execute action a and determine the next state s' and reward r;

[0054] S325: Update the Q value table according to the following formula;

[0055] Q(s,a)=Q(s,a)+α[r+γmaxQ(s',a')-Q(s,a)] (14)

[0056] S326: Determine whether the termination condition is met, if so, end, otherwise return to step S322.

[0057] Due to the adoption of the above technical scheme, the present invention provides an intelligent scheduling method for unmanned equipment on the deck of an aircraft carrier taking into account the degradation of support equipment. The method takes into account the actual operation status of the support equipment and is more suitable for engineering applications. Based on the Q-learning method, the method designs a reward mechanism for the degradation of the support equipment, which can uniformly consider the scheduling time of the unmanned equipment on the deck and the degree of degradation of the support equipment. In addition, based on the ε-greedy exploration strategy, the method designs a variable exploration strategy, which considers the degradation of the support equipment while considering the exploration balance, thereby improving the convergence speed of the improved Q-learning, thereby ensuring that the aircraft carrier can perform the scheduling of unmanned equipment on the deck without direct operation by personnel, thereby reducing personnel risks, enhancing combat flexibility, and improving combat efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 is a flow chart of the method of the present invention;

[0060] Figure 2 This is a schematic diagram of the deck layout in an embodiment of the method of the present invention;

[0061] Figure 3 This is a flow chart of the deck operation of unmanned equipment on the ship in the present invention;

[0062] Figure 4 This is a flowchart of the operation of the unmanned equipment deck on the ship taking into account the degradation of the support equipment disclosed in the present invention. DETAILED DESCRIPTION

[0063] In order to make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0064] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0065] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0066] like Figure 1 The intelligent scheduling method for unmanned equipment on the deck of an aircraft carrier considering the degradation of support equipment shown in the figure specifically includes the following steps:

[0067] S1. Determine the relevant information of the intelligent unmanned dispatching of the deck operation of the ship's unmanned equipment, including the deck layout, the location coordinates of each equipment, the number of unmanned equipment on the ship, the support operation process, and the support time;

[0068] S11. In step S1, the number of unmanned equipment on the ship and the guarantee time can be set by yourself, such as Figure 2 As shown, the deck layout of the "Ford" class aircraft carrier is used as a case demonstration. The number of equipment is 23, including 18 support equipment, numbered 1-18, 4 catapults, numbered 19-22, and the landing runway is numbered 23.

[0069] S12: In step S1, ensure that the operation process is as follows Figure 3As shown, after the unmanned equipment on the ship lands, it is transported to the one-stop support position by a tractor for fault inspection. If there is no fault, the subsequent support operation is carried out directly. If there is a fault, it is repaired on the deck and then the subsequent support operation is carried out. After the fault inspection, grounding, power supply, refueling, oxygenation, nitrogenation, and bomb hanging operations are required. Refueling and oxygenation can be interchanged but not performed at the same time, and can be performed at the same time as nitrogenation and bomb hanging. After completing the above operations, the unmanned equipment on the ship leaves the one-stop support position by a tractor and enters the catapult takeoff.

[0070] S2: Based on the assumptions and constraints of intelligent scheduling of unmanned equipment on the aircraft carrier deck considering the degradation of support equipment, a mathematical model of the corresponding unmanned equipment on the deck is established;

[0071] In order to maintain combat performance, unmanned equipment on the deck needs to perform support operations at the corresponding position before being deployed. According to the deployment plan and support requirements of the unmanned equipment on the deck, the time of support operations is minimized and the deployment rate of unmanned equipment on the deck is increased, that is, efficient deck operation scheduling is achieved. Since there is no participation of support personnel in intelligent unmanned scheduling, the availability of support equipment becomes the key to smooth scheduling. Therefore, the degradation state of the support equipment must be included in the constraint range, and corresponding maintenance activities must be considered to ensure the normal operation of the support equipment. Comprehensive consideration of the intelligent scheduling problem of unmanned equipment on the deck of an aircraft carrier can be regarded as a resource-constrained project scheduling problem under the conditions of satisfying process logic constraints and equipment constraints. Therefore, the following assumptions are first proposed:

[0072] The total amount of various consumable resources stored on the deck is sufficient, and any support mission cannot be interrupted once it begins. The support equipment refers to a one-stop support position, which is initially brand new. When performing support operations, it enters a preventive or maintenance state without affecting the current support operation.

[0073] The Gamma process is used to characterize the degradation phenomena such as operation wear and sea breeze erosion experienced by the security equipment over time, which has the characteristics of monotonicity and non-negative increments. Assume that the degradation increment of the security equipment p at time t1~t2 is It obeys the Gamma distribution, and the probability density function is

[0074]

[0075]

[0076] Among them, the shape parameter is α, the scale parameter is β, the average degradation rate is α / β, and the variance is α / β 2 Therefore, the degradation state value D at time t pt for

[0077]

[0078] According to D pt The protection equipment is divided into three states: pt ∈[0,D P ) when the unmanned equipment on the ship is in normal condition; D pt ∈[D P ,D M ) When the unmanned equipment on the ship is protected for a longer period of time, it is a preventive state; D pt ∈[D M ,+∞), the unmanned equipment on the ship cannot perform support operations and is in a fault state.

[0079] In order to prevent the support equipment from entering a fault state, whether to perform maintenance and the type of maintenance are determined based on the current degradation state of the support equipment. When the support equipment is in a normal state, no maintenance is required and the next support operation can be carried out directly; when the support equipment is in a preventive state, preventive maintenance can be carried out and the degradation state is randomly rolled back; when the support equipment is in a fault state, it needs to be replaced after the fault and the degradation state is reset to zero. When the support equipment is under maintenance, the ship's unmanned equipment can choose other support equipment for the next support operation. The deck operation process of the ship's unmanned equipment considering the degradation of the support equipment is as follows Figure 4 shown.

[0080] The specific process in S2 includes: establishing constraints

[0081] The objective function is

[0082] F=minC max (1)

[0083] Wherein, formula (1) indicates that the objective function is to minimize the maximum completion time, C max It indicates the maximum completion time of all support operations for a wave of unmanned equipment on the ship.

[0084] Constraints

[0085]

[0086]

[0087]

[0088]

[0089] y ijj' +y ij'j =1 (6)

[0090]

[0091]

[0092]

[0093] 0<D P <D M , t P <t M (10)

[0094] where e ij represents the completion time of the ship's unmanned equipment support, (i, j) represents the support task j of the ship's unmanned equipment i, J = {(i, j)|i∈I,j∈J i} represents the set of shipboard unmanned equipment cluster support tasks, I = {1, 2, ..., n} represents the set of shipboard unmanned equipment, J i ={1,2,…,J i} represents the support task set of the ship's unmanned equipment i;

[0095] Formula (3) indicates that the end time of the guarantee task includes its duration and maintenance time, and the maintenance needs to wait until the current guarantee task is completed, where s ij d ij Respectively represent the start time and duration of (i, j). If the preventive maintenance of equipment p is completed after (i, j), then z Pijp =1, otherwise z Pijp = 0, if the guarantee equipment p completes (i, j) and then performs post-failure maintenance, then z Mijp =1, otherwise z Mijp =0,t P Indicates the duration of preventive maintenance of equipment, t M represents the maintenance duration after the failure of the guarantee equipment, P = {1, 2, ..., 18} represents the guarantee equipment set;

[0096] Formula (4) indicates that all the preceding guarantee tasks must be completed before the guarantee task starts. ij represents the set of immediate preceding guarantee tasks of (i, j);

[0097] Formulas (5)-(6) indicate that the order of the guarantee tasks in the exchangeable guarantee task set can be exchanged, but not simultaneously, where Y ij represents the exchangeable support task set of (i, j). If the unmanned equipment i on the ship is providing support at the support equipment p, then x ijp =1, otherwise x ijp = 0, if (i,j) is before (i,j') then y ijj' =1, otherwise y ijj' =0;

[0098] Formula (7) indicates that the same equipment can only support one unmanned equipment on the ship at the same time, Q = {1, 2, ..., m} represents the equipment set;

[0099] Formula (8) indicates that at the same time, each support task of each ship-surface unmanned equipment can only be supported by one device;

[0100] Formula (9) indicates that the number of devices used at the same time must not exceed the total number of devices;

[0101] Formula (10) represents the relationship between preventive maintenance and post-failure maintenance, D P Denotes the preventive maintenance threshold of the equipment, D M Indicates the maintenance threshold after a guaranteed device failure.

[0102] S3, reconstructing the above-mentioned mathematical model of unmanned equipment on the ship deck into a Markov decision process model, improving the ε-greedy exploration strategy in Q learning, and using improved Q learning to train the model;

[0103] S31. Considering that there is only one intelligent agent in the environment, the dispatch center is selected as the intelligent agent.

[0104] S311, State space: The state should reflect the global state of the deck, including equipment number, equipment usage status, shipboard unmanned equipment number, shipboard unmanned equipment usage status, shipboard unmanned equipment currently executing or to be executed support tasks, and support task execution time; the state space is defined as {Ss, Sa}, where S s is the device state, which can be expressed as a triple (S id ,S op ,S d ), S id is the device number, S op The device usage status includes: idle, in use, locked, and repaired. d To ensure the status of the equipment (the status of non-guaranteed equipment is always normal); S a is the state of the unmanned equipment on the ship, which can be expressed as a quaternion (i, A op ,j,d ij ), A op It indicates the operating status of unmanned equipment on the ship, including idle, in use, waiting, and dispatched.

[0105] S312, Action Set: The action should reflect the action directly performed by the unmanned equipment on the ship, that is, a certain unmanned equipment on the ship moves to a certain position, including the number of the unmanned equipment on the ship and the equipment number; the action set is defined as {i, S id}.

[0106] S313. Reward function: The reward function should reflect the feedback obtained from the interaction between the unmanned equipment on the ship and the deck status.

[0107] The system availability is defined as

[0108] R pt =1-D pt (11)

[0109] The reward function takes formula (1) as the target, considers the moving distance reward and the guarantee equipment availability reward, and designs the reward function as follows:

[0110] r=λR l +c(1-λ)R u (12)

[0111] Among them, R r To reward the moving distance, in order to encourage the selection of the nearest location, set

[0112]

[0113] L is the sum of the distances of all mobile unmanned equipment on the ship, which can be calculated using the Euclidean distance. u To reward the availability of security equipment, in order to encourage the use of security equipment with higher availability, it is set to the sum of the availability of all security equipment used. λ is the weight of the two, and considering the different units and data levels of the two, the parameter c is introduced to adjust the order of magnitude.

[0114] S314, Exploration strategy: Considering the impact of learning times and equipment degradation on availability, the ε-greedy exploration strategy is improved. For learning times, when the number of learning times is small, exploration should be the main method. As the number of learning times increases, more accumulated experience should be used. For equipment availability, the equipment with the largest value in the Q value table is selected. When the availability of the equipment is low, in order to reduce the occurrence of equipment failure, the next action should be randomly selected with a greater probability of ε. Otherwise, the action with the largest value in the Q value table should be selected with a greater probability of 1-ε. Therefore, the exploration strategy is designed as

[0115]

[0116] Among them, a, b, c are appropriate parameters, and M is the number of training times.

[0117] S32. The specific steps of training the model using improved Q learning include:

[0118] S321, initializing the learning factor α, discount factor γ and Q value table;

[0119] S322, determine the current state s;

[0120] S323, selecting action a according to the Q value table, and determining the exploration rate ε using the improved ε-greedy exploration strategy;

[0121] S324, execute action a, and determine the next state s' and reward r;

[0122] S325, updating the Q value table according to the following formula;

[0123] Q(s,a)=Q(s,a)+α[r+γmaxQ(s',a')-Q(s,a)] (15)

[0124] S326: Determine whether the termination condition is met, if so, end, otherwise return to step S322.

[0125] S4. Use the Q value table of the last iteration to select states and actions, and obtain the optimal scheduling sequence for intelligent unmanned scheduling of deck operations of unmanned equipment on the ship considering the degradation of support equipment.

[0126] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An intelligent scheduling method for unmanned equipment on an aircraft carrier deck taking into account the degradation of support equipment, characterized in that include: Collect relevant information about the unmanned dispatch of aircraft deck operations, including deck layout, location coordinates of equipment working with aircraft, number of unmanned equipment on the deck, and operation flow of aircraft; Set the assumptions of degradation of support equipment and constraints of unmanned equipment on the ship deck and establish a mathematical model of unmanned equipment on the ship deck; Reconstructing the mathematical model of the ship deck unmanned equipment into a Markov decision process model; Based on the Markov decision process model and in the Markov decision framework, the ε-greedy exploration strategy in Q-learning is improved to enhance the exploration efficiency and strategy adaptability. The Q-value table of the final iteration is used to select states and actions, and the optimal scheduling sequence for unmanned scheduling of aircraft deck operations considering the degradation of support equipment is obtained.

2. According to claim 1, a method for intelligent scheduling of unmanned equipment on the deck of an aircraft carrier taking into account the degradation of support equipment, characterized in that: The assumptions for the degradation of the support equipment are as follows: the total amount of various consumable resources stored on the deck is sufficient; any support task cannot be interrupted after it starts; the support equipment is a one-stop support location and its initial state is brand new; it enters the prevention or maintenance state when performing support operations and does not affect the current support operations. The degradation state of the support equipment is included in the constraint range, and the corresponding maintenance activities are considered to ensure the normal operation of the support equipment.

3. According to claim 1, a method for intelligent dispatching of unmanned equipment on the deck of an aircraft carrier taking into account the degradation of support equipment, characterized in that: Describes the degradation phenomenon of the operating wear and tear and sea breeze erosion experienced by the support equipment over time. It has the characteristics of monotonicity and non-negative increments. The support equipment is divided into three states according to the degradation state value of the support equipment: When the status value falls within the normal range, D pt ∈[0,D P ) When the unmanned equipment on the ship is in normal condition, the support time is normal; When the state value falls within the prevention threshold range, D pt ∈[D P ,D M ), the support time of unmanned equipment on the ship becomes longer, which is a preventive state; When the status value exceeds the fault threshold, D pt ∈[D M ,+∞), the unmanned equipment on the ship cannot perform support operations and is in a fault state; To prevent the support equipment from entering a fault state, determine whether to perform maintenance and the type of maintenance based on the current degradation state of the support equipment. When the support equipment is normal, no maintenance is required and the next support operation can be carried out directly. When the support equipment is in a preventive state, preventive maintenance is carried out and the degradation state is randomly rolled back. When the support equipment is in a fault state, it needs to be replaced after the fault and the degradation state is reset to zero.

4. According to claim 1, a method for intelligent dispatching of unmanned equipment on the deck of an aircraft carrier taking into account the degradation of support equipment, characterized in that: The objective function of the optimal scheduling sequence for unmanned scheduling of aircraft carrier deck operations is: F=minC max (1) Among them, F means that the objective function is to minimize the maximum completion time, C max It represents the maximum completion time of all support operations of a wave of unmanned equipment on the ship, calculated by formula (2); The constraints of the unmanned equipment on the ship are: and ijj' +y ij'j =1 (6) 0<D P <D M , t P <t M (10) In formula (2), e ij represents the completion time of the ship's unmanned equipment support, (i, j) represents the support task j of the ship's unmanned equipment i, J = {(i, j)|i∈I,j∈J i } represents the set of shipboard unmanned equipment cluster support tasks, I = {1, 2, ..., n} represents the set of shipboard unmanned equipment, J i ={1,2,…,J i } represents the support task set of the ship's unmanned equipment i; Formula (3) indicates that the end time of the guarantee task includes its duration and maintenance time, and the maintenance needs to wait until the current guarantee task is completed, where s ij ,d ij Respectively represent the start time and duration of (i, j). If the preventive maintenance of equipment p is completed after (i, j), then z Pijp =1, otherwise z Pijp = 0, if the guarantee equipment p completes (i, j) and then performs post-failure maintenance, then z Mijp =1, otherwise z Mijp =0,t P Indicates the duration of preventive maintenance of equipment, t M represents the maintenance duration after the failure of the guarantee equipment, P = {1, 2, ..., 18} represents the guarantee equipment set; Formula (4) indicates that all the preceding guarantee tasks must be completed before the guarantee task starts. ij represents the set of immediate preceding guarantee tasks of (i, j); Formulas (5)-(6) indicate that the order of the guarantee tasks in the exchangeable guarantee task set can be exchanged, but not simultaneously, where Y ij represents the exchangeable support task set of (i, j). If the unmanned equipment i on the ship is providing support at the support equipment p, then x ijp =1, otherwise x ijp = 0, if (i,j) is before (i,j') then y ijj' =1, otherwise y ijj' =0; Formula (7) indicates that the same equipment can only support one unmanned equipment on the ship at the same time, Q = {1, 2, ..., m} represents the equipment set; Formula (8) indicates that at the same time, each support task of each ship-surface unmanned equipment can only be supported by one device; Formula (9) indicates that the number of devices used at the same time must not exceed the total number of devices; Formula (10) represents the relationship between preventive maintenance and post-failure maintenance, D P Denotes the preventive maintenance threshold of the equipment, D M Indicates the maintenance threshold after a guaranteed device failure.

5. According to claim 4, a method for intelligent dispatching of unmanned equipment on the deck of an aircraft carrier taking into account the degradation of support equipment, characterized in that: Based on the Markov decision process model, when improving the ε-greedy exploration strategy in Q learning under the Markov decision framework: Assume that there is only one intelligent agent in the intelligent scheduling method of unmanned equipment on the aircraft carrier deck, and select the scheduling center as the intelligent agent. The global status of the deck includes the equipment number, equipment usage status, ship deck unmanned equipment number, ship deck unmanned equipment usage status, the current or pending support tasks of the ship deck unmanned equipment, and the execution time of the support tasks; The action directly performed by the ship's surface unmanned equipment is to move a certain ship's surface unmanned equipment to a certain position, which specifically includes the ship's surface unmanned equipment number and equipment number; The feedback obtained by the interaction between the ship's unmanned equipment and the deck status is the reward function, and the system availability is defined as R pt =1-D pt (11) The reward function takes formula (1) as the target, considers the moving distance reward and the guarantee equipment availability reward, and designs the reward function as follows: r=λR l +c(1-λ)R u (12) Among them, R r is the moving distance reward, R u To ensure the available rewards for the device, λ is the weight of the two, and considering the different units and data levels of the two, parameter c is introduced to adjust the order of magnitude; Considering the impact of learning times and equipment degradation on availability, the ε-greedy exploration strategy is improved. For learning times, exploration should be the main method when the number of learning times is small. As the number of learning times increases, more accumulated experience should be used. For equipment availability, the equipment with the largest value in the Q value table is selected. When the availability of the equipment is low, the occurrence of equipment failure is controlled. The next action should be randomly selected with a larger probability of ε. Otherwise, the action with the largest value in the Q value table should be selected with a larger probability of 1-ε. Therefore, the exploration strategy is designed as Among them, a, b, c are appropriate parameters, and M is the number of training times.

6. According to claim 4, a method for intelligent dispatching of unmanned equipment on the deck of an aircraft carrier taking into account the degradation of support equipment, characterized in that: When improving the ε-greedy exploration strategy in Q learning, the specific steps include: S321: Initialize the learning factor α, discount factor γ and Q value table; S322: Determine the current state s; S323: Select action a according to the Q value table, and determine the exploration rate ε using the improved ε-greedy exploration strategy; S324: Execute action a and determine the next state s' and reward r; S325: Update the Q value table according to the following formula; Q(s,a)=Q(s,a)+α[r+γmax Q(s',a')-Q(s,a)] (14) S326: Determine whether the termination condition is met, if so, end, otherwise return to step S322.

Images