Multi-unmanned battle vessel cooperative control optimization method based on machine learning and improved genetic algorithm

Abstract

The invention relates to a multi-unmanned battle vessel cooperative control optimization method based on machine learning and an improved genetic algorithm, and the method comprises the following steps: obtaining unmanned vessel state data and environment data, carrying out the preprocessing, encrypting task operation parameters, and transmitting the encrypted task operation parameters to an unmanned vessel; determining task properties and requirements, and monitoring a state in real time to determine a matching coefficient; constructing a memory function and proficiency, taking the memory function and the proficiency as input feature vectors together with other data, performing decision making on task allocation by using a reinforcement learning algorithm, and performing optimization by using a multi-objective optimization algorithm to obtain an optimized task allocation result; performing feedback reinforcement learning by using an optimization result, and optimizing a task allocation decision; and initializing an unmanned ship cluster, and optimizing the hybrid formation by using an improved genetic algorithm. Compared with the prior art, the method can dynamically adapt to complex and changeable task requirements, improves the accuracy and reliability of task allocation, and remarkably improves the performance and efficiency of the whole formation through real-time formation structure optimization and learning process iteration.

Description

Technical Field

[0001] The present invention relates to the field of coordinated control of multiple unmanned boats, and in particular to a coordinated control optimization method of multiple unmanned combat boats based on machine learning and improved genetic algorithm. Background Art

[0002] With the rapid development of modern military technology, the widespread use of unmanned combat boats has become a key means to improve battlefield situation awareness, precision strike capability and rapid response speed. As technology continues to mature, intelligent collaborative control strategies play an increasingly important role in complex and ever-changing military missions. However, although domestic and foreign research has made significant breakthroughs in the field of ship intelligent task allocation and adaptive formation technology, existing methods and systems still face many challenges in practical applications.

[0003] In the field of intelligent agent task allocation, the current task allocation mechanism still has certain limitations in achieving the generation of UUV task sequences through different methods, such as incomplete model data, neglect of the historical performance of intelligent agents and the accumulation of task execution experience, etc. In the field of adaptive formation, how to adapt to various complex scenarios and find the optimal formation parameter settings are still important problems in intelligent agent adaptive formation technology.

[0004] CN115421492A discloses a method for allocating patrol tasks for a cluster of surface unmanned boats. The USV cluster obtains the angle between the starting point and all patrol points and divides them into groups; it iterates and solves the TSP by improving the genetic algorithm; obtains the solution for each USV to solve n groups of task objectives, and selects the best solution by voting and competition as the initial patrol solution for the cluster; then replans the remaining task routes by polling the navigation costs of the surrounding USVs. This method also only considers the task allocation under a single task, and is difficult to adapt to complex scenarios. Summary of the invention

[0005] The purpose of the present invention is to provide a collaborative control optimization method for multiple unmanned combat boats based on machine learning and improved genetic algorithm, which is based on machine learning algorithm and improved genetic algorithm, and introduces the concepts of memory function and proficiency to improve the accuracy and reliability of task allocation and enhance the performance and efficiency of the overall formation.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm comprises the following steps:

[0008] S1, obtain the status data and environmental data of the unmanned boat, pre-process the data, and encrypt and transmit the checked and corrected mission operation parameters to the unmanned boat;

[0009] S2. Clarify the nature and requirements of the mission, and monitor the status of the unmanned boat's work and mission in real time to determine the matching coefficient;

[0010] S3. Construct a memory function and proficiency, and use the matching coefficient, task requirements, pre-processed unmanned boat state data and environmental data as input feature vectors. Use a reinforcement learning algorithm to make decisions on task allocation, and use a multi-objective optimization algorithm to find the optimal solution to obtain an optimized task allocation result. The memory function is a self-evaluation function formed by tracking the efficiency and quality of each boat in completing tasks based on historical task allocation data. The proficiency is a quantitative evaluation of the experience and performance of each boat in different task types, which is used to reflect the completion of each boat in historical tasks.

[0011] S4. Use the optimization results to conduct feedback reinforcement learning, execute tasks, adjust the reinforcement learning model according to the feedback of task execution results, and optimize task allocation decisions;

[0012] S5. Initialize the unmanned boat cluster and optimize the mixed formation using the improved genetic algorithm based on the optimized task allocation results.

[0013] The unmanned boat status data includes position, speed, battery power and number of completed tasks, and the environmental data includes temperature, humidity, wind speed and wind direction.

[0014] In step S1, the preprocessing includes the following steps:

[0015] Cleaning data: missing value processing, outlier processing and duplicate data processing are performed in sequence;

[0016] Data conversion: perform data standardization and normalization, and use one-hot encoding or label encoding for encoding conversion;

[0017] Feature extraction: Select features that have an important impact on task allocation decisions from the original data, and generate new features based on existing data.

[0018] The encrypted transmission of the checked and corrected mission operation parameters to the unmanned boat is specifically as follows:

[0019] Verify parameter rationality: Check whether the task operation parameters are within a reasonable range, and perform parameter deviation correction for parameters that are not within a reasonable range;

[0020] Convert parameter format: convert task operation parameters into corresponding instruction format;

[0021] Parameter encryption and transmission: The mission operation parameters after format conversion are encrypted and transmitted to the unmanned boat through a secure communication channel.

[0022] The step S2 is specifically as follows:

[0023] Clarify the nature and requirements of tasks: Use classification algorithms and feature extraction algorithms based on task types to extract the features of each task and divide the tasks into combat function tasks and daily non-combat tasks. The task features include priority, required resources, and estimated time.

[0024] Monitor the status of the unmanned boat: Use the real-time monitoring system to confirm the status of the unmanned boat in real time, and determine the matching coefficient based on the current idle status of the unmanned boat and future mission data.

[0025] The step S3 comprises the following steps:

[0026] S31, model initialization: Initialize the reinforcement learning model, set the initial strategy and reward function;

[0027] S32, input feature vector: taking the state characteristics, environmental characteristics, task requirements, memory function, proficiency and matching coefficient of the unmanned boat as input to generate an input feature vector;

[0028] S33, initial task allocation: the reinforcement learning model outputs the initial task allocation result according to the input feature vector;

[0029] S34. Multi-objective optimization decision: The initial task allocation results of the reinforcement learning model are input into the multi-objective optimization module. Based on the real-time data, matching coefficient, task requirements and historical task data, a multi-objective optimization algorithm is used to find the balance point between multiple objectives, and the optimal solution is selected from the non-dominated solution set as the optimized task allocation result.

[0030] The memory function is defined as follows:

[0031]

[0032] Among them, M(S j ) is the unmanned boat S j The memory function of S j is the jth unmanned boat; α, β, γ are weight parameters, corresponding to the task completion time, task quality and energy consumption, respectively, satisfying α+β+γ=1; t i For task T i Completion time; Q i For task T i Finished quality; E i For task T i Energy consumption; N is the unmanned boat S j The number of tasks completed; i For task T i The weight is set according to the priority and importance of the task; T max The longest completion time of all tasks; Qmax The highest completion quality among all tasks; E min The lowest energy consumption among all tasks.

[0033] The proficiency is defined as follows:

[0034]

[0035] Among them, F k (S j ) is the unmanned boat S j Proficiency in task type k; S j is the jth unmanned boat; α, β, γ are weight parameters, corresponding to the task completion time, task quality and energy consumption, respectively, satisfying α+β+γ=1; t i For task T i Completion time; Q i For task T i Finished quality; E i For task T i Energy consumption; N is the unmanned boat S j The number of tasks completed; i For task T i The weight is set according to the priority and importance of the task; T max The longest completion time of all tasks; Q max The highest quality of completion among all tasks; E min The lowest energy consumption among all tasks; t current is the current time point; t i0 For task T i Completion time point; δ ik is the indicator function, k is the task type, when task T i When it belongs to task type k, δ ik =1, otherwise, δ ik =0; λ is the time decay factor.

[0036] The step S4 is specifically as follows:

[0037] Feedback reinforcement learning: During the reinforcement learning training process, the task allocation results after multi-objective optimization are collected and used as reward feedback for the reinforcement learning model. The memory function is used to record the historical task allocation data and update the memory library.

[0038] Task execution and feedback: The task assignment results are transmitted to the unmanned boat in real time through a low-latency, high-reliability communication protocol. The unmanned boat executes the task and collects task execution feedback, including completion time, success rate, resource utilization, task adaptability, task proficiency and energy consumption. An online learning mechanism is adopted to establish a reward feedback mechanism. According to the task completion status and execution effect, immediate rewards are calculated, and long-term reward mechanisms are considered to evaluate the impact of task completion on the overall performance of the system. According to reward feedback, the strategy of the reinforcement learning model is adjusted to optimize task assignment decisions.

[0039] The step S5 is specifically as follows:

[0040] Initialize the population, and according to the optimized task allocation results, use the adaptive genetic algorithm that introduces the adaptive mechanism to dynamically adjust the crossover probability and mutation probability according to the fitness distribution and evolutionary state of the population, perform population iteration, and after the iteration is completed, select the formation configuration scheme with the highest fitness as the optimal solution and output it to the taboo search algorithm;

[0041] The optimal solution obtained by the adaptive genetic algorithm is used as the initial solution of the taboo search algorithm. By simulating the search process of human intelligence, the taboo table is used to record the solutions or search directions that have been visited during the search process, and the optimal solution is obtained through iterative optimization as the final formation configuration plan.

[0042] The adaptive genetic algorithm scores the performance of each formation configuration by combining stability, efficiency and responsiveness through a fitness function, and the fitness function is expressed as:

[0043] Fitness(x)=ω1P stability (x)+ω2P accuracy (x)+ω3P responsiveness (x)+ω4P efficency (x)

[0044] Where x is a formation configuration, P stability , P accuracy , P responsiveness , P efficency They respectively represent the stability of the formation, the accuracy of the formation, the ability to respond to the environment and the efficiency of completing the task. ω1, ω2, ω3 and ω4 are the corresponding weights of the indicators.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] (1) The present invention significantly optimizes the decision-making process of task allocation by constructing a memory function and a proficiency evaluation mechanism. The memory function tracks and evaluates the efficiency and quality of the unmanned boat in completing tasks based on its performance in historical tasks, while the proficiency quantifies the experience and capabilities of the unmanned boat in different task types. This dual evaluation mechanism, as an input feature of the reinforcement learning algorithm, ensures the accuracy of task allocation and improves the reliability of the allocation results.

[0047] (2) The present invention realizes the dynamic adaptability to complex task requirements by combining the real-time monitoring system with the feedback reinforcement learning model. The real-time monitoring system continuously tracks the status of the unmanned boat and dynamically adjusts the matching coefficient according to the task requirements and the current working status of the unmanned boat. Subsequently, through the multi-objective optimization algorithm, multiple decision-making objectives are balanced to select the optimal task allocation plan, ensuring that the system can flexibly respond to changing task environments.

[0048] (3) The present invention optimizes the mixed formation by introducing an improved genetic algorithm, which significantly improves the overall performance of the unmanned boat formation. This optimization process not only effectively improves the rationality of the formation structure, but also further improves the mission execution capability and efficiency of the unmanned boat in complex environments through an iterative learning process.

[0049] (4) The present invention performs strict data preprocessing before transmitting mission parameters, including missing value processing, data standardization, feature extraction and other steps to ensure data quality. In addition, mission operation parameters are encrypted before transmission and transmitted to the unmanned boat through a secure communication channel to improve the security and reliability of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of the method of the present invention;

[0051] Figure 2 A schematic diagram of the intelligent task allocation method for multiple unmanned combat boats based on machine learning;

[0052] Figure 3 To improve the genetic algorithm flow chart;

[0053] Figure 4 The figure is a structural diagram of an intelligent task allocation system in one embodiment. DETAILED DESCRIPTION

[0054] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0055] This embodiment provides a method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithms. It uses a new task allocation mechanism based on machine learning algorithms and introduces the concepts of memory functions and agent proficiency. It can dynamically adapt to complex and changing task requirements, and improve the accuracy and reliability of task allocation through the accumulation of historical data and task execution experience. It integrates improved genetic algorithms into hybrid formation technology to enhance the environmental adaptability and formation performance of agents. Through real-time formation structure optimization and learning process iteration, it aims to achieve dynamic behavior and strategy adjustment of multiple boats, thereby significantly improving the performance and efficiency of the overall formation.

[0056] Specifically, Figure 1 As shown, the method comprises the following steps:

[0057] S1. Obtain the status data and environmental data of the unmanned boat, pre-process the data, and encrypt and transmit the checked and corrected mission operation parameters to the unmanned boat.

[0058] Specifically, S1 includes the following steps:

[0059] S11. Collect data: collect the status data and environmental data of the unmanned boat. The status data of the unmanned boat includes the position, speed, battery power, number of completed tasks, etc. The environmental data includes temperature, humidity, wind speed, wind direction, etc.

[0060] S12. Clean data: perform missing value processing, outlier processing and duplicate data processing in turn.

[0061] S13, data conversion: perform data standardization and normalization, and use one-hot encoding (One-Hot Encoding) or label encoding (Label Encoding) for encoding conversion.

[0062] S14, Feature Extraction: Select features that have an important impact on task allocation decisions from the original data, and generate new features based on the existing data. For example, calculate the length of the navigation path and the navigation time based on the location data, and calculate the remaining endurance time based on the speed and power data.

[0063] S15. Encrypt and transmit the checked and corrected mission operation parameters to the unmanned boat:

[0064] S151. Verify parameter rationality: Check whether the mission operation parameters are within a reasonable range, such as whether the navigation speed exceeds the safety range, whether the mission execution time is reasonable, etc., and perform parameter deviation correction for parameters that are not within the reasonable range;

[0065] S152, convert parameter format: convert the task operation parameters into corresponding instruction formats;

[0066] S153, parameter encryption and transmission: encrypt the mission operation parameters after format conversion and transmit them to the unmanned boat through a secure communication channel.

[0067] like Figure 2 The figure shows a method for intelligent task allocation of multiple unmanned combat boats based on machine learning, including steps S2-S4, which are specifically described as follows.

[0068] S2. Clarify the nature and requirements of the mission, and monitor the working and mission status of the unmanned boat in real time to determine the matching coefficient.

[0069] Specifically, S2 includes the following steps:

[0070] S21. Clarify the nature and requirements of tasks: Use classification algorithms based on task types (such as decision trees) and feature extraction algorithms to extract the characteristics of each task, and divide tasks into combat function tasks and daily non-combat tasks. Task characteristics include priority, required resources, and estimated time.

[0071] S22. Monitor the status of the unmanned boat: Use the real-time monitoring system to confirm the status of the unmanned boat in real time, and determine the matching coefficient based on the current idle status of the unmanned boat and future mission data.

[0072] Description of the matching coefficient calculation method: The matching coefficient value range is 0-1.5. When calculating the matching coefficient, the following three aspects are mainly considered.

[0073] First, current status assessment: real-time monitoring of the current status of the unmanned boat, including key indicators such as whether the unmanned boat is idle, remaining power, speed, and current task load. If the unmanned boat is idle and has sufficient power, the matching coefficient will be higher; on the contrary, if the unmanned boat is performing other tasks or is low on power, the matching coefficient will be reduced accordingly. For example, the unmanned boat is completely idle and has sufficient power: the matching coefficient benchmark value is 0.7-0.9; the unmanned boat is performing other tasks and has insufficient power: the matching coefficient benchmark value is 0.1-0.4.

[0074] Second, future mission demand assessment: by analyzing the number and complexity of future missions, the mission load of the unmanned boat is estimated. If the future mission demand is high and urgent, the currently idle unmanned boats should be given priority and the matching coefficient will increase; if the future mission volume is small or non-urgent, the matching coefficient will be appropriately reduced. For example, if the future mission demand is high and urgent: reduce 0.1-0.2 from the baseline value; if the future mission demand is small or non-urgent: increase 0.1-0.2 from the baseline value.

[0075] Third, real-time status adjustment: If the status of the unmanned boat continues to be good (for example, sufficient power and the mission is progressing smoothly): the matching coefficient can be fine-tuned to increase by 0.05; if the status of the unmanned boat is abnormal (for example, the speed slows down and the power drops sharply): the matching coefficient can be fine-tuned to decrease by 0.05.

[0076] Example analysis:

[0077] Drone A: Currently in idle state, with sufficient power and normal speed. There is a high-priority and urgent task to be performed in the next hour.

[0078] Therefore, the baseline matching coefficient of unmanned boat A is 0.9 (because A is completely idle and has sufficient power); the variation coefficient is -0.15 (due to the urgency of future tasks, its status is saved first). The final matching coefficient is 0.75.

[0079] Unmanned boat B: Currently executing a low-priority mission, the remaining battery power is sufficient, and the speed is normal. The future mission load is light, and there are no urgent tasks.

[0080] Baseline matching coefficient: 0.4 (B is currently performing a task, so its adaptability is slightly lower); Variation coefficient: +0.1 (because future tasks are not urgent, it is suitable for current task allocation). The final matching coefficient is 0.5.

[0081] S3. Construct memory function and proficiency, and use them together with matching coefficient, task requirements, pre-processed unmanned boat status data and environmental data as input feature vectors. Use reinforcement learning algorithm to make decisions on task allocation, and use multi-objective optimization algorithm to find the optimal solution to obtain the optimized task allocation result.

[0082] The memory function refers to a self-evaluation function that records historical task allocation data and tracks the efficiency and quality of each boat's task completion. The specific definition is as follows:

[0083]

[0084] Among them, M(S j ) is the unmanned boat S j The memory function of S j is the jth unmanned boat; α, β, γ are weight parameters, corresponding to the task completion time, task quality and energy consumption, respectively, satisfying α+β+γ=1; t i For task T i Completion time; Q i For task T i Finished quality; E i For task T i Energy consumption; N is the unmanned boat S j The number of tasks completed; i For task T iThe weight is set according to the priority and importance of the task; T max The longest completion time of all tasks; Q max The highest quality of completion among all tasks; E min The lowest energy consumption among all tasks.

[0085] Task weight ω i :Each task has different importance and priority, weight ω i To reflect this. The more important the task, the greater the weight. This is a standardized time-to-completion estimate. i The completion time t i The longest completion time T among all tasks max Comparison. The shorter the completion time, the larger the value, indicating higher efficiency. Completion quality part This is a standardized task quality evaluation. i Finished quality Q i With the highest quality Q in all tasks max Compare. The higher the quality, the greater the value. Energy consumption part This is a standardized energy consumption evaluation. i Energy consumption E i With the lowest energy consumption E among all tasks min Compare. The lower the energy consumption, the larger the value. Weighted sum: The above three parts are weighted summed according to weights α, β, γ to get task T i Finally, the comprehensive evaluation scores of all tasks are calculated according to the weight ω i Sum, get boat S j The memory function value M(S j ).

[0086] Proficiency refers to the quantitative evaluation of each boat's experience and performance in different mission types. It is used to reflect the completion of each boat in historical missions, especially its performance in specific types of missions, and serves as an important reference indicator for optimizing mission allocation decisions. The specific definitions are as follows:

[0087]

[0088] Among them, Fk(Sj) is the proficiency of unmanned boat Sj in task type k; t current is the current time point; t i0 For task T i Completion time point; δ ik is the indicator function, k is the task type, when task T i When it belongs to task type k, δ ik =1, otherwise, δik =0; λ is the time decay factor.

[0089] Comprehensive evaluation score: It is consistent with the evaluation method in the memory function formula, including the standardized evaluation of completion time, task quality and energy consumption, and is weighted and summed according to weights α, β, and γ. Time decay factor exp(-λ(t current -t i0 )):Used to consider the impact of task completion time. The impact of tasks completed earlier on the current proficiency gradually decreases. The decay factor λ controls the decay speed. Weighted summation: The evaluation scores of all tasks belonging to task type k are weighted and summed according to the time decay factor to obtain the result S j Proficiency F on task type k k (S j ).

[0090] Specifically, step S3 includes the following steps:

[0091] S31. Model initialization: Initialize the Deep Q Network (DQN) or Proximal Policy Optimization (PPO) model and set the initial strategy and reward function.

[0092] S32, input feature vector: take the state characteristics, environmental characteristics, task requirements, memory function, proficiency and matching coefficient of the unmanned boat as input to generate an input feature vector.

[0093] S33. Initial task allocation: The reinforcement learning model outputs the initial task allocation result based on the input feature vector.

[0094] S34. Multi-objective optimization decision: The initial task allocation result of the reinforcement learning model is input into the multi-objective optimization module. According to the real-time data, matching coefficient, task requirements and historical task data, a multi-objective optimization algorithm (such as NSGA-II) is used to find the balance point between multiple objectives, and the optimal solution is selected from the non-dominated solution set as the optimized task allocation result.

[0095] In step S3, the input feature vector consists of data in multiple dimensions, including:

[0096] (1) Status characteristics of the unmanned boat: location, speed, battery power, number of completed missions, load status, health status, etc.

[0097] (2) Environmental characteristics: temperature, humidity, wind speed, wind direction, electromagnetic environment, etc.

[0098] (3) Task requirements: task priority, required resources, estimated completion time, task type, etc.

[0099] (4) Memory function: Based on historical task allocation data, evaluate the efficiency and quality of each unmanned boat in completing the task.

[0100] (5) Proficiency: Quantitatively assess each UAV’s experience and performance in different mission types.

[0101] (6) Matching coefficient: calculated based on the current idle state of the unmanned boat and the future mission load.

[0102] In step S3, the action of the reinforcement learning model refers to the specific operation taken by the system on task allocation in a specific state. Specifically, it includes:

[0103] (1) Assigning a specific task: Assign a task to a specific unmanned boat. For example, assign the reconnaissance task to unmanned boat A and the attack task to unmanned boat B.

[0104] (2) Adjusting task priority: The system can adjust the task priority based on the urgency of the task or the status of the UAV. For example, if UAV A is low on battery, it may be adjusted to prioritize low-energy tasks.

[0105] (3) Task reallocation: When a task fails or the status of the UAV changes, the system can choose to reallocate the task. For example, if UAV B fails to perform the task, the system can transfer its task to UAV C.

[0106] In step S3, the decision space of the reinforcement learning model refers to the set of all possible task allocation schemes of the system in a specific state. Specifically, it includes:

[0107] (1) Combination of tasks and unmanned boats: The system needs to find the best allocation between multiple unmanned boats and multiple tasks. Assuming there are 3 unmanned boats (A, B, C) and 3 tasks (reconnaissance, attack, supply), the decision space will include all possible matching combinations of tasks and unmanned boats, and the number of combinations can be calculated by permutations and combinations. For example, for 3 tasks and 3 unmanned boats, each task has 3 options, so the total number of combinations can be calculated by the combination calculation method in permutations and combinations, and the total number of combinations = 3 (unmanned boat selection) ^ 3 (number of tasks) = 27 combinations.

[0108] (2) Task execution order: The system needs to decide the order in which tasks should be executed, which is also part of the decision space. For example, the system can choose to execute the reconnaissance mission first and then the attack mission, or adjust the order based on the urgency of the tasks.

[0109] S4. Use the optimization results to perform feedback reinforcement learning, execute tasks, adjust the reinforcement learning model according to the feedback of task execution results, and optimize task allocation decisions.

[0110] Specifically, step S4 includes the following steps:

[0111] S41. Feedback reinforcement learning: During the reinforcement learning training process, the task allocation results after multi-objective optimization are collected and used as reward feedback for the reinforcement learning model. That is, the reinforcement learning model can gradually learn and internalize the results of multi-objective optimization, continuously adjust and optimize its own strategies, and ultimately achieve comprehensive performance improvement of task allocation. In addition, the memory function is used to record historical task allocation data and update the memory library.

[0112] S42. Task execution and feedback: The task assignment results are transmitted to the unmanned boat in real time through a low-latency, high-reliability communication protocol (such as MQTT, DDS, etc.). The unmanned boat executes the task and collects task execution feedback, including completion time, success rate, resource utilization, task adaptability, task proficiency and energy consumption. An online learning mechanism is adopted to establish a reward feedback mechanism. According to the task completion status and execution effect, immediate rewards are calculated, and long-term reward mechanisms are considered to evaluate the impact of task completion on the overall performance of the system. According to the reward feedback, the strategy of the reinforcement learning model is adjusted to optimize the task assignment decision.

[0113] S5. Initialize the unmanned boat cluster and optimize the mixed formation using the improved genetic algorithm based on the optimized task allocation results.

[0114] Specifically, Figure 3 As shown, step S5 includes the following steps:

[0115] S51. Population initialization: According to the mission requirements and environmental conditions, the basic parameters of the formation, such as formation size, spacing, speed, etc., are initialized, and a set of formation configuration schemes are randomly generated, called "population".

[0116] S52, Fitness Evaluation: The fitness function combines stability, efficiency and responsiveness to score the performance of each formation configuration. A configuration with a higher fitness means that it will perform better in actual applications and has a greater chance of being selected. Its mathematical formula can be expressed as:

[0117] Fitness(x)=ω1P stability (x)+ω2P accuracy (x)+ω3P responsiveness (x)+ω4P efficency (x)

[0118] Where x is a formation configuration, P stability , P accuracy , P responsiveness , P efficencyThey respectively represent the stability of the formation, the accuracy of the formation, the ability to respond to the environment and the efficiency of completing the task. ω2, ω2, ω3 and ω4 are the weights of these indicators, and these weights are adjusted according to the needs of specific applications.

[0119] S53, genetic algorithm operation

[0120] Selection: Based on the fitness of the individuals, an elite retention strategy is adopted to find one or more individuals (elite individuals) with the highest fitness in the current population, and directly enter the next generation population for reproduction to generate a new generation of population;

[0121] Crossover: Perform a crossover operation on the selected parent individuals, simulate the information exchange and fusion between different formation schemes by exchanging some genes, and generate a new formation configuration scheme. The crossover probability is dynamically adjusted according to the fitness distribution and evolutionary state of the population;

[0122] Mutation: Randomly select some individuals for mutation operation, and simulate random adjustments or emergencies in the formation by changing the values ​​of some genes to increase the diversity of the population. The mutation probability is based on.

[0123] S54, Update the population: Add the new formation configuration generated by the genetic algorithm operation to the population, replace the individuals with lower fitness, and form a new population. The process of updating the population is one of the core of the genetic algorithm. By continuously introducing new and potentially better solutions, the population can gradually approach the optimal solution to the problem;

[0124] S55, iterative optimization: repeat the process of S51-S54 until the preset number of iterations is met or when the fitness value of the best individual in the population does not increase significantly after a certain number of iterations, it can be considered that the genetic algorithm has converged, and the iteration is stopped.

[0125] S56. Output the optimal formation configuration: After the iteration is completed, the formation configuration scheme with the highest fitness is selected as the optimal solution and output to the taboo search algorithm.

[0126] S57, initial solution selection: the optimal solution (or a set of high-quality solutions) obtained in the adaptive genetic algorithm stage is used as the initial solution in the taboo search algorithm stage.

[0127] S58. Set taboo table: initialize the taboo table to record the solutions or search directions that have been visited.

[0128] S59, local search:

[0129] Generate candidate solutions: Generate one or more candidate solutions based on the current solution and taboo table information;

[0130] Evaluate candidate solutions: calculate the fitness of candidate solutions;

[0131] Select the best solution: Select the non-tabu candidate solution with the highest fitness as the new current solution.

[0132] S510, updating the taboo table: adding the new solution and the related search direction to the taboo table, and updating the time limit of the taboo table.

[0133] S511, iterative optimization: repeat S59-S510 until the stopping condition is met (such as reaching a preset number of iterations, the optimal solution has no significant improvement within a certain number of iterations, etc.).

[0134] S512, output the optimal formation configuration: select the optimal solution obtained in the taboo search algorithm stage as the final formation configuration solution.

[0135] Adaptive genetic algorithms introduce adaptive mechanisms to dynamically adjust key parameters such as crossover probability and mutation probability according to the fitness distribution and evolutionary state of the population. Taboo search algorithms simulate the search process of human intelligence and use taboo tables to record solutions or search directions that have been visited during the search process, and avoid repeated visits to these solutions or directions in subsequent searches, thereby effectively jumping out of the trap of local optimal solutions.

[0136] The adaptive genetic algorithm and the taboo search algorithm are performed in sequence in a serial combination manner, and the new population obtained by the adaptive genetic algorithm is used as the initial population of the taboo search algorithm. This makes the optimization process of the boat formation more hierarchical and fully utilizes the advantage of the adaptive genetic algorithm in finding better targets in a global scope. In the later stage, the maintenance emergency table of the taboo search algorithm can be fully utilized to avoid repeated searches, effectively prevent falling into local optimal solutions, and further improve the efficiency and accuracy of the search.

[0137] This embodiment also provides a system for cooperative control optimization of multiple unmanned combat boats based on machine learning and improved genetic algorithm. Based on the above method, Figure 4 As shown, it includes the following modules:

[0138] Data collection and processing module: obtains all necessary information from the unmanned boat and the environment; cleans, converts and extracts features from the collected data, and transmits it to the unmanned boat;

[0139] Mission assessment and monitoring module: clarify the nature of the mission, assess the capabilities of the unmanned boat and monitor the status of the unmanned boat;

[0140] Intelligent decision-making and task allocation module: Build memory functions and proficiency, use other data information as input feature vectors, use machine learning algorithms to make decisions on task allocation, and use the multi-objective optimization algorithm NSGA-II to find the optimal solution;

[0141] Reinforcement learning and optimization execution module: Use the optimization results to perform feedback reinforcement learning, execute tasks, and then adjust the strategy of the reinforcement learning model based on the feedback results;

[0142] Ship cluster management and formation optimization module: initialize ship clusters and use improved genetic algorithms to optimize mixed formations;

[0143] Security and communication module: implement data encryption, authentication and authorization mechanisms to prevent data leakage and illegal access; responsible for secure communication between unmanned boats to ensure the reliability and security of data transmission.

[0144] Among them, for the intelligent task allocation system in the collaborative control optimization system of multiple unmanned combat boats, from the perspective of top-level design, the system can be divided into two parts: data service end and task allocation end. The data service end is a key module in the multi-boat intelligent task allocation system, which includes two basic functions: data collection and data processing. It is responsible for collecting various necessary information from unmanned boats and the environment, and pre-processing these data to support subsequent task allocation decisions.

[0145] The data processed by the data server can effectively support the execution of the task allocation algorithm and improve the accuracy and efficiency of task allocation. Most current studies have limited use of historical task data, which is usually only used as a reference and not directly used for task allocation decisions. However, historical task data itself has a strong guiding significance for decision-making. Therefore, this embodiment proposes two concepts of intelligent decision-making memory function proficiency.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the remaining modules described can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0147] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A collaborative control optimization method for multiple unmanned combat boats based on machine learning and improved genetic algorithm, characterized in that: The following steps are involved: S1, obtain the status data and environmental data of the unmanned boat, pre-process the data, and encrypt and transmit the checked and corrected mission operation parameters to the unmanned boat; S2. Clarify the nature and requirements of the mission, and monitor the status of the unmanned boat's work and mission in real time to determine the matching coefficient; S3. Construct a memory function and proficiency, and use the matching coefficient, task requirements, pre-processed unmanned boat state data and environmental data as input feature vectors. Use a reinforcement learning algorithm to make decisions on task allocation, and use a multi-objective optimization algorithm to find the optimal solution to obtain an optimized task allocation result. The memory function is a self-evaluation function formed by tracking the efficiency and quality of each boat in completing tasks based on historical task allocation data. The proficiency is a quantitative evaluation of the experience and performance of each boat in different task types, which is used to reflect the completion of each boat in historical tasks. S4. Use the optimization results to conduct feedback reinforcement learning, execute tasks, adjust the reinforcement learning model according to the feedback of task execution results, and optimize task allocation decisions; S5. Initialize the unmanned boat cluster and optimize the mixed formation using the improved genetic algorithm based on the optimized task allocation results.

2. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: The unmanned boat status data includes position, speed, battery power and number of completed tasks, and the environmental data includes temperature, humidity, wind speed and wind direction.

3. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: In step S1, the preprocessing includes the following steps: Cleaning data: missing value processing, outlier processing and duplicate data processing are performed in sequence; Data conversion: perform data standardization and normalization, and use one-hot encoding or label encoding for encoding conversion; Feature extraction: Select features that have an important impact on task allocation decisions from the original data, and generate new features based on existing data.

4. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: In step S1, the encrypted transmission of the checked and corrected mission operation parameters to the unmanned boat is specifically as follows: Verify parameter rationality: Check whether the task operation parameters are within a reasonable range, and perform parameter deviation correction for parameters that are not within a reasonable range; Convert parameter format: convert task operation parameters into corresponding instruction format; Parameter encryption and transmission: The mission operation parameters after format conversion are encrypted and transmitted to the unmanned boat through a secure communication channel.

5. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: The step S2 is specifically as follows: Clarify the nature and requirements of tasks: Use classification algorithms and feature extraction algorithms based on task types to extract the features of each task and divide the tasks into combat function tasks and daily non-combat tasks. The task features include priority, required resources, and estimated time. Monitor the status of the unmanned boat: Use the real-time monitoring system to confirm the status of the unmanned boat in real time, and determine the matching coefficient based on the current idle status of the unmanned boat and future mission data.

6. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: The step S3 comprises the following steps: S31, model initialization: Initialize the reinforcement learning model, set the initial strategy and reward function; S32, input feature vector: taking the state characteristics, environmental characteristics, task requirements, memory function, proficiency and matching coefficient of the unmanned boat as input to generate an input feature vector; S33, initial task allocation: the reinforcement learning model outputs the initial task allocation result according to the input feature vector; S34. Multi-objective optimization decision: The initial task allocation results of the reinforcement learning model are input into the multi-objective optimization module. Based on the real-time data, matching coefficient, task requirements and historical task data, a multi-objective optimization algorithm is used to find the balance point between multiple objectives, and the optimal solution is selected from the non-dominated solution set as the optimized task allocation result.

7. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: The memory function is defined as follows: Among them, M(S j ) is the unmanned boat S j The memory function of S j is the jth unmanned boat; α, β, γ are weight parameters, corresponding to the task completion time, task quality and energy consumption, respectively, satisfying α+β+γ=1; t i For task T i Completion time; Q i For task T i Finished quality; E i For task T i Energy consumption; N is the unmanned boat S j The number of tasks completed; i For task T i The weight is set according to the priority and importance of the task; T max The longest completion time of all tasks; Q max The highest quality of completion among all tasks; E min The lowest energy consumption among all tasks.

8. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: The proficiency is defined as follows: Among them, F k (S j ) is the unmanned boat S j Proficiency in task type k; S j is the jth unmanned boat; α, β, γ are weight parameters, corresponding to the task completion time, task quality and energy consumption, respectively, satisfying α+β+γ=1; t i For task T i Completion time; Q i For task T i Finished quality; E i For task T i Energy consumption; N is the unmanned boat S j The number of tasks completed; i For task T i The weight is set according to the priority and importance of the task; T max The longest completion time of all tasks; Q max The highest quality of completion among all tasks; E min The lowest energy consumption among all tasks; t current is the current time point; t i0 For task T i Completion time point; δ ik is the indicator function, k is the task type, when task T i When it belongs to task type k, δ ik =1, otherwise, δ ik =0; λ is the time decay factor.

9. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1 is characterized in that: The step S4 is specifically as follows: Feedback reinforcement learning: During the reinforcement learning training process, the task allocation results after multi-objective optimization are collected and used as reward feedback for the reinforcement learning model. The memory function is used to record the historical task allocation data and update the memory library. Task execution and feedback: The task assignment results are transmitted to the unmanned boat in real time through a low-latency, high-reliability communication protocol. The unmanned boat executes the task and collects task execution feedback, including completion time, success rate, resource utilization, task adaptability, task proficiency and energy consumption. An online learning mechanism is adopted to establish a reward feedback mechanism. According to the task completion status and execution effect, immediate rewards are calculated, and long-term reward mechanisms are considered to evaluate the impact of task completion on the overall performance of the system. According to reward feedback, the strategy of the reinforcement learning model is adjusted to optimize task assignment decisions.

10. The method for optimizing the coordinated control of multiple unmanned combat boats based on machine learning and improved genetic algorithm according to claim 1, characterized in that: The step S5 is specifically as follows: Initialize the population, and according to the optimized task allocation results, use the adaptive genetic algorithm that introduces the adaptive mechanism to dynamically adjust the crossover probability and mutation probability according to the fitness distribution and evolutionary state of the population, perform population iteration, and after the iteration is completed, select the formation configuration scheme with the highest fitness as the optimal solution and output it to the taboo search algorithm; The optimal solution obtained by the adaptive genetic algorithm is used as the initial solution of the taboo search algorithm. By simulating the search process of human intelligence, the taboo table is used to record the solutions or search directions that have been visited during the search process, and the optimal solution is obtained through iterative optimization as the final formation configuration plan. The adaptive genetic algorithm scores the performance of each formation configuration by combining stability, efficiency and responsiveness through a fitness function, and the fitness function is expressed as: Fitness(x)=ω1P stability (x)+ω2P accuracy (x)+ω3P responsiveness (x)+ω4P efficency (x) Where x is a formation configuration, P stability , P accuracy , P responsiveness , P efficency They respectively represent the stability of the formation, the accuracy of the formation, the ability to respond to the environment and the efficiency of completing the task. ω1, ω2, ω3 and ω4 are the corresponding weights of the indicators.

Images