A method, device and storage medium for intelligent confrontation of unmanned underwater vehicle based on deep reinforcement learning
Through deep reinforcement learning technology, combined with the force marshalling model and intelligent adversarial model, the problem of low confrontation efficiency of unmanned submarines under complex situations is solved, and more efficient decision-making calculation and adversarial level is achieved.
Patent Information
- Application Number
- CN202410471907.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-19
- Publication Date
- 2025-04-11
- Estimated Expiration
- 2044-04-19
AI Technical Summary
In the prior art, the adversarial efficiency and level of unmanned submarines under complex situations are low, and they fail to effectively design force marshalling and allocation, resulting in low decision-making and computing efficiency.
The intelligent adversarial method of unmanned submarines based on deep reinforcement learning is adopted. By obtaining force marshalling situation information and inputting pre-trained force marshalling model, the force marshalling allocation and load configuration are output; combining the real-time situation information of the external simulation deduction platform, the intelligent adversarial model is input, and the maneuvering action, radar switch action and weapon launch action are output.
It realizes effective decision-making and calculation of unmanned submarines under complex situations, improves the efficiency and level of confrontation, and forms a force marshalling and distribution agent that adapts to multiple scenarios.
Smart Images

Figure CN118446081B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an unmanned underwater vehicle intelligent confrontation method, device and storage medium based on deep reinforcement learning, and belongs to the technical field of unmanned underwater vehicle decision-making. Background Art
[0002] An underwater unmanned submersible, also known as an unmanned submersible, is a device that travels underwater without a human operator and relies on remote control or automatic control. It is mainly used to replace divers or manned small submarines to perform high-risk underwater operations such as deep-sea exploration, rescue, and mine removal. Therefore, it is regarded as an intelligent system and is also called a "diving robot" or "underwater robot."
[0003] Unmanned underwater vehicles have a wide range of applications in the military field. They can be used as an unmanned combat platform weapon in a new concept weapon, and their role is similar to that of drones in some aspects. Unmanned underwater vehicles have the characteristics of miniaturization, intelligence, large maneuverability and good concealment. They can be active in high-threat areas or difficult-to-reach waters of manned platforms, extending the detection and combat range of manned platforms. At the same time, it can also serve as a node in maritime network-centric warfare, providing underwater surveillance, battlefield reconnaissance, target indication, relay communication and other guarantees for mother ships (boats). Accordingly, as a comprehensive system integrating multiple technologies and algorithms, the unmanned underwater vehicle agent can combine knowledge from multiple fields such as machine learning, sensor technology, control systems and communication technology to achieve intelligent perception and autonomous response to the underwater environment.
[0004] However, most of the existing technologies are designed for the path planning or obstacle avoidance scenarios of unmanned underwater vehicles, and no design is made for the relevant force grouping and allocation, resulting in low efficiency and level of confrontation of unmanned underwater vehicles in complex situations. Therefore, it is urgent to provide a new intelligent confrontation method to solve the current technical drawbacks. Summary of the invention
[0005] The purpose of the present invention is to provide an unmanned underwater vehicle intelligent confrontation method, device and storage medium based on deep reinforcement learning, aiming to realize effective decision-making calculation of the unmanned underwater vehicle under complex situations and improve the efficiency and level of confrontation.
[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:
[0007] On the one hand, the present invention provides an unmanned underwater vehicle intelligent confrontation method, which includes:
[0008] Obtain force formation situation information;
[0009] Input the force grouping situation information into a pre-trained force grouping model, and output force grouping allocation and load configuration;
[0010] generating hypothetical mission information according to the force grouping situation information, force grouping allocation and load configuration;
[0011] Obtain real-time situation information from external simulation platforms;
[0012] The real-time situation information and assumed mission information are input into a pre-trained intelligent confrontation model to output maneuvering actions, radar switch actions and weapon launch actions.
[0013] The present invention divides the intelligent confrontation process of the unmanned underwater vehicle into two stages: force organization before deployment and autonomous decision-making after deployment. The hypothetical mission information is generated through the force organization model, and real-time situation information is acquired by obtaining an external simulation and deduction platform, so that the unmanned underwater vehicle intelligent confrontation agent can cope with situations of various complexities and output the optimal confrontation strategy.
[0014] Optionally, the force formation situation information data includes tactical target information, available force information of unmanned underwater vehicles, and target enemy situation information;
[0015] The tactical target information includes: tactical targets, defended targets and strike targets;
[0016] Optionally, the available force information of the unmanned underwater vehicle includes: the number, model, type, formation, name, minimum speed, maximum speed and three-dimensional position coordinates of the unmanned underwater vehicle at the current moment;
[0017] The target enemy information includes: the target enemy's number, model, type, group, name and the three-dimensional position coordinates of the target enemy at the current moment;
[0018] Among them, the tactical objectives include defensive operations, offensive operations and both offense and defense.
[0019] Optionally, the unmanned underwater vehicle intelligent confrontation method further includes: preprocessing the force grouping situation information, including extracting features from basic information and performing zero-filling processing on missing data;
[0020] Among them, feature extraction of basic information includes:
[0021] The orientation, yaw angle, speed, three-dimensional position coordinates, unit type and strike load information of each unmanned underwater vehicle in the force formation situation information are extracted, the different basic information are encoded and mapped, and then spliced with the information of the unmanned underwater vehicle to form the input of the force formation model.
[0022] Optionally, the input layer dimension of the force grouping model is set to eight times the sum of the number of our units and the number of enemy targets; the hidden layer dimension is set to the product of the number of our entities and the number of enemy entities. The neural network calculates the degree of association between each of our units and the enemy's target to take into account the number of unmanned underwater vehicles.
[0023] Optionally, the real-time situation information includes: our force situation, enemy force situation and relative force situation between the enemy and us;
[0024] The said our force situation and enemy force situation respectively include: number, model, formation, name, minimum speed, maximum speed, three-dimensional position coordinates at the current moment, three-dimensional position coordinates at the previous moment, current speed, survival status, RCS, heading angle at the current moment and sonar detection information;
[0025] The relative force situation of the enemy and ourselves includes: relative distance and relative angle.
[0026] Optionally, a one-hot encoder is used to encode the information data corresponding to our force situation, the enemy force situation, and the enemy and our force situation respectively to obtain situation codes of various categories, and the contents of various codes are integrated to generate an input vector of the intelligent confrontation model;
[0027] Among them, the input vector dimension = single force situation dimension × (our force number + enemy force number) + enemy and our force relative situation dimension × our force number × enemy force number.
[0028] Optionally, corresponding unmanned underwater vehicles are allocated according to the target area or the opponent's target entity to generate a force grouping allocation for the unmanned underwater vehicles; the number and type of loads are matched to generate a load configuration; and the force grouping allocation and load configuration are integrated as the force grouping result.
[0029] Optionally, the maneuvering action includes: pursuit, left cut, right cut and evasion;
[0030] The radar switch action includes: turning on the radar switch and turning off the radar switch;
[0031] The weapon launch action includes: no action and turning on the weapon launch switch;
[0032] Among them, turning on the weapon launch switch includes: determining whether the surviving enemy target entity closest to us is within the range of the weapon carried by our platform based on obtaining the enemy target entity, and if so, launching the weapon to attack; otherwise, not launching.
[0033] Optionally, based on the triggering events of our side and the enemy side, calculate the overall rewards of the party, including:
[0034] If the detection payload is turned on, the reward is +5; if the strike payload is launched, the reward is +5; if the target is focused, the reward is +30, if the target is not focused, the reward is -5;
[0035] When the target attacks us, if we pursue, the reward is -10, if we cut left or right, the reward is 0, if we dodge, the reward is +10.
[0036] In a second aspect, the present invention further provides an unmanned underwater vehicle intelligent countermeasure device, which comprises:
[0037] A first situation information acquisition module is configured to acquire force grouping situation information;
[0038] A force grouping generation module is configured to input the force grouping situation information into a pre-trained force grouping model and output force grouping allocation and load configuration;
[0039] A mission scenario module is configured to generate scenario mission information according to the force grouping situation information, force grouping allocation and load configuration;
[0040] A second situation information acquisition module is configured to acquire real-time situation information of an external simulation and deduction platform;
[0041] The intelligent confrontation strategy generation module is configured to input the real-time situation information and assumed mission information into a pre-trained intelligent confrontation model, and output maneuvering actions, radar switch actions, and weapon launch actions.
[0042] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the unmanned underwater vehicle intelligent confrontation method as described in the first aspect are performed.
[0043] Compared with the prior art, the present invention has the following beneficial effects:
[0044] The present invention guides the unmanned underwater vehicle to optimize the learning of strategies through the corresponding situation information coding design, action space design, and reward function design, and forms a reliable force grouping and allocation intelligent body that can adapt to various scenarios through the force grouping and allocation and load configuration design before combat. This method ensures that the unmanned underwater vehicle can make targeted autonomous decisions according to complex situation changes, and improves the overall intelligence level of the unmanned underwater vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 The figure shows a schematic diagram of the task planning and intelligent confrontation process of the present invention;
[0046] Figure 2 The figure shows a schematic diagram of the framework for generating the force grouping scheme in the present invention;
[0047] Figure 3 The figure shows a schematic diagram of the structure of the force grouping model in the present invention;
[0048] Figure 4 Shown is a schematic diagram of the flow of the strength grouping model training in the present invention;
[0049] Figure 5 Shown is a schematic diagram of situation information encoding processing in the present invention;
[0050] Figure 6 The figure shows a schematic diagram of the structure of the intelligent confrontation model in the present invention;
[0051] Figure 7 The figure shows a flow chart of the intelligent adversarial model training in the present invention;
[0052] Figure 8 Shown is a schematic diagram of the reward calculation process in the present invention. DETAILED DESCRIPTION
[0053] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0054] Example 1
[0055] This embodiment introduces an unmanned underwater vehicle intelligent confrontation method, which includes:
[0056] Obtain force formation situation information;
[0057] Input the force grouping situation information into a pre-trained force grouping model, and output force grouping allocation and load configuration;
[0058] generating hypothetical mission information according to the force grouping situation information, force grouping allocation and load configuration;
[0059] Obtain real-time situation information from external simulation platforms;
[0060] The real-time situation information and assumed mission information are input into a pre-trained intelligent confrontation model to output maneuvering actions, radar switch actions and weapon launch actions.
[0061] The grouping allocation in this embodiment ensures the determination of the collaborative relationship within the formation, and the payload configuration is the basis for the unmanned underwater vehicle to make collaborative decisions. The combat scenario is generated based on the initial information of the unmanned underwater vehicle, the initial information of the target and the force grouping result, and then the hypothetical mission information is provided to the unmanned underwater vehicle autonomous decision-making intelligent agent model; further, during the real-time confrontation process, the unmanned underwater vehicle controls dynamic actions and coordinates the formation according to the real-time situation information and hypothetical mission information, so as to achieve decision output under complex situations, thereby improving the efficiency and level of the intelligent confrontation process.
[0062] Example 2
[0063] refer to Figure 1 In order to realize the intelligent confrontation strategy of unmanned underwater vehicles in complex situations, this embodiment specifically divides the intelligent confrontation process of unmanned underwater vehicles into two stages: force organization before dispatching and autonomous decision-making after dispatching.
[0064] 1. Unmanned Underwater Vehicle Force Organization Phase
[0065] In the future autonomous intelligent confrontation, how to realize the grouping allocation and load configuration of our unmanned submersibles in the initial situation based on the enemy information in various scenarios will play a key role in the future submarine confrontation. Among them, the grouping allocation is to divide the unmanned submersibles into groups to build a collaborative relationship within the unmanned submersible formation; the load configuration is to determine the type and quantity of weapons mounted on the unmanned submersibles, which is the basis for the coordinated decision-making of the unmanned submersibles.
[0066] For tasks such as unmanned underwater vehicle patrols in specific scenarios, formations and target area allocation can usually be performed through intelligent optimization algorithms or deep learning methods. However, if intelligent optimization algorithms are used, different optimization goals and constraints need to be constructed for different scenarios, which cannot meet the needs of future autonomous adaptive actions; if deep reinforcement learning is used to dynamically generate force grouping and allocation plans, the effectiveness of the plan needs to be verified through deductions. At the same time, the deduction effect is affected by the ad hoc decision-making model of the deduction process, and it is difficult to determine which process has an impact on the result. Therefore, this method cannot be used directly. Therefore, how to realize the force grouping and allocation of unmanned underwater vehicles in unknown scenarios based on the actual confrontation data in some scenarios has become an urgent problem to be solved.
[0067] Therefore, this embodiment uses a neural network to fit known data to construct a force grouping model based on deep learning that can adapt to various scenarios, and uses this target model to achieve effective and reliable force grouping allocation for unmanned underwater vehicles. Figure 2 The specific implementation process of force grouping in this stage is as follows:
[0068] (1) Dataset acquisition
[0069] The situation data required for the actual force organization stage has the problems of complex structure, large amount of data, low effective information density and dispersion. Such data cannot be directly used as input of neural networks, and needs to be processed to extract key information and transform the data format.
[0070] Specifically, the situation information data required in the force formation phase include tactical target information, available force information of unmanned underwater vehicles, and target enemy information;
[0071] The tactical target information includes: tactical targets, defended targets and strike targets;
[0072] The available force information of the unmanned underwater vehicle includes: the number, model, type, formation, name, minimum speed, maximum speed and the current position coordinates of the unmanned underwater vehicle;
[0073] The target enemy information includes: the target enemy's number, model, type, group, name and the target enemy's current position coordinates;
[0074] Among them, the tactical objectives include defensive operations, offensive operations and both offense and defense.
[0075] For the above-mentioned related data information, please refer to Tables 1 to 3:
[0076] Table 1 Tactical target information
[0077] Serial number Fields Example 1 Tactical goals 1: defensive operations; 2: offensive operations; 3: both offensive and defensive 2 Defend the target 'red_15' 3 Target 'blue_22'
[0078] Table 2 Information on available forces of unmanned underwater vehicles
[0079] Serial number Fields Example 1 serial number 16 2 model 'clarke' 3 type 'submarine' 4 Group 'blue_group15' 5 name 'c2_1' 6 Minimum speed 0 7 Maximum speed 20 8 The x coordinate of the current position -7621.03076171875 9 Current position y coordinate 653.3411865234375 10 Current position z coordinate 0.0
[0080] Table 3 Target enemy information
[0081] Serial number Fields Example 1 serial number 16 2 model 'Arleigh Burke'," (unidentified target model) 3 type 'ship' 4 Group 'blue_group15' 5 name 'c2_1' 6 The x coordinate of the current position -7621.03076171875 7 Current position y coordinate 653.3411865234375 8 Current position z coordinate 0.0
[0082] (2) Preprocessing of relevant data sets
[0083] Specifically, the orientation, yaw angle, speed, three-dimensional position coordinates, unit type and strike load information of each unmanned underwater vehicle is extracted from the situation information as the basic information of the unmanned underwater vehicle, and the different basic information is encoded and mapped, and then the information of all the unmanned underwater vehicles is spliced to form the final input information.
[0084] Secondly, in order to solve the problem of inconsistent input information dimensions caused by different numbers of units, a neural network is designed to be able to cope with enough units. When the number of unmanned underwater vehicles is insufficient, the missing data is padded with zeros, so as to finally realize the coding mapping of the input data processing.
[0085] (3) Construction of force grouping model
[0086] refer to Figure 3 , the model is designed using a fully connected neural network. The input layer dimension is related to the number of entities, and its size is equal to 8×(the number of our units + the number of enemy targets). The number of our units is currently set to 128, and the number of enemy targets is 16, so the neural network input dimension is 1152;
[0087] The hidden layer dimension is set to the number of red entities * the number of blue entities. The neural network calculates the degree of association between each friendly unit and the opponent's target, and each matrix of the number of friendly units × the number of opponent's targets is used to consider the number of unmanned underwater vehicles. Correspondingly, the output dimension of the neural network is 128×16. In addition, the loss function uses binary cross entropy loss.
[0088] (4) Training and reasoning of force grouping models
[0089] like Figure 4 As shown in the figure, the training and reasoning process of the target model is as follows:
[0090] According to the tactical target information, available force information of UUVs and target enemy information input by the user, the UUV force organization model first performs target allocation, allocates available UUVs to the target area or the opponent's target entity, and after completing the target allocation, generates the UUV organization allocation result; then matches the number and type of payloads to generate the payload configuration result; finally, the above results are integrated into a force organization plan.
[0091] The trained intelligent agent can be put into the actual process to complete the force grouping allocation and load configuration. During the reasoning process, the pre-stored neural network model is loaded first, and the parameters of the neural network model of the intelligent agent will not change.
[0092] (5) Output of the force grouping model
[0093] Input the force grouping situation information data into the pre-trained force grouping model to output the force grouping allocation and load configuration. The output of the neural network is numbers of different sizes and distributions. The output data needs to be mapped to obtain the final target allocation result. Each agent selects the opponent target with the greatest correlation, that is, the argmax method is used. Specifically, the force grouping allocation and load configuration information can be shown in Table 4:
[0094] Table 4 Force grouping distribution and load configuration information
[0095]
[0096] Among them, 'red_1', 'red_2', 'red_3', 'red_4' are the names of the unmanned underwater vehicles, 'red_group1', 'red_group2' are the names of the unmanned underwater vehicle groups, and 'load_1' is the type of the mounted weapon. 'red_1', 'red_2' form the group 'red_group1', 'red_3', 'red_4' form the group 'red_group2', 'red_1',
[0097] 'red_2', 'red_3', and 'red_4' all carry 10 'load_1' weapons.
[0098] As a new type of combat force, the formation of unmanned underwater vehicles needs to fully consider their performance characteristics, combat missions and coordination with other combat units. This embodiment can give full play to the advantages of unmanned underwater vehicles and improve the combat effectiveness of the entire combat system through reasonable formation.
[0099] 2. Autonomous decision-making stage of unmanned underwater vehicles
[0100] By constructing an intelligent unmanned underwater vehicle intelligent confrontation model, based on the assumption information generated in the first stage and the real-time situation information from the external simulation and deduction platform, the unmanned underwater vehicle autonomous decision-making agent can realize decision-making calculations under complex situations and improve the efficiency and level of confrontation. Specifically, the relevant assumption information corresponding to the data set at this stage includes the force grouping situation information force corresponding to the force grouping model and the grouping distribution and load configuration information output by the force grouping model. The specific implementation process of the unmanned underwater vehicle autonomous decision-making at this stage is as follows:
[0101] (1) Acquisition of real-time situation information
[0102] The real-time situation information includes: our force situation, enemy force situation and relative force situation between the enemy and us;
[0103] Specifically, the self-force situation and the enemy's force situation include: number, model, formation, name, minimum speed, maximum speed, three-dimensional position coordinates at the current moment, three-dimensional position coordinates at the previous moment, current speed, survival status, RCS, heading angle at the current moment and sonar detection information; the enemy's force situation includes: relative distance and relative angle. See Table 5 and Table 6 for details.
[0104] Table 5 Our / Enemy Force Situation Information
[0105]
[0106]
[0107] Table 6 Information on the relative situation of enemy and our forces
[0108] Serial number meaning Example 1 Relative distance {'5':139763.382,'14':147289.218} 2 Relative Angle {'5':22.43,'14':148.34}
[0109] The key situation information required by the intelligent confrontation model must be coded before it can be used. For the specific process, please refer to Figure 5 As shown in the figure, the one-hot encoder is used to encode the information of our force situation, the enemy force situation and the relative situation of our force and the enemy force respectively, and the situation codes of each category are obtained. Finally, the contents of each category are integrated to obtain the input vector that the network can understand.
[0110] (2) Construction of intelligent adversarial model
[0111] refer to Figure 6 , use the one-hot encoder to abstractly encode all situation information to obtain the input vector that the network can understand. The input vector size calculation formula is as follows:
[0112] Input vector dimension = single force situation dimension × (number of our forces + number of enemy forces) + relative situation dimension of our forces × number of our forces × number of enemy forces.
[0113] The maximum force data of both the enemy and our own forces are set to 16, so the input vector dimension is 1088, and a multi-layer fully connected neural network is set, the network size is 256×256, and the activation function is set to Relu. The output adopts a parallel output structure, which can include maneuvering actions, radar switch actions, and weapon launch actions, and the output dimension is 1×8.
[0114] (3) Training of Intelligent Adversarial Model
[0115] This process uses the PPO algorithm for deep reinforcement learning training. Figure 7 As shown in the figure, during training, a batch of samples are selected from the experience replay library for network parameter update. The PPO algorithm uses the Actor-Critic framework, which contains two networks. The Actor network update part defines the loss and performs pruning, uses a trust region correction method compatible with stochastic gradient descent, and simplifies the algorithm and reduces the need for adaptive correction by eliminating the KL loss.
[0116] (4) Strategy output of intelligent adversarial model
[0117] Obtain real-time situation information from an external simulation platform; input the real-time situation information and the combat assumptions determined in the previous stage into a pre-trained intelligent confrontation model, and output maneuvering actions, radar switch actions, and weapon launch actions.
[0118] Specifically, according to the typical confrontation scenario of unmanned underwater vehicles, the main control actions of each entity are analyzed as maneuvering, detection, and attack. Since maneuvering, attacking, and detection control can also be divided into different control granularities, in order to accelerate convergence, a hierarchical control structure is adopted. The maneuvering strategy selection, detection switch selection, and weapon switch selection are implemented by the reinforcement learning algorithm, and the specific maneuvering strategy, attack target, and weapon launch control are implemented by the underlying knowledge base.
[0119] Design the action of each entity into a multi-dimensional discrete form: [4,2,2].
[0120] Among them, the first dimension controls maneuvers, the second dimension controls radar switches, and the third dimension controls weapon launches. The details are as follows:
[0121] (4.1) Maneuvers
[0122] Since the nearest target poses the greatest threat during the confrontation, it is the most important factor to consider. Therefore, the nearest target is used as the benchmark for maneuver control, and four rule scripts are set relative to the nearest target: chase, left cut, right cut, and hide.
[0123] When outputting each action, reinforcement learning selects one of the four maneuvers in the first dimension according to the real-time situation. The values are 0, 1, 2, and 3, which represent the following maneuvers, respectively, as shown in Table 7:
[0124] Table 7 Maneuvers
[0125]
[0126] (4.2) Radar switch action
[0127] Since turning on radar detection will make it easier for the target to detect you, the decision of whether to turn the radar on or off is also an important factor.
[0128] The second dimension represents the two radar switch actions, with values of 0 and 1, representing turning on the detection switch and turning off the detection switch, respectively.
[0129] Table 8 Radar switch action
[0130]
[0131] (4.3) Weapon firing action
[0132] In the confrontation process, the timing of weapon launch is also an important control factor. The third dimension represents the two weapon launch actions, with values of 0 and 1, representing no action and launching the weapon, respectively.
[0133] Table 9 Weapon firing action
[0134]
[0135] It is worth noting that based on the triggering events of the red and blue sides (our side and the enemy side), the overall reward of the party is calculated. The specific reward design process can be referred to Figure 8 As shown in Table 10:
[0136] Table 10 Reward design
[0137]
[0138]
[0139] Force formation mainly involves the organization and configuration of combat forces, while unmanned submarine intelligent confrontation involves the autonomous decision-making and confrontation capabilities of unmanned submarines when performing tasks. In the context of force formation, unmanned submarines can be grouped according to mission requirements to form a cluster of unmanned submarines with a certain scale and combat capability. This cluster improves the overall combat effectiveness by performing multiple tasks; secondly, in terms of unmanned submarine intelligent confrontation, the intelligent system has a high degree of autonomous decision-making and confrontation capabilities, so that it can take effective countermeasures when facing enemy threats and ensure the smooth completion of the mission.
[0140] Therefore, in the combination of force organization and unmanned underwater vehicle intelligent confrontation, this embodiment fully considers the battlefield environment, combat missions and characteristics of enemy forces, and through reasonable organization and coordinated cooperation, effectively suppresses and strikes the enemy forces.
[0141] Example 3
[0142] This embodiment provides an unmanned underwater vehicle intelligent countermeasure device, which includes:
[0143] A first situation information acquisition module is configured to acquire force grouping situation information;
[0144] A force grouping generation module is configured to input the force grouping situation information into a pre-trained force grouping model and output force grouping allocation and load configuration;
[0145] A mission scenario module is configured to generate scenario mission information according to the force grouping situation information, force grouping allocation and load configuration;
[0146] A second situation information acquisition module is configured to acquire real-time situation information of an external simulation and deduction platform;
[0147] The intelligent confrontation strategy generation module is configured to input the real-time situation information and assumed mission information into a pre-trained intelligent confrontation model, and output maneuvering actions, radar switch actions and weapon launch actions.
[0148] Combination Figures 1 to 8 The unmanned underwater vehicle intelligent confrontation device can execute the unmanned underwater vehicle intelligent confrontation method introduced in the above-mentioned embodiment 1 or 2, and the specific functional implementation of each functional module will not be repeated.
[0149] Example 4
[0150] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the unmanned underwater vehicle intelligent confrontation method as described in Embodiment 1 or 2 are implemented.
[0151] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0152] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0153] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.
[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0155] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. An intelligent countermeasure method for an unmanned underwater vehicle, characterized in that: include: Obtain force formation situation information; Input the force grouping situation information into a pre-trained force grouping model, and output force grouping allocation and load configuration; generating hypothetical mission information according to the force grouping situation information, force grouping allocation and load configuration; Obtain real-time situation information from external simulation platforms; Input the real-time situation information and assumed mission information into a pre-trained intelligent confrontation model, and output maneuvering actions, radar switch actions, and weapon launch actions; The force formation situation information includes: tactical target information, available force information of unmanned underwater vehicles and target enemy information; the tactical target information includes: tactical targets, defense targets and strike targets; the tactical targets include defense operations, attack operations and both offense and defense; the available force information of unmanned underwater vehicles includes: the number, model, type, formation, name, minimum speed, maximum speed and three-dimensional position coordinates of our unmanned underwater vehicles at the current moment; the target enemy information includes: the number, model, type, formation, name and three-dimensional position coordinates of the target enemy at the current moment; The force grouping situation information is input into a pre-trained force grouping model, and force grouping allocation and load configuration are output, including: according to the input tactical target information, available force information of the unmanned underwater vehicle, and target enemy information, the force grouping model completes the target allocation by allocating corresponding unmanned underwater vehicles to the target area or the opponent's target entity, and generates the unmanned underwater vehicle grouping allocation result; the force grouping model matches the number and type of loads to generate the load configuration result; The maneuvering action, radar switch action and weapon launch action are in a multi-dimensional discrete form: [4, 2, 2]; the maneuvering action includes: pursuit, left cut, right cut and evasion; the radar switch action includes: turning on the radar switch and turning off the radar switch; the weapon launch action includes: no action and turning on the weapon launch switch; turning on the weapon launch switch includes: judging whether it is within the range of the weapon carried by our platform based on obtaining the surviving enemy target entity closest to us, and if so, launching the weapon to attack; otherwise, not launching.
2. The unmanned underwater vehicle intelligent confrontation method according to claim 1, characterized in that: Also includes: Preprocessing the force formation situation information, including extracting features from basic information and filling in missing data with zeros; Among them, feature extraction of basic information includes: The orientation, yaw angle, speed, three-dimensional position coordinates, unit type and strike load information of each unmanned underwater vehicle in the force formation situation information are extracted, the different basic information are encoded and mapped, and then spliced with the information of the unmanned underwater vehicle to form the input of the force formation model.
3. The unmanned underwater vehicle intelligent countermeasure method according to claim 1, characterized in that: The input layer dimension of the force grouping model is set to eight times the sum of the number of our units and the number of enemy targets; The hidden layer dimension is set to the product of the number of friendly entities and the number of enemy entities.
4. The unmanned underwater vehicle intelligent confrontation method according to claim 1, characterized in that: The real-time situation information includes: our force situation, enemy force situation and relative force situation between the enemy and us; The said our force situation and enemy force situation respectively include: number, model, formation, name, minimum speed, maximum speed, three-dimensional position coordinates at the current moment, three-dimensional position coordinates at the previous moment, current speed, survival status, RCS, heading angle at the current moment and sonar detection information; The relative force situation of the enemy and ourselves includes: relative distance and relative angle.
5. The intelligent countermeasure method for unmanned underwater vehicles according to claim 4 is characterized in that: A one-hot encoder is used to encode the information data corresponding to our force situation, enemy force situation, and enemy and our force situation respectively to obtain situation codes of various categories, and the contents of various codes are integrated to generate the input vector of the intelligent confrontation model; Among them, the input vector dimension = single force situation dimension × (number of our troops + number of enemy troops) + relative situation dimension of enemy and our troops × number of our troops × number of enemy troops.
6. The unmanned underwater vehicle intelligent confrontation method according to claim 5, characterized in that: Based on the triggering events of our side and the enemy side, the overall rewards of the party are calculated, including: If the detection payload is turned on, the reward is +5; if the strike payload is launched, the reward is +5; If the target is hit, the reward is +30, if the target is missed, the reward is -5; When the target attacks us, if we pursue, the reward is -10, if we cut left or right, the reward is 0, if we dodge, the reward is +10.
7. An intelligent countermeasure device for an unmanned underwater vehicle, characterized in that: include: A first situation information acquisition module is configured to acquire force grouping situation information; A force grouping generation module is configured to input the force grouping situation information into a pre-trained force grouping model and output force grouping allocation and load configuration; A mission scenario module is configured to generate scenario mission information according to the force grouping situation information, force grouping allocation and load configuration; A second situation information acquisition module is configured to acquire real-time situation information of an external simulation and deduction platform; An intelligent confrontation strategy generation module is configured to input the real-time situation information and the assumed mission information into a pre-trained intelligent confrontation model, and output maneuvering actions, radar switch actions, and weapon launch actions; The force formation situation information includes: tactical target information, available force information of unmanned underwater vehicles and target enemy information; the tactical target information includes: tactical targets, defense targets and strike targets; the tactical targets include defense operations, attack operations and both offense and defense; the available force information of unmanned underwater vehicles includes: the number, model, type, formation, name, minimum speed, maximum speed and three-dimensional position coordinates of our unmanned underwater vehicles at the current moment; the target enemy information includes: the number, model, type, formation, name and three-dimensional position coordinates of the target enemy at the current moment; The force grouping situation information is input into a pre-trained force grouping model, and force grouping allocation and load configuration are output, including: according to the input tactical target information, available force information of the unmanned underwater vehicle, and target enemy information, the force grouping model completes the target allocation by allocating corresponding unmanned underwater vehicles to the target area or the opponent's target entity, and generates the unmanned underwater vehicle grouping allocation result; the force grouping model matches the number and type of loads to generate the load configuration result; The maneuvering action, radar switch action and weapon launch action are in a multi-dimensional discrete form: [4, 2, 2]; the maneuvering action includes: pursuit, left cut, right cut and evasion; the radar switch action includes: turning on the radar switch and turning off the radar switch; the weapon launch action includes: no action and turning on the weapon launch switch; turning on the weapon launch switch includes: judging whether it is within the range of the weapon carried by our platform based on obtaining the surviving enemy target entity closest to us, and if so, launching the weapon to attack; otherwise, not launching.
8. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of the unmanned underwater vehicle intelligent confrontation method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Knowledge-driven hierarchical intelligent decision model training method and device
CN117291278A
