Smoothness adjustment method for aircraft control auxiliary prompt neural network strategy model
By retraining the auxiliary prompt neural network strategy model and adding internal excitation based on basic maneuver action count, the problems of unstable jump and poor smoothness of the output result are solved, improving user experience and performance in confrontation.
Patent Information
- Application Number
- CN202411175720.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-17
AI Technical Summary
The auxiliary prompts that the neural network strategy model is prone to problems such as unstable jumps and poor smoothness in the complex decision-making process, resulting in poor user experience.
By retraining the auxiliary prompt neural network strategy model, the internal excitation based on the basic maneuver action count is calculated and added to the original internal excitation, and deep reinforcement learning training is performed to improve the smoothness of the output results.
It effectively improves the smoothness of the output results of the auxiliary prompt neural network strategy model, improves the user experience, and maintains a high winning rate and danger avoidance ability in confrontation.
Smart Images
Figure CN119150947A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of aircraft intelligent auxiliary control design, and specifically relates to a method for adjusting the smoothness of an aircraft control auxiliary prompt neural network strategy model. Background Art
[0002] As artificial intelligence technology becomes increasingly mature, it is gradually being applied to aircraft control. Usually, an auxiliary prompt neural network strategy model is built into the aircraft control system. It takes aircraft situation information as input and outputs auxiliary prompt strategies to assist users in operating and controlling the aircraft.
[0003] The auxiliary prompt neural network strategy model is usually generated by deep reinforcement learning algorithm training and applies fuzzy thrust algorithm. With the development of modern avionics technology, more and more variables need to be considered in the model input, the decision-making process is becoming more and more complicated, the complexity of the fuzzy reasoning system is getting higher and higher, the number of rules and their sources are increasing, and there are often conflicts in variable names between different fuzzy reasoning rules, and confusion or even contradictions in reasoning results. In addition, due to the uninterpretability of the neural network itself and the instability of the network output, the auxiliary prompt neural network strategy model outputs the auxiliary prompt strategy often has the problem of unstable output result jumps and poor output smoothness, resulting in large losses in the application of the auxiliary prompt strategy and poor user experience.
[0004] This application is proposed in view of the above-mentioned technical defects. Summary of the invention
[0005] The purpose of the present application is to provide a method for adjusting the smoothness of an aircraft control auxiliary prompt neural network strategy model to overcome or alleviate at least one of the known technical deficiencies.
[0006] The technical solution of this application is:
[0007] A method for adjusting the smoothness of an aircraft control auxiliary prompt neural network strategy model, comprising:
[0008] The auxiliary prompt neural network strategy model is obtained through training;
[0009] Retrain the auxiliary prompt neural network strategy model;
[0010] When retraining the auxiliary prompt neural network strategy model, the basic maneuvering actions of the auxiliary prompt strategy output by the auxiliary prompt neural network strategy model in each game are counted, and then the internal incentive based on the basic maneuvering action count is calculated, and the internal incentive is added to the original internal incentive, and the auxiliary prompt neural network strategy model is trained by deep reinforcement learning.
[0011] According to at least one embodiment of the present application, in the above-mentioned aircraft control auxiliary prompt neural network strategy model smoothness adjustment method, the auxiliary prompt neural network strategy model is trained, specifically, the auxiliary prompt neural network strategy model is generated by training with a deep reinforcement learning algorithm.
[0012] According to at least one embodiment of the present application, in the above-mentioned aircraft control auxiliary prompt neural network strategy model smoothness adjustment method, the auxiliary prompt neural network strategy model takes aircraft situation information as input and outputs an auxiliary prompt strategy;
[0013] Aircraft situation information includes the speed and altitude of our aircraft, the speed and altitude of the enemy aircraft, and the angle between our aircraft and the enemy aircraft;
[0014] The auxiliary prompting strategies are given by experts.
[0015] According to at least one embodiment of the present application, in the above-mentioned aircraft control auxiliary prompt neural network strategy model smoothness adjustment method, the internal incentive based on the basic maneuver action count is calculated, specifically:
[0016] m = sigmoid(n) / a;
[0017] in,
[0018] m is internal motivation;
[0019] sigmoid() is the squeezing function;
[0020] n is the basic maneuver count;
[0021] a is a calculation constant.
[0022] According to at least one embodiment of the present application, in the above-mentioned aircraft control auxiliary prompt neural network strategy model smoothness adjustment method, when the auxiliary prompt neural network strategy model is retrained, an aircraft environment model is constructed, and the aircraft environment model is used to provide the situation environment of the aircraft, so that the auxiliary prompt strategy is used to control the aircraft to maneuver, and the degree of simulation is controlled.
[0023] This application has at least the following beneficial technical effects:
[0024] A method for adjusting the smoothness of an aircraft control auxiliary prompt neural network strategy model is provided. When the auxiliary prompt neural network strategy model is trained, the logic of the auxiliary prompt neural network strategy model itself is not increased. By adding internal incentives based on basic maneuvering action counts, the auxiliary prompt neural network strategy model is re-trained through deep reinforcement learning, the auxiliary prompt neural network strategy model is adjusted, the smoothness of the auxiliary prompt neural network strategy model output results is improved, and the user experience is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a schematic diagram of a smoothness adjustment method of an aircraft control auxiliary prompt neural network strategy model provided by an embodiment of the present application;
[0026] Figure 2 It is a schematic diagram of the auxiliary prompt neural network strategy model generated by directly training with a deep reinforcement learning algorithm, and outputs the jump instability of the auxiliary prompt strategy;
[0027] Figure 3 It is a schematic diagram of the degree of jump instability of the output auxiliary prompt strategy after the auxiliary prompt neural network strategy model is adjusted by the aircraft control auxiliary prompt neural network strategy model smoothness adjustment method provided in an embodiment of the present application.
[0028] In order to better illustrate the present embodiment, some contents of the drawings may be omitted, which is only used for illustrative purposes and should not be construed as limiting the present application. DETAILED DESCRIPTION
[0029] In order to make the technical solution and advantages of the present application clearer, the technical solution of the present application will be described in further detail in detail and in detail with reference to the accompanying drawings. It can be understood that the specific embodiments described here are only partial embodiments of the present application, which are only used to explain the present application, not to limit the present application. It should be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, and other related parts can refer to the general design.
[0030] In addition, unless otherwise defined, the technical terms or scientific terms used in the description of this application shall have the usual meanings understood by those of ordinary skill in the art to which this application belongs. The words used in the description of this application to indicate orientation are only used to indicate relative directions or positional relationships. When the absolute position of the described object changes, its relative positional relationship may also change accordingly. The word "include" used in the description of this application indicates that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, but does not exclude other elements or objects.
[0031] In addition, it should be noted that, unless otherwise clearly specified and limited, the words "installation", "connection" and similar terms used in the description of this application should be understood in a broad sense. For example, the connection can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. Technical personnel in the field can understand its specific meaning in this application according to the specific circumstances.
[0032] A method for adjusting the smoothness of an aircraft control auxiliary prompt neural network strategy model, such as Figure 1 shown.
[0033] The auxiliary prompt neural network strategy model is obtained by training.
[0034] The auxiliary prompt neural network strategy model is generated by training with deep reinforcement learning algorithm. The auxiliary prompt neural network strategy model takes the aircraft situation information as input and outputs an auxiliary prompt strategy. It can better assist users in operation, control the aircraft to maneuver, has good danger avoidance ability, and has a high winning rate in confrontation, meeting the winning rate requirements.
[0035] The aircraft situation information may specifically include the speed and altitude of our aircraft, the speed and altitude of the opponent's aircraft, and the angle between our aircraft and the opponent's aircraft.
[0036] The auxiliary prompting strategies are given by experts.
[0037] Retrain the auxiliary prompt neural network policy model.
[0038] When retraining the auxiliary prompt neural network strategy model, the basic maneuvering actions of the auxiliary prompt strategy output by the auxiliary prompt neural network strategy model in each game are counted, and then the internal incentive based on the basic maneuvering action count is calculated, and the internal incentive is added to the original internal incentive, and the auxiliary prompt neural network strategy model is trained by deep reinforcement learning.
[0039] The internal excitation based on the basic maneuver count is calculated as follows:
[0040] m = sigmoid(n) / a;
[0041] in,
[0042] m is internal motivation;
[0043] sigmoid() is the squeezing function;
[0044] n is the basic maneuver count;
[0045] a is a calculation constant, and the magnitude of m can be adjusted by a to adapt to the original internal excitation and take a value greater than 0.
[0046] When the auxiliary prompt neural network strategy model is retrained, an aircraft environment model is constructed to provide the situation environment of the aircraft, and the auxiliary prompt strategy is used to control the aircraft to maneuver and control the degree of simulation.
[0047] The auxiliary prompt neural network strategy model is retrained. By watching the aircraft maneuver replay or tracking the n-value curve, an auxiliary prompt neural network strategy model that can smoothly meet the user experience, has good danger avoidance ability, and maintains a high winning rate in confrontation can be obtained.
[0048] In a specific example, the auxiliary prompt neural network strategy model generated by deep reinforcement learning algorithm training is directly used to output the jump instability degree of the auxiliary prompt strategy, such as Figure 2 As shown, the smoothness adjustment method of the aircraft control auxiliary prompt neural network strategy model disclosed in the above embodiment is used to adjust the auxiliary prompt neural network strategy model, and the jump instability degree of the auxiliary prompt strategy is output, such as Figure 3 As shown, the smoothness is significantly improved.
[0049] The method for adjusting the smoothness of the aircraft control auxiliary prompt neural network strategy model disclosed in the above embodiment, after completing the training of the auxiliary prompt neural network strategy model, does not increase the logic of the auxiliary prompt neural network strategy model itself, but re-performs deep reinforcement learning training on the auxiliary prompt neural network strategy model by adding internal incentives based on basic maneuvering action counts, adjusts the auxiliary prompt neural network strategy model, improves the smoothness of the output results of the auxiliary prompt neural network strategy model, and effectively improves the user experience.
[0050] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. Those skilled in the art should understand that the scope of protection of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the scope of protection of the present application.
Claims
1. A method for adjusting the smoothness of a neural network strategy model for aircraft control assistance prompts. It is characterized in that include: The auxiliary prompt neural network strategy model is obtained through training; Retrain the auxiliary prompt neural network strategy model; When retraining the auxiliary prompt neural network strategy model, the basic maneuvering actions of the auxiliary prompt strategy output by the auxiliary prompt neural network strategy model in each game are counted, and then the internal incentive based on the basic maneuvering action count is calculated, and the internal incentive is added to the original internal incentive, and the auxiliary prompt neural network strategy model is trained by deep reinforcement learning.
2. The method for adjusting the smoothness of the aircraft control auxiliary prompt neural network strategy model according to claim 1, It is characterized in that The auxiliary prompt neural network strategy model is obtained by training, specifically, the auxiliary prompt neural network strategy model is generated by training with a deep reinforcement learning algorithm.
3. The smoothness adjustment method of the aircraft control auxiliary prompt neural network strategy model according to claim 1, It is characterized in that The auxiliary prompt neural network strategy model takes aircraft situation information as input and outputs auxiliary prompt strategy; Aircraft situation information includes the speed and altitude of our aircraft, the speed and altitude of the enemy aircraft, and the angle between our aircraft and the enemy aircraft; The auxiliary prompting strategies are given by experts.
4. The method for adjusting the smoothness of the aircraft control auxiliary prompt neural network strategy model according to claim 1, It is characterized in that The internal excitation based on the basic maneuver count is calculated as: m = sigmoid(n) / a; in, m is internal motivation; sigmoid() is the squeezing function; n is the basic maneuver count; a is a calculation constant.
5. The method for adjusting the smoothness of the aircraft control auxiliary prompt neural network strategy model according to claim 1, It is characterized in that When the auxiliary prompt neural network strategy model is retrained, an aircraft environment model is constructed to provide the situation environment of the aircraft, and the auxiliary prompt strategy is used to control the aircraft to maneuver and control the degree of simulation.