Method for constructing and deploying large model-based equipment master control intelligent decision module

Abstract

The present application relates to the technical field of intelligent control and reinforcement learning, and in particular to a method for constructing and deploying an equipment master control intelligent decision module based on a large model. The method comprises the following steps: constructing real equipment, semi-physical simulation, high-precision simulation multi-source data and completing abnormality screening, generating a multi-source weighted data set based on a confidence quantification formula; differentially encoding core scene features and mapping them to a high-dimensional semantic space; building a multi-task Transformer architecture containing a basic framework, multi-dimensional embedding, task prompt enhancement and an independent prediction head, and introducing a parameter isolation mechanism; pre-training using the BCQ algorithm, combining gradient propagation control and gradient similarity alignment to complete multi-task fine-tuning; realizing model lightweight through knowledge distillation and quantization, deploying to an equipment master control system and constructing a closed-loop decision mechanism. The advantages are: improving decision accuracy and generalization ability, accelerating training convergence speed, ensuring deployment real-time performance and stability, and adapting to the intelligent upgrading needs of equipment.

Description

Technical Field

[0001] This invention relates to the field of equipment intelligent control and reinforcement learning technology, and in particular to a method for constructing and deploying an equipment master control intelligent decision-making module based on a large model. Background Technology

[0002] With the increasing intelligence level of equipment, traditional decision-making systems that rely on manually preset rules or simple algorithms can no longer meet the demands. Current development of equipment decision-making technology focuses on two core directions: first, data-driven approaches, which involve collecting equipment operation data and simulation data to train models and using machine learning algorithms to optimize decision-making strategies; however, this approach faces bottlenecks such as "single data types and insufficient scenario coverage"; second, multi-task collaboration approaches, which attempt to enable models to simultaneously handle tasks such as target locking, collaborative protection, and evasion. However, limited by traditional CNN and RNN model architectures, it is difficult to effectively separate and fuse features from multiple tasks, resulting in poor generalization performance.

[0003] The existing equipment decision-making system and related technologies have the following significant shortcomings: 1. Shortcomings in Data Sources and Training Mechanisms: Existing equipment decision-making systems rely on a single data source (such as relying solely on actual equipment data or simulation data), resulting in limited data coverage and insufficient sample size. This makes it impossible to support the training of intelligent decision-making models in complex dynamic environments, limiting the decision-making efficiency and scenario generalization performance of the models. In related technologies for multi-source data training, a weight allocation mechanism adapted to algorithm training has not been established, and the contribution of high-confidence data to model parameter updates has not been strengthened through weight differences. This leads to low-confidence data interfering with training, making it difficult to improve model decision accuracy. The core of offline reinforcement learning algorithm training lies in the fact that "data quality determines model performance." High-confidence data (such as actual equipment data reflecting the real state of equipment) contains more reliable decision-making experience, while low-confidence data (such as high-precision simulation data with simulation bias) is prone to introducing noise. The lack of data weight allocation further restricts model performance. Simultaneously, the input of multi-source heterogeneous data (actual equipment, semi-physical simulation, and high-precision simulation data) suffers from format incompatibility and difficulty in feature fusion, resulting in low model training efficiency.

[0004] 2. Model architecture cannot adapt to multi-task decision-making needs: Traditional decision-making models (such as those based on rule engines and shallow neural networks) struggle to capture the sequential dependencies of "historical state-action-current decision" during the decision-making process, exhibiting weak multi-task generalization ability and failing to simultaneously adapt to various decision-making tasks such as target locking, collaborative protection, and avoidance. Some decision-making models built using recurrent neural networks (RNNs) attempt to capture the correlation between historical states and current decisions by leveraging the temporal characteristics of RNNs, but suffer from the "vanishing gradient problem" in long sequences, failing to effectively capture the correlation between long-term historical states (such as equipment status over the past 100 steps) and current decisions. Furthermore, these models typically design dedicated output layers only for a single task (such as target locking), requiring redesign and retraining of the model structure for multi-task scenarios, resulting in high development costs and weak generalization ability.

[0005] 3. Multi-task training and deployment face technical bottlenecks: Gradient conflicts (inconsistent gradient update directions across different tasks) are prone to occur during multi-task training, leading to difficulties in model convergence and decreased decision accuracy. Furthermore, existing technologies lack gradient coordination mechanisms, resulting in slow model convergence during multi-task training (2-3 times faster than single-task training). Simultaneously, existing models have high computational complexity during deployment, lacking lightweight optimization, making it difficult to meet the stringent latency requirements of equipment control systems for real-time decision-making, and thus unsuitable for practical real-time applications.

[0006] 4. Unmanned operation requirements are difficult to meet: Existing equipment control systems rely on manual operation for core decision-making actions (such as target selection and manual single-lever target search), resulting in slow decision response and the risk of human error, thus failing to achieve unmanned operation. The traditional "manual decision-making + main control execution" model cannot adapt to highly dynamic scenarios due to human response delays (usually ≥1s), and the decision output dimension is singular, only providing task initiation instructions. There is no standardized interaction interface with the equipment control system, and decision instructions must be manually forwarded to the main control system for execution, failing to form a fully automated closed loop of "perception-decision-execution".

[0007] Meanwhile, advancements in related technologies have provided feasible paths for optimizing equipment decision-making systems: reinforcement learning technology is maturing in the decision-making field, and offline reinforcement learning avoids the dependence on real-time data for online training by using historical datasets to train models; the Transformer architecture, with its powerful sequence dependency capture capabilities, is widely used in natural language processing, autonomous driving decision-making, and other fields, providing an architectural reference for equipment decision-making. Furthermore, the unmanned operation of equipment has become a development trend, urgently requiring intelligent decision-making modules to replace manual decision-making and overcome the technical bottlenecks of traditional decision-making systems. Therefore, it is urgent to propose an intelligent decision-making module construction and deployment scheme that balances data reliability, model adaptability, training efficiency, and real-time deployment, breaking through traditional technical limitations and supporting the intelligent and unmanned upgrade of equipment main control systems. Summary of the Invention

[0008] To address the aforementioned problems, this invention provides a method for constructing and deploying an intelligent decision-making module for equipment control based on a large model.

[0009] The purpose of this invention is to provide a method for constructing and deploying an intelligent decision-making module for equipment control based on a large model, which specifically includes the following steps: S1. Collect data from multiple sources and remove outliers from each source; then, based on the matching degree between the data and the real scene and the stability of the data, calculate the confidence level of each type of data and assign weights to form a multi-source weighted dataset. S2. Differentiatedly encode the core task scenario features and concatenate them into a feature vector of a unified dimension. Then, through the embedding layer, map the feature vector to a high-dimensional semantic space and output a high-dimensional feature vector. S3. Build a multi-task Transformer large model architecture, including a task prompting enhancement module, a basic architecture module, a multi-dimensional embedding module, and a task flow output layer; through multi-dimensional embedding, high-dimensional feature vectors are uniformly mapped to the model adaptation dimension to obtain enhanced feature vectors containing task prior knowledge; after the enhanced feature vectors are input into the basic architecture module to complete the core feature calculation, they are distributed to each independent prediction head of the task flow output layer to output the state prediction results and action prediction results that match the task; S4. The model is pre-trained using an offline reinforcement learning algorithm, and then fine-tuned by inputting encoded multi-task data. The gradient coordination mechanism is used to improve the model's decision-making accuracy and training stability. S5. Lighten the trained model and deploy the lightweight model to the equipment main control system to realize intelligent decision-making of the equipment main control.

[0010] Preferably, the multi-source data in step S1 includes physical data, semi-physical simulation data, and high-precision simulation data; The specific methods for removing abnormal data include: collecting actual data from the equipment sensors to remove abnormal values ​​caused by sensor malfunctions; simulating N types of scenarios using a hardware-in-the-loop simulation platform to generate hardware-in-the-loop simulation data and removing noise data from the hardware-in-the-loop simulation data; and generating high-precision simulation data for M types of scenarios based on a digital twin model to remove logically contradictory data where the target speed exceeds the physical limit. The weighting formula is: Weight = Matching degree × 0.6 + Stability × 0.4; Among them, matching degree represents the degree of matching between the data and the real scene, and stability represents the stability of the data; the weight of the actual data is 0.5, the weight of the semi-physical simulation data is 0.3, and the weight of the high-precision simulation data is 0.2.

[0011] Preferably, the core task scenario features in step S2 include the type of incoming target, the direction of attack, the attributes of the cooperating object, and the priority of the protected target; The differential encoding specifically includes: the incoming target type is converted into a 10-dimensional vector using one-hot encoding; the incoming direction is converted into a 2-dimensional vector using normalized encoding; the cooperative object attributes are converted into a 16-dimensional vector using embedding encoding; the protection target priority is converted into a 1-dimensional vector using numerical mapping; the encoded data are then concatenated to form a 30-dimensional unified vector; the input to the embedding layer is a 30-dimensional unified vector, the hidden layer dimension is set to 128 dimensions and a ReLU activation function is used, and the output is a 128-dimensional high-dimensional feature vector.

[0012] Preferably, step S3 also includes a task parameter isolation mechanism, as follows: all task-specific embedding layers and prediction layers are managed through the ModuleDict container, and parameter isolation is achieved by using the task name as the key; during training, only the corresponding layer parameter update of the current iteration task is activated, and the parameters of other tasks are frozen by setting requires_grad=False to avoid parameter interference between tasks.

[0013] Preferably, the basic architecture module in step S3 uses GPT-2 as the core basic model, and switches the adaptation mode through the special_embedding switch parameter: when the parameter is True, independent embedding layers and prediction layers are configured for different tasks to adapt to heterogeneous tasks; when the parameter is False, each task shares the layer structure to adapt to homogeneous tasks. The multi-dimensional embedding module includes a state embedding layer, an action embedding layer, a cumulative reward embedding layer, a time step embedding layer, and a type embedding layer, with each embedding layer outputting a uniform 512 dimensions. The task prompt enhancement module sets up a prompt state embedding layer and a prompt action embedding layer, and the two-layer structure is completely consistent with the state embedding layer and action embedding layer in the multi-dimensional embedding module; the task-specific prompt information is a fragment of the task's historical best decision trajectory, spliced ​​at the front of the input sequence; The independent prediction head includes a state prediction layer and an action prediction layer. The state prediction layer adopts a linear layer structure with an input dimension of 512 dimensions and an output dimension that matches the original state dimension of the task. The action prediction layer adopts a two-layer MLP architecture. The first layer has an input dimension of 512 dimensions and an output dimension of 256 dimensions, and the second layer has an input dimension of 256 dimensions and an output dimension that matches the action dimension of the task. The GELU activation function is used between the two layers.

[0014] Preferably, the input dimension of the state embedding layer is 128-dimensional, which is consistent with the dimension of the high-dimensional feature vector output in step S2; The input dimension of the action embedding layer matches the task action space dimension; The time-step embedding layer uses sine and cosine position coding to capture temporal dependencies; The type embedding layer is used to distinguish the element types in the sequence.

[0015] Preferably, the multi-source data in step S1 includes physical data, semi-physical simulation data, and high-precision simulation data; the multi-source weighted dataset includes 50% physical data, 30% semi-physical simulation data, and 20% high-precision simulation data. The offline reinforcement learning algorithm in step S4 is the BCQ algorithm; The pre-training process consists of 100 rounds, and the reward function used is: reward value = decision accuracy × 0.7 + decision delay coefficient × 0.3.

[0016] Preferably, the gradient coordination mechanism in step S4 includes gradient flow propagation control and gradient similarity alignment loss; The gradient flow propagation control specifically involves setting up one gradient propagation controller every two Trunk layers in the Transformer model. The controller optimizes the gradient path by scaling the gradient. The gradient similarity alignment loss is specifically as follows: by calculating the cosine similarity of gradients for different tasks, when the similarity is <0.6, the gradient direction is corrected by adjusting the coefficients of the loss function to constrain the consistency of gradients for different tasks.

[0017] Preferably, the lightweight processing in step S5 adopts knowledge distillation and quantization; knowledge distillation uses the trained large model as the teacher model, and the training parameter scale is 1 / 5 of the teacher model's student model; quantization adopts dynamic quantization, which quantizes the model parameters from 32-bit floating-point numbers to 16-bit floating-point numbers, thereby reducing the amount of computation.

[0018] Compared with the prior art, the present invention can achieve the following beneficial effects: 1. Enhancing Data Utilization Reliability and Scene Coverage Completeness: This invention overcomes the limitations of existing technologies that only use two types of data sources with equal weights for training. It innovatively integrates three types of multi-source data: real-world data, semi-physical simulation data, and high-precision simulation data. Data confidence is evaluated using a quantitative formula of "data-to-real-scene matching degree × 0.6 + data stability × 0.4," assigning differentiated weights of 0.5, 0.3, and 0.2 to the three types of data. This design not only expands the scene coverage through the complementarity of multi-source data but also reduces the interference of low-confidence data on the training process by strengthening the weight of high-confidence data (real-world data), providing high-quality data support for accurate model decision-making.

[0019] 2. Enhanced Multi-Task Adaptability and Generalization Performance: This invention abandons the existing design of RNN architecture, which only supports single-task decision-making. Instead, it adopts the Transformer (GPT-2) architecture, combined with parameterized task adaptation switches (special_embedding), task-specific hint mechanisms, and independent prediction heads, to achieve flexible adaptation and synchronous decision-making across multiple tasks. This eliminates the need for reconstructing the model structure and performing full training for different tasks, significantly reducing development costs. Simultaneously, it effectively improves the model's generalization ability in multi-task scenarios, adapting to more diverse equipment control task requirements.

[0020] 3. Accelerate model training convergence and improve training efficiency: This invention optimizes the gradient propagation path through Gradient Flow Transformer (GFT) to avoid gradient vanishing or exploding problems; simultaneously, it introduces Gradient Similarity Alignment Loss (SPMLoss) to constrain gradient consistency across different tasks, alleviating gradient conflicts in multi-task training. Compared to existing multi-task offline reinforcement learning algorithms, this invention significantly improves model convergence speed, greatly shortens the training cycle, and reduces model development and iteration costs.

[0021] 4. Ensuring Real-Time Deployment and Adapting to the Actual Needs of Equipment Control Systems: This invention employs a lightweight strategy combining knowledge distillation (training a student model with only 1 / 5 the size of the original model) and FP16 dynamic quantization, reducing the single-step decision latency of the model to 85ms. This significantly outperforms existing technologies and fully meets the stringent real-time decision-making requirements of equipment control systems. Furthermore, through standardized decision command frame design and industrial Ethernet protocol adaptation, efficient data interaction between the model and the equipment control system is achieved, ensuring the feasibility and stability of engineering deployment.

[0022] In summary, this invention significantly improves the decision-making accuracy, generalization ability, training efficiency, and real-time response performance of the equipment main control intelligent decision-making module. It is also better suited to the intelligent and unmanned upgrade needs of the equipment main control system in complex dynamic environments, and has outstanding practical value and promotion prospects. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method for constructing and deploying an equipment main control intelligent decision-making module based on a large model, according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the multi-task Transformer large model structure provided in an embodiment of the present invention.

[0025] Figure 3 This is a gradient coordination flowchart provided according to an embodiment of the present invention. Detailed Implementation

[0026] In the following description, embodiments of the invention will be described with reference to the accompanying drawings. In the description below, the same modules are denoted by the same reference numerals. Where the same reference numerals are used, their names and functions are also the same. Therefore, their detailed description will not be repeated.

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0028] To address the shortcomings of existing equipment decision-making systems in terms of data, models, training, deployment, and unmanned adaptation, this invention provides a method for constructing and deploying an intelligent decision-making module for equipment master control based on a large model. By introducing semi-physical simulation data to construct a multi-source heterogeneous data system, establishing a differentiated weight allocation mechanism based on data confidence, designing a Transformer model architecture that supports multi-task feature separation and fusion, adding a multi-task gradient coordination mechanism, and performing lightweight optimization of the model, while constructing a standardized interactive interface with the equipment master control system, the following objectives are ultimately achieved: (1) Improve data utilization efficiency and model decision-making accuracy, and solve the problem of low-confidence data interfering with training; (2) Achieve unified modeling and efficient decision-making for multiple tasks such as target locking, collaborative protection, and avoidance, and improve the model's generalization ability; (3) Accelerate the model convergence speed during multi-task training and reduce deployment latency to meet real-time requirements; (4) Replace manual core decision-making actions, forming a fully automatic closed loop of "perception-decision-execution" to adapt to the unmanned equipment requirements.

[0029] See flowchart Figure 1 Specifically, it includes the following steps: S1. Construction and weight allocation of multi-source datasets: Collect real-world data, semi-physical simulation data and high-precision simulation data, and remove outlier data respectively. Then, calculate the confidence of each type of data based on the matching degree between the data and the real scene and the data stability, and assign weights to form a multi-source weighted dataset. S11. Multi-source data acquisition and filtering: Acquire real-world data, semi-physical simulation data, and high-precision simulation data, and remove outlier data; Specifically, by equipping sensors to collect real-world data, abnormal values ​​caused by sensor malfunctions are removed; a hardware-in-the-loop simulation platform is used to simulate N types of scenarios to generate hardware-in-the-loop simulation data, and redundant noise data caused by electromagnetic interference is removed from the hardware-in-the-loop simulation data; high-precision simulation data for M types of scenarios is generated based on digital twin models, and logical inconsistencies such as target speeds exceeding physical limits are removed. In some embodiments, the methods for collecting and filtering the three types of data are as follows: (1) Real-world data: The equipment's operating data over a period of time is collected using equipment sensors such as radar and photoelectric detectors. The 3σ principle or sliding window filtering method is used to remove abnormal values ​​caused by sensor failures. (2) Hardware-in-the-loop simulation data: Data is generated by simulating N types of scenarios using a hardware-in-the-loop simulation platform that includes physical equipment components and a virtual environment. Low-pass filtering or threshold filtering methods are used to remove noise data (such as redundant data caused by electromagnetic interference). (3) High-precision simulation data: M types of scenario data are generated based on a digital twin model. The speed threshold range of the corresponding equipment is preset to remove logically contradictory data (such as the target speed exceeding the physical limit).

[0030] S12. Data Confidence Assessment and Weighting: The confidence of various types of data is calculated by weighting "data matching degree with real scene + data stability". The weight of each type of data is determined according to the weight formula: weight = matching degree × 0.6 + stability × 0.4; where the weight of real-world data is 0.5, the weight of semi-physical simulation data is 0.3, and the weight of high-precision simulation data is 0.2, and the total weight is 1, thus forming a multi-source weighted dataset. In some embodiments, the matching degree between the data and the real scene is: 95% for real-world data, 85% for semi-physical simulation, and 70% for high-precision simulation; Data stability is as follows: 90% for actual data, 80% for semi-physical simulation, and 65% for high-precision simulation.

[0031] S2. Scene Feature Encoding and Embedding Mapping: The core task scene features are differentially encoded and concatenated into a feature vector of a unified dimension. Then, the feature vector is mapped to a high-dimensional semantic space through an embedding layer to output a high-dimensional feature vector. S21. Encode the features of the four core task scenarios, and concatenate the encoded features into a 30-dimensional unified vector; Specifically, the type of incoming target (such as missiles or drones) is converted into a 10-dimensional vector using one-hot encoding; Incoming direction (e.g., azimuth 0°-360°, elevation -10°-90°): converted into a 2D vector using normalized encoding; Collaboration object attributes (such as friendly equipment model and distance): are converted into 16-dimensional vectors using embedding encoding; Protection target priority (e.g., high, medium, low): Transformed into a 1-dimensional vector using numerical mapping (3→high, 2→medium, 1→low); In some embodiments, the normalization encoding of the incoming direction adopts a linear normalization method, mapping the azimuth angle 0°~360° and the elevation angle -10°~90° to the [0,1] interval respectively; the embedding encoding of the collaborative object attribute is implemented by end-to-end training with the model, the input dimension of the embedding layer is consistent with the feature dimension of the collaborative object attribute, and the output dimension is 16-dimensional. S22. Embedding layer design: Construct a feature embedding layer, input a 30-dimensional uniform vector, set the hidden layer dimension of the embedding layer to 128 dimensions, and use the ReLU activation function to map the 30-dimensional feature vector to a 128-dimensional high-dimensional semantic space, providing high-quality input features for the subsequent Transformer model.

[0032] S3. Build a multi-task Transformer large model architecture, sequentially constructing a task prompting enhancement module, a basic architecture module, a multi-dimensional embedding module, and a task flow output layer; using GPT-2 as the base model and completing the core parameter configuration, high-dimensional feature vectors are uniformly mapped to the model adaptation dimension through multi-dimensional embedding, and task-specific prompting mechanisms are combined with task prior knowledge to obtain enhanced feature vectors; the enhanced feature vectors are input into the basic architecture module to complete the core feature calculation, and then distributed to the independent prediction heads of the task flow output layer through the task parameter isolation mechanism, outputting state prediction results and action prediction results that match the task, avoiding parameter interference from multi-task training; See Figure 2 The Transformer model consists of the following components and hierarchical relationships: The task prompt enhancement module sets up a prompt state embedding layer and a prompt action embedding layer with a structure completely consistent with the state embedding layer and the action embedding layer, and is dedicated to processing prompt data. The role of the task prompt enhancement module is to introduce prior knowledge of the task, enhance the task adaptability, and embed task-specific prompts. The multi-dimensional embedding module includes a state embedding layer, an action embedding layer, a cumulative reward embedding layer, a time step embedding layer, and a type embedding layer. The state embedding layer has an input dimension of 128 dimensions, which is consistent with the output dimension of the embedding layer in step S22. The input dimension of the action embedding layer matches the dimension of the task action space. The cumulative reward embedding layer has an input dimension of 1 dimension. The time step embedding layer uses sine and cosine position encoding to capture temporal dependencies. The type embedding layer is used to distinguish the element types of "reward (R)-state (S)-action (A)" in the sequence. The basic architecture module uses GPT-2 as the core basic model, with a hidden layer dimension of 512, 8 attention heads, and 6 network layers. The "heterogeneous task independent layer" or "homogeneous task shared layer" mode can be flexibly switched through the special_embedding switch parameter. The task flow output layer includes multiple independent prediction heads, which are used to output decision results according to task flow. Each independent prediction head includes a state prediction layer and an action prediction layer. The state prediction layer adopts a linear layer structure, with a 512-dimensional vector as input and the output dimension matching the original state dimension of the task. The action prediction layer adopts a two-layer MLP architecture (512-dimensional → 256-dimensional → task action dimension), with the GELU activation function used in between to adapt to the continuous action space. In some embodiments, a parameter isolation management layer may also be included, which manages all task-specific embedding and prediction layers through a ModuleDict container, and stores parameters independently using the task name as the key.

[0033] Specifically, the steps include the following: S31. Core Architecture Selection and Configuration: Select a suitable basic model architecture and design parameterized task adaptation switches to achieve flexible compatibility with homogeneous and heterogeneous tasks, providing an architectural foundation for subsequent multi-task decisions; specifically including: The GPT-2 architecture is used as the basic decision model. The hidden layer dimension (hidden_size) is set to 512, the number of attention heads (num_heads) is 8, and the number of network layers (num_layers) is 6. The task adaptation mode is controlled by the special_embedding parameter. When the parameter is True, independent embedding and prediction layers are configured for different tasks to adapt to heterogeneous tasks. When the parameter is False, different tasks share the layer structure to adapt to homogeneous tasks.

[0034] As an alternative implementation of this step, a hybrid architecture of "Transformer + CNN" can be used to build the basic decision model: CNN is used to extract static features of the equipment (such as target shape features), and Transformer is used to extract temporal features (such as historical state sequences). The two types of features are then concatenated and input into the task prediction head. The advantage of this alternative is a 5% improvement in the accuracy of static feature extraction; the disadvantage is a 15% increase in model complexity and a deployment latency of 110ms, requiring further optimization to meet the real-time requirement of ≤100ms for the equipment's main control system. In contrast, the pure Transformer architecture selected in this invention better balances decision accuracy and real-time requirements.

[0035] S32. Construction of a Multi-Dimensional Embedding System: Based on the model input dimension requirements determined in S31, a multi-dimensional embedding system is constructed to uniformly map heterogeneous inputs to a low-dimensional vector space, eliminating the differences between different types of inputs and providing a standardized input foundation for large Transformer models; specifically including: A multi-dimensional embedding system is constructed, comprising state embedding (embed_state), action embedding (embed_action), cumulative reward (RTG) embedding (embed_return), timestep embedding (embed_timestep), and type embedding (embed_type). The state embedding layer has an input dimension of 128 dimensions, consistent with the output dimension of the embedding layer in step S22. The action embedding layer has an input dimension matching the task action space dimension. The cumulative reward embedding layer has an input dimension of 1 dimension. The timestep embedding layer uses sine and cosine positional encoding to capture temporal dependencies. The type embedding layer is used to distinguish the element types of "reward (R)-state (S)-action (A)" in the sequence. The output dimension of each embedding layer is uniformly 512 dimensions, realizing a unified mapping of heterogeneous inputs.

[0036] S33. Enhanced Task-Specific Hint Mechanism: Building upon the standardized input in S32, prior task knowledge is introduced to improve the model's adaptability to different tasks. Task-specific hint information is appended before the input sequence, including fragments of the historical best decision trajectory for that task. A hint state embedding layer (prompt_embed_state) and a hint action embedding layer (prompt_embed_action) with structures identical to those in step S32 are set up to specifically process the hint data, ensuring semantic consistency between the hint information and the original input in the vector space. This helps the model quickly locate the core features of the task and reduces ineffective learning.

[0037] S34. Independent prediction head design: Based on the adaptation requirements of heterogeneous tasks (step S31), an independent prediction module is customized for each task to achieve accurate output; specifically, an independent prediction head containing a state prediction layer and an action prediction layer is configured for different tasks: (1) The state prediction layer (predict_state) adopts a linear layer structure, and the output dimension matches the original task state dimension; (2) The action prediction layer (predict_action) adopts a two-layer MLP architecture (including GELU activation function) to enhance the fitting ability of continuous action space, and the output dimension is consistent with the task action dimension; In some embodiments, the state prediction layer adopts a linear layer structure with an input dimension of 512 dimensions (matching the output dimension of S32) and an output dimension consistent with the original state dimension of the corresponding task; the action prediction layer adopts a two-layer MLP architecture, with the first layer having an input dimension of 512 dimensions and an output dimension of 256 dimensions, and the second layer having an input dimension of 256 dimensions and an output dimension matching the task action dimension. The GELU activation function is used between the two layers to enhance the fitting ability to the continuous action space.

[0038] Step S3 also includes a task parameter isolation mechanism, as follows: all task-specific embedding and prediction layers are managed through the ModuleDict container, and parameter isolation is achieved by using the task name as the key; during training, only the corresponding layer parameter update of the current iteration task is activated, and the parameters of other tasks are frozen by setting requires_grad=False to avoid parameter interference between tasks.

[0039] In summary, the principle behind this step is as follows: In multi-task reinforcement learning scenarios, the state dimension (state_dim) and action dimension differ significantly across tasks. Forcing the same set of embedding and prediction layers will cause the model to struggle to adapt to the uniqueness of each task, leading to a generalization bottleneck. Therefore, this framework designs independent prediction heads, assigning dedicated embedding and prediction layers to each task to achieve accurate adaptation to heterogeneous tasks. The model's neural network architecture is based on the Transformer (GPT-2 architecture) decision model (PDT), integrating multi-task prompting mechanisms and gradient optimization strategies.

[0040] S4. Multi-task gradient coordination training: The model is pre-trained using an offline reinforcement learning algorithm, and then fine-tuned by inputting encoded multi-task data. A gradient coordination mechanism is used to improve the model's decision-making accuracy and training stability. This gradient coordination mechanism includes gradient propagation control and gradient similarity alignment. Specifically, it includes: S41. Offline reinforcement learning pre-training: Offline reinforcement learning algorithm (BCQ, Batch-ConstrainedQ-Learning) is used for pre-training; The reward function is designed as follows: Reward value = Decision accuracy × 0.7 + Decision delay coefficient × 0.3. Maximizing the cumulative reward value achieves the decision-making goal of high accuracy and low delay. Decision accuracy It is the matching ratio between the model's current decision action and the true optimal action / expert-labeled action, calculated using the following formula: ; In the formula, This represents the total number of decision samples in the current batch / round; This indicates the number of samples in the current batch / round where the model decision is consistent with the standard optimal decision; the optimal standard decision comes from the optimal standard decision information in the corresponding decision scenario in the multi-source weighted dataset (which can be obtained through the labels inherent in the data itself or extracted based on the characteristics of high-quality data). Since lower latency is better, it needs to be normalized to [0,1] and positively correlated with reward (lower latency, larger coefficient, higher reward). A max-min normalization inverse mapping is used; the decision latency coefficient... The calculation formula is as follows: ; In the formula, This indicates the actual time taken for a single decision. This represents the minimum decision delay among all samples; This represents the maximum decision delay across all samples. The model was pre-trained for 100 epochs using the multi-source weighted dataset obtained from S1. The batch size was set to 64, the learning rate was 1e-4, and the Adam optimizer was used. After training, the model's decision accuracy reached 75%, achieving initial convergence.

[0041] S42. Multi-task fine-tuning and gradient coordination: Input the three types of task data encoded in S2 for multi-task fine-tuning, and at the same time use a gradient coordination mechanism to ensure training effect; (1) Gradient flow propagation (GFT) control: A gradient propagation controller is set every 2 layers in the Trunk layer of the Transformer model. The controller optimizes the gradient path by gradient scaling. In some embodiments, when the absolute value of the gradient is greater than 5, the gradient is scaled down to 5 to avoid gradient vanishing or exploding problems. (2) Gradient similarity alignment loss (SPMLoss): Calculate the cosine similarity of gradients of different tasks; when the similarity is <0.6, adjust the gradient direction by adjusting the coefficient of the loss function (the adjustment coefficient is set to 1.0) to constrain the consistency of gradients of different tasks; After fine-tuning, the model's decision accuracy improved to 92%, and the convergence speed was 40% faster than the solution without gradient coordination. The gradient coordination flowchart is as follows: Figure 3 As shown.

[0042] As an alternative implementation of the gradient coordination mechanism in this step, a combination strategy of "gradient pruning + task priority weight" can also be adopted: parameters whose absolute gradient value exceeds the threshold (such as 1.0) are pruned, while high-priority tasks (such as target locking tasks) are assigned a gradient weight of 0.6 and low-priority tasks are assigned a gradient weight of 0.4.

[0043] The advantages of this alternative are its simplicity and 20% reduction in development cycle; however, its disadvantages include limited gradient conflict mitigation and 8% lower multi-task decision-making accuracy compared to the GFT+SPMLoss scheme of this invention. Therefore, the gradient coordination mechanism of this invention is more suitable for high-precision equipment decision-making scenarios.

[0044] S5. Model Lightweighting and Deployment: The trained model is lightweighted and deployed to the equipment's main control system, constructing a closed-loop decision-making mechanism of "decision-execution-feedback-optimization" to achieve intelligent decision-making in the equipment's main control system; specifically including: S51. Lightweight Model Processing: Knowledge distillation is employed, using the large model trained in S4 as the teacher model, and training a student model with a parameter size of 1 / 5 that of the teacher model; ensuring that the student model retains more than 90% of the decision accuracy despite the reduction in parameter size; In some embodiments, the distillation temperature is set to 5, the distillation loss weight is set to 0.7, and the student model loss weight is set to 0.3. The model parameters are quantized from 32-bit floating-point numbers (FP32) to 16-bit floating-point numbers (FP16). A dynamic quantization method is used to quantize the weights and activation values ​​during the model inference stage, achieving the goal of reducing the amount of computation by 50%.

[0045] As an alternative to the lightweight optimization method in this step, model pruning can also be used to compress the model: removing attention heads and neurons with a contribution of <0.1 from the Transformer while retaining the core structure. The advantage of this alternative is a 60% reduction in computation, making it more efficient than the method in this invention. The disadvantage is that the robustness of the model decreases after pruning, and the decision accuracy fluctuates by ±5% in extreme scenarios, requiring an additional robustness verification step. In contrast, the knowledge distillation + quantization scheme of this invention can achieve lightweighting while maintaining controllable accuracy loss (retaining more than 90% of the decision accuracy), which is more in line with the stability requirements of the equipment's main control system.

[0046] S52. Equipment Main Control System Deployment and Adaptation: Deploy the lightweight model to the embedded chip of the equipment main control system; use the industrial Ethernet protocol to realize data interaction between the decision module and the main control system, and control the data interaction latency to ≤10ms; design standardized decision instruction frames, which include fields such as "task type, target ID, control parameters, and execution time limit"; after deployment, perform real-time verification, and reduce the single-step decision latency of the model to 85ms, meeting the latency requirement of ≤100ms of the equipment main control system.

[0047] S53. Closed-loop decision-making mechanism construction: A closed-loop mechanism of "decision-execution-feedback-optimization" is formed based on feedback information from the main control system; feedback information includes command execution status (success / failure) and current equipment status (target lock-on status, search range); the decision module dynamically adjusts subsequent decision-making strategies based on feedback information; at the same time, the target selection head outputs target priority to achieve automatic target lock when multiple targets are attacking; the target search control head outputs parameters such as search angular velocity and elevation adjustment to achieve automatic search control of the radar antenna, replacing manual decision-making operations.

[0048] The key technical points of this invention are: (1) Multi-source data confidence weighting mechanism: The confidence of data is evaluated by “matching degree + stability”, and the differentiated weight allocation is realized to solve the problem of low confidence data interference; (2) Transformer multi-task adaptation architecture: Combining task-specific special embedding and dedicated prediction head, “shared feature extraction + task-specific output” is realized to improve the multi-task generalization ability; (3) Heterogeneous data alignment and gradient coordination integrated design: Data alignment is realized by feature encoding + shared backbone, and gradient conflict is solved by combining GFT and SPMLoss to ensure training stability; (4) Lightweight deployment optimization scheme: Through knowledge distillation + quantization processing, the computational complexity is reduced on the premise of ensuring accuracy to meet the real-time requirements of equipment.

[0049] This invention differs significantly from the closest existing technologies (RNN-based single-task equipment decision-making models, and other existing multi-task reinforcement learning algorithms such as DecisionTransformer, Prompt-DT, and Harmo-DT) in core performance and technical implementation. Specific advantages are compared below: (1) In terms of data source and weight, the existing technology only uses two types of data, namely "real data + high-precision simulation data", and does not consider the difference in data confidence. It uses equal weight for training, which results in limited scene coverage and low-confidence data easily interferes with the model. However, the present invention integrates three types of data, namely "real data + semi-physical simulation data + high-precision simulation data", evaluates the data confidence according to the formula "matching degree × 0.6 + stability × 0.4", and assigns differentiated weights of 0.5 to real data, 0.3 to semi-physical simulation data and 0.2 to high-precision simulation data, thereby improving the scene coverage and decision accuracy.

[0050] (2) In terms of multi-task generalization ability, existing technologies based on RNN architecture only support single-task (such as target locking) decision-making. When facing other tasks such as collaborative defense and avoidance, the model structure needs to be redesigned and full training needs to be carried out, resulting in high development costs and weak generalization ability. This invention is based on Transformer architecture, combined with task-specific Special Embedding and dedicated prediction head, which can make multi-task decisions at the same time without reconstructing the model, improving development efficiency and increasing generalization scenarios.

[0051] (3) In terms of training convergence efficiency, this invention optimizes the gradient propagation path by using Gradient Flow Transformer (GFT) and combines gradient similarity alignment loss (SPMLoss) to constrain gradient consistency, thereby improving the convergence speed by 50% compared with existing multi-task offline reinforcement learning algorithms.

[0052] (4) In terms of deployment real-time dimension, the present invention reduces the single-step decision latency to 85ms through knowledge distillation (student model with training parameters only 1 / 5 of the original model) and FP16 quantization, which meets the real-time combat deployment requirements of equipment and reduces the latency by 83% compared with the existing technology.

[0053] In summary, this invention is superior to existing technologies in terms of data utilization, task adaptation, training efficiency, and engineering deployment, and is better able to meet the intelligent decision-making needs of equipment main control systems in complex and dynamic combat environments.

[0054] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0055] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for constructing and deploying an intelligent decision-making module for equipment control based on a large model, characterized in that: Specifically, the steps include the following: S1. Collect multi-source data and perform abnormal data removal processing on each; then, based on the matching degree between the data and the real scene and the data stability, calculate the confidence of various types of data by weight and assign weights to form a multi-source weighted dataset; the multi-source data includes real-world data, semi-physical simulation data and high-precision simulation data; S2. Differentiatedly encode the core task scenario features and concatenate them into a feature vector of a unified dimension. Then, through an embedding layer, map the feature vector to a high-dimensional semantic space and output a high-dimensional feature vector. The core task scenario features include incoming target type, incoming direction, cooperative object attributes, and protection target priority. S3. Build a multi-task Transformer large model architecture, including a task prompting enhancement module, a basic architecture module, a multi-dimensional embedding module, and a task flow output layer; through multi-dimensional embedding, high-dimensional feature vectors are uniformly mapped to the model adaptation dimension to obtain enhanced feature vectors containing task prior knowledge; after the enhanced feature vectors are input into the basic architecture module to complete the core feature calculation, they are distributed to each independent prediction head of the task flow output layer to output the state prediction results and action prediction results that match the task; S4. The model is pre-trained using an offline reinforcement learning algorithm, and then fine-tuned by inputting encoded multi-task data. The gradient coordination mechanism is used to improve the model's decision-making accuracy and training stability. S5. Lighten the trained model and deploy the lightweight model to the equipment main control system to realize intelligent decision-making of the equipment main control.

2. The method for constructing and deploying an equipment main control intelligent decision-making module based on a large model according to claim 1, characterized in that: The specific methods for abnormal data removal in step S1 include: collecting actual data from the equipment sensors to remove abnormal values ​​caused by sensor malfunctions; simulating N types of scenarios using a hardware-in-the-loop simulation platform to generate hardware-in-the-loop simulation data and removing noise data from the hardware-in-the-loop simulation data; generating high-precision simulation data for M types of scenarios based on a digital twin model and removing logically contradictory data where the target speed exceeds the physical limit. The weighting formula is: Weight = Matching degree × 0.6 + Stability × 0.4; Among them, matching degree represents the degree of matching between the data and the real scene, and stability represents the stability of the data; the weight of the actual data is 0.5, the weight of the semi-physical simulation data is 0.3, and the weight of the high-precision simulation data is 0.

2.

3. The method for constructing and deploying an equipment main control intelligent decision-making module based on a large model according to claim 1, characterized in that: The differential encoding in step S2 specifically includes: the incoming target type is converted into a 10-dimensional vector using one-hot encoding; the incoming direction is converted into a 2-dimensional vector using normalized encoding; the cooperative object attribute is converted into a 16-dimensional vector using embedding encoding; the protection target priority is converted into a 1-dimensional vector using numerical mapping; the encoded data are then concatenated to form a 30-dimensional unified vector; the input to the embedding layer is a 30-dimensional unified vector, the hidden layer dimension is set to 128 dimensions and the ReLU activation function is used to output a 128-dimensional high-dimensional feature vector.

4. The method for constructing and deploying an equipment main control intelligent decision-making module based on a large model according to claim 1, characterized in that: Step S3 also includes a task parameter isolation mechanism, as follows: all task-specific embedding and prediction layers are managed through the ModuleDict container, and parameter isolation is achieved by using the task name as the key; during training, only the corresponding layer parameter update of the current iteration task is activated, and the parameters of other tasks are frozen by setting requires_grad=False to avoid parameter interference between tasks.

5. The method for constructing and deploying an equipment main control intelligent decision-making module based on a large model according to claim 4, characterized in that: The infrastructure module in step S3 uses GPT-2 as the core basic model and switches the adaptation mode through the special_embedding switch parameter: when the parameter is True, independent embedding and prediction layers are configured for different tasks to adapt to heterogeneous tasks. When the parameter is False, all tasks share the same layer structure to adapt to homogeneous tasks; The multi-dimensional embedding module includes a state embedding layer, an action embedding layer, a cumulative reward embedding layer, a time step embedding layer, and a type embedding layer, with each embedding layer outputting a uniform 512 dimensions. The task prompt enhancement module sets up a prompt state embedding layer and a prompt action embedding layer, and the two-layer structure is completely consistent with the state embedding layer and action embedding layer in the multi-dimensional embedding module; the task-specific prompt information is a fragment of the task's historical best decision trajectory, spliced ​​at the front of the input sequence; The independent prediction head includes a state prediction layer and an action prediction layer. The state prediction layer adopts a linear layer structure with an input dimension of 512 dimensions and an output dimension that matches the original state dimension of the task. The action prediction layer adopts a two-layer MLP architecture. The first layer has an input dimension of 512 dimensions and an output dimension of 256 dimensions, and the second layer has an input dimension of 256 dimensions and an output dimension that matches the action dimension of the task. The GELU activation function is used between the two layers.

6. The method for constructing and deploying an equipment main control intelligent decision-making module based on a large model according to claim 5, characterized in that: The input dimension of the state embedding layer is 128-dimensional, which is consistent with the dimension of the high-dimensional feature vector output in step S2. The input dimension of the action embedding layer matches the task action space dimension; The time-step embedding layer uses sine and cosine position coding to capture temporal dependencies; The type embedding layer is used to distinguish the element types in the sequence.

7. The method for constructing and deploying an equipment main control intelligent decision-making module based on a large model according to claim 1, characterized in that: The multi-source data in step S1 includes physical data, semi-physical simulation data, and high-precision simulation data; the multi-source weighted dataset includes 50% physical data, 30% semi-physical simulation data, and 20% high-precision simulation data. The offline reinforcement learning algorithm in step S4 is the BCQ algorithm; The pre-training process consists of 100 rounds, and the reward function used is: reward value = decision accuracy × 0.7 + decision delay coefficient × 0.

3.

8. The method for constructing and deploying an equipment master control intelligent decision-making module based on a large model according to claim 1, characterized in that: The gradient coordination mechanism in step S4 includes gradient flow propagation control and gradient similarity alignment loss; The gradient flow propagation control specifically involves setting up one gradient propagation controller every two Trunk layers in the Transformer model. The controller optimizes the gradient path by scaling the gradient. The gradient similarity alignment loss is specifically as follows: by calculating the cosine similarity of gradients for different tasks, when the similarity is <0.6, the gradient direction is corrected by adjusting the coefficients of the loss function to constrain the consistency of gradients for different tasks.

9. The method for constructing and deploying an equipment master control intelligent decision-making module based on a large model according to claim 1, characterized in that: The lightweighting process in step S5 employs knowledge distillation and quantization. Knowledge distillation uses a large, trained model as the teacher model, and the training parameter size is 1 / 5 of that of the student model. Quantization uses dynamic quantization to convert the model parameters from 32-bit floating-point numbers to 16-bit floating-point numbers, thereby reducing the computational load.

Images