Electromagnetic interference fault prediction method fusing mixed loss and gating LoRA

Abstract

The invention provides an electromagnetic interference fault prediction method fusing mixed loss and gating LoRA. According to the method, a gating vector is dynamically generated by constructing a dual feedback signal fusing internal uncertainty and external knowledge correctness of a model, and the vector performs rank dimension level fine adjustment on a low rank matrix of the LoRA. According to the mechanism, the knowledge injection process is converted from global and indiscriminate updating into an adaptive process which is independently controlled according to knowledge importance and a model cognitive state, so that key fault features can be subjected to key learning, and meanwhile, the damage to the original capability of the model is avoided. It can be seen that the method integrates the advantages of model uncertainty evaluation and efficient parameter fine adjustment, and a new electromagnetic interference fault prediction normal form capable of remarkably improving prediction accuracy, robustness and interpretability is formed.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and electromagnetic compatibility (EMC) fault prediction, specifically involving an EMC fault prediction method that integrates hybrid loss and gated LoRA. This method constructs a hybrid loss that fuses internal model uncertainties and external correctness signals to drive rank-dimensional gating of low-rank adaptive weights, thereby improving the model's ability to accurately predict and interpret potential faults in complex electromagnetic environments. Background Technology

[0002] As electronic devices become increasingly complex and integrated, electromagnetic interference (EMI) has become one of the main limiting factors affecting their reliability and safety. In critical systems such as aerospace, rail transportation, smart grids, and autonomous driving, external electromagnetic environments or internal circuit crosstalk can intrude into electronic devices through conduction and radiation, leading to data transmission errors, logic malfunctions, and even hardware damage. Therefore, developing technologies that can accurately predict potential faults caused by electromagnetic interference is of significant engineering value for ensuring the stable operation of critical systems.

[0003] Currently, the technical solutions used for electromagnetic interference fault prediction mainly include the following three categories: The first category is simulation methods based on physical models. This method requires establishing accurate electromagnetic models of the electronic components, circuits, and the entire system, and then performing simulation analysis using numerical calculation tools such as the finite element method. The advantage of this method is its ability to reveal the physical mechanisms of faults. Its disadvantages are: 1) The modeling process is complex, especially for large systems with complex structures; 2) It consumes enormous computational resources and is too time-consuming, making it unsuitable for real-time prediction; 3) The simulation results are highly dependent on the accuracy of the model parameters and cannot reflect the impact of changes in operating conditions and environmental uncertainties during actual operation.

[0004] The second category is prediction methods based on traditional machine learning. This method utilizes algorithms such as Support Vector Machines (SVM) and Random Forests to analyze collected electromagnetic signals and bus data to identify fault modes. Compared to physical simulation, this type of method can handle actual operating data and has higher computational efficiency. Its main limitations are: 1) Performance heavily relies on manually designed features; the quality of feature extraction determines the upper limit of model performance; 2) The model's generalization ability is limited; when the operating environment or fault mode changes, the prediction performance will significantly decrease.

[0005] The third category is prediction methods based on deep learning. These methods employ deep models such as recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) to automatically extract features from raw time-series data. Their advantage lies in the absence of complex feature engineering and their ability to handle high-dimensional, non-linear data. However, these methods have inherent drawbacks: 1) poor model interpretability; their predictions are often considered "black boxes," failing to provide a basis for fault tracing; 2) poor performance when dealing with weakly correlated signals caused by electromagnetic interference, which are highly random and sudden, making it difficult to capture implicit causal relationships, resulting in insufficient prediction accuracy when facing novel or rare interference.

[0006] To overcome the limitations of the aforementioned methods, researchers have begun exploring the application of large language models (LLMs) for fault prediction. LLMs possess powerful contextual understanding, logical reasoning, and knowledge integration capabilities, and theoretically can achieve deeper fault prediction by analyzing textual data such as system logs and monitoring reports and integrating domain knowledge. However, efficiently and accurately integrating electromagnetic domain expertise into a general-purpose LLM remains a core technical challenge. Existing efficient parameter fine-tuning techniques (such as low-rank adaptation, LoRA) typically employ a globally uniform update strategy when performing knowledge injection, applying indiscriminate adjustments to all parameters to be updated. This non-differentiated update approach cannot adaptively adjust based on the importance of the knowledge itself or the model's unfamiliarity with that knowledge, potentially leading to insufficient learning of key fault features or damage to the model's original general capabilities due to over-correction.

[0007] In summary, current technologies for electromagnetic interference fault prediction using large-scale language models lack a mechanism for adaptive and refined knowledge injection. Designing a novel parameter fine-tuning framework that dynamically adjusts the update process based on knowledge content and model state is key to solving these problems.

[0008] To address this technological gap, this invention proposes a solution. This solution dynamically generates a gating vector using a hybrid loss function that integrates internal model uncertainty assessment and external knowledge accuracy feedback. This vector acts on the low-rank matrix of LoRA, independently enabling and scaling each rank dimension. This mechanism enables adaptive control of the knowledge injection process, allowing for focused learning of key fault knowledge during fine-tuning while stabilizing the model's existing knowledge system, thereby improving the accuracy, robustness, and interpretability of electromagnetic interference fault prediction. Summary of the Invention

[0009] The purpose of this invention is to address the problems in existing fault prediction methods, such as complex physical modeling, weak generalization ability of traditional machine learning, poor interpretability of deep learning models, and the lack of adaptive fine-tuning mechanisms for knowledge injection in large language models. This invention proposes an adaptive parameter fine-tuning framework that integrates hybrid loss and gated low-rank adaptation. This method dynamically generates a gate vector by constructing a dual feedback signal that integrates internal model uncertainty and external knowledge correctness. This vector performs fine-tuning of the low-rank matrix of LoRA at the rank dimension level. This mechanism transforms the knowledge injection process from a global, undifferentiated update into an adaptive process independently controlled based on knowledge importance and model cognitive state. This allows for focused learning of key fault features while avoiding damage to the model's original capabilities. It can be seen that this invention integrates the advantages of model uncertainty assessment and efficient parameter fine-tuning, forming a new paradigm for electromagnetic interference fault prediction that significantly improves prediction accuracy, robustness, and interpretability.

[0010] To achieve the above objectives, this invention provides an electromagnetic interference fault prediction method that integrates hybrid loss and gated LoRA, the specific steps of which are as follows: Step 1: Construct an electromagnetic interference fault dataset and perform linguistic preprocessing; Step 2: Initialize the large language model and integrate the low-rank adaptation (LoRA) module; Step 3: Construct a hybrid loss function that combines uncertainty and knowledge accuracy in the fusion model; Step 4: Design a rank-dimensional gated vector generation mechanism based on hybrid loss; Step 5: Perform adaptive fine-tuning training to optimize model parameters; Step 6: Use the fine-tuned model to predict electromagnetic interference faults.

[0011] In step 1, "Constructing an electromagnetic interference fault dataset and performing language-based preprocessing," the aim is to transform multi-source, heterogeneous system operation data into natural language text format that a large language model can understand and process. The raw data includes, but is not limited to, timestamps, bus messages, sensor voltage / current readings, error frame counts, and system log text. The preprocessing process converts these structured or semi-structured data entries into descriptive statements with clear semantics, based on a pre-defined template. For example, a data entry containing {"ts": 1678886400, "bus_v": 4.8, "error_code": "0x1A"} can be converted to: "At timestamp 1678886400, the bus voltage is 4.8V, and the system reports error code 0x1A." Finally, the processed text sequence is paired with corresponding expert-annotated fault labels (such as "normal," "power fluctuation interference," "instantaneous interruption of communication bus," etc.) to form a dataset for model fine-tuning.<Prompt, Label> Dataset.

[0012] In step 2, "initializing a large language model and integrating a low-rank adaptation module," this invention selects a pre-trained large language model (such as Llama, GPT, etc.) as the base model. To achieve efficient parameter fine-tuning, at specific layers of the model (such as query layers with self-attention mechanisms)... s and keys The low-rank adaptation (LoRA) module is integrated into the original pre-trained weight matrix. (dimension is) Next to it, a low-rank factorization matrix is ​​connected in parallel. . (dimension is) )and (dimension is) ) is a trainable low-rank matrix, where rank much smaller During forward propagation, the output of the corresponding layer of the model... Calculated by the following formula (where (This is the input feature vector of the corresponding layer of the model).

[0013] During fine-tuning, the original weights Keep frozen, only update a much smaller number of parameters matrix and .

[0014] In step 3, which describes "constructing a hybrid loss function for the fusion model's uncertainty and knowledge correctness," this invention designs a hybrid loss function. This loss function is used to simultaneously evaluate the "correctness" and "uncertainty" of the model's predictions, and uses this as a signal to guide subsequent gating mechanisms. The loss function consists of two parts: Loss of knowledge accuracy The standard cross-entropy loss function is used to measure the model's prediction results. Consistency with the true label y. The calculation formula is:

[0015] in, This represents the total number of fault categories. The one-hot encoded vector of the real label Bit element, The model predicts that the sample belongs to the first... The probability value of the class, where This loss term ensures that the model optimizes in the direction of learning the correct domain knowledge.

[0016] Model uncertainty loss The entropy of the predicted probability distribution is used to quantify the model's uncertainty regarding the current input. High entropy indicates that the model's judgment of the likelihood of multiple categories is relatively average, i.e., high uncertainty. The calculation formula is:

[0017] in, This represents the total number of fault categories. The model predicts that the sample belongs to the first... The probability value of the class, where .

[0018] The uncertainty loss term reflects the model's "cognitive state" regarding its own knowledge system.

[0019] Final hybrid loss function It is the weighted sum of the two:

[0020] Among them, hyperparameters This is used to balance the relative importance of the two loss terms. Furthermore, the method for measuring model uncertainty is not unique. In an alternative implementation, Variance based on Monte Carlo Dropout can be used for estimation. Specifically, during the model inference phase, the Dropout layer in the model is activated multiple times (e.g., T times), each time generating a slightly different predicted probability distribution. By calculating the variance of these T predictions across each category, a variance vector can be obtained, whose magnitude or trace can serve as a measure of uncertainty. This method can more robustly reflect uncertainty at the model parameter level, but its computational cost is higher than that of the entropy method. Hyperparameters The value of determines the "sensitivity" of the gating mechanism: a higher ... Value-based gating focuses more on error correction and is suitable for scenarios with a complete knowledge base and requiring rapid convergence; lower value-based gating... The value makes gating focus more on the "confusion" of the model itself, which is suitable for exploratory learning or processing noisy data.

[0021] In the section described in step 4, "Designing a rank-dimensional gated vector generation mechanism based on hybrid loss," the core innovation of this invention lies in utilizing the hybrid loss calculated in the previous step. Dynamically generate a rank-dimensional gate vector The dimension of this vector is related to the rank of LoRA. Same. Gating vector The generation process can be formalized as follows:

[0022] in, It is a small multilayer perceptron that takes scalar loss values Mapped to a dimensional vector; It is the Sigmoid activation function, which activates each element of the output vector. The constraint is within the interval (0, 1), where For the rank dimension index of the LoRA module, . Each element in This corresponds to a rank dimension of the LoRA bypass matrix, and its value represents the importance of that dimension in the current learning task or the necessity of updating it. A value close to 1 indicates that the dimension needs to be fully utilized and updated, while a value close to 0 indicates that the dimension should be suppressed to protect the model's original capabilities or avoid ineffective updates.

[0023] In step 5, "perform adaptive fine-tuning training", this step will use the gate vector generated in step 4. This is applied to the LoRA update process to achieve adaptive fine-tuning. Specifically, the original LoRA forward propagation formula is modified by changing the gate vector... Embedded in the form of a diagonal matrix:

[0024] in, It is a vector Convert to elements on the main diagonal The diagonal matrix. This operation is equivalent to applying LoRA. Each rank dimension is independently "scaled" or "on / off". During backpropagation and parameter updates, the gradient will be based on... (or other standard training objective function) is used for calculation, but the gradient propagation path and the... The influence of the matrix will be affected by the diagonal matrix. The model dynamically adjusts itself. Thus, when faced with samples of high uncertainty and incorrect predictions, it generates a gating vector that allows more information to flow through, thereby enhancing the learning of new knowledge; conversely, it suppresses updates and maintains stability.

[0025] In step 6, "predicting electromagnetic interference faults using the fine-tuned model," after multiple rounds of adaptive fine-tuning training, the model containing the optimized LoRA parameter matrices A and B is deployed. In practical applications, the system operation data collected on-site is processed in real-time using the method described in step 1 to generate input text. This text is then input into the fine-tuned model, which directly outputs the fault type prediction result for the current system state. Because the fine-tuning process achieves refined injection of key fault knowledge, this model exhibits higher prediction accuracy, robustness, and the ability to identify novel fault modes compared to traditional methods and standard fine-tuning methods when facing complex and variable electromagnetic environments.

[0026] Specifically, within each training batch, this adaptive fine-tuning process comprises the following sub-steps: (a) Forward Propagation and Loss Calculation: For each sample in the batch, the model first performs a standard forward propagation to calculate the predicted probability. Subsequently, based on and real labels Calculate the mixed loss .

[0027] (b) Gated vector generation: (or its average value over the batch) is input into the MLP network defined in step 4 to generate the gating vector corresponding to that batch or sample. .

[0028] (c) Gated gradient calculation: using the gated forward propagation formula applied to LoRA:

[0029] With standard training objectives (usually) To optimize the objective, backpropagation is performed to calculate the gradients of matrices A and B. and It is worth noting that, due to The calculation depends on ,and It itself depends on the model output. To simplify the calculation, when the gradient is backpropagated to A and B, it can be... It is treated as a fixed constant, so that it does not participate in the calculation of the gradient chain, and thus only serves as a weight factor to adjust the gradient magnitude.

[0030] (d) Parameter update: Use an optimizer (such as AdamW) based on the calculated gradients. and Update parameters A and B of the LoRA module.

[0031] Through the above cycle, the model continuously self-evaluates and dynamically adjusts throughout the training process, achieving differentiated and refined learning of knowledge from different samples.

[0032] Through the above steps, a complete process for constructing and optimizing an electromagnetic interference fault prediction model based on a large-scale language model can be completed: First, multi-source, heterogeneous system operation data is processed into language to construct a training corpus suitable for the model; then, a parameter-efficient low-rank adaptation (LoRA) module is integrated into the pre-trained large-scale language model; next, a hybrid loss that integrates knowledge correctness and model uncertainty is constructed, and a gating vector is dynamically generated using this as a signal; finally, the rank dimension of the LoRA is adaptively adjusted using this gating vector to achieve accurate injection of key fault knowledge during training, resulting in an optimized fault prediction model. This invention is applicable to fault prediction and health management of complex electronic systems affected by electromagnetic interference in fields such as aviation, aerospace, automotive electronics, and industrial control, and has extremely high application and promotion value due to its good preservation of the original model's capabilities and excellent interpretability.

[0033] The advantages of this invention compared to the prior art are: Existing fault prediction methods either rely on physical models that are difficult to model accurately and have poor generalization ability, or on traditional deep learning methods that lack interpretability and require large amounts of labeled data. Standard LLM fine-tuning, on the other hand, risks indiscriminate updating of new knowledge and is prone to "catastrophic forgetting." This method combines the powerful reasoning ability of large language models with efficient parameter fine-tuning. Through an innovative hybrid loss and gating mechanism, it transforms the static, indiscriminate parameter update process into a dynamic, adaptive knowledge injection process. Secondly, by constructing dual feedback signals of uncertainty and knowledge correctness, this method enables the model to "perceive" its mastery of different knowledge during fine-tuning, thereby achieving focused learning of key fault features and effective suppression of redundant information, significantly improving training efficiency and model interpretability. Finally, this method adopts a rank-dimensional gating low-rank adaptation strategy, which can finely control the influence of injected knowledge down to a single rank dimension, effectively avoiding the destruction of the model's original general capabilities. While improving the accuracy of fault prediction in specific domains, it maximizes the model's generalization performance. In summary, this invention achieves a high degree of balance between prediction accuracy, model interpretability, and generalization ability, making it an advanced technical paradigm suitable for intelligent fault prediction in complex electromagnetic environments. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating the overall steps of the present invention.

[0035] Figure 2 This refers to the data preprocessing and language serialization process of this invention.

[0036] Figure 3 This is a schematic diagram illustrating the electromagnetic interference fault prediction using a finely tuned model according to the present invention. Detailed Implementation

[0037] To make the technical problems, technical solutions, and advantages of this invention clearer, a detailed description will be provided below in conjunction with the accompanying drawings.

[0038] To make the technical problems, solutions, and advantages of this invention clearer, a detailed description will be provided below in conjunction with a specific application scenario in the automotive electronics field (electromagnetic interference fault prediction for in-vehicle electronic systems). This embodiment takes an "autonomous vehicle in-vehicle electronic system" as the research object, and verifies the effectiveness of the method of this invention by addressing faults such as CAN bus communication anomalies, sensor data distortion, and ECU logic disorder caused by electromagnetic interference in this system.

[0039] This invention provides an electromagnetic interference fault prediction method that integrates hybrid loss and gated low-rank adaptation. Its core is to solve the technical problem of "strong randomness and complex fault modes" in electromagnetic interference faults in autonomous vehicle electronic systems through refined knowledge injection. For example... Figure 1 As shown, the specific construction steps are as follows: Step 1: Construct an electromagnetic interference fault dataset and perform linguistic preprocessing. The purpose of this step is to transform the multi-source heterogeneous data from autonomous vehicles into a large text corpus that can be processed by a language model, providing high-quality training data for model fine-tuning and a high-quality, standardized data foundation for subsequent model fine-tuning and knowledge injection. Specifically, as follows... Figure 2 As shown. Formally, let Representing raw datasets from different sensors, logs, and records, the goal of this step is to generate a unified fault text dataset. Each data item Each is one right, It is natural language text that describes the system's state and environment. These are the corresponding fault category labels. This process must ensure two key attributes: first, the generated text accurately and completely reflects the temporal relationships and feature associations in the original data; second, the linguistic format is conducive to stimulating the reasoning ability of LLM, enabling it to learn potential fault modes rather than surface statistical features.

[0040] In specific implementation, it is recommended to process according to the following embodiments. It is preferred to use autonomous driving scenario data (such as real road test, electromagnetic compatibility (EMC) anechoic chamber simulation test) as input, which may include: (1) Time-series sensor data: CAN bus voltage (0-5V), engine ECU input current (0-20A), LiDAR / millimeter-wave radar signal-to-noise ratio (10-40dB), inertial measurement unit (IMU) acceleration value (-5~5m / s²), electromagnetic interference power spectral density collected by spectrum analyzer (-120~-60dBm / Hz); (2) Discrete event log: CAN bus retransmission count, CRC check error count, ECU fault code (such as P0135, U0073), sensor data packet loss rate, equipment restart record, etc.; (3) Manual observation and maintenance record: "When the vehicle millimeter-wave radar operating frequency (77GHz) is superimposed with the surrounding base station signal, the CAN bus communication delay exceeds 100ms" "When the high-voltage harness is energized, the LiDAR data jumps" and other scenario descriptions. Language-based preprocessing can employ template-based serialization methods. For example, the following template can be designed: "At the timestamp, the autonomous vehicle is in [driving state: constant speed / acceleration / deceleration / stationary], and the system status is as follows: the average CAN bus voltage is [value A] V, and the peak voltage is [value B] V; the engine ECU input current is [value C] A; the lidar signal-to-noise ratio is [value D] dB, and the data packet loss rate is [value E]%; during this period, there were [M] CAN bus retransmissions and [N] CRC check errors, and the ECU reported fault code [fault code]; the current electromagnetic environment is: the center interference frequency is [frequency] MHz (source: [radar / high voltage harness / external base station]), and the field strength is [field strength] dBμV / m; the final fault label for this condition is [fault label]." This template transforms multidimensional raw data points and events within a time window into a coherent natural language description. After this processing, the entire raw dataset is converted into a corpus containing thousands to tens of thousands of (description text, fault label) pairs, which serves as input for subsequent LLM adaptive fine-tuning.

[0041] Step 2: Initialize the large language model and integrate the low-rank adaptation (LoRA) module. The goal of this step is to select a base model for automotive electronic scenario adaptation and integrate the LoRA module to lay the foundation for efficient parameter fine-tuning. By introducing efficient parameter fine-tuning techniques, a foundation is laid for subsequent adaptive knowledge injection, thereby limiting modifications to the large model (LLM) to a small number of controllable parameters, so that subsequent gating mechanisms can operate efficiently and accurately.

[0042] (I) Selection and Introduction of Large Language Model (LLM) Base The Llama-2-7B open-source model (adapted to edge computing scenarios, with a moderate number of parameters and fast inference speed) was selected as the base model. This model has been pre-trained on massive amounts of text and has strong contextual reasoning capabilities, which can be quickly transferred to vehicle fault prediction tasks.

[0043] (II) Integration and Parameter Introduction of Low-Rank Adaptive Module To maximize the preservation of the original general-purpose capabilities of the LLM while reducing computational overhead and incorporating new knowledge, this embodiment introduces the Low-Rank Adaptation (LoRA) technique. This technique does not modify the original weight matrix of the LLM. (dimension is) Instead of directly modifying it, a "bypass" is connected in parallel next to it. This bypass consists of two trainable low-rank matrices A (with dimensions of 1). ) and B (dimension is The product of ) constitutes the rank, where Much smaller than the original dimension and The core idea of ​​LoRA is to adjust the original weights. Update Use low-rank decomposition To approximate this. During the fine-tuning process, Keep the system frozen and train only those with a very small number of parameters. and Therefore, the model's output at specific layers (such as self-attention layers or feedforward network layers) Modified to:

[0044] in, For input, This is a fixed scaling factor used to control the intensity of bypass updates. With this setting, modifications to the model are concentrated on the low-rank matrix. and This greatly reduces the number of parameters that need to be trained.

[0045] (III) Definition of Trainable Parameter Space To ensure that the fine-tuning process is efficient and controllable, this embodiment defines the trainable parameter space. Defined as a low-rank matrix in all LoRA modules and The set of:

[0046] in, It is a collection of layers in the model that integrate LoRA modules. Within this space, each set of parameters... Through forward propagation of the model, the incremental modifications to the capabilities of the original model are collectively determined. By optimizing only this low-dimensional parameter space This method transforms the complex problem of full-parameter fine-tuning into a low-cost, high-efficiency optimization task, providing a prerequisite for the subsequent introduction of rank-dimensional dynamic gating mechanisms. The final trainable space is not only small in size, but also, through the structured design of LoRA, ensures minimal interference with the model's original knowledge system.

[0047] Step 3: Construct a hybrid loss function that combines uncertainty and knowledge correctness in the fusion model. The goal of this step is to design a hybrid loss function adapted to automotive electronic fault prediction, providing dual feedback signals for the gating mechanism. The primary approach is to propose a hybrid loss function that can simultaneously evaluate the "correctness" and "uncertainty" of the model's predictions. This serves as the quantization signal guiding subsequent adaptive gating mechanisms. The loss function integrates external supervision signals and the model's internal cognitive state, providing a dual basis for refined knowledge injection.

[0048] (I) Fault Category Definition (C Value) The total number of automotive electronic electromagnetic interference fault categories is C=5, and the specific categories are as follows: 1. Normal (no electromagnetic interference); 2. Power supply fluctuation interference (caused by electromagnetic radiation from high-voltage wiring harness); 3. CAN bus momentary interruption (caused by crosstalk between radar and bus signals); 4. Radar sensor interference (caused by superposition of signals from external base stations); 5. ECU logic disorder (caused by superposition of electromagnetic interference from multiple sources).

[0049] (II) Loss Function Formula and Parameter Definition a) Loss of knowledge accuracy ) Cross-entropy loss is used, and the formula is as follows:

[0050] Parameter definition: For fault category index ( (corresponding to the above 5 types of faults). One-hot encoding of the actual label (e.g., "CAN bus momentary interruption") ,the remaining ); The model predicts that the sample belongs to the first... The probability of such failures (e.g.) The model indicates a 92% certainty that the CAN bus experienced a momentary interruption.

[0051] b) Model uncertainty loss ( ) The Shannon entropy, which uses the predicted probability distribution, is given by the following formula:

[0052] Consistent with the above, ; If the model is affected by "radar sensor interference" ( The predicted probability of ) If the probabilities of other categories are uniformly distributed (each 0.175), then the entropy value is high (great uncertainty), and this sample needs to be studied more carefully.

[0053] c) Mixed loss function

[0054] Final hybrid loss function It is the weighted sum of the two above, and its formal definition is:

[0055] Among them, hyperparameters This is used to balance the relative importance of the two loss terms: knowledge accuracy and model uncertainty.

[0056] Furthermore, the method for measuring model uncertainty is not unique. In an alternative implementation, Variance can be estimated using Monte Carlo Dropout-based methods. Specifically, during the model inference phase, the Dropout layer in the model is enabled for forward propagation multiple times (e.g., T times), generating a slightly different predicted probability distribution each time. By calculating the variance of these T predictions across each category, a variance vector can be obtained, whose magnitude or trace can be used as a measure of uncertainty. This method can more robustly reflect the uncertainty at the model parameter level, but its computational cost is relatively higher than that of the entropy method.

[0057] Step 4: Design a rank-dimensional gated vector generation mechanism based on hybrid loss. Step four is the core technical step in achieving adaptive knowledge injection in this invention. Its fundamental goal is to construct a gated generation network adapted to the characteristics of automotive electronic faults, dynamically adjusting the LoRA rank dimension update intensity, thereby establishing a bridge from the macroscopic learning state to the microscopic parameter adjustment. This is achieved by using the hybrid loss calculated in step three, which integrates the model's "cognitive state" and "external supervision." This is transformed into a rank-dimensional gated vector that enables fine-grained control of the low-rank adaptation (LoRA) module. This mechanism gives the model the ability to self-regulate during fine-tuning, which is key to the shift from "undifferentiated updates" to "differentiated learning".

[0058] At the core of this mechanism is a trainable gated generative network. The function of this network is to convert data into scalar values. Mapped to the rank of the LoRA module Same dimensions dimensional vector The specific implementation method is as follows: (I) Network Structure and Formal Definition: In this embodiment, the gated generation network preferably employs a lightweight multilayer perceptron (MLP). This MLP receives a single... The value is taken as input, passes through one or more non-linear hidden layers, and finally outputs a... A vector of dimension 1. The entire generation process can be formally represented as:

[0059] in: : Input scalar mixed loss. When processing a batch of data, all samples within that batch can be used. The average value is used to obtain a more stable gating signal.

[0060] , and , : These represent the weight matrix and bias vector of the first and second layers in the MLP, respectively. These parameters are trainable and will be updated by the optimizer along with the parameters of the LoRA module during the adaptive fine-tuning process in step five. This allows the gating mechanism itself to learn how to generate the most effective adjustment method based on the loss signal.

[0061] To correct the linear unit, which is used as the activation function of the hidden layer, nonlinear processing capability is introduced.

[0062] The sigmoid activation function is applied element-wise to the final output of the MLP to ensure the gating vector... Each element (in All are precisely constrained within Within the range.

[0063] (II) Gating Vector Functional and physical significance: Output dimensional vector It is a dynamically generated set of "switches" and "knobs" for each rank dimension of LoRA. Each element They all have clear regulatory functions: Importance assessment: The value quantifies the first The importance of the rank dimension when processing the current data sample (or batch). When the model faces a sample with high uncertainty and incorrect predictions, With higher values, the gating network tends to generate a graph containing more high-value elements. Vectors are equivalent to "opening up" more learning channels.

[0064] Update intensity scaling: The value is directly used as a scaling factor to adjust the gradient flow through the first [value] during backpropagation. The intensity of a rank dimension. A value close to 1. Allowing gradients to pass through with almost no decay promotes significant updates to the parameters in the corresponding dimension; while a gradient close to 0... This will cause the gradient to approach zero, freezing the parameters of that dimension and thus protecting the existing knowledge it carries.

[0065] Through the above design, this step transforms a previously homogeneous LoRA update process into a heterogeneous and dynamic one. It no longer simply updates all parameters using a single global learning rate, but rather... Each parallel knowledge channel is configured with an independent, real-time gate that changes according to the learning state. This design allows the model to focus on the knowledge dimensions that truly need learning or correction, avoiding wasting computational resources on irrelevant or already mastered features. When processing known or simple samples, generating low-value gating vectors effectively suppresses unnecessary parameter updates, thus greatly mitigating the "catastrophic forgetting" problem that may occur during fine-tuning. Analysis of the gating vectors generated under different types of fault samples... The pattern can be used to infer which types of fault features LoRA has learned in different rank dimensions, providing a new perspective for model prediction and understanding.

[0066] In summary, the rank-dimensional gated vector generation mechanism based on hybrid loss designed in this step is the core hub connecting model state awareness and parameter adaptive adjustment, providing a solid technical foundation for achieving efficient, accurate and stable domain knowledge injection.

[0067] Step 5: Perform adaptive fine-tuning training to optimize model parameters. This step is the execution and convergence phase of the methodology of this invention. Its core objective is to perform gated adaptive fine-tuning to optimize model parameters based on the in-vehicle hardware environment (edge ​​computing unit). Utilizing the complete framework constructed in the preceding steps, a gated adaptive training loop is used to optimize the model's trainable parameter space (i.e., the matrix of all LoRA modules). , The parameters of the gating generator network are iteratively optimized until the model achieves the preset performance index on the electromagnetic interference fault prediction task.

[0068] The key to this process lies in using the rank-dimensional gated vector generated in step four. It can be dynamically embedded into the standard training process (forward propagation, back propagation, parameter update) to achieve refined and adaptive adjustment of the knowledge injection process.

[0069] (a) Gated LoRA forward propagation: First, the forward propagation formula of the standard LoRA is modified. For each layer that integrates the LoRA module, its output... No longer by Instead of computation, it is done by using a gated vector. As a diagonal matrix Embedded within it for calculation:

[0070] in: It is the original, frozen weight matrix of a large language model. It is the input for each layer of the LoRA module. and It is a trainable LoRA low-rank matrix. It is a fixed scaling factor.

[0071] It is the vector generated in step four. The resulting diagonal matrix. This operation is equivalent to using... Each element This involves independently scaling the corresponding rows of matrix A (or the corresponding columns of matrix B). This is precisely the mathematical embodiment of the independent control of the rank dimension level in this invention.

[0072] (II) Iterative process of adaptive fine-tuning training: During training, for each batch of data extracted from the dataset in step one, the following series of sub-steps are performed: (a) Forward propagation and loss calculation: For each sample in the batch, the model first performs a standard forward propagation (without gating) to calculate the initial predicted probability distribution. Subsequently, based on and real labels The loss values ​​of the two cores are calculated in parallel: Loss of knowledge accuracy The primary optimization objective is used to drive the model to learn the correct fault classification.

[0073] Mixed loss As the source of gating signals, the current learning requirements of the quantization model.

[0074] (b) Gated vector generation: All samples in this batch The average value is input into the gating generation network defined in step four to generate the shared gating vector for this batch. Using batch averages can yield a more stable and representative gating signal.

[0075] (c) Gated backpropagation and gradient calculation: Loss of knowledge correctness As the ultimate optimization objective, backpropagation is performed. During this process, since the forward propagation formula has been modified to a gated form, the gradient is propagated back to the matrix... and At that time, its path and size will be affected Dynamic adjustment.

[0076] To simplify computation and stabilize training, the gate vector is used when backpropagating gradients to A and B. It is treated as a fixed constant, so that it does not participate in the calculation of the gradient chain. Thus, As a regulator only Compared to and The weighting factor of the gradient magnitude, and Its own generative network then through The gradient is optimized independently.

[0077] (d) Parameter update: Using an optimizer (such as AdamW), the trainable parameters of both parts are updated simultaneously based on the gradients calculated in the previous step: Parameters of the LoRA module: matrix and .

[0078] Parameters of the gated generative network: the weights and biases of its internal MLP.

[0079] Step Six: Predict electromagnetic interference faults using the fine-tuned model. This step represents the application and value realization stage of the technical solution of this invention. Its goal is to deploy the trained model on the edge computing unit of an autonomous vehicle to achieve real-time fault prediction. The model, obtained after adaptive fine-tuning in step five and infused with refined domain knowledge, is deployed in a real or quasi-real-world environment to efficiently and accurately predict electromagnetic interference faults based on real-time system operation data. Specific implementation details are as follows: Figure 3 As shown, this step includes the following steps: (I) Model Deployment and Inference Preparation First, the fully trained and converged model is solidified by packaging the Llama-2-7B base model (with frozen weights), optimized LoRA parameters (A and B matrices), and a language-based processing script into an inference engine (ONNX format, with 30% faster inference speed). It is deployed in the vehicle's central gateway module, communicating in real-time with the CAN bus and sensor cluster.

[0080] (II) Real-time Data Processing and Input Generation Real-time acquisition of CAN bus voltage, radar sensor data, and electromagnetic interference field strength (sampling frequency 10Hz). Following the vehicle-specific template in step one, the acquired data within a 1-second time window is converted into natural language text (e.g., "At timestamp 1678886500, the autonomous vehicle is accelerating; the average CAN bus voltage is 4.9V, with a peak of 5.2V; the LiDAR signal-to-noise ratio is 25.3dB, and the data packet loss rate is 3.5%; during this period, there were 4 CAN bus retransmissions and 2 CRC check errors, and the ECU reported fault code U0073; current electromagnetic environment: center interference frequency is 800MHz (source: 5G base station), field strength is 55dBμV / m;"). The text is input into the inference engine, performing forward propagation (no gating, no loss calculation, only prediction), outputting the probability distribution of 5 types of faults. The category with the highest probability is selected as the prediction result and sent to the cockpit display module via the vehicle bus (e.g., "Predicted fault: radar sensor interference, confidence level 96.3%").

[0081] In summary, this step seamlessly applies the model trained by this invention to real-world scenarios. Through a closed-loop process of "data acquisition - language processing - model inference - result interpretation," it ultimately delivers a high-precision, robust, and well-interpretable electromagnetic interference fault prediction capability, constituting the complete technical solution of this invention. Parts of this invention not described in detail are well-known technologies in the field.

[0082] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting electromagnetic interference faults by integrating hybrid loss and gated LoRA, characterized in that, The steps include the following: Step 1: Construct an electromagnetic interference fault dataset and perform linguistic preprocessing; Step 2: Initialize the large language model and integrate the low-rank adaptive LoRA module; Step 3: Construct a hybrid loss function that combines uncertainty and knowledge correctness in the fusion model. ; Step 4: Design a rank-dimensional gated vector generation mechanism based on hybrid loss; Step 5: Perform adaptive fine-tuning training to optimize model parameters; Step 6: Use the fine-tuned model to predict electromagnetic interference faults.

2. The electromagnetic interference fault prediction method integrating hybrid loss and gated LoRA according to claim 1, characterized in that: In step 1, specifically: the multi-source, heterogeneous system operation data is converted into a natural language text format that can be understood and processed by a large language model; the raw data includes timestamps, bus messages, sensor voltage / current readings, error frame counts, and system log text; The preprocessing step transforms these structured or semi-structured data entries into descriptive statements with clear semantics, based on a pre-defined template. Finally, the processed text sequence is paired with corresponding expert-annotated fault labels to form a dataset for model fine-tuning.<Prompt, Label> Dataset.

3. The electromagnetic interference fault prediction method based on a fusion of hybrid loss and gated LoRA as described in claim 1 or 2, characterized in that: In step 2, specifically: a pre-trained large-scale language model is selected as the base model; to achieve efficient parameter fine-tuning, a low-rank adaptive LoRA module is integrated into a specific layer of the model; the specific layer includes a query with a self-attention mechanism. s and keys Matrix; module in the original pre-trained weight matrix Next, a low-rank decomposition matrix is ​​connected in parallel. and It is a trainable low-rank matrix, where, The dimension is , The dimension is , The dimension is ,rank much smaller During forward propagation, the output of the corresponding layer of the model. Calculated by the following formula, where, It is the input feature vector of the corresponding layer of the model: ； During fine-tuning, the original weights Keep frozen, only update a much smaller number of parameters matrix and .

4. The electromagnetic interference fault prediction method integrating hybrid loss and gated LoRA according to claim 1, characterized in that: Loss of knowledge accuracy The standard cross-entropy loss function is used to measure the model's prediction results. Consistency with the true label y; the calculation formula is: ； in, This represents the total number of fault categories; The one-hot encoded vector of the real label Bit element, The model predicts that the sample belongs to the first... The probability value of the class, where, This loss term ensures that the model optimizes in the direction of learning the correct domain knowledge. Model uncertainty loss The entropy of the predicted probability distribution is used to quantify the model's uncertainty regarding the current input; high entropy indicates that the model's judgment of the likelihood of multiple categories is relatively average, i.e., high uncertainty; the calculation formula is: ； in, This represents the total number of fault categories; The model predicts that the sample belongs to the first... The probability value of the class, where, The uncertainty loss term reflects the model's cognitive state regarding its own knowledge system.

5. The electromagnetic interference fault prediction method based on hybrid loss and gated LoRA as described in claim 4, characterized in that: Hybrid loss function It is a weighted sum of model uncertainty and knowledge correctness: ； Among them, hyperparameters Used to balance the relative importance of the two loss terms.

6. The electromagnetic interference fault prediction method based on hybrid loss and gated LoRA according to claim 1, characterized in that: In step 4, specifically: the gating vector The generation process is formalized as follows: ； in, It is a small multilayer perceptron that takes scalar loss values Mapped to a dimensional vector; It is the Sigmoid activation function, which activates each element of the output vector. The constraint is within the interval (0, 1), where, For the rank dimension index of the LoRA module, Each element in The value corresponds to a rank dimension of the LoRA bypass matrix. The magnitude of the value represents the importance of the dimension in the current learning task or the necessity of updating it. A value close to 1 indicates that the dimension needs to be fully utilized and updated, while a value close to 0 indicates that the dimension should be suppressed to protect the original capabilities of the model or avoid invalid updates.

7. The electromagnetic interference fault prediction method based on hybrid loss and gated LoRA according to claim 1, characterized in that: In step 5, specifically: the generated gating vector This is applied to the LoRA update process to achieve adaptive fine-tuning; specifically, the original LoRA forward propagation formula is modified by changing the gate vector. Embedded in the form of a diagonal matrix: 。 8. The electromagnetic interference fault prediction method according to claim 7, characterized in that: It is a vector Convert to elements on the main diagonal The diagonal matrix; this operation is equivalent to the operation on LoRA. Each rank dimension is independently scaled or switched; during backpropagation and parameter updates, the gradient will be based on... The calculation is performed, but the gradient propagation path and its relationship with the target are different. The influence of the matrix will be affected by the diagonal matrix. The model dynamically adjusts itself; when faced with samples with high uncertainty and incorrect predictions, it generates a gating vector that allows more information to flow through, thereby enhancing the learning of new knowledge; conversely, it inhibits updates and maintains stability.

9. The electromagnetic interference fault prediction method based on hybrid loss and gated LoRA according to claim 1, characterized in that: In step 6, specifically: after multiple rounds of adaptive fine-tuning training, the model containing the optimized LoRA parameter matrices A and B is deployed; in practical applications, the system operation data collected on-site is processed in real time to generate input text. Inputting this text into the fine-tuned model will directly output the predicted fault type for the current system state.

10. The electromagnetic interference fault prediction method according to claim 9, characterized in that: Within each training batch, the adaptive fine-tuning process comprises the following sub-steps: (a) Forward Propagation and Loss Calculation: For each sample in the batch, the model first performs a standard forward propagation to calculate the predicted probability. Subsequently, based on and real labels Calculate the mixed loss ; (b) Gated vector generation: The input is fed into an MLP network to generate gating vectors corresponding to batches or samples. ; (c) Gated gradient calculation: using the gated forward propagation formula applied to LoRA: ； Using the standard training objective as the optimization objective, backpropagation is performed to calculate the gradients of matrices A and B. and ;because The calculation depends on ,and It itself depends on the model output. To simplify the calculation, when the gradient is backpropagated to A and B, It is treated as a fixed constant, so that it does not participate in the calculation of the gradient chain, and thus only serves as a weight factor to adjust the gradient magnitude; (d) Parameter update: Use the optimizer based on the calculated gradients and Update parameters A and B of the LoRA module.

Images