Private generation type large model optimization system based on hybrid genetic algorithm

Abstract

The invention discloses a private generation type large model optimization system based on a hybrid genetic algorithm, belongs to the technical field of artificial intelligence, and aims to solve the technical problems of how to compress the model scale, simplify the feature dimension to improve the reasoning efficiency and maintain the model precision as much as possible. Comprising a multi-source heterogeneous data fine adjustment module, a large model construction module, a hybrid genetic algorithm optimization module, a diffusion type encoder feature extraction module, a course type noise robustness improvement module, an activation function optimization module and a feature dimension reduction and low-rank compression module, the large model is gradually and deeply optimized from the aspects of data, structure, training, robustness, activation function and model compression, and a generative large model which can fully learn financial data of the new energy vehicle enterprise and is excellent in performance, reliable in robustness and friendly to resources is produced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a private generative large model optimization system based on a hybrid genetic algorithm. Background Technology

[0002] With the development of the new energy vehicle industry, related companies have accumulated massive amounts of financial data (such as sales revenue, costs, subsidies, and financing). How to utilize artificial intelligence models to model this private data to assist in financial forecasting, risk assessment, and decision support has become an important issue. Large-scale generative models (such as Large Language Models, LLM) have the potential to provide profound insights in such financial scenarios due to their powerful learning and generative capabilities. However, applying large models to the internal data of new energy vehicle companies faces several challenges: Firstly, corporate financial data comes from diverse sources and has heterogeneous formats, including structured numerical reports, text-based financial analyses, and time series data. Traditional methods often process a single data source individually, failing to fully integrate multi-source information, resulting in an insufficient understanding of the overall business picture. Directly mixing all data for training can lead to model convergence difficulties or bias towards a particular data source due to distribution differences. Secondly, training a high-performance generative large model typically requires fine-tuning of the model structure and hyperparameters. Standard gradient descent training is prone to getting stuck in local optima or is extremely sensitive to hyperparameter settings, resulting in high costs for manual parameter tuning. In addition, the massive parameter scale of large models makes global optimization difficult, and automatically searching for better-performing model configurations becomes a challenge. Furthermore, financial data often contains noise and anomalies (such as data entry errors, extreme market fluctuations, etc.). Models trained using conventional methods lack robustness to noise and exhibit instability in practical applications. Simple data cleaning is insufficient to cover all anomalies; therefore, it is necessary to improve the model's noise resistance during training to ensure its robustness to abnormal inputs.

[0003] Furthermore, the architecture and activation function selection of large models significantly impact training effectiveness and performance. While the traditional ReLU activation function is simple and efficient, it suffers from issues such as non-differentiability and "neuron death," limiting model performance. Newer activation functions like Swish (SiLU), due to their smoother operation, promise to improve gradient propagation, but their effectiveness needs to be validated in specific scenarios and their parameters optimized. Finally, large models often have a massive parameter scale, and deploying them directly in an enterprise's local environment consumes significant storage and computational resources. How to compress model size, simplify feature dimensions to improve inference efficiency, and simultaneously maintain model accuracy as much as possible is a key technical challenge that needs to be addressed. Summary of the Invention

[0004] The technical objective of this invention is to address the above-mentioned shortcomings by providing a private generative large model optimization system based on a hybrid genetic algorithm, in order to solve the technical problem of how to compress the model size, simplify the feature dimensions to improve inference efficiency, and at the same time maintain the model accuracy as much as possible.

[0005] This invention discloses a private generative large model optimization system based on a hybrid genetic algorithm, comprising:

[0006] A multi-source heterogeneous data fine-tuning module is used to collect multi-source heterogeneous financial data and perform data preprocessing on the multi-source heterogeneous financial data. Through data preprocessing, data alignment, data cleaning, format conversion and feature extraction are performed to obtain fused features, and a general large model is trained based on the fused features.

[0007] A large model building module is used to build a large model. The module initializes the large model for training based on the fusion features output by the multi-source heterogeneous data fine-tuning module. Generative objectives are added during the model training initialization process to obtain a pre-trained large model. The large model is a Transformer model containing an encoder and a decoder.

[0008] A hybrid genetic algorithm optimization module is used to optimize the large model and its hyperparameters output by the large model construction module through a hybrid genetic algorithm.

[0009] A diffusion encoder feature extraction module is used to extract robust features of financial data through the principle of diffusion model.

[0010] A curriculum-based noise robustness enhancement module is used to gradually increase the noise in the training data through a curriculum learning strategy.

[0011] An activation function optimization module is used to introduce and optimize the Swish activation function in a large model.

[0012] The feature dimensionality reduction and low-rank compression module is used to compress the feature representation and parameters of the large trained model, including feature dimensionality reduction and low-rank decomposition of the weight matrix.

[0013] Preferably, the multi-source heterogeneous data fine-tuning module is used to perform the following operations:

[0014] Multi-source data convergence and alignment: financial data is collected from different data sources, including structured, semi-structured, and unstructured data. Data cleaning and format conversion operations are performed on the financial data from different data sources, and the converted financial data is then aligned.

[0015] Heterogeneous feature representation and fusion: For different types of financial data, the aligned financial data is converted into feature representations that are adapted to the large model through the corresponding feature representation methods, and the feature representations are fused to obtain fused features;

[0016] Pre-training module fine-tuning and adaptation: Based on the fusion features, the general large model is fine-tuned to obtain the trained general large model and the processed fusion features.

[0017] Preferably, the large model building module is used to perform the following operations:

[0018] Construct a large model, which is a Transformer model including an encoder and a decoder. The encoder is used to extract features from the input data, and the decoder is used to generate prediction results based on the output of the encoder.

[0019] Model Training and Initialization: The large model constructed based on the transfer learning strategy is trained and initialized. During model training and initialization, the large model is pre-trained based on open-domain data and fine-tuned based on the enterprise's private data. The fine-tuned large model is then trained based on fusion features to fine-tune the model parameters, resulting in a pre-trained large model. During the training of the fine-tuned large model based on fusion features, a self-supervised learning-mask model reconstruction training task is performed on the large model to adjust the model parameters until convergence.

[0020] Generative capability optimization: Generative objectives are incorporated into model training. The large model generates analysis report paragraphs based on financial data and synthesizes reasonable new data from the learned knowledge.

[0021] Preferably, the hybrid genetic algorithm optimization module is used to perform the following operations:

[0022] Chromosome encoding and initial population: Define the key hyperparameters of the large model as chromosome representations for the genetic algorithm, and randomly initialize the population to obtain individuals with several different model configurations;

[0023] Fitness evaluation: Define a fitness function to evaluate the quality of each model configuration individual. Fitness is a comprehensive performance index of the large model on the validation set, including the accuracy of the prediction task, the loss value, and the penalty term for model complexity. For each population individual, train or adjust the model according to the configuration encoded by the population individual and calculate the validation set performance as the fitness score of the population individual.

[0024] Genetic operator operations: Based on fitness, the genetic algorithm is applied to the individuals in the population to generate candidate configurations for the next generation. High-fitness individuals are retained by performing selection operations. Crossover is performed on the selected high-fitness individuals. For each pair of individuals, a part of their chromosomes is exchanged to form a new population offspring configuration. Mutation operations are performed on the individuals obtained by crossover to randomly change the genes on the chromosomes—learning rate or network layer number genes—with a specified probability.

[0025] Hybrid local optimization strategy: During the iteration process of the genetic algorithm, a gradient descent local optimization strategy is introduced to complement the global evolution. For each candidate configuration individual, when evaluating fitness, a specified number of gradient descent training rounds are performed on the large model to optimize the weights.

[0026] Iterative evolution and termination conditions: Fitness evaluation, genetic operator operations and hybrid local optimization strategies are repeatedly applied to evolve the population for multiple generations. The model configuration of the large model is gradually optimized. After the iteration terminates, the individual with the highest fitness is selected from the final population, and the corresponding model configuration is taken as the optimal configuration. Based on the optimal configuration, the model is fully trained on the full training data to obtain the optimized final large model.

[0027] Preferably, the diffusion encoder feature extraction module is used to perform the following operations:

[0028] The progressively noisy data diffusion process: Define the diffusion process, and for any financial data sample, generate a noisy version of the fusion feature according to a Gaussian distribution;

[0029] Denoising encoder model training: The diffusion encoder is trained by training a parametric model. The input of the parametric model is the noisy version of the fused features, and the output is the denoised and reconstructed fused features. The training objective is to minimize the reconstruction error, which is the difference between the denoised and reconstructed fused features and the fused features corresponding to the financial data samples. In the diffusion model, the intermediate layer is extracted as the feature representation. The implicit representation vector extracted by the encoder from the noisy version of the fused features is regarded as the feature of the input data in a noisy environment.

[0030] Robust feature extraction and output: After model training, a diffusion encoder is used to extract feature representations for each new input data. A step size is selected at a specific intermediate time. The noisy version of the fused features is input into the encoder to obtain the latent representation. The latent representation is then output as the feature vector of the financial data sample.

[0031] As a preferred option, the course-based noise robustness enhancement module is used to perform the following operations:

[0032] Noise Course Design: Set up a course plan for noise injection during the training process. In the first stage of model training, use the original data for training. In the second stage of model training, add low-intensity noise to the input data. In the third stage of model training, add high-intensity noise to the input data. In the subsequent stages of model training, gradually add noise of a specified intensity to the input data until the final stage when the model learns in a strong noise environment. Set stopping conditions for each stage. When the model performance tends to stabilize under the noise level, proceed to the next stage.

[0033] Stepwise noise training: Following a predetermined course, noise is gradually introduced into the model training process. At each stage, data corresponding to the noise level is used to train the model parameters.

[0034] Robustness assessment and performance balance: After each stage of course training, the performance of the model under different noise conditions is evaluated, including the performance indicators under no noise, medium noise, and high noise conditions.

[0035] Preferably, the activation function optimization module is used to perform the following operations:

[0036] Swish activation function: Replace the original activation function in the large model with the Swish function;

[0037] Parametric Swish Adjustment: After replacing the original activation function in the large model with the Swish function, a parameter β is introduced in each layer, and the large model adjusts the shape of the activation function itself during training.

[0038] Training monitoring and performance improvement: Train large models and monitor metrics during training to evaluate the effectiveness of activation function optimization.

[0039] Preferably, the feature reduction and low-rank compression module is used to perform the following operations:

[0040] High-dimensional feature reduction: For high-dimensional feature representations generated in the middle of a large model and feature vectors of input data, dimensionality reduction techniques are applied to extract the main information. Dimensionality reduction techniques include principal component analysis and autoencoder hybrid dimensionality reduction methods.

[0041] Low-rank decomposition of model weights: For the parameter matrix inside a large model, a low-rank approximation method is used for compression;

[0042] Model weight training and calibration: For the model with compressed weights, use the parameters of the compressed model as initial values ​​and run gradient descent for a small number of epochs on the original training data to restore the accuracy lost by the large model during compression.

[0043] The private generative large model optimization system based on hybrid genetic algorithm of the present invention has the following advantages: the modules cooperate with each other to gradually optimize the large model in terms of data, structure, training, robustness, activation function and model compression, and finally produce a generative large model that can fully learn financial data, perform well, is robust and reliable, and is resource-friendly. This model and system can be used in application scenarios such as financial indicator prediction, business risk warning and intelligent report generation for new energy vehicle companies, so as to improve the company's ability to analyze and utilize its own data and make intelligent decisions. Detailed Implementation

[0044] The present invention will be further described below with reference to specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0045] This invention provides a private generative large model optimization system based on a hybrid genetic algorithm, which addresses the technical problem of how to compress model size, simplify feature dimensions to improve inference efficiency, and at the same time maintain model accuracy as much as possible.

[0046] Example:

[0047] This invention discloses a private generative large model optimization system based on a hybrid genetic algorithm, comprising a multi-source heterogeneous data fine-tuning module, a large model construction module, a hybrid genetic algorithm optimization module, a diffusion encoder feature extraction module, a curriculum-based noise robustness enhancement module, an activation function optimization module, and a feature dimensionality reduction and low-rank compression module.

[0048] The multi-source heterogeneous data fine-tuning module is used to collect multi-source heterogeneous financial data and perform data preprocessing on the multi-source heterogeneous financial data. Through data preprocessing, data alignment, data cleaning, format conversion and feature extraction are performed to obtain fused features, and the general large model is trained based on the fused features.

[0049] As a specific implementation of the multi-source heterogeneous data fine-tuning module, this module is used to perform the following operations:

[0050] (1) Multi-source data convergence and alignment: financial data is collected from different data sources, including structured data, semi-structured data and unstructured data. Data cleaning and format conversion operations are performed on the financial data from different data sources, and the format-converted financial data is then aligned.

[0051] (2) Heterogeneous feature representation and fusion: For different types of financial data, the aligned financial data is converted into feature representations that are adapted to the large model through the corresponding feature representation methods, and the feature representations are fused to obtain fused features;

[0052] (3) Fine-tuning and adaptation of pre-training module: Fine-tuning of general large model based on fusion features to obtain the trained general large model and the processed fusion features.

[0053] The multi-source heterogeneous data fine-tuning module is responsible for collecting, aligning, and fine-tuning multi-source heterogeneous financial data, providing comprehensive and clean training corpora for large-scale model training. Since the data from new energy vehicle companies comes from multiple systems and departments with varying formats, this module unifies and preprocesses the data, and performs initial fine-tuning to adapt the model to the multi-source data distribution. The main functions of this module are as follows:

[0054] (1) Multi-source data aggregation and alignment: Obtain financial-related data from various sources within the enterprise, including structured numerical tabular data (such as financial statement indicators), semi-structured data (such as sales records and budget entries), and unstructured text data (such as financial comments). For data from different sources, perform cleaning and format conversion, such as handling missing values ​​and outliers, and standardizing time formats and units of measurement. Then, align the heterogeneous data according to business entities or time dimensions to build a comprehensive dataset, providing a multi-dimensional and consistent input view for subsequent model training.

[0055] (2) Heterogeneous Feature Representation and Fusion: For different types of data, appropriate feature representation methods are adopted to convert them into an input form acceptable to the model: numerical indicators are standardized and Z-score regularized to eliminate differences in units; time series data are extracted using a sliding window to extract sequence features; and text data are extracted using a pre-trained model in the financial field using a large language model to extract semantic vector representations. Heterogeneous data are mapped to a unified feature space for use by the model, realizing feature-level fusion of financial information.

[0056] (3) Fine-tuning and Adaptation of the Pre-trained Model: Based on the fused features, the pre-existing basic model is fine-tuned to initially adapt it to the distribution and patterns of the enterprise's multi-source data. Specifically, a general large model (such as a pre-trained large language model) is used as a starting point and further trained on the processed enterprise data. On the one hand, this allows the model to quickly inherit knowledge from the enterprise's financial data and improve its sensitivity to professional domains; on the other hand, through joint training with multi-source data, the model receives a more comprehensive knowledge injection. After this step, the output preliminary model and the processed data will serve as the foundation for the subsequent construction of a large model.

[0057] The large model building module is used to build a large model. It performs model training and initialization on the large model based on the fusion features output by the multi-source heterogeneous data fine-tuning module. Generative objectives are added during the model training and initialization process to obtain a pre-trained large model. The large model is a Transformer model containing an encoder and a decoder.

[0058] In this embodiment, the large model building module is used to perform the following operations:

[0059] (1) Construct a large model, which is a Transformer model including an encoder and a decoder. The encoder is used to extract features from the input data, and the decoder is used to generate prediction results based on the output of the encoder.

[0060] (2) Model training and initialization: The large model constructed is trained and initialized based on the transfer learning strategy. During the initialization of model training, the large model is pre-trained based on open domain data and fine-tuned based on the enterprise's private data. The fine-tuned large model is trained based on fusion features to fine-tune the model parameters and obtain the initially trained large model. Among them, when the fine-tuned large model is trained based on fusion features, the self-supervised learning-mask model reconstruction training task is performed on the large model to adjust the model parameters until convergence.

[0061] (3) Generative capability optimization: Generative objectives are added to the model training. The large model generates analysis report paragraphs based on financial data and synthesizes reasonable new data from the learned knowledge.

[0062] This module is used to design and build the architecture of a generative large model and to train and initialize the model based on the results of the S1 stage. While ensuring the model learns sufficiently from the financial data, it employs an advanced network structure to provide the necessary capacity and generative capabilities. This invention uses a Transformer-based deep neural network as the large model architecture, whose self-attention mechanism can effectively capture long-range correlations between various elements in the financial data. The main functions of the module are as follows:

[0063] (1) Model Architecture Design: Based on the characteristics and application requirements of financial data from new energy vehicle companies, a Transformer model with an encoder-decoder architecture was constructed: the encoder extracts the latent representation of the input data (including numerical sequences and text descriptions), and the decoder generates prediction results or text output based on the encoded representation. The network consists of multiple layers of self-attention and feedforward sublayers stacked together, with residual connections and layer normalization used after each layer to ensure training stability. The model parameter scale is set in the range of hundreds of millions to billions, thus possessing sufficient expressive power to characterize the complex patterns of financial data—increasing the model parameter scale has been proven to bring about a leap in model performance; for example, GPT-3 exhibits stronger creativity and understanding compared to smaller models. By reasonably increasing the model depth and width, it is ensured that the model has sufficient capacity to fit complex financial feature relationships.

[0064] (2) Model Training and Initialization: After the model architecture is determined, the model is trained and initialized. This patent utilizes a transfer learning strategy: first, the model is pre-trained using large-scale open-domain data to learn general language and pattern representations; then, the training is fine-tuned on private enterprise data to enable the model to master domain-specific knowledge. This is similar to pre-training on publicly available corpora such as financial news and industry reports, and then fine-tuning the parameters using internal data from new energy vehicle companies. Through two-stage training, the model possesses general language understanding capabilities and incorporates specialized knowledge from the enterprise's financial domain. In addition, for numerical features, an embedding layer is used to map them into vectors, which are then fused with text features and input into the model, thereby achieving the construction of a large multimodal fusion model. During the training process, the comprehensive dataset prepared in stage S1 is used to perform a self-supervised learning-mask model reconstruction training task on the model, adjusting the model parameters until convergence.

[0065] (3) Generative Capability Optimization: Generative objectives are incorporated into the training process, allowing the model to generate analytical report paragraphs based on financial data and simulate the financial indicator sequence for several future quarters. This enables the model to synthesize reasonable new data from its learned knowledge. The model output becomes more diversified, supporting the answering of financial questions and the generation of risk warning prompts, thus expanding the model's application scope. After processing by this module, a preliminarily trained large model is obtained, possessing the basic ability to characterize and generate corporate financial data.

[0066] The hybrid genetic algorithm optimization module is used to optimize the large model and its hyperparameters, which are initially trained and output by the large model building module, using a hybrid genetic algorithm.

[0067] As a specific implementation of the hybrid genetic algorithm optimization module, this module is used to perform the following operations:

[0068] (1) Chromosome encoding and initial population: Define the key hyperparameters of the large model as the chromosome representation of the genetic algorithm, and randomly initialize the population to obtain several individuals with different model configurations;

[0069] (2) Fitness evaluation: Define a fitness function to evaluate the quality of each model configuration individual. Fitness is a comprehensive performance index of the large model on the validation set, including the accuracy of the prediction task, the loss value and the penalty term for model complexity. For each population individual, train or adjust the model according to the configuration encoded by the population individual and calculate the validation set performance as the fitness score of the population individual.

[0070] (3) Genetic operator operation: Based on the fitness level, the genetic algorithm is applied to the population individuals to generate the next generation candidate configuration. The high fitness population individuals are retained by performing the selection operation. Crossover is performed on the selected high fitness population individual pairs. For each population individual pair, a part of their respective chromosomes is exchanged to form a new population offspring configuration. The crossover individuals are subjected to mutation operation to randomly change the gene-learning rate or network layer number gene on the chromosome with a specified probability.

[0071] (4) Hybrid local optimization strategy: During the iteration process of the genetic algorithm, a gradient descent local optimization strategy is introduced to complement the global evolution. For each candidate configuration individual, when evaluating fitness, a specified number of gradient descent training rounds are performed on the large model to optimize the weights.

[0072] (5) Iterative evolution and termination conditions: Repeatedly evaluate fitness, perform genetic operator operations and hybrid local optimization strategies to evolve the population for multiple generations. The model configuration of the large model is gradually optimized. After the iteration terminates, the individual with the highest fitness is selected from the final population, and the corresponding model configuration is taken as the optimal configuration. Based on the optimal configuration, the model is fully trained on the full training data to obtain the optimized final large model.

[0073] This embodiment employs a hybrid genetic algorithm to optimize the large model and its hyperparameters constructed in stage S2, aiming to further improve model performance and find a better model configuration. This invention combines a genetic algorithm with gradient descent local optimization techniques to accelerate model optimization convergence to the global optimum. The specific functions of the module are as follows:

[0074] (1) Chromosome Encoding and Initial Population: First, the chromosome representation of the genetic algorithm is defined: the key hyperparameters of the model (learning rate, regularization coefficient, number of layers, number of neurons per layer). The population is randomly initialized to obtain several individuals with different model configurations. To improve the quality of the initial population, the default configuration of the S2 stage is introduced as part of the population individuals, thus having a certain performance foundation at the beginning of the search.

[0075] (2) Fitness Evaluation: A fitness function is defined to evaluate the performance of each individual (model configuration). Fitness is a comprehensive performance metric of the model on the validation set: accuracy of the prediction task, loss value, and a penalty term for model complexity, thereby achieving multi-objective optimization. For each individual in the population, the model is trained or adjusted according to its encoded configuration, and the validation set performance is calculated as the fitness score for that individual. Since the cost of a single full training of a large model is high, a strategy of shortening the number of training generations is adopted to speed up fitness evaluation.

[0076] (3) Genetic Operator Operations (Selection, Crossover, Mutation): Based on fitness levels, genetic algorithm evolutionary operations are applied to individuals in the population to generate candidate configurations for the next generation. First, a selection operation is performed, retaining individuals with high fitness through a roulette wheel selection process to increase the probability of selecting individuals with high fitness. Next, crossover is performed on the selected pairs of individuals, exchanging parts of their chromosomes to create new offspring configurations. Then, mutation is performed on the individuals obtained from the crossover, randomly changing certain genes on the chromosomes—learning rate or network layer number genes—with a specified probability to introduce diversity and prevent getting trapped in local optima. After selection, crossover, and mutation, a new generation of model configurations is obtained.

[0077] (4) Hybrid Local Optimization Strategy: This invention introduces a gradient descent local optimization strategy during the genetic algorithm iteration process, complementing global evolution. This module employs a hybrid optimization of GA and gradient descent: for each candidate configuration individual, when evaluating its fitness, the model undergoes a specified number of gradient descent training rounds to optimize the weights, making the fitness evaluation more accurate. Therefore, this module significantly improves the efficiency and effectiveness of the model's configuration search through hybrid optimization.

[0078] (5) Iterative Evolution and Termination Condition: Repeat steps (2) to (4) continuously to allow the population to evolve through multiple generations, gradually optimizing the model configuration. Stop when the fitness improvement is below the threshold for several consecutive generations. After termination, select the individual with the highest fitness from the final population; its corresponding model configuration is considered the optimal configuration. Then, train the model on the full training data according to this configuration to obtain the optimized final model. Through this genetic algorithm optimization module, the key hyperparameters of the model are automatically tuned, and the network structure is also improved, which is more efficient and reliable than manual experience-based tuning. The hybrid genetic algorithm used in this invention can find a global near-optimal solution in a large search space, avoiding the defect of conventional gradient methods getting trapped in local optima.

[0079] Using 200,000 anonymized financial customer service conversations from an internal company as the experimental subject, the total data volume was approximately 12GB; 92% was used as the training set and 8% as the validation set. The base model was a six-layer Encoder-DecoderTransformer (512 hidden dimensions, 8 multi-heads), followed by a genetic algorithm combined with gradient local optimization to search for the optimal network structure and hyperparameters. Based on the publicly available steps, the specific operations are as follows:

[0080] (1) Chromosome Design and Population Initialization: The five key hyperparameters—number of network layers, hidden dimension, learning rate, activation function type, and Dropout ratio—are encoded into a fixed-length 18-bit "chromosome." The first 4 bits represent the number of layers from 4 to 12, the next 6 bits map to the hidden dimension from 256 to 1024, and the last 4 bits are mapped to 1×10 through exponential mapping. -5 ~5×10 -4 The learning rate is used, with a single bit used to distinguish between the GELU and Swish activation functions, and the last 3 bits discretely representing the Dropout ratio of 1% to 35%. The system randomly generates 64 individuals in the encoding space as the initial population.

[0081] (1) Fitness function and evolution mechanism: The fitness function takes into account the validation set loss, BLEU score and model size, and is defined as follows:

[0082] F = 0.6(1-loss) + 0.3BLEU - 0.1log10(#parameter / 106)F

[0083] =0.6(1-\text{loss})+0.3\text{BLEU}

[0084] -0.1\log_{10}(\#text{parameter} / 10 6 )

[0085] = 0.6(1-loss) + 0.3BLEU - 0.1log10(#parameter / 106)

[0086] The evolutionary process employs tournament selection, single-point crossover, and an 8% mutation rate. To avoid slow convergence of pure genetic evolution, three rounds of AdamW fine-tuning are performed on each individual before each generation evaluation, complementing global search with local gradient descent. The population iteration limit is set at 20 generations or termination occurs if the fitness improvement is less than 0.5% over two consecutive generations.

[0087] (3) Experimental Environment and Results: All experiments were completed on two NVIDIA A800 GPUs (80GB each), with a total training time of approximately 6 hours. The final optimal individual code "01011000111011" obtained through iteration corresponds to an eight-layer network with 675 hidden dimensions and a learning rate of 2.7 × 10⁻⁶. -4 The activation functions were Swish and Dropout (18%). Compared to the baseline model, BLEU score improved from 28.1 to 34.7, validation loss decreased from 2.31 to 1.92, and the number of parameters was reduced from 110M to 93M, representing improvements of 23.5%, 16.9%, and 15.5%, respectively.

[0088] The diffusion encoder feature extraction module is used to extract robust features from financial data based on the diffusion model principle.

[0089] As a specific implementation of the feature extraction module of the diffusion encoder, this module is used to perform the following operations:

[0090] (1) Gradual Noise-Adding Data Diffusion Process: Define the diffusion process. For any financial data sample, generate a noisy version of the fusion feature according to a Gaussian distribution.

[0091] (2) Denoising encoder model training: The diffusion encoder is trained by training a parameterized model. The input of the parameterized model is the noisy version of the fusion feature, and the output is the denoised and reconstructed fusion feature. The training objective is to minimize the reconstruction error. The reconstruction error is the difference between the denoised and reconstructed fusion feature and the fusion feature corresponding to the financial data sample. In the diffusion model, the intermediate layer is extracted as the feature representation. The implicit representation vector extracted by the encoder from the noisy version of the fusion feature is regarded as the feature of the input data in the noisy environment.

[0092] (3) Robust feature extraction and output: After model training, a diffusion encoder is used to extract feature representations for each new input data. The step size at a specific intermediate time is selected, and the noisy version of the fused features is input into the encoder to obtain the latent representation. The latent representation is then output as the feature vector of the financial data sample.

[0093] This module extracts robust features from data using the diffusion model principle, providing high-quality feature representations for subsequent model training and inference. The diffusion encoder learns the internal structure of the data through a process of progressive noise addition and denoising, obtaining representations of key features in the denoising and reconstruction task, thereby extracting robust features that are insensitive to noise. The main functions of this module include:

[0094] (1) Gradual Noise-Adding Data Diffusion Process: First, define the diffusion process, which involves gradually adding noise to the original input data. For any financial data sample x0 (a feature vector or sequence), generate noisy versions x1, x2, ..., x according to a Gaussian distribution.T Where t steps correspond to sample x t In x t-1 Random noise of a specified intensity is added to the base data, with the noise intensity increasing with t. The diffusion process constructs a path from the original data to complete noise, laying the foundation for training the denoising model.

[0095] (2) Denoising Encoder Model Training: Train the diffusion encoder to reconstruct the original input from noisy data. One implementation is to train a parameterized model f. θ Let its input be a noisy sample x t The output is the denoised reconstruction. (Attempting to approximate the original noise-free x0). The training objective is to minimize the reconstruction error. Because the model needs to reconstruct meaningful original data from inputs that are overwhelmed with noise, it is forced to learn the underlying structure and key features of the data. Furthermore, in the diffusion model, intermediate layers are extracted as feature representations: the encoder is applied to x... t Extracted hidden representation vector h t This is considered a feature of the input data in a noisy environment. This is because: when t is moderate, h... t It contains the key information of the original data while filtering out most of the noise. In this way, the trained encoder can be regarded as a feature extractor, mapping the original data into robust low-dimensional feature vectors.

[0096] (3) Robust Feature Extraction and Output: After model training, for each new input data, a diffusion encoder is used to extract its feature representation. The step size t at intermediate time points is selected. ^ Noisy sample x t ^ Input encoder to obtain hidden representation h t The features extracted from the original data are then used as feature vectors for that sample. These feature vectors encompass the main variations in the original data but are insensitive to noise, and are used for subsequent large-scale model training (noise-robust training) to improve the model's ability to grasp key patterns. Compared to traditional feature extraction methods, the diffusion encoder, by explicitly modeling the impact of noise during training, extracts features that are more capable of ignoring anomalous perturbations, thus helping the overall system maintain performance in noisy environments.

[0097] The course-based noise robustness enhancement module is used to gradually increase the noise in the training data through a course learning strategy.

[0098] As a specific implementation of the course-based noise robustness enhancement module, this module is used to perform the following operations:

[0099] (1) Noise course design: Set up a course plan for noise injection during the training process. In the first stage of model training, use the original data for training. In the second stage of model training, add low-intensity (signal-to-noise ratio: SNR≥30db) noise to the input data. In the third stage of model training, add high-intensity (signal-to-noise ratio: 10dB≤SNR≤30db) noise to the input data. In the subsequent stages of model training, gradually add noise of a specified intensity to the input data until the model learns in a strong noise environment in the final stage. Set the stopping conditions for each stage. When the model performance tends to stabilize under the noise level, enter the next stage.

[0100] (2) Gradual noise training: According to the predetermined course, noise is gradually introduced into the model training process. At each stage, data of the corresponding noise level is used to train the model parameters.

[0101] (3) Robustness assessment and performance balance: After each stage of course training, the performance of the model under different noise conditions is evaluated, including the performance indicators under no noise (signal-to-noise ratio: SNR≥30dB), medium noise (signal-to-noise ratio: 20dB≤SNR≤30dB) and high noise (signal-to-noise ratio: 10dB≤SNR≤20dB).

[0102] This module enhances the model's robustness to noise and anomalies by gradually increasing the noise in the training data using a curriculum-based learning strategy. Curriculum learning borrows from the human concept of progressive learning, training the model from easy to difficult data. This module applies this idea to noise-enhanced training, allowing the model to gradually adapt to the transition from noiseless to noisy data, significantly improving its tolerance to noise without sacrificing original performance. The specific process of this module is as follows:

[0103] (1) Noise Curriculum Design: A curriculum scheme for noise injection during training is set. In the early stages of training, the model mainly encounters clean data or data samples with very small perturbations. Subsequently, the difficulty is gradually increased according to the curriculum: as training progresses, the noise level in the training data is increased round by round. The first stage uses the original data for training; the second stage begins to add slight noise (1% Gaussian noise) to the input data; the third stage increases the noise intensity to 5%; and so on, gradually increasing the difficulty until the final stage when the model can still learn in a strong noise environment. At the same time, a stopping condition is set for each stage: when the model performance tends to stabilize at the noise level, it automatically enters the next stage.

[0104] (2) Gradual Noise Training: Noise is gradually introduced into the model training process according to a predetermined curriculum. At each stage, data corresponding to the noise level is used to train the model parameters. Because the initial stage is almost noise-free, the model can first learn the basic patterns; subsequently, as the noise increases, the model further learns how to ignore noise interference based on the patterns it has already mastered. Through this gradual training strategy, the model is not immediately overwhelmed by a large amount of noise interference, avoiding the convergence difficulties caused by training directly on noisy data. Simultaneously, the model continuously updates its parameters to adapt to higher noise levels, forming a memory of the noise patterns. Furthermore, whenever a higher noise level is reached, the learning rate is appropriately reduced by 2% for robust convergence, and the training iterations are increased by 1% to ensure the model is fully adapted to that noise level.

[0105] (3) Robustness Assessment and Performance Balancing: After each stage of the course training, the model's performance under different noise conditions is evaluated, including performance metrics under noise-free (original data), moderate noise, and high noise conditions. The goal of course-based training is to significantly improve performance on noisy data without significantly reducing performance on noise-free data. Therefore, it was found that the model's accuracy on noise-free data decreased significantly after training at a certain noise level. Appropriately extending the training in the low-noise stage or adding mixed noise-free samples for training was recommended to consolidate the basic capabilities. Ultimately, the model trained with course-based noise robustness training maintains robust predictive ability across a range from clean data to extremely noisy data. Compared to the model without this module, the course-trained model shows a significant decrease in error rate under noisy conditions, while the accuracy remains essentially unchanged under clean conditions, ensuring both robustness and accuracy. This module is particularly important for the financial data scenarios of new energy vehicle companies, as financial data is affected by data entry errors, abnormal transactions, etc. Robust models can effectively avoid prediction inaccuracies caused by individual abnormal data points, providing more reliable support for decision-making.

[0106] To verify the robustness of curriculum-based noise training for OCR financial text, this embodiment selects 50,000 scanned financial statements, which are then transcribed into approximately 650,000 text segments after OCR. The curriculum strategy divides noise injection into five stages: First, three rounds of pre-training are performed using noise-free data. Once the loss is verified to be stable, a low-noise stage is entered, where Gaussian noise with a signal-to-noise ratio ≥30dB is added to all samples. Subsequently, the training transitions sequentially to medium noise (20-30dB), high noise (10-20dB), and specified high noise (fixed at 18dB, injected only into 25% of samples). Each stage is trained for two rounds (only one round for high noise), with BLEU or the loss convergence threshold used as transition conditions.

[0107] The AdamW optimizer was used throughout the model training process, with a learning rate of 2×10⁻⁶. -4The batch size was 32, and gradient accumulation was performed in 2 steps. After the course, in a noisy (15dB) scenario, the model's BLEU score improved from 21.8 in a single noisy training session to 29.3, an increase of 34.4%; while in the noise-free condition, the BLEU score only decreased by 0.2 percentage points, demonstrating that the course-based strategy significantly enhances noise robustness with almost no loss to the original performance.

[0108] The activation function optimization module is used to introduce and optimize the Swish activation function in large models.

[0109] As a specific implementation of the activation function optimization module, this module is used to perform the following operations:

[0110] (1) Swish activation function: Replace the original activation function in the large model with the Swish function;

[0111] (2) Parametric Swish adjustment: After replacing the original activation function in the large model with the Swish function, a parameter β is introduced in each layer, and the large model adjusts the shape of the activation function itself during training.

[0112] (3) Training monitoring and performance improvement: Train large models and monitor metrics during the training process to evaluate the effect of activation function optimization.

[0113] This module introduces and optimizes the Swish (SiLU) activation function in large models, aiming to replace traditional activation functions and improve the model's training convergence and expressive power. It is defined as: Swish(x) = x * sigmoid(x), where sigmoid is the Sigmoid function. When sigmoid(x) takes values ​​between 0 and 1, Swish performs a non-linear transformation on the input, scaling it according to the value. It approximates an identity mapping when x is positive and provides a non-zero output when x is negative (unlike ReLU, which directly outputs 0). The entire function is smooth, continuous, and differentiable with respect to x. The main implementation steps of this module are as follows:

[0114] (1) Swish activation function: Replace the original activation function (such as ReLU, Leaky ReLU, etc.) in the large model with the Swish function. In deep network structures such as Transformer, the activation function usually appears after each feedforward fully connected layer. Replace the ReLU in these positions with Swish, so that the output of each layer is calculated as h = Swish(W*x+b). Since Swish is similar to ReLU in shape (both suppress negative values ​​and maintain linear growth for positive values) but is smoother, it injects nonlinear characteristics into the model with almost no change to the network structure. Swish can also work normally for existing batch normalization, residual and other structures. In addition, for other nonlinear units, use the SiLU form of activation instead of the traditional Sigmoid / Tanh.

[0115] (2) Parameterized Swish Adjustment: f(x) = x * sigmoid(β * x + b), where β is a trainable parameter. When β = 1, it is the standard Swish function; when β = 0, it degenerates into the identity function. In this application, a β parameter is introduced for each layer, allowing the network to adjust the shape of the activation function during training. Specifically, β is initialized to 1, and the gradient with respect to β is calculated and updated during backpropagation. Through this parameterization, the model adjusts the activation response level according to the feature distribution of different layers, further improving the flexibility of the activation function and the expressive power of the model.

[0116] (3) Training Monitoring and Performance Improvement: Train the model and monitor the metrics during the training process to evaluate the effect of activation function optimization. Typically, the following can be observed: faster convergence speed and smoother loss descent in the early stages of training; and improved performance on the validation set. This is because the smoothness of the Swish function makes the gradient during backpropagation more stable, unlike the ReLU function where the gradient is 0 in the negative region. Furthermore, the Swish function output is more sensitive to small changes in the input (especially in the region where x is close to 0, the derivative is not constant), giving the model a more nuanced ability to characterize input changes, thus improving the sensitivity to small fluctuations in financial data prediction scenarios. In contrast, while traditional ReLU is computationally simple, its gradient is zero when x < 0, limiting the representational power of some network units; Sigmoid / Tanh, on the other hand, exhibits gradient vanishing when x is large. Swish effectively combines the advantages of ReLU and Sigmoid: it retains the linear pass-through characteristic for positive values ​​while providing a specified degree of flowing gradient in the negative region, thereby comprehensively improving model performance. With the optimization of this module, large models can usually achieve lower validation loss within the same training period.

[0117] For large trained models, the feature dimensionality reduction and low-rank compression modules are used to compress the feature representations and parameters of the large model, including feature dimensionality reduction and low-rank decomposition of the weight matrix.

[0118] As a specific implementation of the feature dimensionality reduction and low-rank compression module, this module is used to perform the following operations:

[0119] (1) Dimensionality reduction of high-dimensional features: For the high-dimensional feature representations generated in the middle of the large model and the feature vectors of the input data, dimensionality reduction techniques are applied to extract the main information. Dimensionality reduction techniques include principal component analysis (PCA) and autoencoder hybrid dimensionality reduction method.

[0120] (2) Low-rank decomposition of model weights: For the parameter matrix inside a large model, a low-rank approximation method is used for compression;

[0121] (3) Model weight training and correction: For the model with compressed weights, the parameters of the compressed model are used as the initial values, and gradient descent is run on the original training data for a small number of epochs to restore the accuracy lost by the large model during the compression process.

[0122] After model training, this module compresses and simplifies the model's feature representations and parameters, including feature dimension reduction and low-rank decomposition of the weight matrix. This significantly reduces model complexity and improves inference efficiency and deployment feasibility while minimizing performance loss. The specific implementation steps are as follows:

[0123] (1) High-dimensional feature reduction: For the high-dimensional feature representations generated in the middle of the large model and the feature vectors of the input data, dimensionality reduction techniques are applied to extract the main information. After the financial data is fused, a high-dimensional feature representation is formed. This module adopts a hybrid dimensionality reduction method of principal component analysis (PCA) and autoencoder. PCA decomposition is performed on the feature vectors of all training samples, and several principal components with a cumulative contribution rate of more than 95% are selected as new feature bases. In this way, the original n-dimensional feature vector of each sample is compressed into k-dimensional (k is much smaller than n) principal component features, while retaining most of the variance information. For features with significant nonlinear relationships, autoencoder dimensionality reduction is used: a bottleneck structure autoencoder is trained to make the dimension of the intermediate hidden layer much smaller than the input dimension, so that the hidden layer output is used as a low-dimensional feature. Through feature dimensionality reduction, redundant related information and noise components in the features can be removed, reducing the data scale of subsequent processing, which is conducive to improving the model inference speed and reducing the risk of overfitting.

[0124] (2) Low-rank decomposition of model weights: For large-scale parameter matrices within a large model, a low-rank approximation method is used for compression. The weight matrices of deep models often have parameter redundancy, and their actual effective rank is much smaller than the matrix dimension. Therefore, this module performs singular value decomposition (SVD) on these matrices, truncating low-contribution singular values ​​to approximate the original matrix. Specifically, for a weight matrix W (of size m×n), its SVD decomposition is calculated as: W=U×Σ×V T Where Σ represents the diagonal singular value matrix, then the largest r singular values ​​in Σ are selected (along with the corresponding first r columns U and first r rows V). T Discard the remaining smaller singular values. Thus, W is approximated as U. r ×Σ r ×V rT , among which, U r It is m×r, V rT It is r×n, Σ r It is an r×r diagonal matrix. By choosing an appropriately small r, a matrix with storage of m×n is compressed to storage of m×r plus r×n. When r is much smaller than the smaller of m and n, the compression ratio is quite considerable. Different rank thresholds are set for the weights of each layer to ensure that each layer retains a sufficient energy proportion. At the same time, the corresponding bias terms are also processed when performing low-rank approximation. Low-rank compression is performed on the weights of fully connected layers and embedded layers, which account for the majority of parameters, because these layers usually have the highest parameter proportion and the greatest compression benefit.

[0125] (3) Model Retraining and Correction: After dimensionality reduction, the model's input or intermediate representation changes, requiring a fixed dimensionality reduction transformation. The remaining part of the model is then retrained to adapt to the new feature space. Therefore, this module uses the compressed model's parameters as initial values ​​and performs gradient descent on the original training data for a small number of epochs (10% of the original epochs) to restore the accuracy lost during compression. After this step, the model's performance approaches the level before compression, but the model size and inference time are significantly reduced. Through this module's processing, the final model has significantly reduced parameters and computational overhead. This is because the dense layer matrix after low-rank decomposition significantly reduces storage requirements and improves model inference speed; the dimensionality-reduced features result in smaller vector dimensions required for each calculation, reducing matrix operations. Overall, the reduced model size makes it easier to deploy in existing enterprise hardware environments, including resource-constrained local servers or even in-vehicle terminals, thus enabling real-world applications. At the same time, appropriate compression can also play a regularization role, preventing the model from overfitting the training data and improving generalization ability. Compared to the uncompressed large model, the compressed model achieves an order-of-magnitude improvement in inference efficiency, while the prediction accuracy only slightly decreases or remains almost unchanged (after retraining correction). Therefore, this invention successfully solves the resource bottleneck of deploying large models in enterprise private environments through feature dimensionality reduction and low-rank compression modules, providing new energy vehicle companies with an AI model solution that combines performance and efficiency.

[0126] This invention achieves comprehensive optimization of a private large-scale model, from data to model to deployment. First, based on fine-tuning and fusion of multi-source heterogeneous data, the model can ingest richer financial information, improving its ability to capture complex business patterns and significantly enhancing prediction accuracy. Second, the introduction of large-scale model construction and global optimization using hybrid genetic algorithms more effectively mines the optimal combination of model structure and hyperparameters compared to traditional manual parameter tuning, improving model performance while reducing development cycle. Third, through diffusion coding and course learning noise processing, the model's noise resistance and robustness are greatly improved, maintaining stability and reliability even under abnormal data or perturbation inputs. Furthermore, improved activation functions such as Swish are used to accelerate model convergence and improve accuracy. Finally, combining feature reduction and low-rank approximation techniques to compress the model significantly reduces model parameter redundancy and runtime overhead, facilitating efficient deployment of the model in local environments. This invention provides new energy vehicle companies with a systematic and comprehensive AI modeling optimization method that can fully mine the value of corporate financial data while ensuring data privacy, improving the development efficiency, prediction accuracy, and practical application effects of large-scale models. Simultaneously, this invention also provides a system architecture corresponding to the above methods. The system consists of the following functional modules: a multi-source heterogeneous data fine-tuning module, a large model construction module, a hybrid genetic algorithm optimization module, a diffusion encoder feature extraction module, a curriculum-based noise robustness enhancement module, a Swish activation function optimization module, and a feature dimensionality reduction and low-rank compression module. Each module works collaboratively, following the above-described workflow, to jointly optimize the private generative large model.

[0127] The above provides a detailed description of a private generative large model optimization system based on a hybrid genetic algorithm provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A private generative large model optimization system based on a hybrid genetic algorithm, characterized in that, include: A multi-source heterogeneous data fine-tuning module is used to collect multi-source heterogeneous financial data and perform data preprocessing on the multi-source heterogeneous financial data. Through data preprocessing, data alignment, data cleaning, format conversion and feature extraction are performed to obtain fused features, and a general large model is trained based on the fused features. A large model building module is used to build a large model. The module initializes the large model for training based on the fusion features output by the multi-source heterogeneous data fine-tuning module. Generative objectives are added during the model training initialization process to obtain a pre-trained large model. The large model is a Transformer model containing an encoder and a decoder. A hybrid genetic algorithm optimization module is used to optimize the large model and its hyperparameters output by the large model construction module through a hybrid genetic algorithm. A diffusion encoder feature extraction module is used to extract robust features of financial data through the principle of diffusion model. A curriculum-based noise robustness enhancement module is used to gradually increase the noise in the training data through a curriculum learning strategy. An activation function optimization module is used to introduce and optimize the Swish activation function in a large model. The feature dimensionality reduction and low-rank compression module is used to compress the feature representation and parameters of the large trained model, including feature dimensionality reduction and low-rank decomposition of the weight matrix.

2. The private generative large model optimization system based on hybrid genetic algorithm according to claim 1, characterized in that, The multi-source heterogeneous data fine-tuning module is used to perform the following operations: Multi-source data convergence and alignment: financial data is collected from different data sources, including structured, semi-structured, and unstructured data. Data cleaning and format conversion operations are performed on the financial data from different data sources, and the converted financial data is then aligned. Heterogeneous feature representation and fusion: For different types of financial data, the aligned financial data is converted into feature representations that are adapted to the large model through the corresponding feature representation methods, and the feature representations are fused to obtain fused features; Pre-training module fine-tuning and adaptation: Based on the fusion features, the general large model is fine-tuned to obtain the trained general large model and the processed fusion features.

3. The private generative large model optimization system based on hybrid genetic algorithm according to claim 1, characterized in that, The large model building module is used to perform the following operations: Construct a large model, which is a Transformer model including an encoder and a decoder. The encoder is used to extract features from the input data, and the decoder is used to generate prediction results based on the output of the encoder. Model Training and Initialization: The large model constructed based on the transfer learning strategy is trained and initialized. During model training and initialization, the large model is pre-trained based on open-domain data and fine-tuned based on the enterprise's private data. The fine-tuned large model is then trained based on fusion features to fine-tune the model parameters, resulting in a pre-trained large model. During the training of the fine-tuned large model based on fusion features, a self-supervised learning-mask model reconstruction training task is performed on the large model to adjust the model parameters until convergence. Generative capability optimization: Generative objectives are incorporated into model training. The large model generates analysis report paragraphs based on financial data and synthesizes reasonable new data from the learned knowledge.

4. The private generative large model optimization system based on hybrid genetic algorithm according to claim 1, characterized in that, The hybrid genetic algorithm optimization module is used to perform the following operations: Chromosome encoding and initial population: Define the key hyperparameters of the large model as chromosome representations for the genetic algorithm, and randomly initialize the population to obtain individuals with several different model configurations; Fitness evaluation: Define a fitness function to evaluate the quality of each model configuration individual. Fitness is a comprehensive performance index of the large model on the validation set, including the accuracy of the prediction task, the loss value, and the penalty term for model complexity. For each population individual, train or adjust the model according to the configuration encoded by the population individual and calculate the validation set performance as the fitness score of the population individual. Genetic operator operations: Based on fitness, the genetic algorithm is applied to the individuals in the population to generate candidate configurations for the next generation. High-fitness individuals are retained by performing selection operations. Crossover is performed on the selected high-fitness individuals. For each pair of individuals, a part of their chromosomes is exchanged to form a new population offspring configuration. Mutation operations are performed on the individuals obtained by crossover to randomly change the genes on the chromosomes—learning rate or network layer number genes—with a specified probability. Hybrid local optimization strategy: During the iteration process of the genetic algorithm, a gradient descent local optimization strategy is introduced to complement the global evolution. For each candidate configuration individual, when evaluating fitness, a specified number of gradient descent training rounds are performed on the large model to optimize the weights. Iterative evolution and termination conditions: Fitness evaluation, genetic operator operations and hybrid local optimization strategies are repeatedly applied to evolve the population for multiple generations. The model configuration of the large model is gradually optimized. After the iteration terminates, the individual with the highest fitness is selected from the final population, and the corresponding model configuration is taken as the optimal configuration. Based on the optimal configuration, the model is fully trained on the full training data to obtain the optimized final large model.

5. The private generative large model optimization system based on hybrid genetic algorithm according to claim 1, characterized in that, The diffusion encoder feature extraction module is used to perform the following operations: The progressively noisy data diffusion process: Define the diffusion process, and for any financial data sample, generate a noisy version of the fusion feature according to a Gaussian distribution; Denoising encoder model training: The diffusion encoder is trained by training a parametric model. The input of the parametric model is the noisy version of the fused features, and the output is the denoised and reconstructed fused features. The training objective is to minimize the reconstruction error, which is the difference between the denoised and reconstructed fused features and the fused features corresponding to the financial data samples. In the diffusion model, the intermediate layer is extracted as the feature representation. The implicit representation vector extracted by the encoder from the noisy version of the fused features is regarded as the feature of the input data in a noisy environment. Robust feature extraction and output: After model training, a diffusion encoder is used to extract feature representations for each new input data. A step size is selected at a specific intermediate time. The noisy version of the fused features is input into the encoder to obtain the latent representation. The latent representation is then output as the feature vector of the financial data sample.

6. The private generative large model optimization system based on hybrid genetic algorithm according to claim 1, characterized in that, The curriculum-based noise robustness enhancement module is used to perform the following operations: Noise Course Design: Set up a course plan for noise injection during the training process. In the first stage of model training, use the original data for training. In the second stage of model training, add low-intensity noise to the input data. In the third stage of model training, add high-intensity noise to the input data. In the subsequent stages of model training, gradually add noise of a specified intensity to the input data until the final stage when the model learns in a strong noise environment. Set stopping conditions for each stage. When the model performance tends to stabilize under the noise level, proceed to the next stage. Stepwise noise training: Following a predetermined course, noise is gradually introduced into the model training process. At each stage, data corresponding to the noise level is used to train the model parameters. Robustness assessment and performance balance: After each stage of course training, the performance of the model under different noise conditions is evaluated, including the performance indicators under no noise, medium noise, and high noise conditions.

7. The private generative large model optimization system based on hybrid genetic algorithm according to claim 1, characterized in that, The activation function optimization module is used to perform the following operations: Swish activation function: Replace the original activation function in the large model with the Swish function; Parametric Swish Adjustment: After replacing the original activation function in the large model with the Swish function, a parameter β is introduced in each layer, and the large model adjusts the shape of the activation function itself during training. Training monitoring and performance improvement: Train large models and monitor metrics during training to evaluate the effectiveness of activation function optimization.

8. The private generative large model optimization system based on hybrid genetic algorithm according to claim 1, characterized in that, The feature reduction and low-rank compression module is used to perform the following operations: High-dimensional feature reduction: For high-dimensional feature representations generated in the middle of a large model and feature vectors of input data, dimensionality reduction techniques are applied to extract the main information. Dimensionality reduction techniques include principal component analysis and autoencoder hybrid dimensionality reduction methods. Low-rank decomposition of model weights: For the parameter matrix inside a large model, a low-rank approximation method is used for compression; Model weight training and calibration: For the model with compressed weights, use the parameters of the compressed model as initial values ​​and run gradient descent for a small number of epochs on the original training data to restore the accuracy lost by the large model during compression.