Small-scale long-text Chinese generative dialogue model used in field of energy power data security

Abstract

The invention discloses a small-scale long-text Chinese generative dialogue model used in the field of energy power data security. The small-scale long-text Chinese generative dialogue model comprises a user-defined Tokenizer module, an embedded layer, a multi-layer lightweight Transform block and an output layer, the user-defined Tokenizer module is used for generating a vocabulary list from an input statement and encoding vocabularies to form a Token sequence; and in an embedding layer, mapping the Token sequence into an embedding vector, inputting the embedding vector into a multi-layer lightweight Transform block, obtaining a weighted vector of a semantic and context relationship of each Token, inputting the weighted vector into an output layer, and generating a final text output by the output layer. Training and reasoning of the long text Chinese generative dialogue model can be completed on a low-video-memory device. Compared with the prior art, the method not only reduces hardware requirements, but also improves the efficiency and applicability of the Chinese dialogue generation task.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and electric power information security, and in particular relates to a training and reasoning system for a long-text Chinese generative dialogue model in a low-graphics memory environment suitable for energy and electric power data processing. Background Art

[0002] Large language models have shown potential in the power industry due to their excellent natural language processing and logical reasoning capabilities. However, since existing large language models often rely on a large amount of computing resources, they are difficult to be widely used in actual power systems.

[0003] Traditional large model architectures, such as GPT and LLaMA, have the ability to handle complex tasks, but they still have many limitations in the power sector. Power dispatching and real-time data processing require models to have fast response and stability, and to support various multimodal data (such as text, images, and time series). However, existing LLMs lack training data for specific applications in the power industry, and the models lack an industry knowledge base, making it difficult to meet actual needs in terms of accuracy and security.

[0004] Current large language models usually rely on model compression and distillation techniques to reduce computing resource requirements in order to adapt to computing constraints in power market applications. However, these technologies often require professional technical support, have complex operating procedures, and are difficult for power companies and ordinary users to implement independently. In the power market, due to the diversity of power data and the complexity of the Tokenization process, the existing word segmentation and fine-tuning processes are difficult to complete efficiently in a resource-constrained environment. Therefore, achieving efficient deployment of Chinese generation models suitable for the power market under low-resource conditions is still a major technical challenge at present. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to efficiently deploy a Chinese language generation model suitable for the power market under the condition of limited resources. By adopting lightweight model structure design and video memory optimization technology, the present invention successfully realizes efficient Chinese language generation model training and real-time reasoning on consumer-grade devices, meeting the needs of the power market in real-time data processing, price prediction, intelligent scheduling and other applications.

[0006] This paper proposes a long text Chinese generative dialogue model. Through lightweight model design, custom Chinese word segmenter Tokenizer, video memory optimization, and instruction fine-tuning, it realizes efficient training and reasoning of Chinese generative language models on consumer-grade devices with a minimum of 5GB video memory. The system is suitable for scenarios such as data analysis, abnormal reporting, fault diagnosis, and automatic question and answer in the energy and power industry, providing a low-cost and efficient solution for data security and natural language processing applications in the energy and power industry.

[0007] The long text Chinese generative dialogue model of the present invention comprises a custom Tokenizer module, an embedding layer, a multi-layer lightweight Transformer block and an output layer; the custom Tokenizer module is used to generate a vocabulary from an input sentence and encode the vocabulary to form a Token sequence;

[0008] The formula is:

[0009]

[0010] Among them, T is the Token sequence, D is the input corpus, and t i The i-th Token, Indicates the generated Token t1, t2, ... t i-1 Generate the current Tokent under the condition i The probability of

[0011] In the embedding layer, the Token sequence is mapped into an embedding vector and input into a multi-layer lightweight Transformer block to obtain the weighted vector of the semantics and contextual relationship of each Token, which is input into the output layer, and the output layer generates the final text output.

[0012] In the present invention, an improved fine-tuning scheme suitable for power market tasks is designed by combining the innovations of LoRA fine-tuning technology and LoftQ framework. After using LoftQ quantization, the model storage requirement is reduced by 75%, and the video memory requirement is reduced to 45% of the original, so that low-resource devices can run efficiently, and the fine-tuning efficiency, model compression performance and generation quality are optimized.

[0013] This paper introduces an innovative method for optimizing data processing models in the power industry. This method constructs a novel training-free framework by introducing distributed task decomposition and dynamic cooperation mechanism, which is specially designed for processing long sequence data. A divide-and-conquer strategy is adopted to achieve comprehensive document understanding. The distributed cooperation mechanism realizes dynamic scheduling of weights among multiple devices, thereby improving the operating efficiency of the model on low-memory devices by 40%, providing technical support for cross-device collaboration. This framework effectively solves the problems of low efficiency and poor adaptability of traditional generative models on low-memory devices.

[0014] The specific steps include:

[0015] Step 1: construct a Chinese corpus of power market data, remove noise, remove duplicates, and perform word and sentence segmentation on the Chinese corpus;

[0016] Step 2, generating a vocabulary based on the Chinese corpus;

[0017] Step 3: Segment the Chinese corpus based on the BPE algorithm to generate Tokenized text containing characters, words, and phrases. The quality of segmentation can be evaluated by the following formula:

[0018]

[0019] BPE_Score: This indicator represents the performance of the BPE (Byte Pair Encoding) word segmentation model, which reflects the compression efficiency and degree of vocabulary reuse in the word segmentation process. The closer the value is to 1, it means that there are many types of tokens generated (each word may be further subdivided), and the closer the value is to 0, it means that the number of unique tokens reused in the word segmentation process is small. Unique_Tokens: refers to the total number of different word segmentation units that appear in the text after word segmentation. It reflects the diversity of unique word segmentation units generated by the BPE model on a specific dataset. Total_Tokens: refers to the total number of all word segmentation units in the text after word segmentation (including repeated units). This indicator measures the total amount of word segmentation units generated after the entire text is processed by word segmentation, thereby evaluating the compactness and efficiency of word segmentation.

[0020] Furthermore, the multi-layer lightweight Transformer block contains 5 layers of Transformer blocks, each layer includes self-attention and feedforward networks, uses shared weights and low-precision processing; the low precision refers to FP16 or BF16, and the self-attention mechanism captures context information by calculating the dependency between each token. The formula is:

[0021]

[0022] Where Q, K, and V represent query, key, and value matrices, respectively. k is the dimension of the key matrix.

[0023] Further, the multi-layer lightweight Transformer block and the output layer are trained to obtain a pre-trained multi-layer lightweight Transformer block and the output layer. The training process is as follows:

[0024] Step 1: Build training set data

[0025] The token sequence obtained by processing with a custom Tokenizer;

[0026] Step 2: In the multi-layer lightweight Transformer block, the autoregressive mechanism is used to allow the model to learn the contextual relationship and sequential information in the power market context by gradually predicting the next token, so that the multi-layer lightweight Transformer block has the ability to understand and generate Chinese text. In the output layer, the autoregressive language model gradually generates text to ensure the coherence and logical consistency of the output content. The autoregressive target is expressed as:

[0027]

[0028] Where X = {x1, x2, …, x n} represents a Token sequence, x <t represents all Token sequences before time t, x t is the target Token at the current time t. The model is based on x <t Conditional generation of x t , p(x t |x <t ) is the conditional probability at the current time t.

[0029] The goal of multi-layer lightweight Transformer blocks and output layer training is to minimize the cross entropy loss, which is formulated as:

[0030]

[0031] Where N is the number of samples, is the target value, is the predicted value.

[0032] Furthermore, a fine-tuning dataset is constructed, including load forecasting, scheduling optimization, and price trend analysis instructions; the pre-trained multi-layer lightweight Transformer block is further trained to obtain a trained multi-layer lightweight Transformer block.

[0033] The long text Chinese generative dialogue model of the present invention also includes a reward model; used to evaluate the final text generated by the output layer. Under the guidance of the reward model, the proximal policy optimization PPO algorithm is used to perform reinforcement learning optimization on the lightweight Transformer model. The target expectation formula of PPO optimization is:

[0034]

[0035] Among them, γ t (θ) is the strategy ratio, is the advantage function, clip(γ t (θ),1-ε,1+ε) to the probability ratio γ t(θ) is clipped and restricted to the interval [1-ε,1+ε].

[0036] Furthermore, use FP16 / BF16 format to load lightweight Transformer model weights and load lightweight Transformer model layers in batches.

[0037] Beneficial effects: The model and training method proposed in the present invention have significant beneficial effects, and can complete the training and reasoning of the long-text Chinese generative dialogue model on a low-memory (5GB) device. Compared with the prior art, the present invention not only reduces the hardware requirements, but also improves the efficiency and applicability of Chinese dialogue generation tasks, and has the following advantages:

[0038] 1. High-quality output generated by Chinese dialogue: This paper realizes the adaptive generation optimization of the model in the power field through instruction fine-tuning and reinforcement learning based on human feedback (RLHF). Experiments show that after RLHF fine-tuning, the content generated by the model is closer to the user needs in the power market and system scheduling, which significantly improves the user experience and interaction efficiency. In addition, this paper further optimizes the generation quality of the model in the power field application by dynamically adjusting the generation strategy.

[0039] Reasons for the advantage: Custom Tokenizer and instruction fine-tuning technology enhance the Chinese expression and dialogue generation effects of the model, enabling the model to capture the complex language characteristics and semantic relationships in the Chinese corpus.

[0040] 2. Efficient fine-tuning and adaptive generation optimization: The present invention achieves adaptive generation optimization of the model through instruction fine-tuning and reinforcement learning based on human feedback (RLHF). Experiments show that the dialogue content generated by the model after RLHF fine-tuning is closer to user needs and improves user experience. In addition, the present invention optimizes the generation quality of the model by dynamically adjusting the generation strategy.

[0041] Reasons for the advantage: RLHF optimization enables the model to combine human feedback for generation improvements, making the model's generated content more in line with user expectations. At the same time, adaptive optimization further improves the model's performance in Chinese dialogue scenarios.

[0042] 3. Data privacy and localized deployment: This invention supports the training and reasoning of power market data on consumer-grade hardware devices. The data is completely stored locally, effectively protecting the privacy of user data and avoiding the delay and security risks of cloud transmission. This localized deployment has significant advantages for application scenarios that focus on data privacy and real-time performance (such as power system scheduling, risk assessment, etc.).

[0043] Reasons for the advantages: The training and reasoning capabilities of low video memory enable the present invention to complete text generation tasks in the power field on local devices, avoiding cloud dependence and improving data privacy protection and real-time response.

[0044] 4. Reduce the cost of using the model: The lightweight design of the present invention significantly saves video memory, storage and computing resources, and the deployment and use costs are greatly reduced compared to traditional large-scale language models. Consumer-grade hardware can meet the training and reasoning requirements of the Chinese dialogue model in the power market, providing a low-cost, high-performance generative dialogue solution for small and medium-sized power companies and research institutions. The total parameter calculation formula of the model is:

[0045]

[0046] Among them, d model is the embedding dimension, n layers is the number of model layers, n vocab is the vocabulary size.

[0047] Reasons for the advantages: By reducing the number of parameters and optimizing the use of video memory in the model architecture design, the present invention enables ordinary computing devices to undertake model training and reasoning, greatly reducing deployment and maintenance costs, and contributing to the popularization of generative dialogue technology in the power field.

[0048] 5. Strong adaptability to complex Chinese corpus: This invention enhances the adaptability of the model in power system and market data processing by constructing a lightweight generation model suitable for power data and combining instruction fine-tuning and reward model optimization. Compared with existing lightweight models, this invention has better generation quality and information coherence in specific tasks in the power industry.

[0049] Reasons for the advantages: The adaptive design of Tokenizer on power data and instruction fine-tuning technology have improved the model's ability to generate power data, making the model highly adaptable in processing complex data, diversity and real-time performance in the power market.

[0050] 6. Powerful task adaptation capability: Through dynamic sharding and multi-device collaboration, the model can predict the load conditions in multiple areas in real time and generate accurate scheduling recommendations. With the help of a distributed collaboration mechanism, the model can efficiently analyze abnormal signals in sensor data to ensure the accuracy and timeliness of detection results. By decomposing scheduling tasks into small subtasks that run in parallel, the model significantly reduces computing time and ensures global optimization of the scheduling solution.

[0051] Advantages: Distributed task decomposition and dynamic cooperation mechanism enhances the parallel processing capability when handling complex tasks, performs video memory optimization and dynamic weight loading, significantly reduces the demand for hardware, and enables low-memory devices to run smoothly. The introduction of task priority scheduling mechanism effectively improves the timeliness of tasks and resource utilization. Through global optimization design, the overall performance of cross-regional scheduling tasks is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart for training, fine-tuning, optimizing, and deploying a small-scale, long-text Chinese generative dialogue model with low video memory. DETAILED DESCRIPTION

[0053] like Figure 1 As shown, the present invention provides a long-text Chinese generative dialogue model suitable for a low-memory environment, which is particularly designed for real-time data processing and intelligent analysis requirements in the power market. The long-text Chinese generative dialogue model of the present invention includes an input layer, a custom Tokenizer module, an embedding layer, a multi-layer lightweight Transformer block and an output layer.

[0054] 1. Input layer: receives preprocessed and segmented electricity market text data.

[0055] 2. Custom Tokenizer module: Uses word segmentation methods such as Byte-Pair Encoding (BPE) to generate a vocabulary of specific terms suitable for the power market to improve the understanding of texts in the power field.

[0056] The vocabulary is generated by training a custom Tokenizer. It contains common words and terms in the electricity market and is the basis for encoding the input text. The vocabulary determines how each token (i.e., word or phrase) is represented as a digital identifier. The token sequence is generated by the custom Tokenizer module. The input electricity market text is first segmented and encoded by the Tokenizer module. Each word or phrase in the text is converted into a corresponding Token ID with reference to the vocabulary, forming a sequence of Token IDs, namely the "Token sequence".

[0057] 3. Embedding layer: The embedding layer receives the token sequence generated by the Tokenizer and maps it into an embedding vector for subsequent processing by the Transformer layer. The embedding vector is a numerical representation of the token in a high-dimensional space, which makes it easier for the model to understand the semantics and contextual relationship of each token. It can be expressed as:

[0058] E=X·W e +b e

[0059] Among them: the input Token sequence matrix X and the embedding layer weight matrix W e Multiply and add the bias term b e , generate the embedding vector E for subsequent Transformer processing.

[0060] In short, the token sequence is generated by the Tokenizer module after segmenting and encoding the input text according to the vocabulary, and the embedding layer converts this sequence into an embedding vector for further processing by the model. The mapping process can be expressed as:

[0061] E=T·W e

[0062] Among them, T is the Token sequence matrix, W e is the embedding matrix and E is the generated embedding vector.

[0063] 4. Multi-layer lightweight Transformer block: The embedding layer maps the token sequence to an embedding vector (numerical representation), which is passed as input to the subsequent Transformer block. The dimension of the embedding vector is usually much richer than the representation of the token ID. Such a high-dimensional representation can capture more semantic information. The Transformer block (including the self-attention mechanism and the feedforward network) receives the embedding vector and further processes the relationship between these vectors. When processing the embedding vector, the self-attention mechanism can dynamically calculate the dependencies between tokens and generate a weighted representation of each token in different contexts, so as to understand the sentence structure and semantics. By processing the embedding vector, the Transformer block builds the contextual representation of each token layer by layer, so that the model can generate more coherent and contextual output content. After being processed by multiple layers of Transformer blocks, the information carried by the embedding vector is gradually refined and enriched, preparing for subsequent text generation. It contains 5 layers of Transformer blocks, each layer includes self-attention and feedforward networks. The multi-head self-attention mechanism calculates and merges the query, key and value matrices. The formula is:

[0064] Multihead(Q,K,V)=Concat(head1,…,head h )W o

[0065] in:

[0066] head i =Attention(QW i Q ,KW iK ,VW i V )

[0067] W i Q ,W i K ,W i V , respectively, are the projection matrix, W O is the output matrix.

[0068] Using shared weights and low-precision (FP16 / BF16) processing, the parameter volume is controlled within 0.25B, and the formula is:

[0069] FFN(x)=ReLU(xW1+b1)W2+b2

[0070] Among them, x is the input, W1, W2 are weight matrices, and b1, b2 are bias terms.

[0071] 5. Output layer: Generate text or dialogue content related to the electricity market.

[0072] In general: Input data: The embedding vector generated by the embedding layer represents the token sequence of the input text. Output of the Transformer block: The weighted vector containing the semantics and contextual relationship of each token. Output flow: The contextual weighted vector after the Transformer block is passed to the output layer, which generates the final text output.

[0073] A small-scale long-text Chinese generative dialogue model training process for the field of energy and power data security, including building a vocabulary containing power market-specific terms through data cleaning and word segmentation to improve the model's understanding of power semantics; then designing a lightweight model architecture, limiting the number of parameters to 0.25B, combining shared weights and FP16 / BF16 low-precision technology, so that the model can run efficiently in a 5GB video memory environment. After self-supervised pre-training, the model has basic text generation capabilities, and then adapts to specific tasks in the power market through instruction fine-tuning (SFT), and further optimizes the generation quality by combining user feedback reinforcement learning (RLHF). The video memory optimization technology is used in the inference stage, so that the model can achieve efficient inference with a minimum of 0.6GB video memory, which is suitable for local deployment on multiple devices and meets the low-cost and high-efficiency application requirements in the power industry.

[0074] The small-scale long-text Chinese generative dialogue model of the present invention covers the complete process from data processing, model structure construction, training to reasoning, aiming to achieve efficient operation of the long-text Chinese generative dialogue model in the power market under a low-memory environment.

[0075] The training process of the small-scale long-text Chinese generative dialogue model of the present invention includes the following steps: The following is the detailed implementation process of each step:

[0076] Step 1: Data cleaning and custom Tokenizer training

[0077] 1. Data acquisition: Acquire and build a Chinese corpus containing electricity market data, with a data volume of approximately 600,000 items, covering key tasks and topics in the electricity market such as price forecasting, load management, and dispatch analysis.

[0078] 2. Data cleaning:

[0079] Denoising: Remove irrelevant information and abnormal data contained in the corpus to ensure the quality of the data set.

[0080] Deduplication: Remove repeated sentences or content to ensure data diversity and validity.

[0081] Word segmentation and sentence segmentation: Perform word segmentation and sentence segmentation on Chinese text to prepare more accurate word segmentation data for Tokenization.

[0082] 3.Tokenizer training:

[0083] Vocabulary construction: The Byte-Pair Encoding (BPE) method is used to generate a custom vocabulary suitable for the power market corpus, covering common power market terms.

[0084] Word segmentation model training: The data is segmented based on the BPE algorithm to generate tokenized text containing characters, words, and phrases. The final vocabulary size is set to 40,000 to ensure that the model has a high Chinese expression ability in a low-memory environment. The BPE word segmentation formula process is:

[0085]

[0086] Among them, V is the vocabulary of the corpus, D is the corpus, freq(w) is the word frequency, and len(C(w)) is the decomposition length of the word.

[0087] The formula for the participle-subword ratio is:

[0088]

[0089] It mainly measures the ratio of the number of subwords to the original words in the text after word segmentation. This ratio is used to measure the degree of refinement of word segmentation.

[0090] Step 2: Build on a lightweight Transformer model architecture

[0091] 1. Model architecture design:

[0092] The model architecture design of the present invention is based on a lightweight Transformer model. By optimizing the parameter quantity and introducing video memory management technology, it can run efficiently in a low video memory environment (minimum 5GB).

[0093] The model architecture design is based on a lightweight Transformer model. By reducing the number of layers, parameters, shared weights, and low-precision optimization, it adapts to the requirements of low-resource environments in power market applications, and supports real-time reasoning and deployment of Chinese generation dialogues and text generation tasks.

[0094] Step 3: Pre-training based on lightweight Transformer model

[0095] In the data processing phase of step 1, the Tokenizer training is completed. Specifically, this step has performed data cleaning, word segmentation, and sentence segmentation on the Chinese data of the power market. The Tokenizer training is performed through a customized Tokenizer module, such as the Byte-Pair Encoding (BPE)-based method, which ensures that the model can obtain a set of professional vocabulary suitable for the power market, so that the subsequent steps do not need to repeat the Tokenizer training. The core task of the model is to use this vocabulary to perform in-depth text understanding and generation during the pre-training and fine-tuning phases to meet the specific application needs of the power market.

[0096] 1. Training data preparation: Use the electricity market Tokenized dataset processed by a custom Tokenizer.

[0097] 2. Training configuration:

[0098] Batch size: Set the batch size to 2 to accommodate the 5GB graphics memory device limitation.

[0099] Precision processing: Use FP16 or BF16 half-precision floating point numbers to load weights to further reduce video memory usage. The weight scaling formula for mixed precision training is:

[0100] W mixed =W FP32 ·scale

[0101] Among them, the mixed W mixed Weight value under accuracy, W FP32 The original 32-bit floating point (FP32) weight value, scale scaling factor, used to adjust the FP32 precision weights to a range suitable for low-precision representation.

[0102] 3. Pre-training strategy:

[0103] The mechanism of the autoregressive language model mainly works in the multi-layer lightweight Transformer block and the output layer. It is used to train the model to generate coherent text content. In the multi-layer Transformer block, the autoregressive mechanism allows the model to learn contextual relationships and sequential information in the power market context by gradually predicting the next token, so that the model has the ability to understand and generate Chinese text. In the output layer, the autoregressive language model gradually generates text to ensure the coherence and logical consistency of the output content. This mechanism is the core technology for the model to generate content related to the power market, enabling it to maintain content accuracy and contextual coherence during the generation process.

[0104] Self-supervised learning: Using the autoregressive language model method, the model is trained to gradually predict the next token and learn the data context relationship, so that the model has basic power market text generation capabilities.

[0105] Breakpoint saving and recovery: Save the model state periodically during training so that it can be restored when the device loses power or is interrupted.

[0106] 4. Training effect: After 15 rounds of training, the model can generate grammatically correct and coherent Chinese sentences about the power market, and has basic language understanding and generation capabilities.

[0107] Step 4: Command Fine Tuning (SFT)

[0108] 1. Construction and dynamic expansion of fine-tuning dataset: Build and create a prescriptive dialogue dataset for the power market, covering typical tasks such as load forecasting, dispatch optimization, and price trend analysis. While inheriting the traditional static data construction method, it introduces dynamic expansion capabilities. By integrating the real-time data dynamics of the power market (such as price fluctuations, load forecast updates, etc.), the prescriptive dialogue data is continuously collected and generated. The optimization objective formula for prescriptive fine-tuning is:

[0109] L SFT =L task +λL reg

[0110] Among them, L task is the task loss, L reg is the regularization loss and λ is the balancing factor.

[0111] 2. Instruction fine-tuning process:

[0112] Adopt: LoRA (Low-Rank Adaptation) fine-tuning solution and LoftQ combined technology

[0113] The LoftQ framework is used for model quantization and low-rank initialization. The model size is significantly compressed through 4-bit hybrid quantization, which improves the storage and computing efficiency of the model. The LoRA low-rank adapter is combined to optimize the weights of the key task layers. The hierarchical weight sharing mechanism reduces the underlying weight updates and further reduces the memory requirements. The quantization strategy can be expressed as:

[0114]

[0115] Among them, x is the original weight value and s is the quantization step size.

[0116] Training rounds: After 6 rounds of fine-tuning training, the model has the ability to understand and generate electricity market tasks and can generate content that meets the requirements of different tasks.

[0117] Step 5: Reinforcement Learning with Human Feedback (RLHF) Optimization

[0118] RLHF optimization is an external optimization process and is not directly embedded in the model structure layer. Instead, it is an optimization step performed after model pre-training and fine-tuning. The "model" referred to in RLHF optimization refers to a lightweight Transformer model that has undergone autoregressive pre-training and instruction fine-tuning. The output content is generated by the output layer of the model. It uses the generated content to compare with the reward model of user feedback to guide the optimization of the model's generation strategy. RLHF optimization is used to improve the actual application effect of model-generated content. The gradient update formula of reinforcement learning is:

[0119]

[0120] Among them, π θ For strategy, A t is the advantage function.

[0121] After the entire model training is completed, RLHF helps the model generate content that is more in line with the needs of the power market through a feedback mechanism based on a reward model. By combining actual user feedback on the output content, RLHF optimization significantly improves the coherence, practicality and accuracy of the model-generated content, making the model closer to the specific needs of the power market scenario.

[0122] Through LoftQ quantization weights and LoRA fine-tuning adapters, the reward model is used to score the generated content, and the model generation strategy is adjusted in combination with the PPO algorithm. This optimization scheme significantly improves the efficiency of reinforcement learning and the adaptability of the model to task scenarios, and reduces the hardware resource requirements during the RLHF process. The specific reward expectations and optimization goals are as follows:

[0123]

[0124]

[0125] Among them, R(θ) is the expected reward, π θ is the current strategy, A(x) is the advantage function, Represents the strategy π θ Find the expectation of all sampled trajectories x.

[0126] 1. Reward model training:

[0127] The reward model is an independently constructed model specifically designed to evaluate the quality of generated content. Its main task is to score and provide feedback on the generated text content, thereby helping to guide the RLHF optimization process. The reward model is an "evaluator" that evaluates whether the generated content meets the needs of the power market based on pre-defined criteria such as coherence, accuracy, and practicality. The reward model does not generate content, but provides quality feedback to the RLHF process.

[0128] Collect a set of manually annotated data, usually including correct responses and high-quality output examples in power market-related tasks. These examples may cover typical conversations or responses in power markets such as load forecasting, dispatch optimization, and real-time data analysis, and are used to train a reward model that evaluates the quality of generated content to ensure that the output content meets the actual needs of the power market.

[0129] 2.RLHF optimization process:

[0130] Under the guidance of the reward model, the PPO (Proximal Policy Optimization) algorithm is used to perform reinforcement learning optimization on the main model, that is, the lightweight Transformer model (that is, the model after pre-training and instruction fine-tuning (SFT)).

[0131] Feedback signal: The lightweight Transformer model adjusts the generated content based on user feedback to ensure that the generated content conforms to human interaction habits and power market task requirements.

[0132] 3. Optimization effect: After RLHF optimization, the content generated by the lightweight Transformer model is closer to the application scenarios of the power market, meets actual needs, and improves the user experience.

[0133] Step 6: Low GPU memory inference optimization

[0134] This module is an independent component that aims to optimize the use of video memory during the inference phase, so that the inference process of the entire model can be efficiently executed on devices with limited video memory. The formula for calculating video memory usage is:

[0135]

[0136] Optimize video memory usage by activating necessary weight layers. ACTIVE_Weight represents the effective weight part after quantization, including only the weight currently involved in calculation or activation. Total_Weight represents the total number or total amount of all weights.

[0137] The present invention introduces a "low video memory reasoning optimization module" in the reasoning stage. It is not part of the output layer, but exists as an independent component throughout the entire reasoning stage. By adopting optimization technologies such as FP16 / BF16 low-precision weight loading, layered on-demand loading, and batch reasoning, and at the same time, using distributed task decomposition and dynamic cooperation mechanisms, the huge power data processing task is decomposed into multiple sub-tasks that can be executed in parallel. In each task module, only the necessary model weights and activation values ​​are loaded to avoid wasting video memory. After each subtask is completed, the model merges the results through a dynamic cooperation mechanism to ensure the coherence and consistency of the output content. Combined with distributed video memory scheduling, efficient reasoning on low video memory devices (minimum 5GB) is achieved. The core of the distributed task processing framework lies in task sharding and collaborative merging. The specific implementation steps are as follows

[0138] The task decomposition stage is: according to the task data volume and hardware capacity, the complex task TTT is dynamically decomposed into multiple subtasks:

[0139] Decompose the total task T into n subtasks {t1, t2, …, t n}:

[0140]

[0141] Each subtask t i Allocate processing to different devices to reduce the load on a single device.

[0142] Dynamic cooperation mechanism merging stage: After the task is completed, the output of each subtask {O1, O2, ..., O n} are merged into the final result O:

[0143] O=Reduce(O1,O2,…O n )

[0144] Video memory optimization stage: total video memory occupancy M total By executing subtasks in parallel on k devices, and the video memory of each device occupies M device satisfy:

[0145]

[0146] Among them, M model is the total video memory requirement of the model, M overhead For additional system overhead.

[0147] This module significantly reduces the memory consumption during inference, ensuring that the model can run efficiently on devices with limited memory.

[0148] This enables real-time output of generative Chinese dialogues for long texts. In addition, the sharding inference mechanism for dynamically loading model weights further reduces video memory usage and improves task adaptability.

[0149] 1. Reasoning memory optimization:

[0150] Weight loading: Use FP16 / BF16 format to load model weights during inference, reducing the video memory requirement to 0.6GB during inference.

[0151] Layered on-demand loading: Load model layers in batches according to actual needs to further reduce video memory usage and improve inference speed. The video memory usage optimization formula is:

[0152]

[0153] Among them, Active_Layers is the number of currently activated layers, and Layer_Size is the video memory usage of each layer.

[0154] 2. Batch Reasoning:

[0155] By processing multiple requests in batch inference, the inference efficiency is improved, which is suitable for the high concurrency requirements in the online deployment scenarios of the power market.

[0156] 3. Loading time optimization:

[0157] In the initial stage of reasoning, the loading time is shortened by loading weights in segments, improving the user experience and ensuring that the model can quickly respond to power market data queries and task requirements.

[0158] Through the above detailed implementation steps, the present invention successfully completed the comprehensive training and reasoning process of the long text Chinese generative dialogue model under limited video memory conditions, meeting the needs of the power market for intelligent data processing and real-time analysis. This implementation significantly reduces the demand for hardware resources and provides a flexible and efficient local deployment option, which is very suitable for cost-sensitive application scenarios in the power industry.

[0159] The innovative features of the present invention and the effects produced are as follows:

[0160] 1. Structural design of lightweight long-text Chinese generative dialogue model

[0161] The model of the present invention adopts a lightweight design of 0.25B, combined with a shared weight mechanism and FP16 / BF16 low-precision format, so that the model can run efficiently in a low-memory (5GB) environment. This structure significantly reduces the memory requirements and supports the completion of Chinese generation model training and reasoning tasks related to the power market on consumer-grade devices. The reasoning time complexity formula of the model is:

[0162] T inference =O(n 2 ·d model )

[0163] Where n is the length of the input sequence, d model is the embedding dimension.

[0164] 2. Custom Tokenizer module for Chinese

[0165] The present invention constructs a vocabulary adapted to the Chinese characteristics of the power market through a custom Tokenizer module and BPE technology, enhancing the model's understanding and generation of power industry terms and sentences. The model adopts a specific word segmentation and encoding strategy to achieve efficient processing of power texts in a resource-limited environment.

[0166] 3. Application of instruction fine tuning (SFT) technology

[0167] The present invention uses an innovative instruction fine-tuning (SFT) technology to deeply optimize the current model. This optimization gives the model extremely high flexibility, enabling it to effectively respond to various different power instructions. To achieve this goal, we have specially constructed a customized power market instruction data set. By fine-tuning the model on this data set, the adaptability and expressiveness of the model in power market-related tasks are significantly enhanced. Specifically, these tasks cover areas such as load forecasting, scheduling optimization, and real-time data analysis. In terms of load forecasting, the fine-tuned model can more accurately predict power demand, thereby assisting power companies to more effectively allocate and dispatch resources. In terms of scheduling optimization, the model can propose the most optimized power dispatching plan based on the real-time power market conditions to improve the operating efficiency and reliability of the power system. As for real-time data analysis, the model can quickly process and analyze a large amount of power data, providing solid support for decision-making in the power market. With the help of this instruction fine-tuning technology, the model of the present invention not only shows stronger adaptability in theory, but also proves its excellent performance in practical applications. This makes the model of the present invention have broad application potential in the field of power market, and can provide strong technical support for the stable operation and optimized management of power systems.

[0168] 4. Reinforcement Learning with Human Feedback (RLHF) Optimization

[0169] This paper introduces a method to improve the quality of generated content using Reinforcement Learning with Human Feedback (RLHF). Through reward model guidance and PPO algorithm, the quality of power market dialogue content is significantly improved, ensuring that the content meets human interaction needs. This technology enables the model to adjust the generated content based on user feedback to adapt to different scenarios in the power market.

[0170] By deeply integrating LoftQ and LoRA, and combining the LoftQ quantization framework and LoRA low-rank adaptation technology, we have significantly optimized the memory usage and fine-tuning efficiency, ensuring that the model still has high performance in a low-resource environment. The hybrid quantization formula of the weight is expressed as:

[0171]

[0172] Among them, s1 and s2 are the step sizes of different quantization levels.

[0173] A hierarchical optimization strategy is adopted to achieve efficient generation capability in multi-task scenarios through LoftQ quantization weights, LoRA adapter hierarchical update and task priority classification.

[0174] 5. Low-Video Memory Reasoning Optimization Module

[0175] The present invention significantly improves the efficiency of video memory usage through a series of innovative video memory optimization designs in the inference stage. These optimization designs include but are not limited to FP16 / BF16 weight loading technology, layered on-demand loading mechanism and shared weight management strategy. Specifically, FP16 / BF16 weight loading technology can store and load model weights in half-precision floating point format, thereby significantly reducing video memory usage. The layered on-demand loading mechanism dynamically loads and unloads different parts of the model according to the actual needs during the inference process, further optimizing the use of video memory. The shared weight management strategy reduces redundant storage by sharing weights between multiple layers, further reducing video memory requirements. Through these comprehensive optimization measures, the present invention successfully reduces the video memory requirement in the inference stage to only 0.6GB. This significant video memory optimization enables the model to efficiently perform inference tasks on devices with extremely limited video memory. This is of great significance for consumer-grade devices because it provides these devices with the ability to generate power market dialogue solutions in practical applications. Through the optimization design of the present invention, consumer-grade devices are no longer limited by high video memory requirements and can be more widely used in various scenarios, thereby providing users with a more convenient and efficient power market dialogue experience.

[0176] In addition, batch inference technology is introduced to further improve the efficiency of inference. Batch inference allows the model to process multiple inputs at the same time, thereby reducing the overhead of single input processing and making full use of the parallel computing capabilities of the GPU. By reasonably setting the batch size, the present invention significantly shortens the response time and improves the throughput of the system without sacrificing the accuracy of inference. The application of this technology enables the present invention to maintain efficient and stable performance output when processing large amounts of data.

[0177] In terms of loading time optimization, the present invention adopts a dynamic loading mechanism for model parameters. This mechanism allows the model to load corresponding parameters only when needed, rather than loading all parameters at once at startup. This not only reduces the startup time, but also enables the model to respond to user requests more quickly. In addition, the present invention further shortens the loading time of the model by optimizing the storage format and loading algorithm of the model, ensuring that there is almost no delay during use.

[0178] 6. Multi-device adaptability and cross-device deployment capabilities

[0179] By designing a flexible model architecture and interface, the present invention can easily adapt to devices with different performance. At the same time, the present invention supports cross-device deployment, and users can seamlessly switch between different devices to continue unfinished conversations or queries without restarting the model. This cross-device continuity experience greatly improves the convenience and consistency of users when using the power market dialogue model. The small-scale long-text Chinese generative dialogue model of the present invention can adapt to multi-device deployment and use distributed computing to improve training and reasoning efficiency. Using a distributed task processing framework, through task decomposition and result merging, efficient processing of complex power data generation tasks is achieved. On low-memory devices, the pressure on video memory is reduced through modular subtasks and cross-device weight distribution strategies. Through the adaptation and adjustment of video memory management, not only training in a low-resource environment is achieved, but also consistent performance across devices is guaranteed, supporting efficient operation on different devices in the power market.

[0180] 7. Introduce distributed task decomposition and dynamic cooperation mechanism

[0181] By breaking down the huge power data processing task into small subtasks that can be executed on low-memory devices, each subtask only loads the necessary model weights and activation values, and releases the video memory immediately after the task is completed. In addition, the granularity of the task slicing is dynamically adjusted according to the complexity of the input data, providing solid technical support for the efficient operation of low-memory devices. This invention is applicable to devices with at least 5GB of video memory, allowing more hardware resources to participate in the training and inference process of the model. Combining FP16 / BF16 low-precision weight loading and hierarchical on-demand loading strategies, we further reduce the video memory usage. Dynamically loading the required model weights during task execution effectively reduces the peak demand for video memory, and coordinates task execution across multiple devices to integrate the results of each subtask. This ensures the coherence and consistency of the generated content, while improving the adaptability of the model to complex tasks in the power industry. Through the distributed task processing mechanism, the model can flexibly adapt to hardware devices of different sizes and support deployment on a variety of devices with video memory ranging from 5GB to 9GB and above. This enhances the scalability of the model in different application scenarios in the power industry. This method can efficiently process complex tasks in the power market in a low-memory, multi-device environment, significantly improving the training and reasoning efficiency of the model. At the same time, this method reduces hardware resource requirements, reduces deployment costs, and promotes the application and popularization of generative dialogue models in the power industry.

[0182] 8. Complete Chinese generation model training process

[0183] The present invention proposes a complete training process, covering data cleaning, Tokenizer training, model pre-training, instruction fine-tuning and RLHF optimization, which is particularly suitable for the autonomous construction and optimization of the power market generation model. This process efficiently achieves the actual effect of the Chinese generation model in the power market in a low-memory environment, and becomes a key part of the power generation model training in a low-resource environment.

[0184] The above-mentioned invention points demonstrate the innovations of the present invention in the design of low-graphics memory Chinese generation model, reasoning optimization and adaptability. Through these technical means, the training and reasoning of long-text Chinese generation dialogue models can be efficiently run on consumer-grade devices.

Claims

1. A long text Chinese generative dialogue model, characterized in that: Includes custom Tokenizer module, embedding layer, multi-layer lightweight Transformer block and output layer; The custom Tokenizer module is used to generate a vocabulary from the input sentence and encode the vocabulary to form a Token sequence; In the embedding layer, the Token sequence is mapped into an embedding vector and input into a multi-layer lightweight Transformer block to obtain the weighted vector of the semantics and contextual relationship of each Token, which is input into the output layer, and the output layer generates the final text output.

2. According to claim 1, a long text Chinese generative dialogue model is characterized in that: The training of the custom Tokenizer specifically includes the following steps: Step 1: construct a Chinese corpus of power market data, remove noise, remove duplicates, and perform word and sentence segmentation on the Chinese corpus; Step 2, generating a vocabulary based on the Chinese corpus; Step 3: Segment the Chinese corpus based on the BPE algorithm to generate Tokenized text containing characters, words and phrases.

3. According to claim 1, a long text Chinese generative dialogue model is characterized in that: The multi-layer lightweight Transformer block contains 5 layers of Transformer blocks, each layer includes self-attention and feed-forward networks, uses shared weights and low-precision processing; the low precision refers to FP16 or BF16.

4. According to claim 1, a long text Chinese generative dialogue model is characterized in that: The multi-layer lightweight Transformer block and the output layer are trained to obtain a pre-trained multi-layer lightweight Transformer block and the output layer. The training process is as follows: Step 1: Build training set data The token sequence obtained by processing with a custom Tokenizer; Step 2: In the multi-layer lightweight Transformer block, an autoregressive mechanism is used to allow the long-text Chinese generative dialogue model to gradually predict the next token, learn the contextual relationship and sequential information in the power market context, so that the multi-layer lightweight Transformer block has the ability to understand and generate Chinese text. In the output layer, an autoregressive mechanism is used to gradually generate text to ensure the coherence and logical consistency of the output content.

5. According to claim 4, a long text Chinese generative dialogue model is characterized in that: The autoregressive target is expressed as: Among them, X = {x1, x2, ..., xn} represents the token sequence, x <t represents all Token sequences before time t, x t is the target Token at the current time t. The model is based on x <t Conditional generation of x t , p(x t |x <t ) is the conditional probability at the current time t.

6. According to claim 4, a long text Chinese generative dialogue model is characterized in that: Build fine-tuning data sets, including load forecasting, dispatch optimization, and price trend analysis instructions; Fine-tune the pre-trained multi-layer lightweight Transformer block to obtain a trained multi-layer lightweight Transformer block.

7. A long text Chinese generative dialogue model according to claim 6, characterized in that: The LoRA fine-tuning adapter and LoftQ quantization weights are combined to fine-tune the pre-trained multi-layer lightweight Transformer block. Specifically, the LoftQ framework is used to quantize and low-rank initialize the pre-trained multi-layer lightweight Transformer model, and the multi-layer lightweight Transformer block size is compressed by 4-bit hybrid quantization; the LoRA low-rank adapter is combined to optimize the weights of the key task layers, and the hierarchical weight sharing mechanism is used to reduce the bottom-level weights. The calculation formula for weight adjustment is as follows: W=W0+△W=W0+AB T Among them, W0 is the original weight, △W is the optimized weight, and A and B are low-rank matrices.

8. According to claim 7, a long text Chinese generative dialogue model is characterized in that: A reward model is also included; used to evaluate the final text generated by the output layer.

9. A long text Chinese generative dialogue model according to claim 8, characterized in that: Under the guidance of the reward model, the proximal policy optimization (PPO) algorithm is used to perform reinforcement learning optimization on the lightweight Transformer model.

10. A long text Chinese generative dialogue model according to claim 9, characterized in that: Load lightweight Transformer model weights using FP16 / BF16 format, and load lightweight Transformer model layers in batches.

Images