Large model real-time training system based on human brain mechanism

Abstract

The invention discloses a large model real-time training system based on a human brain mechanism, and the system comprises an information input module which is used for receiving and processing various types of data, and carrying out the standardization processing; the temporary storage module is used for temporarily storing data for a certain time length; the importance evaluation module is used for evaluating the importance of the input data in the temporary storage module in real time; the memory classification storage module is used for classifying and storing the data into short-term, medium-term and long-term memories according to the importance of the data; the real-time fine tuning or training module is used for performing small-batch real-time training and updating model parameters by adopting a LoRA technology; and the model updating control module is used for monitoring the performance of the model in real time and automatically rolling back when abnormity occurs. According to the large model real-time training system based on the human brain mechanism, through a real-time fine tuning or training mechanism, the response speed and accuracy of the large model are remarkably improved, resource consumption is reduced, and meanwhile, the model can better adapt to a dynamic environment through an introduced memory and forgetting mechanism.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and particularly relates to a large model real-time training system based on the human brain mechanism. Background Art

[0002] With the rapid development of artificial intelligence technology, large models (such as LLMs) have been widely used in many fields such as natural language processing, image recognition, and speech recognition. However, there are many problems in the actual application of existing large models: the separation of model training and application makes it difficult to optimize in real time; the cost of offline training resources is high; there is a lack of efficient memory and forgetting mechanisms; and the importance of information cannot be dynamically evaluated. These problems seriously limit the performance and application scope of large models. Summary of the Invention

[0003] The present invention provides a large model real-time training system based on the human brain mechanism to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:

[0004] A large model real-time training system based on the human brain mechanism, comprising:

[0005] An information input module, configured to receive and process various types of data and perform standardization processing;

[0006] A temporary storage module, configured to temporarily store data for a certain duration by simulating the working memory mode of the human brain;

[0007] An importance evaluation module, configured to evaluate the importance of the input data in the temporary storage module in real time; comprehensively considering novelty, complexity, frequency, and application value;

[0008] A memory classification and storage module, configured to classify and store data into short-term, medium-term, and long-term memories according to the importance of the data, and manage it through the RAG mechanism;

[0009] A real-time fine-tuning or training module, configured to adopt the LoRA technology to perform small-batch real-time training and update the model parameters;

[0010] A model update control module, configured to monitor the model performance in real time and automatically roll back when an anomaly occurs.

[0011] Furthermore, the information input module receives data through a RESTful API interface, including text, images, audio, and video, performs word segmentation and vectorization processing on text data, performs normalization and size adjustment on image data, and performs spectral analysis on audio data.

[0012] Furthermore, the temporary storage module is implemented by using a Redis in-memory database, which is set according to the theory of human cognitive science and temporarily stores the input data for 1 hour.

[0013] Furthermore, the importance assessment module uses the following formula for scoring:

[0014] Real-time score = 0.3 × novelty score + 0.2 × complexity score + 0.2 × frequency score + 0.3 × application value score.

[0015] Furthermore, the specific calculation methods for the novelty score, complexity score, frequency score, and application value score are as follows:

[0016] Construct a knowledge base, and use cosine similarity to calculate the similarity between the input data and the historical data in the knowledge base as the novelty score;

[0017] Use a pre-trained language model based on the Transformer architecture to calculate the perplexity of the input data as the complexity score;

[0018] Evaluate the frequency by statistically counting the historical usage frequency of the input data in the past 30 days. Adopt a weighted counting method, assign a higher weight to the recent usage frequency, and the weight of the earlier usage frequency gradually decays. Calculate the frequency score of the input data in this way;

[0019] Determine it by collecting the feedback scores of users on the input data, and perform a weighted average on the scores of each dimension to obtain a comprehensive application value score.

[0020] Furthermore, the memory classification and storage module uses Redis, MongoDB, and Neo4j to store short-term, medium-term, and long-term memories respectively. The short-term memory stores for 24 hours, the medium-term memory stores for 30 days, and the long-term memory stores permanently.

[0021] Furthermore, the fine-tuning or training module adopts the LoRA technology, processes 20 - 50 high-importance data per batch. The model is based on Transformer, and only updates the parameters of the Query and Value layers of the model, and the training frequency is adjusted adaptively.

[0022] Furthermore, the model update control module monitors the perplexity and user satisfaction of the model in real time. When the perplexity exceeds 5% or the user satisfaction is lower than 90%, it automatically rolls back to the previous stable version.

[0023] Furthermore, the real-time fine-tuning or training module also includes an automatic annotation mechanism, which uses the feedback signal in the interaction as a weak supervision signal without manual annotation. [[ID=�3]]

[0024] Furthermore, the real-time fine-tuning or training module also includes a progressive knowledge distillation mechanism, which improves the performance of the small model by transferring the knowledge of the large model to the small model.

[0025] The advantages of the present invention lie in the large model real-time training system based on the human brain mechanism. Through real-time fine-tuning or training mechanisms, it significantly improves the response speed and accuracy of the large model, reduces resource consumption, and the introduced memory and forgetting mechanisms enable the model to better adapt to dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0027] Figure 1 It is a schematic diagram of an intelligent graphical programming language application system based on large model technology of the present invention;

[0028] Figure 2 It is a detailed flowchart of the importance assessment score of the present invention;

[0029] Figure 3 It is a detailed diagram of the implementation of real-time fine-tuning or training LoRA of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The following will describe in detail the embodiments of the present application. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0031] As Figure 1 shown, a large model real-time training system based on the human brain mechanism of the present application includes: an information input module 101, a temporary storage module 102, an importance assessment module 103, a memory classification storage module 104, a real-time fine-tuning or training module 105, and a model update control module 106.

[0032] Among them, the information input module 101 is used to receive and process various types of data and perform standardized processing. The temporary storage module 102 is used to temporarily store data for a certain period of time by simulating the working memory mode of the human brain. The importance evaluation module 103 is used to evaluate the importance of the input data in the temporary storage module 102 in real time, comprehensively considering novelty, complexity, frequency, and application value. The memory classification and storage module 104 is used to classify and store data into short-term, medium-term, and long-term memories according to the importance of the data and manage it through the RAG mechanism. The real-time fine-tuning or training module 105 is used to adopt the LoRA technology to perform small-batch real-time training and update the model parameters. The model update control module 106 is used to monitor the model performance in real time and automatically roll back when an anomaly occurs. The above modules are specifically described below.

[0033] In an embodiment of the present application, the information input module 101 receives data through a RESTful API interface, and the input data includes various forms of data such as text, images, audio, and video. The information input module 101 can perform corresponding preprocessing according to different data types. For example, text is standardized through word segmentation and vectorization, images are normalized and resized, and audio is standardized through spectrum analysis.

[0034] In an embodiment of the present application, the temporary storage module 102 is implemented using a Redis in-memory database. It is set according to the theory of human cognitive science and temporarily stores the input data for 1 hour. Specifically, bionic working memory refers to the short-term temporary storage of input data, with a time limit of 1 hour, set according to the experimental data of human cognitive science. This time period can effectively simulate the way the brain processes information in the short term and complete necessary real-time fine-tuning or training within this time range. If it is not marked as highly important or stored in long-term memory within 1 hour, it will be deleted.

[0035] In an embodiment of the present application, the importance evaluation module 103 uses the following formula for scoring: Real-time score = 0.3 × Novelty + 0.2 × Complexity + 0.2 × Frequency + 0.3 × Application value. Among them, the weights of each factor are determined by large-scale experimental data, as Figure 2 shown. This scoring mechanism can comprehensively consider multi-dimensional factors and evaluate the overall importance of the input data.

[0036] In an embodiment of the present application, the specific calculation methods of novelty, complexity, frequency, and application value are as follows:

[0037] Build a knowledge base and use cosine similarity to calculate the similarity between the input data and the historical data in the knowledge base as the novelty evaluation score.

[0038] Specifically, text data is collected from open datasets, scientific research paper repositories, technical blogs, social media, books, and other knowledge sources. The data sources include large-scale corpora (such as Wikipedia, arXiv, GitHub, etc.), and it is ensured that the data covers various fields to enhance the diversity of the knowledge base. Duplicate data, noise, and irrelevant text are removed, and lexical normalization is performed (such as removing stop words, punctuation marks, and other meaningless content). Preprocessing tasks such as word segmentation, part-of-speech tagging, and named entity recognition (NER) are carried out on the text. The text is transformed into a vector representation, and word vectorization is performed using methods such as TF-IDF or Word2Vec to ensure that the text can calculate similarity. An efficient retrieval mechanism is established using inverted index technology to ensure that relevant knowledge base information can be quickly retrieved for the current input data.

[0039] Cosine similarity is used to measure the similarity between the input text and other texts in the knowledge base. The formula is as follows:

[0040]

[0041] Where A and B are the vector representations of the input data and a certain text in the knowledge base respectively.

[0042] The text data in the knowledge base comes from open-source academic resources, public datasets, and online knowledge bases. The knowledge base contains at least 1 million records to ensure the comprehensiveness of the evaluation.

[0043] Calculation steps:

[0044] a. Select text data related to the content of the current input data from the knowledge base.

[0045] b. Calculate the cosine similarity between the input data and each relevant text.

[0046] c. Use the similarity score as an evaluation indicator for novelty.

[0047] Similarity threshold: The evaluation of novelty depends on the similarity score. If the similarity is lower than a certain threshold (for example, 0.3), then the input is considered to have a high degree of novelty.

[0048] The perplexity of the input data is calculated using a pre-trained language model based on the Transformer architecture (such as BERT, etc.) as the complexity evaluation score. A pre-trained language model based on the Transformer architecture is selected to ensure the accuracy of the calculation. According to the analysis of a large number of text samples, the perplexity threshold is determined to reasonably divide the complexity levels.

[0049] The perplexity formula is as follows:

[0050]

[0051] Among them, is the predicted probability of the model for the i-th word, and N is the text length. The perplexity is used to measure the difficulty of the language model in generating text. A low perplexity indicates that the text generated by the model is more fluent and natural, with lower complexity. The complexity of the text can be divided into three levels: low, medium, and high according to the perplexity value:

[0052] Low complexity: perplexity < 50.

[0053] Medium complexity: 50 ≤ perplexity < 100.

[0054] High complexity: perplexity ≥ 100.

[0055] Calculation steps:

[0056] 1. Use a pre-trained language model to perform a generation task on the input text.

[0057] 2. Calculate the perplexity according to the generated probability distribution.

[0058] 3. Determine the complexity level according to the perplexity value.

[0059] The frequency is evaluated by statistically analyzing the historical usage frequency of the input data in the past 30 days. A weighted counting method is adopted, with higher weights assigned to the recent usage frequencies, and the weights of the earlier usage frequencies gradually decaying. In this way, the frequency score of the input data is calculated.

[0060] When statistically analyzing the historical usage frequency, the data within the past 30 days is selected for calculation. The statistics adopt a weighted counting method, with higher weights assigned when the recent usage frequency is higher. The occurrence frequency of each input data is recorded through weighted counting, with higher weights given to the newer data and the weights of the historical data gradually decaying.

[0061] Calculation steps:

[0062] 1. Record the usage times and usage times of each input data.

[0063] 2. Use the weighted counting method to calculate the historical frequency of each data point.

[0064] 3. Use the frequency value as the evaluation index of the frequency factor.

[0065] The weight decay rate of the historical data can be determined according to a time decay function (such as exponential decay).

[0066] It is determined by collecting the feedback scores of users on the input data. The scores of each dimension are weighted and averaged to obtain a comprehensive application value score.

[0067] Specifically, user feedback is collected through the user interface. Users can directly give ratings through a rating button (on a scale of 1 - 5), or indirectly evaluate the application value through methods such as questionnaires and system behavior data, covering multiple dimensions including satisfaction, ease of use, and accuracy. Ratings for each dimension are collected through different feedback channels. A 1 - 5 rating system is adopted, where 1 represents very dissatisfied and 5 represents very satisfied. Satisfaction, ease of use, and accuracy are selected according to the actual application scenario, and the weights are set based on experimental results.

[0068] Calculation steps:

[0069] 1. Collect feedback data from users (including ratings, comments, behavior data, etc.).

[0070] 2. Calculate the weighted average for each dimension according to the rating criteria to obtain a comprehensive application value rating.

[0071] 3. If there is less user feedback, combine historical data and model predictions to infer the application value.

[0072] In the implementation of this application, the memory classification storage module 104 uses Redis, MongoDB, and Neo4j to store short - term, medium - term, and long - term memories respectively. The short - term memory is stored for 24 hours, the medium - term memory is stored for 30 days, and the long - term memory is stored permanently. In the memory classification storage module 104, a small neural network is introduced for data classification to ensure that data is appropriately stored according to different time scales. The memory classification storage module 104 specifically determines the data type based on the importance rating of the input data.

[0073] The memory classification storage module 104 is managed by RAG for easy access by large models. The RAG (Retrieval - Augmented Generation) mechanism is used in this system for efficient access to different types of stored data. By establishing indexes and tags, RAG can quickly retrieve the required data from different storage systems and combine it with the current model input data to improve the efficiency and accuracy of generation tasks.

[0074] In an embodiment of the present application, the real-time fine-tuning or training module 105 adopts the LoRA technology, processes 20 - 50 high-importance data items per batch, the model is based on Transformer, and only updates the parameters of the Query and Value layers of the model, with the training frequency adjusted adaptively. The LoRA (Low-Rank Adaptation) technology is an efficient method for fine-tuning large-scale models. By only updating the parameters of the low-rank part of the model, LoRA can adapt the model to new tasks without significantly increasing the computational cost. This method reduces the update of all model parameters in traditional methods by introducing low-rank matrices as the representation of parameter updates, thereby improving computational efficiency.

[0075] In the Transformer architecture, the Query and Value layers play a crucial role in the information processing and transmission of the model. Especially in tasks based on the attention mechanism, the Query and Value layers directly affect the representation of input data and the output of the model. By updating the parameters of these two layers, the model can quickly adapt to new tasks without changing the entire model architecture, improving the response speed and accuracy of the model. Updating the parameters of the Query and Value layers can optimize the performance of the model in generation tasks, enabling it to more effectively extract and represent information when processing input data. Compared with updating other layers (such as the Encoder or Decoder layers), updating these two layers can reduce the computational complexity while maintaining the improvement of model performance, especially in Transformer-based tasks such as natural language generation and question-answering systems.

[0076] In the case of small-batch data, it can significantly improve the accuracy and speed of generation tasks while avoiding overfitting of model performance.

[0077] In an embodiment of the present application, the real-time fine-tuning or training module 105 further includes an automatic annotation mechanism, which uses the feedback signal in the interaction as a weak supervision signal without manual annotation.

[0078] The automatic annotation mechanism uses the interaction behavior of the robot as a weak supervision signal. The specific types of feedback signals include:

[0079] Sensor data: Sensor readings obtained by the robot in the environment, such as collision detection, distance measurement, image capture, etc.

[0080] Operation records: Action logs executed by the robot, such as movement paths, joint angle changes, task instruction executions, etc.

[0081] Performance metrics: Metrics reflecting the execution effect of the robot's tasks, such as task completion time, energy consumption, success rate, etc.

[0082] Other interaction information: such as abnormal event records, responses to environmental changes, interaction information with other robots or humans, etc., which can also be used as feedback signals.

[0083] By analyzing the robot's behavior data, important operation patterns and concerns are identified using statistical methods (such as behavior sequence analysis). For example, if the robot repeatedly adjusts its path in a certain area, it may indicate the presence of obstacles or navigation difficulties in that area, which need to be marked as key points. Through data cleaning and denoising, irrelevant or abnormal sensor readings and operation records are removed, and only valid feedback signals are retained. The accuracy of the marked information can be improved through multi-source data fusion (such as combining sensor data, historical operations, and environmental models).

[0084] The automatic annotation mechanism can greatly reduce the cost of manual annotation, and at the same time generate effective annotations in real time through the robot's interaction behavior, promoting the rapid optimization of the model in practical applications. Through this mechanism, the robot can obtain feedback and self-regulate during real-time operations, enhancing its adaptive ability and intelligence level.

[0085] In the embodiments of the present application, the real-time fine-tuning or training module 105 further includes a progressive knowledge distillation mechanism, which improves the performance of the small model by transferring the knowledge of the large model to the small model. Progressive Knowledge Distillation is an effective model compression method that transfers the knowledge learned in the large model (teacher model) to the small model (student model), thereby improving the performance of the small model. The teacher model transfers knowledge to the student model, improving the model's performance on new data while controlling the computational cost.

[0086] In the embodiments of the present application, the model update control module 106 monitors the perplexity and user satisfaction of the model in real time. When the perplexity exceeds 5% or the user satisfaction is lower than 90%, it automatically rolls back to the previous stable version.

[0087] Perplexity calculation is performed after generating text, using a language model based on the Transformer architecture to evaluate the rationality of the generated text. The specific formula is:

[0088]

[0089] where P represents the prediction probability of the language model, is the i-th word of the generated text.

[0090] User satisfaction: is measured by analyzing the user's ratings on the interface and other indirect behaviors (such as task completion time, error rate, etc.). The rating uses a scale of 1-5, and the behavior data is used to infer the satisfaction by analyzing the user's clicks, dwell time, and error rate.

[0091] Rollback Strategy: When the perplexity exceeds 5% or the user satisfaction is lower than 90%, the system will roll back to the previous stable version. During the rollback process, the updated parameters will be reset to the parameters of the previous version, and at the same time, the unvalidated data updates in the recent period will be cleared. After the rollback, the system will re-evaluate the performance of the model and decide whether to continue training or resume the current state.

[0092] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form. Any technical solutions obtained by using equivalent replacements or equivalent transformations fall within the protection scope of the present invention.

Claims

1. A large model real-time training system based on the human brain mechanism, characterized in that, Including: An information input module for receiving and processing various types of data and performing normalization processing; A temporary storage module for temporarily storing data for a certain period of time in a way that simulates the working memory of the human brain; An importance evaluation module for evaluating the importance of the input data in the temporary storage module in real time; A memory classification storage module for classifying and storing data into short-term, medium-term, and long-term memories according to the importance of the data and managing it through the RAG mechanism; A real-time fine-tuning or training module for using the LoRA technology to perform small-batch real-time training and update model parameters; A model update control module for monitoring the model performance in real time and automatically rolling back when an anomaly occurs.

2. The large model real-time training system based on the human brain mechanism according to claim 1, characterized in that The information input module receives data through a RESTful API interface, including text, images, audio, and video, performs word segmentation and vectorization processing on text data, performs normalization and size adjustment on image data, and performs spectral analysis on audio data.

3. The large model real-time training system based on the human brain mechanism according to claim 1, characterized in that The temporary storage module is implemented using a Redis in-memory database, which is set according to the theory of human cognitive science and temporarily stores the input data for 1 hour.

4. The large model real-time training system based on the human brain mechanism according to claim 1, characterized in that The importance evaluation module uses the following formula to perform real-time importance scoring: Real-time score = 0.3 × novelty score + 0.2 × complexity score + 0.2 × frequency score + 0.3 × application value score.

5. The large model real-time training system based on the human brain mechanism according to claim 4, characterized in that The specific calculation methods of the novelty score, complexity score, frequency score, and application value score are as follows: Build a knowledge base and use cosine similarity to calculate the similarity between the input data and the historical data in the knowledge base as the novelty score; Use a pre-trained language model based on the Transformer architecture to calculate the perplexity of the input data as the complexity score; Evaluate the frequency by statistically counting the historical usage frequency of the input data in the past 30 days. Adopt a weighted counting method, assign higher weights to the recent usage frequency, and the weights of the earlier usage frequencies gradually decay. Calculate the frequency score of the input data in this way; Determined by collecting the feedback scores of users on the input data, and perform a weighted average on the scores of each dimension to obtain a comprehensive application value score.

6. The large model real-time training system based on the human brain mechanism according to claim 1, characterized in that The memory classification storage module uses Redis, MongoDB, and Neo4j to store short-term, medium-term, and long-term memories respectively. The short-term memory stores for 24 hours, the medium-term memory stores for 30 days, and the long-term memory stores permanently.

7. The real-time training system for large models based on the human brain mechanism according to claim 1, characterized in that, The real-time fine-tuning or training module adopts the LoRA technology, processes 20-50 high-importance data items per batch, the model is based on Transformer, and only updates the parameters of the Query and Value layers of the model, with the training frequency adjusted adaptively.

8. The large model real-time training system based on the human brain mechanism according to claim 1, wherein The model update control module monitors the perplexity and user satisfaction of the model in real time. When the perplexity exceeds 5% or the user satisfaction is lower than 90%, it automatically rolls back to the previous stable version.

9. The large model real-time training system based on the human brain mechanism according to claim 1, wherein The real-time fine-tuning or training module further includes an automatic annotation mechanism, which uses the feedback signal in the interaction as a weak supervision signal without manual annotation.

10. The large model real-time training system based on the human brain mechanism according to claim 1, wherein The real-time fine-tuning or training module further includes a progressive knowledge distillation mechanism, which improves the performance of the small model by transferring the knowledge of the large model to the small model.

Images