Government affair big language model continuous learning and knowledge updating method

Abstract

The invention discloses a government affair large language model continuous learning and knowledge updating method, which comprises the following steps of: constructing a dynamic knowledge synchronization system, docking a department system through RESTfulAPI or WebSocket, extracting policy core information by using an NLP technology such as Transform architecture OCR and BERT-BiLSTM-CRF, constructing a knowledge graph containing'effective time-application range-efficacy hierarchy 'dimensions through Neo4j, and establishing a dynamic knowledge synchronization system through the NLP technology. Wherein the OCR recognition accuracy is not lower than 98%; incremental learning, elastic weight merging and batch progressive learning are optimized, the low-rank matrix dimension of the LoRA module is 8-64, the elastic weight merging attenuation coefficient is 0.1-0.5, and the proportion of old knowledge samples to new knowledge samples in batch learning is 30%-50%; a security mechanism is established and comprises data desensitization, localization computing power and triple verification, and the semantic similarity of the triple verification needs to be not lower than 90%; collaborative evaluation is completed, a cross-department middle station is built, the synchronization delay of the important policies of the middle station does not exceed one minute, the policy interpretation accuracy is set to be not lower than 95%, and the model response speed does not exceed one second, and iterative optimization is carried out according to the evaluation result.

Description

Technical Field

[0001] This invention belongs to the field of government knowledge management technology, specifically a method for continuous learning and knowledge updating of a large government language model. Background Technology

[0002] With the advancement of digitalization, the demand for government knowledge services is increasing, and the public expects to obtain real-time, accurate, and practical government information. However, existing government knowledge service systems have significant shortcomings and are unable to meet current needs: Lagging and confusing knowledge updates: Existing systems largely rely on static pre-trained data, and knowledge updates depend on periodic manual operations. This makes it impossible to capture dynamic changes in policies, regulations, and procedures in real time, resulting in insufficient timeliness of knowledge. Furthermore, the system lacks an effective mechanism to distinguish between old and new knowledge, easily leading to confusion between outdated and current policies, potentially misleading users and impacting the quality of government services.

[0003] Insufficient semantic matching capability: Government data is mostly in the form of written corpus, while user inquiries are often expressed in spoken language. Existing systems lack effective semantic mapping technology, making it difficult to bridge the "semantic gap." This results in a failure to accurately match user needs with government knowledge, leading to low service response accuracy.

[0004] Continuous learning capacity loss: When updating knowledge in the model, existing solutions mostly adopt full parameter fine-tuning, which is not only inefficient in computation, but also prone to "catastrophic forgetting"—that is, the process of learning new knowledge overwrites old knowledge, resulting in a significant drop in the response accuracy of cross-period business consultations, and failing to guarantee the effective retention of historical government knowledge by the model.

[0005] Lack of security and credibility: Government data involves sensitive information, and existing systems lack a sound data security processing mechanism; at the same time, the policy interpretations output by the models lack effective verification and traceability methods, which can easily lead to "illusory" content (i.e., fabricated policy clauses or misinterpretations), reducing users' trust in government services.

[0006] In summary, the shortcomings of existing government knowledge service systems in terms of knowledge updating, semantic matching, continuous learning, and security and reliability have become key issues restricting the high-quality development of government services. There is an urgent need for a method for continuous learning and knowledge updating of a large government language model that can overcome the above bottlenecks. Summary of the Invention

[0007] The purpose of this invention is to provide a method for continuous learning and knowledge updating of a large-scale government affairs language model, comprising the following steps: (1) Constructing a dynamic knowledge synchronization system: Build a multi-protocol-adaptive real-time government knowledge acquisition interface cluster. The interface cluster includes a RESTful API interface pool, a WebSocket long connection channel, and a file sharing protocol adaptation module, which respectively connect to the heterogeneous policy release systems of various departments (including but not limited to Web portals, OA systems, and dedicated business platforms) to achieve real-time acquisition of all policy documents; adopt a distributed OCR processing engine to perform optical character recognition on the acquired policy documents. The engine supports multi-threaded parallel processing, and a single node can process ≥50 pages of documents per second. Improve the recognition quality through image preprocessing (including noise reduction, tilt correction, and resolution enhancement), and then stream... The water-based NLP processing workflow extracts core information from text information, including but not limited to policy name, effective date, expiration date, scope of application, level of effectiveness, responsible department, policy clauses, and reward and punishment measures. Based on the extracted core information, a spatiotemporally related structured knowledge graph is constructed. The knowledge graph includes an entity layer (policy entities, departmental entities, regional entities, etc.), a relationship layer (publication relationship, application relationship, effectiveness relationship, etc.), and an attribute layer. The attribute layer uses "effective date - expiration date - scope of application - level of effectiveness - update frequency" as the core dimension. Through timestamp marking and version number management, it achieves accurate differentiation between old and new government information, multi-dimensional indexing, and source tracing. (2) Optimize the incremental learning technology path: Multiple LoRA low-rank adaptation modules are embedded between the Transformer encoder and decoder of the government affairs big language model. The modules are distributed in the attention layer and the feedforward network layer. By freezing more than 90% of the core parameters of the model, only the parameters of the LoRA modules are fine-tuned. Combined with the feature distillation technology of the intermediate layer of the model, the catastrophic forgetting when the model learns new knowledge is further avoided. A dynamic elastic weight merging algorithm is introduced. The algorithm includes a historical parameter importance evaluation module and an update constraint execution module. The former calculates the contribution of historical parameters to government affairs knowledge through the Fisher information matrix, and the latter calculates the contribution of historical parameters to government affairs knowledge through the Fisher information matrix. Differential protection constraints are applied to parameters based on contribution (the higher the contribution, the stronger the constraint); an "adaptive batch progressive learning" strategy is adopted, and the policy update frequency monitoring device is used to count the policy update frequency of each domain in real time. The data in the structured knowledge graph is divided into dynamic batches (daily update batch, weekly update batch, monthly update batch, quarterly update batch). The ratio of old knowledge samples to new knowledge samples in each batch of learning data is dynamically adjusted based on domain similarity (30% when similarity ≥ 80%, and 50% when similarity < 50%). The learning weights of old and new knowledge are balanced by the knowledge distillation loss function to ensure that the model retains ≥ 90% of historical knowledge. (3) Establish a secure and reliable update mechanism: Construct a multi-level data desensitization pipeline, and perform rule-based desensitization (using "6 bits for the first digit and 4 bits for the last digit, and masking the middle 7 bits" for the policy documents and core information obtained in step (1), intelligent desensitization (using place name word segmentation technology to retain address information to the municipal administrative unit, automatically identifying and hiding street address numbers), and sensitive entity replacement (using "position + number" to replace the names of personnel involved in confidential matters); the desensitized data enters the security audit platform, which verifies the legality of the data source through blockchain evidence storage (comparing the digital signature of the publishing department), and checks the integrity of the content through the text integrity verification algorithm (calculating the file hash value and comparing it with the publishing source). Only when the audit pass rate is ≥99% can it enter the subsequent process; the entire process is carried by a full-stack domestic computing power facility, including AI training servers based on Kunpeng / Feiteng chips, Kylin / Euler operating system, Renmin University Kingbase / DaMeng database and Huawei Cloud / Alibaba Cloud. The government cloud-dedicated cluster uses the SM4 national cryptographic algorithm for data transmission encryption (128-bit key length), and employs a dual-layer encryption scheme of "encrypted partition + file-level encryption" for storage. A closed-loop triple verification system is constructed: "model output - official database comparison - manual review." The first layer compares the policy interpretation results generated by the model with the official policy database using a pre-trained semantic similarity model (such as SimBERT). A similarity of ≥90% passes directly, while <90% triggers the second layer of manual review (cross-review by at least two government experts). After successful review, a blockchain traceability tag (including the original policy document hash value, generation time, and reviewer information) is added to the interpretation results. A real-time early warning system is deployed. This system monitors the factual consistency (compared to the knowledge base), logical coherence (based on causal relationship detection), and domain relevance (through a domain classification model) of the model output in real time, identifying and issuing warnings for "illusory" content. The warning response time is ≤100ms. (4) Improve the collaboration and evaluation framework: Construct a distributed cross-departmental knowledge sharing platform. The platform adopts a microservice architecture, including departmental intelligent agent clusters (1-3 deployed in each department), regional aggregation intelligent agents (divided by provincial / municipal level), and global distribution intelligent agents. Knowledge synchronization is achieved through a message queue based on Kafka. Multiple intelligent agents use a dedicated communication protocol for the government domain (supporting hierarchical access permissions). Establish a three-dimensional comprehensive evaluation system of "technical indicators + business indicators + user feedback". Quantitative technical indicators include policy interpretation accuracy (target ≥ 95%), model response speed (target ≤ 1 second), knowledge update latency (target ≤ 2 hours), and illusion rate (target ≤ 0.5%). The service indicators include policy coverage completeness (target ≥ 99%) and cross-departmental knowledge association accuracy (target ≥ 90%). Qualitative indicators are collected through government service terminals (APP / web page) pop-up questionnaires, telephone follow-ups and departmental seminars, including user service satisfaction (target ≥ 90%) and policy interpretation practicality score (target ≥ 4.5 / 5 points). The evaluation results are used to generate optimization schemes through the intelligent decision-making system, dynamically adjusting the knowledge collection frequency of step (1) (e.g., adjusting the high-attention areas from once a day to once every 4 hours) and the batch learning parameters of step (2) (e.g., learning rate, batch sample ratio), forming a closed-loop iterative mechanism of "collection-learning-evaluation-optimization".

[0008] As an improvement, the real-time data collection interface cluster in step (1) adopts a "master-slave" architecture design. The master interface is responsible for regular data collection, and the slave interface serves as a redundant backup. When the master interface fails, it automatically switches within 10 seconds. When the interface is connected to the policy release system of each department, the OAuth2.0 protocol is used for identity authentication, and access control is achieved through API keys. When the policy release system detects that there are new files, revised content, or invalidation announcements, the file download process is automatically triggered. The download process supports breakpoint resume and generates metadata records containing file release time, source department, original URL, status label ("new", "revised", "invalid"), file format, and size. The metadata is synchronized to the metadata layer of the knowledge graph in real time.

[0009] As an improvement, in step (1), the OCR technology adopts a multimodal deep learning OCR model based on the Transformer architecture. The model training set contains 1 million+ government affairs samples (covering scanned documents, PDF screenshots, handwritten annotations, and mixed documents with electronic signatures). It supports the recognition of commonly used government fonts such as Song, Fangsong, Heiti, and Kaiti, and has an adaptive resolution adjustment function (supporting 150dpi-1200dpi). The recognition accuracy for fuzzy text (clarity ≤50%) is ≥98%. For policy documents containing tables and formulas, the table structure recognition module (based on TableTransformer) is used to realize the synchronous extraction of text and structure, and the table recognition accuracy is ≥95%.

[0010] As an improvement, the specific process of NLP technology to extract core information in step (1) includes: using a pre-trained BERT-BiLSTM-CRF model for entity recognition, the model is fine-tuned on 50,000+ labeled government data, the entity recognition F1 value is ≥96%, the effective time is automatically converted to ISO8601 standard format (such as YYYY-MM-DD) after recognition, and the applicable scope entity is associated with the national standard administrative division coding library; using a bidirectional relationship extraction model based on attention mechanism, the relationship weight between entities is calculated through multi-head attention, and core relationships such as "policy name-effective time", "policy name-applicable scope", "policy name-effectiveness level" and "policy name-responsible department" are established, and the relationship extraction accuracy is ≥94%; using an improved TextCNN model to classify policy texts, the model contains three types of convolution kernels (1×2, 1×3, and 1×4, 256 of each), which are connected to the classifier through global max pooling, and the effectiveness level classification results include laws, administrative regulations, departmental rules, local regulations, local government rules, and normative documents, with a classification accuracy of ≥97%.

[0011] As an improvement, the structured knowledge graph in step (1) is stored in a distributed graph database Neo4j cluster with more than 3 nodes (1 master and 2 slaves), supporting data sharding and load balancing; the graph database constructs a multi-layer index system, including a B+ tree spatiotemporal index based on effective time, an R tree spatial index based on applicable scope, and a full-text index based on entity name, which can filter policy knowledge effective within a specific time range (accurate to the day) and administrative region within 1 second; it has an intelligent process of "automatic archiving of expired policies", which triggers an expiration check at 3 am every day. For policies that have exceeded the expiration time, they are automatically migrated from the active knowledge layer to the archived knowledge layer (stored using the LZ4 compression algorithm). The archived knowledge retains complete attributes and relationships, supports retrieval by the dimension of "expiration time + department", and the retrieval response time is ≤3 seconds.

[0012] As an improvement, in step (2), the dimension r of the low-rank matrix of the LoRA low-rank adaptation module is dynamically determined according to the model size (r=8-16 when the model parameters are ≤1 billion, r=16-32 when the model parameters are 1 billion-10 billion, and r=32-64 when the model parameters are >10 billion). The module parameters are initialized using the Xavier initialization method (the A matrix is ​​initialized to a uniform distribution of [-0.01, 0.01], and the B matrix is ​​initialized to a 0 matrix). The AdamW optimizer is used during fine-tuning, with the learning rate set to 1e-4-5e-4 (a smaller value is used when the similarity between new knowledge and historical knowledge is high). The batch size is 16-32, the training rounds are 10-50, and the cosine annealing learning rate scheduling strategy is adopted. Only the A and B matrix parameters of the LoRA module are updated (the dimension of the A matrix is ​​d×r, the dimension of the B matrix is ​​r×d, and d is the dimension of the hidden layer of the model). After fine-tuning, the LoRA parameters are merged with the original model parameters through model fusion technology. After merging, the model performance loss is ≤2%.

[0013] As an improvement, the protection constraints of the elastic weight merging algorithm in step (2) are as follows: calculate the importance score of the parameters related to historical government knowledge through the Fisher information matrix, and select the parameters with the top 30% scores as key protection objects; set the decay coefficient α for the key protection parameters (α takes the value of 0.1-0.5, and the higher the importance, the smaller α is); during the learning of new knowledge, the update amount of historical parameters = α × (new knowledge gradient - historical parameter gradient), and limit the update amplitude through gradient pruning (threshold set to 1.0); the single update amplitude of historical parameters does not exceed 20% of the initial value, and the cumulative update amplitude does not exceed 50% of the initial value; the parameter update process records logs in real time (including update time, parameter name, values ​​before and after update, and knowledge domains involved), and supports backtracking queries.

[0014] As an improvement, the batch division criteria for the "adaptive batch progressive learning" strategy in step (2) are as follows: daily updated policies (such as epidemic prevention and control notices) are divided into batches per day (each batch has ≤100 samples), weekly updated policies (such as market supervision dynamics) are divided into batches per week (each batch has ≤500 samples), monthly updated policies (such as statistical data releases) are divided into batches per month (each batch has ≤2000 samples), and quarterly updated policies (such as planning documents) are divided into batches per quarter. After each batch of data is input, the model is fine-tuned 1-3 times. After fine-tuning, it is tested on the validation set (the validation set contains 30% old knowledge samples and 70% new knowledge samples, and the samples cover all government affairs fields). The validation evaluation indicators include accuracy (target ≥95%), F1 value (target ≥0.95), and historical knowledge retention rate (target ≥90%). If the accuracy improvement is less than 1% in two consecutive rounds of fine-tuning, fine-tuning is stopped and a fine-tuning report (including parameter changes, performance comparison, and domain performance) is generated.

[0015] As an improvement, step (3) includes domestically produced computing facilities: AI training servers based on Kunpeng 920 / Feiteng 2000+ chips (single node computing power ≥200TFLOPS, memory ≥256GB, storage ≥10TBSSD), Kirin V10 / Euler 2.0 operating system (supporting security hardening and auditing), Renmin University Kingbase KES / DaMeng DM8 database (supporting distributed transactions and high availability), Huawei Cloud Government Dedicated Cloud / Alibaba Government Cloud Government Cloud Platform (compliant with Level 3 Information Security Protection Standard); the data transmission process adopts SM4 national cryptographic algorithm encryption (key length 128 bits, encryption mode is CBC), the key is managed through a domestic key management system (supporting full lifecycle management of key generation, distribution, rotation, and destruction), and the key is automatically rotated every 72 hours; the storage process adopts a dual-layer encryption scheme of "LUKS encrypted partition + file-level SM4 encryption" to ensure the security of static data storage.

[0016] As an improvement, the synchronization frequency of the cross-departmental knowledge sharing platform in step (4) is divided into three levels: important policies (marked by each department through tags, including policies involving people's livelihood, major corporate interests, and emergency management) are synchronized in real time and pushed through a WebSocket long connection with a synchronization delay of ≤1 minute; regular policies are synchronized on a timed basis with a synchronization cycle of once per hour; low-frequency updated policies (such as annual plans) are synchronized daily (at 2 a.m.); only incrementally updated knowledge data is transmitted during synchronization (the increment is determined by data fingerprint comparison), and the incremental data is compressed in Protocol Buffers format (compression rate ≥50%); the synchronization process has breakpoint resume and integrity verification functions, and triggers multi-level alarms when synchronization fails (sends an SMS to the administrator if not restored within 10 minutes, and automatically dials an alarm phone if not restored within 30 minutes), and automatically records the synchronization log (including synchronization time, data volume, status, and time consumed).

[0017] Beneficial effects Through a multi-protocol compatible interface cluster and dynamic knowledge graph, the system achieves real-time collection of full-volume policy documents across departments. Combined with high-precision OCR and NLP technologies, it ensures that the accuracy of core information extraction exceeds 95%. The spatiotemporal indexing and version management mechanism of the knowledge graph enables millisecond-level differentiation between new and old policies, allowing for rapid location of valid policies at specific times, regions, and levels, thus solving the problems of lagging updates and inefficient retrieval of traditional government knowledge.

[0018] Employing LoRA low-rank fine-tuning and dynamic elastic weighting algorithms, the new knowledge learning efficiency is improved by 3-5 times while freezing 90% of the core parameters, and the historical knowledge forgetting rate is controlled within 10%. The adaptive batch learning strategy balances the learning weights of new and old knowledge by dynamically adjusting the sample ratio, enabling the model to maintain an interpretation accuracy of over 95% even in scenarios with high-frequency policy updates, overcoming the "catastrophic forgetting" problem in large-scale incremental learning.

[0019] The fully domestically produced computing infrastructure (from chips to cloud platforms) meets the Level 3 requirements of the Information Security Protection Scheme. Combined with multi-level desensitization (accuracy ≥99%) and national cryptographic encryption technology, it ensures the absolute security of government data transmission and storage. Triple verification and blockchain traceability mechanisms increase the credibility of policy interpretation results to 99.9%, and the real-time early warning system controls the response time for identifying "illusion" content to within 100ms, completely solving the security and trust bottleneck of AI applications in the government sector.

[0020] The cross-departmental knowledge-sharing platform achieves full-domain knowledge collaboration through a multi-agent architecture, with a synchronization delay of ≤1 minute for important policies, accelerating the process by over 90% compared to traditional methods. A three-dimensional evaluation system (technology + business + user) and a dynamic optimization mechanism form a closed loop, ensuring 99% policy coverage completeness and over 90% user satisfaction, thus driving the upgrade of government services from "passive response" to "proactive adaptation."

[0021] The model's response time is ≤1 second, and the knowledge update latency is ≤2 hours, representing an efficiency improvement of over 100 times compared to traditional manual update methods. Through precise policy interpretation and intelligent correlation, the time required for businesses and the public to handle affairs is reduced by more than 60%, and the pressure on government departments for manual consultation is reduced by 70%, significantly optimizing the government service experience and helping the construction of digital government move towards intelligence and precision. Attached Figure Description

[0022] Figure 1 This is a diagram of the dynamic knowledge synchronization system of the present invention; Figure 2 This is a diagram illustrating the incremental learning technology roadmap of the present invention. Figure 3 This is a diagram of the secure and reliable update mechanism of the present invention; Figure 4 This is a diagram illustrating the synergy and evaluation framework of the present invention.

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1-4 As shown, a method for continuous learning and knowledge updating of a government affairs big data language model is presented, the method including: A multi-protocol adaptive real-time government knowledge collection interface cluster was built, including a RESTful API interface pool, a WebSocket long-connection channel, and a file-sharing protocol adaptation module. This cluster connects to heterogeneous policy publishing systems across various departments (including web portals, OA systems, and dedicated business platforms) to obtain a full set of policy documents. The interface cluster adopts a master-slave architecture, with the master interface handling routine data collection and the slave interfaces serving as redundant backups. Automatic failover occurs within 10 seconds in case of master interface failure. When the interface connects with the policy publishing systems of various departments, OAuth2.0 is used for authentication, and API keys are used for access control. When a new file, revised content, or expired announcement is detected in the policy publishing system, a file resuming download process is automatically triggered, generating metadata records containing the file's publication time, source department, original URL, status label ("new," "revised," "expired"), file format, and size. This metadata is synchronized in real-time to the metadata layer of the knowledge graph. Through this interface cluster and synchronization mechanism, full real-time collection of policy documents across departments can be achieved, solving the problem of lagging basic data acquisition in traditional government knowledge updates.

[0025] A multimodal deep learning OCR model based on the Transformer architecture is used to perform optical character recognition on acquired policy documents. The model's training set contains over 1 million samples from the government sector (covering scanned documents, PDF screenshots, handwritten annotations, and mixed documents with electronic signatures). It supports the recognition of commonly used government fonts such as Song, Fangsong, Heiti, and Kaiti, and features adaptive resolution adjustment (supporting 150dpi-1200dpi). Image preprocessing, including noise reduction, tilt correction, and resolution enhancement, improves recognition quality. For blurred text (clarity ≤50%), the recognition accuracy is ≥98%. For policy documents containing tables and formulas, a TableTransformer-based table structure recognition module is used to simultaneously extract text and structure, achieving a table recognition accuracy of ≥95%. This high-precision OCR technology provides a high-quality text foundation for subsequent core information extraction, avoiding knowledge bias caused by text recognition errors.

[0026] A pipelined NLP processing workflow is used to extract core information from the text extracted by OCR (including but not limited to policy name, effective date, expiration date, scope of application, level of effectiveness, responsible department, policy clauses, and reward / punishment measures). A pre-trained BERT-BiLSTM-CRF model, finely tuned on over 50,000 labeled government data points, is used for entity recognition, achieving an F1 score ≥96%. The effective date is automatically converted to the ISO8601 standard format (e.g., YYYY-MM-DD), and the scope of application is associated with the national standard administrative division code library. A bidirectional relationship extraction model based on an attention mechanism is employed, using multi-head attention to calculate the association weights between entities, establishing relationships such as "policy name - effective date," "policy name - scope of application," and "policy name - level of effectiveness." The core relationships, such as "policy name - responsible department," were extracted with an accuracy rate of ≥94%. An improved TextCNN model was used to classify policy texts. This model contains 1×2, 1×3, and 1×4 convolutional kernels (256 of each), connected to the classifier via global max pooling. The hierarchical classification results included laws, administrative regulations, departmental rules, local regulations, local government rules, and normative documents, with a classification accuracy rate of ≥97%. This high-accuracy NLP core information extraction process ensures no omissions or mismatches of key policy information, laying the foundation for knowledge structuring and guaranteeing the accuracy of subsequent knowledge applications.

[0027] Based on the extracted core information, a spatiotemporally related structured knowledge graph is constructed. The knowledge graph includes an entity layer (policy entities, departmental entities, regional entities, etc.), a relationship layer (publication relationship, application relationship, validity relationship, etc.), and an attribute layer. The attribute layer uses "effective time - expiration time - scope of application - validity level - update frequency" as its core dimensions. The structured knowledge graph is stored in a distributed graph database Neo4j cluster with more than 3 nodes (1 master and 2 slaves), supporting data sharding and load balancing. The graph database constructs a multi-layered index system, including a B+ tree spatiotemporal index based on effective time, an R tree spatial index based on scope of application, and a full-text index based on entity name. It can filter policy knowledge effective within a specific time range (accurate to the day) and administrative region within 1 second. It has an intelligent process of "automatic archiving of expired policies," which triggers an expiration check at 3:00 AM every day. Policies that have exceeded their expiration time are automatically migrated from the active knowledge layer to the archived knowledge layer (stored using the LZ4 compression algorithm). Archived knowledge retains complete attributes and relationships, supports retrieval by the dimension of "expiration time + department," and has a retrieval response time of ≤3 seconds. The spatiotemporally related structured knowledge graph, along with efficient indexing and automatic archiving mechanisms, enables accurate differentiation between old and new policies and rapid multi-dimensional retrieval, solving the problems of traditional knowledge confusion and inefficient retrieval. At the same time, it provides structured and ordered knowledge data for model learning.

[0028] Multiple LoRA low-rank adaptation modules are embedded between the Transformer encoder and decoder of the government affairs big language model. These modules are distributed across the attention layer and feedforward network layer. By freezing over 90% of the model's core parameters, only the LoRA module parameters are fine-tuned. Combined with feature distillation technology in the intermediate layers, this further mitigates catastrophic forgetting when the model learns new knowledge. The low-rank matrix dimension *r* of the LoRA low-rank adaptation modules is dynamically adjusted according to the model size (*r* = 8-16 for model parameters ≤ 1 billion, *r* = 16-32 for 1 billion-10 billion, and *r* = 32-64 for > 10 billion). The module parameters adopt Xavier initialization... Initialization method (matrix A initialized with a uniform distribution of [-0.01, 0.01], matrix B initialized with 0); during fine-tuning, the AdamW optimizer was used, with a learning rate set to 1e-4-5e-4 (the smaller value is used when the similarity between new and historical knowledge is high), batch size of 16-32, training epochs of 10-50, and a cosine annealing learning rate scheduling strategy; only the A and B matrix parameters of the LoRA module were updated (matrix A has a dimension of d×r, matrix B has a dimension of r×d, where d is the dimension of the hidden layer of the model), and after fine-tuning, the LoRA parameters were merged with the original model parameters through model fusion technology, and the performance loss of the merged model was ≤2%. The combination of LoRA low-rank fine-tuning and feature distillation technology significantly reduces the amount of parameter updates, improves the efficiency of new knowledge learning (3-5 times better than full parameter fine-tuning), effectively avoids catastrophic forgetting, and ensures the model's retention of historical knowledge.

[0029] A dynamic elastic weight merging algorithm is introduced, which includes a historical parameter importance assessment module and an update constraint execution module. The former calculates the contribution of historical parameters to government knowledge using the Fisher information matrix, while the latter applies differentiated protection constraints to the parameters based on their contribution (the higher the contribution, the stronger the constraint). The protection constraints of the elastic weight merging algorithm are as follows: the importance score of historical government knowledge-related parameters is calculated using the Fisher information matrix, and the parameters with the top 30% scores are selected as key protection targets; a decay coefficient α is set for key protection parameters (α ranges from 0.1 to 0.5, with α decreasing as importance increases); during the new knowledge learning process, the update amount of historical parameters = α × (new knowledge gradient - historical parameter gradient), and the update amplitude is limited by gradient pruning (threshold set to 1.0); the single update amplitude of historical parameters does not exceed 20% of the initial value, and the cumulative update amplitude does not exceed 50% of the initial value; the parameter update process is logged in real time (including update time, parameter name, values ​​before and after the update, and the knowledge domain involved), supporting backtracking queries. The dynamic elastic weight merging algorithm further reduces the risk of historical knowledge being covered by differential parameters, ensuring that the model retains ≥90% of historical political knowledge when learning new knowledge.

[0030] An "adaptive batch progressive learning" strategy is adopted, using a policy update frequency monitor to statistically analyze the policy update frequency in various fields in real time. The data in the structured knowledge graph is divided into dynamic batches (daily, weekly, monthly, and quarterly batches). The batch division criteria for the "adaptive batch progressive learning" strategy are as follows: daily updated policies (such as epidemic prevention and control notices) are divided into batches per day (each batch having ≤100 samples); weekly updated policies (such as market supervision updates) are divided into batches per week (each batch having ≤500 samples); monthly updated policies (such as statistical data releases) are divided into batches per month (each batch having ≤2000 samples); and quarterly updated policies (such as planning documents) are divided into batches per quarter. The ratio of old knowledge samples to new knowledge samples in each batch of learning data is... For example, the model dynamically adjusts its learning weights based on domain similarity (30% when similarity is ≥80%, and 50% when similarity is <50%), and balances the learning weights of old and new knowledge using a knowledge distillation loss function to ensure that the model retains ≥90% of historical knowledge. After each batch of data input, the model undergoes 1-3 rounds of fine-tuning, followed by testing on a validation set (containing 30% old knowledge samples and 70% new knowledge samples, covering all government domains). Validation evaluation metrics include accuracy (target ≥95%), F1 score (target ≥0.95), and historical knowledge retention rate (target ≥90%). If the accuracy improvement is less than 1% in two consecutive rounds of fine-tuning, fine-tuning is stopped, and a fine-tuning report is generated (including parameter changes, performance comparison, and domain performance). The adaptive batch learning strategy balances the learning weights of old and new knowledge by reasonably dividing batches and dynamically adjusting the sample ratio, enabling the model to maintain an interpretation accuracy of over 95% even in scenarios with high-frequency policy updates. This further optimizes the model's continuous learning ability and avoids learning imbalances caused by differences in knowledge update frequency.

[0031] A multi-level data anonymization pipeline was constructed, sequentially performing rule-based anonymization (using a "first 6 digits and last 4 digits retained, middle 7 digits masked" approach for ID card numbers and mobile phone numbers), intelligent anonymization (using place name word segmentation technology to retain address information down to the municipal administrative unit level, automatically identifying and hiding street and house numbers), and sensitive entity replacement (replacing the names of personnel with classified information using "position + number"). This multi-level data anonymization process effectively protects personal privacy and classified information, with a sensitive information residue rate of ≤0.1%, ensuring data processing compliance.

[0032] After anonymization, the data enters the security audit platform. The platform verifies the legality of the data source through blockchain evidence storage (comparing the digital signature of the publishing department) and checks the content integrity through a text integrity verification algorithm (calculating the file hash value and comparing it with the publishing source). Only data with an audit pass rate of ≥99% can proceed to the next stage. This security audit mechanism ensures that the data entering the system is reliable and complete, preventing false or tampered data from interfering with model learning from the source.

[0033] The entire process utilizes a fully domestically produced computing infrastructure, including AI training servers based on Kunpeng 920 / Phytium 2000+ chips (single-node computing power ≥200 TFLOPS, memory ≥256GB, storage ≥10TB SSD), Kirin V10 / Euler 2.0 operating systems (supporting security hardening and auditing), Renmin University Kingbase KES / DM8 databases (supporting distributed transactions and high availability), and Huawei Cloud Government Dedicated Cloud / Alibaba Government Cloud platforms (compliant with Level 3 Information Security Protection Standards). Data transmission employs the SM4 national cryptographic algorithm for encryption (128-bit key length, CBC encryption mode), with keys managed through a domestically produced key management system (supporting full lifecycle management of key generation, distribution, rotation, and destruction), and keys automatically rotated every 72 hours. Storage employs a dual-layer encryption scheme of "LUKS encrypted partition + file-level SM4 encryption" to ensure the security of static data storage. The combination of fully domestically produced computing power and multi-layer encryption technology meets Level 3 Information Security Protection Standards requirements, ensuring absolute security for government data transmission and storage, and safeguarding data sovereignty.

[0034] A closed-loop triple verification system is constructed, consisting of "model output - official database comparison - manual review." The first layer compares the policy interpretation results generated by the model with the official policy database using a pre-trained semantic similarity model (such as SimBERT). If the similarity is ≥90%, the model passes directly; otherwise, a second layer of manual review (cross-reviewed by at least two experts in the government sector) is triggered. After passing the review, a blockchain traceability tag (including the hash value of the original policy document, generation time, and reviewer information) is added to the interpretation results. A real-time early warning system is deployed. This system monitors the factual consistency (compared to the knowledge base), logical coherence (based on causal relationship detection), and domain relevance (through a domain classification model) of the model output in real time, identifying and issuing warnings about "illusory" content. The warning response time is ≤100ms. This triple verification and real-time early warning mechanism increases the credibility of policy interpretation results to 99.9%, quickly identifying and avoiding model "illusions," and completely solving the security and trust bottleneck of AI applications in the government sector.

[0035] A distributed, cross-departmental knowledge-sharing platform is constructed, employing a microservice architecture. This platform includes departmental intelligent agent clusters (1-3 deployed per department), regional aggregation intelligent agents (divided by provincial / municipal level), and global distribution intelligent agents. Knowledge synchronization is achieved through a Kafka-based message queue, and communication between multiple intelligent agents utilizes a government-specific communication protocol (supporting hierarchical access permissions). The synchronization frequency of the cross-departmental knowledge-sharing platform is divided into three levels: important policies (labeled by each department, including policies related to people's livelihood, significant corporate interests, and emergency management) are synchronized in real-time via WebSocket long-connection push, with synchronization delay... ≤1 minute; regular policies are synchronized on a scheduled basis, once per hour; low-frequency updated policies (such as annual plans) are synchronized daily (at 2 AM); only incrementally updated knowledge data is transmitted during synchronization (increments are determined by data fingerprint comparison), and incremental data uses Protocol Buffers compression format (compression rate ≥50%); the synchronization process has breakpoint resume and integrity verification functions, and triggers multi-level alarms when synchronization fails (sending an SMS to the administrator if not restored within 10 minutes, and automatically dialing an alarm phone if not restored within 30 minutes), and automatically records synchronization logs (including synchronization time, data volume, status, and time taken). The cross-departmental knowledge sharing platform achieves full-domain knowledge collaboration through a multi-agent architecture and hierarchical synchronization strategy, accelerating the synchronization of important policies by more than 90% compared to traditional methods, breaking down departmental knowledge barriers, and ensuring that model nodes in each department obtain unified and up-to-date government knowledge.

[0036] A three-dimensional comprehensive evaluation system integrating technical indicators, business indicators, and user feedback was established. Quantitative technical indicators include policy interpretation accuracy (target ≥95%), model response speed (target ≤1 second), knowledge update latency (target ≤2 hours), and illusion rate (target ≤0.5%). Business indicators include policy coverage completeness (target ≥99%) and cross-departmental knowledge association accuracy (target ≥90%). Qualitative indicators were collected through methods such as pop-up questionnaires on government service terminals (APP / webpage), telephone follow-ups, and departmental seminars, including user satisfaction with services (target ≥90%) and policy interpretation practicality score (target ≥4.5 / 5). This three-dimensional evaluation system comprehensively measures system performance from multiple dimensions—technical, business, and user—ensuring objective and comprehensive evaluation results and providing accurate basis for system optimization.

[0037] The evaluation results generate optimization plans through an intelligent decision-making system, dynamically adjusting the knowledge collection frequency (e.g., from once a day to once every 4 hours for high-interest areas) and batch learning parameters (e.g., learning rate, batch sample ratio), forming a closed-loop iterative mechanism of "collection-learning-evaluation-optimization". This closed-loop iterative mechanism allows the system to continuously optimize based on actual operational results, ensuring policy coverage completeness of 99% and user satisfaction exceeding 90%, driving the upgrade of government services from "passive response" to "proactive adaptation". Simultaneously, the model response speed is ≤1 second and the knowledge update latency is ≤2 hours, improving efficiency by more than 100 times compared to the traditional manual update mode. Through precise policy interpretation and intelligent association, the time for businesses and the public to handle affairs is reduced by more than 60%, and the pressure on government departments for manual consultation is reduced by 70%, significantly optimizing the government service experience and providing strong support for the intelligent and precise construction of digital government.

[0038] Contents not described in detail in this specification are prior art known to those skilled in the art. Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for continuous learning and knowledge updating of a large-scale government language model, characterized in that, Includes the following steps: (1) Construct a dynamic knowledge synchronization system: Build a real-time government knowledge collection interface, and connect the real-time collection interface to the policy release system of various departments to obtain the latest policy documents in real time; OCR technology is used to perform optical character recognition on policy documents to extract text information. Then, NLP technology is used to extract core information from the text information. The core information includes at least the policy name, effective date, scope of application, and level of effectiveness. Based on the extracted core information, a structured knowledge graph is constructed. The structured knowledge graph uses "effective date-scope of application-level of effectiveness" as the core attribute dimension to distinguish and index new and old government knowledge. (2) Optimize incremental learning technology path: Introduce LoRA low-rank adaptation module in government affairs big language model, freeze the core parameters of the model, and only fine-tune the parameters of LoRA low-rank adaptation module to avoid catastrophic forgetting when the model learns new knowledge; introduce elastic weight merging algorithm to apply protective constraints to the parameters related to historical government affairs knowledge in the model and limit the update range of historical parameters during the learning of new knowledge; adopt "batch progressive learning" strategy, divide the data in the structured knowledge graph into multiple batches of learning data according to the policy update frequency, input the data into the model in stages according to batches, and control the ratio of old knowledge samples to new knowledge samples in each batch of learning data to 30%-50% to balance the learning weight of new and old knowledge; (3) Establish a secure and reliable update mechanism: perform data desensitization processing on the policy documents and core information obtained in step (1). The desensitization processing includes at least masking the ID card number and mobile phone number contained in the documents and retaining the address information up to the municipal administrative unit. Security audits are conducted on the anonymized data to verify the legality of the data source and the integrity of the content; domestically produced computing power facilities are used to support the model training, knowledge storage, and update processes; a three-tier verification process of "model output - official database comparison - manual review" is constructed. First, the policy interpretation results generated by the model based on new knowledge are compared with the official policy database in terms of semantic similarity. If the similarity is less than 90%, manual review is triggered. After the review is passed, a source tracing mark is added to the interpretation results, and the source tracing mark is associated with the corresponding original policy document. Deploy a real-time early warning system. The real-time early warning system identifies and warns of "illusionary" content generated by the model by monitoring the semantic consistency and factual accuracy of the model's output content. (4) Improve the collaboration and evaluation framework: Establish a cross-departmental knowledge sharing platform. The platform realizes the full-domain synchronization of government knowledge of each department through a multi-agent message transmission mechanism. The multi-agent message transmission mechanism includes departmental agents, aggregation agents and distribution agents. Departmental agents collect policy knowledge of their own departments, aggregation agents integrate full-domain knowledge, and distribution agents push updated knowledge to model nodes of each department. Construct a comprehensive evaluation system of "technical indicators + user feedback". Quantitative monitoring indicators include policy interpretation accuracy and model response speed. The policy interpretation accuracy target is ≥95%, and the model response speed target is ≤1 second. Qualitative indicators are collected through user satisfaction with services via pop-up questionnaires on government service terminals. Based on the evaluation results, dynamically optimize the knowledge collection frequency of step (1) and the batch learning parameters of step (2) to achieve iterative optimization of the update strategy.

2. The method according to claim 1, characterized in that, In step (1), the real-time data collection interface uses RESTful API or WebSocket protocol to connect with the policy release system of each department. When the policy release system detects that there are new files, revised content or invalidation announcements, the file download process is automatically triggered, and the release time, source department and status label of the file are recorded. The status labels include "new", "revised" and "invalid".

3. The method according to claim 1, characterized in that, In step (1), the OCR technology adopts a deep learning OCR model based on the Transformer architecture. The model supports text extraction from scanned documents, PDF screenshots, and handwritten annotation policy documents, and has an accuracy rate of ≥98% for recognizing fuzzy text.

4. The method according to claim 1, characterized in that, The specific process of extracting core information using NLP technology in step (1) includes: using the BERT-BiLSTM-CRF model for entity recognition to locate entities such as policy name, effective date, scope of application, and level of effectiveness; using a relation extraction model based on attention mechanism to establish the association relationship of "policy name-effective date", "policy name-scope of application", and "policy name-level of effectiveness"; and using the TextCNN model to classify the policy text and determine the level of effectiveness, which includes laws, administrative regulations, departmental rules, and local regulations.

5. The method according to claim 1, characterized in that, In step (1), the structured knowledge graph is stored in the graph database Neo4j. The graph database supports spatiotemporal indexes based on "effective time", which can quickly filter policy knowledge that is effective within a specific time range. It also supports the function of "automatic archiving of expired policies". When a policy exceeds its expiration time, it will be automatically migrated from the active knowledge layer to the archived knowledge layer.

6. The method according to claim 1, characterized in that, In step (2), the dimension r of the low-rank matrix of the LoRA low-rank adaptation module is 8-64. The module parameters are initialized using the Xavier method. During the fine-tuning process, the learning rate is set to 1e-4-5e-4, and only the A and B matrix parameters of the LoRA module are updated. The dimension of matrix A is d×r, and the dimension of matrix B is r×d, where d is the dimension of the hidden layer of the model.

7. The method according to claim 1, characterized in that, The protective constraints of the elastic weight merging algorithm in step (2) are as follows: set the decay coefficient α for the parameters related to historical government knowledge, with α ranging from 0.1 to 0.

5. During the learning of new knowledge, the update amount of historical parameters = α × (new knowledge gradient - historical parameter gradient), and the single update amplitude of historical parameters does not exceed 20% of the initial value.

8. The method according to claim 1, characterized in that, In step (2), the batch division criteria for the "batch progressive learning" strategy are as follows: the daily update policy is divided into batches per day, the weekly update policy is divided into batches per week, and the monthly update policy is divided into batches per month. After each batch of data is input, the model is fine-tuned 1-3 times. After fine-tuning, it is tested on the validation set. The validation set contains 30% old knowledge samples and 70% new knowledge samples. If the accuracy improvement is less than 1%, the fine-tuning is stopped.

9. The method according to claim 1, characterized in that, Step (3) The domestically produced computing facilities include servers based on Kunpeng chips, the Kylin operating system and Huawei Cloud government dedicated cloud platform. The data transmission process adopts the SM4 encryption algorithm, and the storage process adopts the national cryptographic level encryption storage scheme.

10. The method according to claim 1, characterized in that, In step (4), the synchronization frequency of the cross-departmental knowledge sharing platform is divided into two levels: important policies adopt real-time synchronization with a synchronization delay of ≤1 minute; regular policies adopt timed synchronization with a synchronization cycle of once per hour, and only incrementally updated knowledge data is transmitted during synchronization.

Images