A government affair data collection method and system of an artificial intelligence large language model

Abstract

The application provides a government affair data acquisition method and system of an artificial intelligence large language model, and the method comprises the following steps: S1, government affair data source access and heterogeneous data standardization processing; S1.1, the server acquires access information and data element information of cross-department government affair data sources; S1.2, the server constructs a government affair heterogeneous data mapping model based on government affair data element information, and sets a heterogeneous data mapping function. Through the construction of the government affair heterogeneous data mapping model, the standardized conversion of cross-department government affair data of different databases such as MySQL, Oracle and MongoDB is realized, the heterogeneous data data puzzle type integration problem is effectively solved, and the data sharing barrier between government departments is broken.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for collecting government data using an artificial intelligence large language model. Background Technology

[0002] In the process of digital transformation of government affairs, government data collection is a core and fundamental link in government data analysis, government service optimization, and government decision support. With the development of big language model (LLM) technology, applying big language model to government data collection has become an important direction to improve collection efficiency and expand collection dimensions.

[0003] Existing technologies, such as the data collection method, device, and medium based on a large language model (Announcement No. CN120632184A), extract keyword matching URLs using a large language model and determine crawling strategies based on website historical update patterns to achieve data collection; and the automated multi-source data collection method based on a large language model (Announcement No. CN120450015B), which generates collection strategies by analyzing historical data characteristics and data source status, expands the collection scope using knowledge graphs, and completes data standardization processing. However, when these general data collection methods are applied to government data collection scenarios, they expose many adaptability issues and cannot meet the professionalism, compliance, and accuracy requirements of government data collection. Specific problems are as follows: Data supply layer issues: Cross-departmental data is stored in different databases such as MySQL, Oracle, and MongoDB, with inconsistent structures and standards, creating a jigsaw puzzle-like integration challenge; fragmented data storage makes cross-departmental sharing and collaboration difficult, resulting in long collection chains and poor timeliness. There is also dirty data such as duplicate records, format errors (e.g., invalid dates), and missing key fields; manual correction of historical data is costly and difficult to annotate, directly reducing the accuracy and robustness of LLM data collection.

[0004] Compliance and security issues: Sensitive government data (privacy, confidentiality) requires local / private deployment, and 76% of provincial government big data models need to run on the government extranet, which drives up computing costs; at the same time, it puts forward hard requirements for LLM lightweight and low resource adaptation.

[0005] The data collection process is difficult to implement fine-grained access control, data anonymization and auditing; existing solutions often do not incorporate technologies such as differential privacy and federated learning, which may easily violate the red lines of the Personal Information Protection Law.

[0006] Issues with the semantic layer of government documents: Policy documents and service guides are characterized by long documents, clause-based structures, dense terminology, and rigorous logic. Traditional LLM models have low accuracy in extracting key information from long texts and are prone to losing business rule constraints. Implicit business knowledge such as approval processes, required documents, and discretionary standards are not structured and injected, making it difficult for the model to understand the causal relationships in government data collection scenarios such as missing documents, corrections, and process transitions. The terminology systems and data collection rules vary greatly across different fields (market supervision, social security, taxation), and the model lacks domain adaptability, resulting in high migration costs. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the defects of the existing technology. The present invention proposes a method and system for collecting government data using an artificial intelligence large language model.

[0008] To address the aforementioned data supply layer issues: Cross-departmental data is stored in different databases (MySQL, Oracle, MongoDB), with inconsistent structures and standards, creating a jigsaw puzzle-like integration challenge; fragmented data structures hinder cross-departmental sharing and collaboration, resulting in long collection chains and poor timeliness. There is also dirty data with duplicate records, format errors, and missing key fields; manual correction of historical data is costly and difficult to annotate, directly reducing the accuracy and robustness of LLM data collection. Compliance and security layer issues: Sensitive government data requires local and private deployment; 76% of provincial government large-scale models need to run on the government extranet, driving up computing costs; simultaneously, it imposes strict requirements on the lightweight nature and low resource adaptability of LLM. Fine-grained access control, data anonymization and auditing are difficult to achieve during the collection process; existing solutions often lack differential privacy and federated learning technologies, easily violating the Personal Information Protection Law. Government semantic layer issues: Policy documents and service guides are characterized by long documents, clause-based structures, dense terminology, and rigorous logic; traditional LLM methods have low accuracy in extracting key information from long texts and are prone to losing business rule constraints. The implicit business knowledge regarding approval processes, required documents, and discretionary standards has not been structurally injected, making it difficult for the model to understand the causal relationships in government data collection scenarios involving missing documents, document corrections, and process transitions. Furthermore, significant differences in terminology and data collection rules across different fields result in a lack of domain adaptability and high migration costs for the model. The technical solution adopted in this invention is: A method for collecting government data using an artificial intelligence large language model, the method comprising the following steps: S1. Access to government data sources and standardization of heterogeneous data: The server described in S1.1 obtains access information and data element information from cross-departmental government data sources; The server described in S1.2 constructs a heterogeneous government data mapping model based on government data metadata. Define heterogeneous data mapping functions For government raw data from different databases Standardization transformation is performed to obtain a standardized government data set. ,in ; The server described in S1.3 supports standardized government data sets. Standardize data formats and align fields to generate a standardized index table for government data. S2. Refined Cleaning and Quality Assessment of Government Data: The server described in S2.1 constructs a multi-dimensional cleansing rule base based on the characteristics of government data. ; S2.2 describes the server calling the cleaning rule base. Standardized government data sets Dirty data identification is performed to obtain a dirty data set. With the dataset to be cleaned ; The server described in S2.3 handles dirty data sets. The cleaning operation was performed, and missing fields were filled using a government affairs domain interpolation completion algorithm to obtain the cleaned government affairs dataset. ; The server described in S2.4 constructs a government data quality assessment model. Define the quality assessment function Calculate the cleaned government data set Quality rating ,in ; The server setting quality score threshold described in S2.5 ,like Then Determined as a qualified government data set ,like Then The sample is then sent back to step S2.2 for a second cleaning. S3, Lightweight Large Language Model Government Affairs Adaptation Training: The server described in S3.1 acquires the pre-trained large language model. A lightweight compression algorithm combining model pruning, quantization, and knowledge distillation is used to compress the data. Compression is performed to obtain a lightweight basic model. ; The server described in S3.2 constructs a labeled dataset for the government affairs sector. With government general knowledge graph ; The server described in S3.3 uses a small-sample fine-tuning algorithm to... and Inject lightweight base model Adaptive fine-tuning is performed in the field of government affairs to obtain a lightweight large language model for government affairs. ; S4. Government data collection under fine-grained compliance and security control: The server described in S4.1 constructs a fine-grained permission control model based on the government data permission system. Set the permission allocation function Based on the identity information of user U, assign corresponding government data collection permissions to them; The server described in S4.2 constructs a government sensitive information identification model. For qualified government data sets Sensitive information is identified to obtain a set of sensitive information. ; The server described in S4.3 uses a differential privacy de-identification algorithm to de-identify the sensitive information set. Data anonymization was performed by introducing Laplace noise to obtain a qualified de-identified government data set. ; The server described in S4.4 calls the government lightweight large language model. Based on the user's permission scope The task of collecting government data was carried out, and preliminary results of the data collection were obtained. ; The server described in S4.5 performs real-time auditing and logging of operations during the data collection process, generating a government data collection operation log. ; S5. Optimization and verification of data collection results with enhanced government semantics: The server described in S5.1 constructs a government affairs business rule base. Knowledge graphs of implicit business in government affairs ; The server described in S5.2 will and Injecting lightweight large language models into government affairs Set up a government affairs semantic optimization function Regarding the preliminary collection results The semantic correction and completion of government affairs data were performed to obtain optimized data collection results. ,in ; The server described in S5.3 constructs a government data collection and verification model. Set the verification function From the two dimensions of accuracy and compliance Verification was performed, and a verification score was obtained. ,in ; The server setting verification scoring threshold described in S5.4 ,like Then Confirmed as the final government data collection result ;like < Then Feedback is sent to the lightweight large language model for government affairs for secondary optimization; The server described in S5.5 will collect the final government data results. With the collection operation log The data is associated with the storage and then pushed to the corresponding government service terminals.

[0009] Preferably, in step S1.1, the access information includes database type, access address, and access permissions, and the data element information includes data structure, data type, field definition, and data encoding standard; In step S2.1, the cleaning rule base includes rules for cleaning duplicate data, rules for correcting format errors, and rules for completing missing fields; In step S3.2, the government affairs domain labeled dataset includes labeled information of government affairs long texts, government affairs terminology, and government affairs business data, and the government affairs general knowledge graph includes government affairs entities, entity attributes, and relationships between entities; In step S4.5, the operation log includes the user who collected the data, the time of collection, the range of data collected, and the collection operation behavior. In step S5.1, the implicit business knowledge graph of government affairs includes the causal relationship of approval process, material requirements, and discretionary standards.

[0010] Preferably, in step S1.2, the government heterogeneous data mapping model The construction process is as follows: The server extracts features from government data metadata from different databases to obtain data element feature vectors. ,in For data structure features, For data type characteristics, Data encoding standard features; The server is configured with a standardized template for government data. This includes standardized data structures, standardized data types, standardized field definitions, and standardized data encoding; The server uses a cosine similarity algorithm to calculate the feature vector of data elements. With standardized template feature vector similarity Construct a heterogeneous data mapping table; The server sets a heterogeneous data mapping function based on a mapping table. ,in For the mapping weight matrix, As the bias term, it is applied using the gradient descent algorithm. and Training and optimization are conducted to transform raw government data into standardized data. In step S2.4, the government data quality assessment model Quality assessment function The calculation formula is: in: A data integrity score is assigned to characterize the completeness of government data fields. ; The data format standardization score represents the degree to which government data conforms to standardized templates. ; The data deduplication rate is scored to characterize the degree to which duplicate records are eliminated in government data. ; The data validity score represents the degree to which government data values ​​conform to the logic of government business operations. ; Let be the weighting coefficient, satisfying ,and It will be dynamically adjusted according to the needs of the government sector.

[0011] Preferably, in step S3.1, the execution process of the lightweight compression algorithm is as follows: Model pruning: The server prunes the pre-trained large language model Analyze the neural network weights and select those with weight values ​​less than the pruning threshold. Remove the connections to obtain the pruned model. ; Model quantization: The server will The 32-bit floating-point weights are quantized to 8-bit integers, and the quantization function is set. Inverse quantization function The quantized model is obtained. ; Knowledge distillation: The server will As a teacher model As a student model, the distillation loss function is defined. ,in For cross-entropy loss, Output for the teacher model, Output for the student model. Using the temperature coefficient, a lightweight base model is obtained by transferring knowledge from the teacher model to the student model through distillation training. .

[0012] Preferably, in step S4.3, the execution process of the differential privacy de-identification algorithm is as follows: The server determines the set of sensitive information. Privacy Budget Set a privacy budget allocation strategy, and Assign to each sensitive field; The server sets the value of each sensitive field. Laplace noise was added to obtain the desensitized values. ,in Global sensitivity for query functions targeting sensitive fields. For noise that conforms to a Laplace distribution, its probability density function is: ; The server handles the anonymized values. Perform a reasonableness check, if If the data falls within the scope of government business logic, it is retained; otherwise, noise is added again for desensitization, ultimately resulting in a qualified desensitized government dataset. .

[0013] Preferably, in step S5.3, the government data collection and verification model verification function The calculation formula is: in: The accuracy of the data collection results is scored to characterize the degree of matching between the collected results and the original government data, as well as the accuracy of key information extraction. ; The compliance score for the collected results indicates the degree to which the collected results comply with government data security rules, business rules, and laws and regulations. ; To verify the weighting coefficients, the following conditions must be met: ,and The security level of government data will be dynamically adjusted according to the level of government data security. Sensitive government data will be subject to further adjustments. The value should not be lower than 0.6.

[0014] Preferably, the method further includes an adaptive migration step in the government sector: The server obtains a subset of domain-specific labels for the target government affairs domain. With domain knowledge graph ; The server will use a lightweight large language model for government affairs. The parameters of the general government affairs layer are fixed, and only the parameters of the domain adaptation layer are fine-tuned. The gradient descent algorithm is used to minimize the fine-tuning loss function. ; After the server completes fine-tuning, a lightweight large language model for the target government domain is obtained. This enables data collection and adaptation across government sectors.

[0015] A government data collection system based on an artificial intelligence large language model, the system including a government data collection server, the server including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described methods for collecting government data based on an artificial intelligence large language model; The system also includes a communication connection with the government data collection server: Cross-departmental government data source terminal: used to store government raw data, provide government data source access information and data element information to the government data collection server, receive data collection requests from the server and feed back government raw data; Government security management terminal: used to configure government data security rules, fine-grained access control policies, and privacy budget parameters; receive sensitive information identification results and collection operation logs from government data collection servers; and realize security management and auditing of government data collection. Government Service Terminal: Used to build a government service rule base and a government service implicit business knowledge graph, receive the final collection results from the government data collection server, manually verify the collection results and provide feedback, thereby achieving closed-loop optimization of the collection results. Preferably, the government data collection server includes multiple functional modules, which work together to collect government data. The functional modules include: Heterogeneous data standardization module: This module is used to perform the steps of accessing government data sources and standardizing heterogeneous data, build a mapping model for heterogeneous government data, and realize the standardized transformation of raw government data. Government Data Cleaning and Assessment Module: Used to perform refined cleaning and quality assessment steps for government data, build a cleaning rule base and quality assessment model, and realize the cleaning and quality screening of dirty government data; Lightweight Model Training Module: This module is used to perform the government affairs adaptation training steps of the lightweight large language model. It compresses and fine-tunes the pre-trained large language model for the government affairs domain to obtain a lightweight large language model for government affairs. Compliance and security data collection module: used to execute government data collection steps under fine-grained compliance and security control, build access control model and sensitive information identification model, and realize de-identified data collection and operation auditing; Semantic Enhancement Optimization and Verification Module: This module is used to perform optimization and verification steps for the collection results of government semantic enhancement, injecting government business rules and implicit knowledge to achieve optimization and dual verification of the collection results. Domain Adaptive Migration Module: Used to execute adaptive migration steps for government domains, enabling rapid model adaptation across government domains; Data storage and push module: Used to associate and store the final collection results and operation logs, push the collection results to government business terminals, and receive manual verification feedback.

[0016] Preferably, the cross-departmental government data source terminals include relational database terminals and non-relational database terminals. The relational database terminals include MySQL terminals and Oracle terminals, and the non-relational database terminals include MongoDB terminals. The government data collection server communicates with each terminal via the government extranet in encrypted form, using the national cryptographic SM4 algorithm to encrypt the transmitted data and the national cryptographic SM2 algorithm for authentication.

[0017] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs a heterogeneous government data mapping model, which realizes the standardized conversion of cross-departmental government data from different databases such as MySQL, Oracle, and MongoDB. It effectively solves the problem of jigsaw puzzle-style integration of heterogeneous data and breaks down data sharing barriers between government departments. It also optimizes the design of the government data collection link, significantly shortens the collection link of cross-departmental government data, improves the timeliness of government data acquisition, and increases the collection efficiency by more than 60% compared with existing general methods.

[0018] This invention constructs a fine-grained government data access control model, which can achieve refined allocation of data collection permissions based on government departments, positions, and personnel; it embeds differential privacy desensitization technology to perform irreversible security processing on sensitive government information, avoiding privacy leaks; and it realizes real-time auditing and logging of operational behavior during the collection process, enabling full-process traceability of collection operations, fully complying with the requirements of laws and regulations such as the Personal Information Protection Law and the Data Security Law, effectively avoiding compliance risks in government data collection.

[0019] This invention injects implicit business knowledge graphs, such as government business rule bases, approval processes, and discretionary standards, into a lightweight large language model, effectively improving the model's ability to understand long government documents, professional terminology, and clause-based logic, increasing the accuracy of key information extraction by more than 65%. A domain-adaptive fine-tuning module is designed for different government domains. When migrating across market supervision, social security, taxation, and other fields, only the domain knowledge graph and labeled subsets need to be replaced, without needing to re-optimize the model, reducing migration costs by more than 80% and enabling rapid collection and adaptation of government data across multiple domains. Attached Figure Description

[0020] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein: Figure 1 This is a flowchart illustrating the steps of the government data collection method of the present invention. Figure 2 This is a flowchart of step S1 of the government data collection method of the present invention; Figure 3 This is a flowchart of step S2 of the government data collection method of the present invention; Figure 4 This is a flowchart of step S3 of the government data collection method of the present invention; Figure 5 This is a flowchart of step S4 of the government data collection method of the present invention; Figure 6 This is a flowchart of step S5 of the government data collection method of the present invention; Figure 7 This is a system architecture block diagram of the government data collection method of the present invention. Detailed Implementation

[0021] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0022] Please see Figures 1-7 The following detailed description of a method and system for collecting government data using an artificial intelligence large language model according to the present invention is provided in conjunction with specific embodiments. This embodiment takes data collection in the field of social security government affairs as an example to elaborate on each step of the method and each module of the system. The scope of protection of the present invention is not limited to this embodiment and can be extended to all government affairs fields such as market supervision, taxation, and civil affairs.

[0023] System Architecture Setup The government data collection system for the artificial intelligence large language model of this invention takes the government data collection server as the core execution entity. The server adopts the domestic Kunpeng 920 processor, is equipped with 64GB of memory and 2TB solid-state drive, and is deployed on the government extranet to realize localized and private operation. The server runs on CentOS 7.9 operating system, uses Python 3.9 as the development language, and builds the model training and data processing environment based on the PyTorch 2.0 deep learning framework and TensorFlow 2.15 machine learning framework.

[0024] The system also includes three types of terminals that communicate encrypted with the government data collection server via the government extranet: Cross-departmental government data source terminals include: a MySQL database terminal for the social security department (stores information on social security participants and social security payment records), an Oracle database terminal for the civil affairs department (stores household registration information), and a MongoDB database terminal for the public security department (stores identity information). Each terminal is equipped with a national cryptographic SM4 encryption module to encrypt data transmission. Government Security Control Terminal: Deployed in government data security management departments, it is equipped with an access control system and a sensitive information audit system. It is used to configure access control policies and privacy budget parameters for social security data, and to achieve full-process auditing of data collection operations. Government Service Terminal: Deployed in social security service departments, equipped with a social security service rule management system and knowledge graph construction tools to build a social security service rule base and a social security implicit service knowledge graph, enabling manual verification and feedback of the collected results.

[0025] The government data collection server is internally built with seven functional modules: heterogeneous data standardization module, government data cleaning and evaluation module, lightweight model training module, compliance and security collection module, semantic enhancement, optimization and verification module, domain adaptive migration module, and data storage and push module. Each module is decoupled and works collaboratively based on a microservice architecture, and the modules communicate with each other through RESTful APIs.

[0026] Specific Implementation Steps for Government Data Collection Methods This implementation method takes the collection of social security participants' data in the field of social security administration as the specific data collection task. The data collection users are the staff of the social security business handling departments. The scope of data collection includes the participants' identity information, household registration information, and social security payment records. The government data collection server acts as the executing entity and performs the following steps: Step 1: Access to government data sources and standardization of heterogeneous data This step is executed by the heterogeneous data standardization module. Its core objective is to enable the access and standardized conversion of heterogeneous government data across social security, civil affairs, and public security departments, thereby solving the problem of heterogeneous data integration.

[0027] Step 1.1: Obtaining Data Source Information The server sends data source access requests to cross-departmental government data source terminals of social security, civil affairs, and public security departments via the government extranet. After each terminal verifies the server's identity (using the national cryptographic SM2 algorithm for authentication), it returns data source access information and data element information. Social Security MySQL Terminal: Access address 192.168.1.100, access permission is read-only, data structure is a two-dimensional table, data types include string (person's name, ID number), value (payment amount), date (payment time), field definitions include insured person ID, name, ID number, payment base, payment amount, payment time, data encoding is UTF-8; Civil Affairs Oracle Terminal: Access address 192.168.1.101, access permission is read-only, data structure is a two-dimensional table, data types include string (person's name, ID number, household registration address) and date (date of birth), field definitions include household registration ID, name, ID number, date of birth, household registration address, data encoding is GBK; The public security MongoDB terminal has the following access address: 192.168.1.102. The access permission is read-only. The data structure is document type. The data types include strings (person's name, ID number, ethnicity) and booleans (whether the person is a permanent resident). The field definitions include identity ID, name, ID number, ethnicity, and permanent resident identifier. The data encoding is UTF-8.

[0028] The server parses the access information and data element information of each data source and stores them in the government data element information database.

[0029] Step 1.2: Construction of a Heterogeneous Data Mapping Model for Government Affairs The server builds a mapping model for heterogeneous government data. The specific process is as follows: Feature extraction: Feature extraction is performed on the data metadata from various data sources to obtain data metadata feature vectors, such as the feature vector of the social security MySQL terminal. Feature vector of the civil affairs Oracle terminal Feature vectors of public security MongoDB terminals ; Standardized template settings: Set up standardized templates for social security participant data. The standardized data structure is a two-dimensional table, and the standardized data types are strings (name, ID number, registered address, ethnicity), numbers (contribution base, contribution amount), dates (date of birth, payment time), and booleans (permanent resident identifier). The standardized fields are defined as unified insured person ID, name, ID number, date of birth, ethnicity, registered address, permanent resident identifier, contribution base, contribution amount, and payment time. The standardized data encoding is UTF-8. Similarity calculation: The cosine similarity algorithm is used to calculate the feature vector of each data element and the feature vector of the standardized template. Similarity, for example , , Furthermore, a heterogeneous data mapping relationship table is constructed to clarify the mapping relationship between each data source field and the standardized field (e.g., the social security MySQL participant ID is mapped to the standardized unified participant ID, and the civil affairs Oracle household registration ID is associated with the standardized unified participant ID). Mapping function training: Setting up a mapping function for heterogeneous data ,in The mapping weight matrix is ​​10×8. The bias term is 10-dimensional. It takes raw government data from various data sources as input and standardized template data as output, and applies a gradient descent algorithm (learning rate...). =0.001, iteration count 1000) and Perform training and optimization until the loss function is optimized. The heterogeneous data mapping model is obtained when the model converges (MSE < 0.001). .

[0030] Step 1.3: Heterogeneous data standardization transformation The server calls the trained mapping model. Through mapping function Government raw data from various data sources Perform standardization conversion: Convert GBK encoded data in the Civil Affairs Oracle terminal to UTF-8 encoding; Convert document-type data in the public security MongoDB terminal into two-dimensional table data; Fields from various data sources are aligned and merged, and the insured person ID, household registration ID, and identity ID are integrated into a unified insured person ID, enabling cross-data source data association and matching based on the ID card number; Standardize the data format, for example, unify the payment time and date of birth to the YYYY-MM-DD format.

[0031] The final standardized government data set Dstd is obtained. This dataset has a two-dimensional table structure, contains 10 standardized fields, and a total of 100,000 data records of insured persons. The server also generates a standardized index table for government data, which records the original data source and field mapping relationship of each standardized data, so as to facilitate data traceability.

[0032] Step 2: Refined cleaning and quality assessment of government data This step is performed by the government data cleaning and assessment module. Its core objective is to achieve refined cleaning of social security participant data, remove dirty data, conduct quality assessment, and improve the quality of government data.

[0033] Step 2.1: Building a Multi-Dimensional Cleaning Rule Base The server builds a multi-dimensional cleansing rule base based on the characteristics of social security government data. Specifically, it includes: Duplicate data cleaning rules: Use the ID number as a unique identifier. If multiple data entries have the exact same ID number, they are considered duplicate data, and the most recently updated data entry is retained. Format error correction rules: The ID number must be 18 digits / letters, otherwise it will be judged as a format error and marked; the payment time and date of birth must conform to the YYYY-MM-DD format and the values ​​must be within a reasonable range (e.g., the date of birth is between 1900 and 2025), otherwise it will be judged as a format error and corrected. Missing field completion rules: If core fields such as name and ID number are missing, they are considered critical missing and marked as dirty data; if non-core fields such as ethnicity and registered address are missing, they are filled by interpolation based on the first 6 digits (address code) and the 17th digit (gender code) of the ID number.

[0034] Step 2.2: Dirty Data Identification The server calls the cleaning rule library. Standardized government data sets Perform a line-by-line scan to identify the dirty data set. With the dataset to be cleaned : Dirty Data Collection There are a total of 1200 records, including 300 duplicate records, 200 records with incorrect ID number formats, and 700 records with missing core fields. Data set to be cleaned There are a total of 98,800 records, including 2,500 records with missing non-core fields, 1,800 records with incorrect date formats, and 500 records with other minor formatting issues.

[0035] Step 2.3: Cleaning dirty data Server for dirty data sets With the dataset to be cleaned Perform the cleaning operation separately: Duplicate data: According to the cleaning rules, retain the latest updated record and delete the rest of the duplicate records; Formatting errors: Records with incorrect 18-digit ID numbers will be manually marked (cannot be automatically corrected), while records with incorrect date formats will be automatically corrected (e.g., 2025 / 05 / 20 will be corrected to 2025-05-20). Missing fields: Records with missing core fields are marked as invalid data and removed from the dataset; records with missing non-core fields are completed using government interpolation completion algorithms, for example, based on the ID card number address code 330106, the household registration address is completed as Xihu District, Hangzhou City, Zhejiang Province.

[0036] After cleaning, the cleaned government data set is obtained. A total of 97,500 valid data records were recorded.

[0037] Step 2.4: Government Data Quality Assessment Server-side government data quality assessment model Define the quality assessment function: The weighting coefficients are set based on the needs of social security government data. =0.3 (data integrity) =0.25 (formatting specifications) =0.2 (data deduplication rate) =0.25 (data validity).

[0038] The server calculates scores for each dimension separately: Data integrity score : 98 points, only 0.5% of the non-core fields have completed information, and the core fields are not missing; Formatting Standardization Scoring : 95 points, with only 0.3% of the date fields having minor formatting corrections, and the remaining fields conforming to standardized formats; Data deduplication rate score 100 points; all duplicate data has been removed. Data validity score The score was 96, with only a slight adjustment to the contribution base of 0.2% (which is consistent with the logic of social security operations).

[0039] Substitute into the quality assessment function to calculate the quality score: Step 2.5: Qualified Dataset Screening The server sets a quality scoring threshold for social security government data. ,because Therefore, the cleaned government data set Determined as a qualified government data set A total of 97,500 valid data records of social security participants were collected.

[0040] Step 3: Lightweight Large Language Model Government Affairs Adaptation Training This step is performed by the lightweight model training module. The core objective is to perform lightweight compression and adaptive fine-tuning of the pre-trained large language model in the social security domain, so as to obtain a lightweight large language model for social security that meets the deployment requirements of the government extranet.

[0041] Step 3.1: Lightweight Compression of Pre-trained Large Language Model The server selects the open-source pre-trained large language model Llama2-7B as the pre-trained model. A lightweight compression algorithm combining model pruning, quantization, and knowledge distillation is used to compress the model. The specific process is as follows: Model pruning: Set pruning threshold The neural network weights of Llama2-7B were analyzed, and connections with weight values ​​less than 1e-4 were removed, resulting in the removal of approximately 30% of redundant connections, thus obtaining the pruned model. The number of model parameters decreased from 7 billion to 4.9 billion; Model quantization: The 32-bit floating-point weights are quantized to 8-bit integers, and the quantization function is set. Inverse quantization function The quantized model is obtained. The model storage space was reduced from 28GB to 7GB; Knowledge distillation: Using Llama2-7B as the teacher model, As a student model, the distillation loss function is defined. The temperature coefficient Using general government text data (policy documents, service guides) as training data, a 10-epoch distillation training was performed (batch size=32, learning rate=0.5%). =0.0001), until the distillation loss function converges, thus obtaining the lightweight basic model. The model has 4.9 billion parameters, 7GB of storage space, and its inference speed is 3 times faster than the original model. Its computing power consumption is reduced by 75%, meeting the low computing power requirements of local deployment on the government extranet.

[0042] Step 3.2: Construction of Datasets and Knowledge Graphs in the Government Affairs Domain The server builds a labeled dataset in the social security field. With government general knowledge graph : Annotated datasets in the social security field We collected 500,000 pieces of text data in the social security field, including policy documents, service guides, and data on insured persons. These were then manually annotated (by social security business experts). The annotations included key information from long government documents, definitions of government terminology, and field relationships in social security business data, forming a small sample annotated dataset (50,000 annotations) to meet the needs of small sample fine-tuning. General knowledge graph for government affairs The system is built using the Neo4j graph database, including government entities (insured persons, social security departments, and types of insurance premiums), entity attributes (name and ID number of insured persons, name and responsibilities of social security departments), and relationships between entities (insured persons - payment - types of social security premiums, social security departments - processing - social security services), totaling 100,000 entity nodes and 200,000 relationship edges.

[0043] Step 3.3: Adaptive Fine-tuning in the Government Sector The server uses a small-sample fine-tuning algorithm to adjust the labeled dataset in the social security field. With government general knowledge graph Injecting the lightweight base model Mlight for adaptive fine-tuning in the social security domain: The entities and relations of the knowledge graph are converted into vector representations that the model can recognize. The TransE algorithm is used to map the entities and relations to a low-dimensional vector space (dimension d=128). The labeled dataset and knowledge graph vectors are fed into a lightweight base model. The underlying parameters of the model are fixed, and only the parameters of the top fully connected layer are fine-tuned. Set the fine-tuning loss function The Adam optimizer (learning rate) is used. =0.0001, weight decay wd=1e-5) for 5 epochs of fine-tuning training until the loss function converges.

[0044] After fine-tuning, a lightweight large language model for social security was obtained. This model retains the inference speed and low computational consumption of the lightweight basic model, while also possessing professional semantic understanding capabilities in the social security field.

[0045] Step 4: Government Data Collection under Fine-Grained Compliance and Security Management This step is executed by the compliance and security data collection module. Its core objective is to achieve secure data collection of social security participants while ensuring fine-grained access control and sensitive information desensitization, and to complete the audit and record-keeping of the data collection operation.

[0046] Step 4.1: Building a fine-grained access control model and allocating permissions The server is based on the social security government data access control system, and a fine-grained access control model is constructed. Set the permission allocation function This function takes the user's identity information (department, position, employee number) as input and the scope of government data collection as output.

[0047] The user for this data collection task is a staff member of the social security business handling department (employee ID SB001, position: social security business review). The server obtains the user's identity information through the government security management terminal. After the permission management model is used to allocate the user's permission, it is determined that the user can only collect social security participants' data within this jurisdiction (Xihu District, Hangzhou City), and cannot collect sensitive data from other jurisdictions (such as data of high-income people with payment amounts exceeding 100,000 yuan).

[0048] Step 4.2: Identification of Sensitive Government Information Server builds a model for identifying sensitive government information. The model is built based on the BERT-BiLSTM-CRF algorithm and uses a social security qualified government data set. As input, identify the set of sensitive information. ,include: Personal privacy information: ID number, name, registered address, date of birth; Sensitive social security information: contribution base, contribution amount, and contact information of the insured person (if any).

[0049] Step 4.3: Differential Privacy Desensitization Processing The server uses a differential privacy de-identification algorithm to de-identify sensitive information sets. The anonymization process strictly adheres to the requirements of the Personal Information Protection Law. The specific process is as follows: Privacy Budget Allocation: Set the total privacy budget =1.0, and allocate it proportionally to each sensitive field: ID number =0.3, Name =0.2, contribution base =0.2, Payment Amount =0.3; Laplace noise addition: Add noise conforming to a Laplace distribution to the values ​​of each sensitive field, for example: Contribution base Global sensitivity Add noise After obtaining the desensitized payment base ; Payment amount Global sensitivity Add noise The amount paid after desensitization ; ID card number anonymization: Partially mask the ID card number (keeping the first 6 and last 4 digits, and replacing the middle 8 digits with *), for example, 330106199005201234 is anonymized to 330106********1234; Reasonableness verification: The anonymized values ​​are validated using social security business logic to ensure that the anonymized contribution base and amount are within a reasonable range for social security contributions (e.g., the range of social security contribution base in Hangzhou in 2025 is 3957-24021). If they exceed the range, noise is added again to obtain a qualified anonymized government data set. .

[0050] Step 4.4: Execution of Government Data Collection The server calls the lightweight large language model of social security. Based on the access rights of user SB001 (Xihu District, Hangzhou), a data collection task for social security participants was performed on the anonymized and qualified government data set Ddes: Based on social security business needs, the model extracts key fields from the collected results: Unified Insured Person ID, Name (mask), ID Card Number (mask), Registered Address, Contribution Base (de-sensitized), Contribution Amount (de-sensitized), and Contribution Time; The model categorizes and organizes the collected data, grouping it according to the type of insurance contribution (pension insurance, medical insurance, and unemployment insurance) to obtain preliminary government data collection results. A total of 82,000 data entries of social security participants in Xihu District, Hangzhou City were collected.

[0051] Step 4.5: Data Collection Operation Audit and Traceability The server performs real-time auditing and logging of all operations during the data collection process, generating government data collection operation logs. The log content is stored in a structured format, including: User who collected the data: SB001 (Social Security Business Processing Department, Enrollment Business Review Post); Data collection time: 2025-08-20 09:30:00-2025-08-20 09:45:00; Data collection scope: Social security participants in Xihu District, Hangzhou City (82,000 records); Data collection operations include: data source access, data standardization, data cleaning, sensitive information de-identification, and key information extraction. Data processing results: Preliminary collection results The data volume is 82,000 records, and the user has no permission to perform out-of-bounds operations.

[0052] The server pushes the operation log (Lop) to the government security management terminal in real time, enabling full traceability of the data collection process.

[0053] Step 5: Optimization and Verification of Data Collection Results with Enhanced Government Semantics This step is performed by the semantic enhancement and optimization verification module. Its core objective is to optimize the government semantics of the collected results by injecting social security business rules and tacit knowledge, and to perform dual verification of accuracy and compliance to obtain the final collected results.

[0054] Step 5.1: Construction of Social Security Business Rules and Implicit Knowledge Graph The server obtains a social security business rule base constructed by social security business experts through government service terminals. Knowledge Graph of Implicit Social Security Business : Social Security Business Rules Database It includes 50 core business rules, such as social security contribution rules (pension insurance contribution rates are 8% for individuals and 16% for employers), participation conditions rules (being 16 years of age or older and not yet reaching the statutory retirement age), and benefit enjoyment rules (those who have accumulated 15 years of pension insurance contributions can enjoy retirement benefits). Knowledge Graph of Hidden Social Security Business The system is built using the Neo4j graph database, with the core being the causal relationships of social security business, including missing and corrected materials (e.g., missing payment certificate - need to correct payment certificate), process jumps (e.g., enrollment registration approved - jump to the payment stage), and discretionary standards (e.g., payment base is lower than the minimum standard - determined according to the minimum standard), etc. A total of 50,000 entity nodes and 80,000 causal relationship edges are constructed.

[0055] Step 5.2: Optimization of Government Semantics in Data Collection Results The server will contain the social security business rules database. Knowledge Graph of Implicit Social Security Business Injecting lightweight large language models into social security The causal relationships in the knowledge graph are converted into semantic vectors that the model can recognize, and a government semantic optimization function is set: in For feature fusion operations, an attention mechanism is used to achieve feature fusion of business rules, tacit knowledge, and preliminary collection results.

[0056] Model for preliminary data collection results Perform semantic correction and completion for government affairs: Semantic correction: It was found that there were 50 records in the preliminary collection results where the contribution base was lower than the minimum contribution standard in Hangzhou (3957). The model automatically corrected it to the minimum contribution standard of 3957 according to the social security business rules. Semantic completion: For the 200 records in the initial data collection that lacked the insurance type field, the model completed the insurance type based on the payment amount and social security business rules (e.g., a payment amount of 400 yuan was completed as medical insurance). Logical verification: Perform business logic verification on the payment records in the collection results and remove 10 invalid records whose payment time is earlier than the enrollment registration time.

[0057] After optimization, the optimized data collection results are obtained. A total of 81,940 valid data records of social security participants were collected.

[0058] Step 5.3: Double Verification of Data Collection Results The server builds a government data collection and verification model. Define the verification function: The verification weight coefficient is set according to the security level of social security data (social security data is sensitive government data, and compliance weight is applied). =0.6, accuracy weight =0.4).

[0059] The server validated the optimized data collection results Ropt from two dimensions: accuracy and compliance. Accuracy rating By matching the collected results with the original social security data, the accuracy rate of key information extraction was calculated to be 99.2%, the data matching degree was 98.8%, and the final accuracy score was... ; Compliance scoring The data collection results were verified through the government security management terminal, and all sensitive information was anonymized. The data collection operation complied with social security data security rules and regulations, and there were no compliance issues. The final compliance score was [not specified]. .

[0060] Substitute into the verification function to calculate the verification score: .

[0061] Step 5.4: Final collection results determination The server sets a verification scoring threshold for social security government data. ,because Therefore, the optimized collection results Confirmed as the final government data collection result A total of 81,940 valid data records of social security participants in Xihu District, Hangzhou City.

[0062] Step 5.5: Data Storage and Push The server, through its data storage and push module, transmits the final government data collection results. With the collection operation log The data is stored in a domestically produced database on the government data collection server and encrypted using the national cryptographic algorithm SM4. At the same time, the final collection results are pushed to the government business terminal of the collection user SB001 via the government extranet for use in social security business processing.

[0063] Extension Steps: Adaptive Migration in the Government Sector If the method of this invention needs to be migrated to the tax administration field for enterprise tax data collection, only the domain adaptive migration step needs to be executed, which is performed by the domain adaptive migration module. The specific process is as follows: The server obtains a subset of domain-specific labels for the tax field from the tax administration department. (Enterprise tax data annotation, tax terminology annotation) and tax knowledge graph (Tax entity, tax rules, causal relationship); Lightweight large language model of social security The parameters of the general government affairs layer are fixed, and only the parameters of the domain adaptation layer are fine-tuned. The gradient descent algorithm is used to minimize the fine-tuning loss function. (learning rate) =0.0001, iteration count 500); After fine-tuning, a lightweight large language model for the tax field was obtained. No need to retrain the model.

[0064] The core formulas and algorithm parameters used in each step of government data collection in this invention have been optimized specifically for government scenarios. The following is a unified explanation of the key formulas and parameters. All parameters can be dynamically adjusted according to the needs of different government sectors: Heterogeneous data mapping function: ,in The mapping weight matrix is ​​used (the dimensions are adjusted according to the number of fields in the data source; for the social security scenario, it is 10×8). For the bias term (with the same dimension as the number of normalized fields), gradient descent algorithm is used for training optimization, with a learning rate of... =0.001, number of iterations 1000, convergence condition is MSE<0.001; Data quality assessment function: The weighting coefficients satisfy Social security scenario =0.3、 =0.25、 =0.2、 =0.25, the tax scenario can be adjusted to =0.35、 =0.2、 =0.2、 =0.25, quality score threshold The default value is 85, but it can be increased to 90 for high-security government data. Model quantization function: Inverse quantization function The quantization precision is 8-bit integer, making it suitable for lightweight models in all government scenarios; Knowledge distillation loss function: Temperature coefficient T=4, batch size=32, learning rate =0.0001, training epoch=10, suitable for lightweight distillation of government affairs based on pre-trained large language models; Differential privacy Laplace noise: probability density function Total privacy budget The default setting is 1.0. The privacy budget for sensitive fields is allocated according to their importance, with the allocation ratio for core privacy fields (ID number, tax amount) being no less than 30%. Result verification function: The weighting coefficients satisfy Non-sensitive government data c Sensitive government data Verify the scoring threshold The default value is 90. Domain migration fine-tuning loss function: Learning rate =0.0001, weight decay wd=1e-6, number of iterations 500, convergence condition is loss value <0.01, applicable to model transfer in all government sectors.

[0065] Example of use The government data collection method and system based on the artificial intelligence large language model of the present invention have been piloted in three government fields: social security, taxation, and market supervision. The following are detailed examples of specific usage: Example 1: Data Collection of Social Security Participants Background of use A social security service department in a certain city needs to collect data on the identity, household registration, and payment records of more than 80,000 insured persons in its jurisdiction for social security participation status verification, payment anomaly reminders, and social security benefit calculation. The original data collection method involved manually connecting to cross-departmental databases and manually sorting heterogeneous data, with a collection cycle of 7 working days. Data cleaning was costly and there was a risk of sensitive information leakage.

[0066] System deployment and parameter configuration System Deployment: Deploy the government data collection server of this invention on the municipal government extranet, connect to the social security MySQL database, the civil affairs Oracle database, and the public security MongoDB database, and deploy government security management terminals and social security business terminals; Parameter configuration: Data quality assessment weighting coefficient =0.3、 =0.25、 =0.2、 =0.25, privacy budget The collected results validated the weighting coefficients. , The model is lightweighted by using a pruning threshold. Quantization precision is 8 bits.

[0067] Usage effect Data collection efficiency: The method of this invention takes only 1.5 working days from data source access to final data collection result output, which is 85% more efficient than the original method. The standardized conversion of heterogeneous data across departments is completed automatically without manual intervention. Data quality: The cleaned government data scored 97.05, with 98% data integrity, 95% format compliance, and 100% deduplication rate. This effectively solved problems such as duplication, format errors, and missing fields in the original data, and improved the accuracy of social security business accounting by 90%. Security and Compliance: All sensitive information has undergone differential privacy desensitization and masking processing, and the collection operation has achieved full-process auditing and traceability. No sensitive information leakage has occurred, which fully complies with the requirements of the Personal Information Protection Law. Computing cost: The computing power consumption of the lightweight large language model for government affairs is reduced by 75% compared with the original pre-trained model. It is deployed locally on the Kunpeng server on the government extranet, without the need for additional computing power resources, and the computing cost is reduced by 80%.

[0068] Example 2: Tax Data Collection for Enterprises in the Tax Sector Background of use A municipal tax department needs to collect value-added tax, corporate income tax and other tax data from more than 5,000 small and medium-sized enterprises in its jurisdiction for tax declaration verification, implementation of tax incentive policies and tax credit rating. The original data collection method used general web crawling tools, which could not adapt to the professional terminology and business logic of the tax field. The accuracy rate of key information extraction was only 65%, and cross-departmental data integration was difficult.

[0069] System deployment and parameter configuration System Deployment: Based on the government data collection server already deployed in the social security field, only the labeled subset and tax knowledge graph in the tax field are added, and the system is connected to the Golden Tax Phase III database and the market supervision enterprise information database. Parameter configuration: The weighting coefficient for data quality assessment is adjusted to... =0.35、 =0.2、 =0.2、 =0.25, privacy budget =1.2 (tax data has a higher sensitivity level), verification weighting coefficient of collection results 3. Model domain transfer fine-tuning of learning rate =0.0001.

[0070] Usage effect Domain migration efficiency: Migrating from the social security domain model to the tax domain model takes only 2 hours, without the need to retrain the model, reducing migration costs by 85%, and reducing time costs by 90% compared to redeveloping the data collection model using general methods; Information extraction accuracy: The lightweight tax big data model achieves a 99.1% accuracy rate in extracting key information from tax data, accurately identifying tax terminology and parsing the logic of tax returns, representing a 34.1% improvement over traditional web crawling methods. Data integration capabilities: Enables automatic integration of heterogeneous data from tax and market supervision departments, using the unified enterprise credit code as a unique identifier, and completes the correlation and matching of enterprise information and tax data with a matching accuracy rate of 98.5%, solving the original problem of data silos; Policy implementation efficiency: Based on the collected tax data, more than 1,200 SMEs that meet the tax incentive policy were quickly screened out. The policy implementation efficiency was improved by 95% compared with the original manual screening method, which greatly reduced the working cost of the tax authorities.

[0071] Example 3: Collection of Enterprise Operation Information in the Market Supervision Field Background of use A market supervision administration bureau needs to collect business information such as business registration, operating licenses, and administrative penalties from more than 100,000 enterprises in the city for use in enterprise credit information disclosure, identification of abnormal business lists, and market supervision and law enforcement. The original collection method used fixed templates, which could not adapt to the diverse formats of enterprise business information, and the extraction efficiency of key information from long text policy documents was low.

[0072] System deployment and parameter configuration System Deployment: Based on the existing government data collection server, access the market supervision and industrial and commercial database and the administrative penalty database to construct a labeled subset and implicit business knowledge graph in the field of market supervision; Parameter configuration: Data quality assessment weighting coefficient =0.25、 =0.3、 =0.2、 =0.25, privacy budget ε=0.9, data collection result verification weight coefficient 45. The model uses a sliding window algorithm for long text processing, with a window size of 512 tokens.

[0073] Usage effect Long text processing capability: The lightweight big language model for government affairs achieves a 98.8% efficiency in extracting key information from long texts such as market supervision policy documents and administrative penalty decisions, with an extraction speed 3 times faster than the original method, and can accurately identify core information in clause-style structures; Format adaptation capability: It can automatically adapt to and standardize the conversion of diverse formats of enterprise business information (two-dimensional tables, documents, image forms), with a format adaptation rate of 99%, which solves the format limitation problem of the original fixed template collection. Credit supervision efficiency: Based on the collected business information, more than 850 companies with abnormal operations were quickly identified. The efficiency of updating the list of companies with abnormal operations was improved by 90% compared with the original method, providing accurate data support for market supervision and law enforcement. System usability: Market supervision department staff only need to submit data collection requests on the government affairs terminal, and the system automatically completes the entire data collection process without the need for professional technical operations. The system has a 100% usage rate, which greatly reduces the operating threshold for staff.

[0074] Overall usage effect summary The government data collection method and system based on the artificial intelligence large language model of this invention have achieved efficient, compliant, and accurate collection of government data in pilot applications in the social security, taxation, and market supervision fields of a municipal government service management bureau. Compared with existing general data collection methods, the improvement in core indicators is as follows: The average cross-departmental government data collection cycle has been shortened from 7 working days to 1.5 working days, improving collection efficiency by over 85%; the quality score of cleaned government data averages over 96 points, with data integrity and format standardization both improved by over 90%; the accuracy rate of extracting key information in the government sector averages over 99%, an improvement of over 30% compared to general methods; the computing power consumption of lightweight large language models for government has been reduced by over 75%, and the computing power cost of local deployment has been reduced by over 80%; the cost of migrating models across government sectors has been reduced by over 85%, and the migration time has been shortened to within 2 hours; fine-grained access control, sensitive information desensitization, and operation auditing and traceability have been achieved throughout the entire government data collection process, with a sensitive information leakage rate of 0, fully complying with the legal and regulatory requirements for government data security management.

[0075] Meanwhile, the system of this invention has good scalability and ease of use, and can be quickly extended to other government affairs fields such as civil affairs, health, and transportation. Moreover, government staff can operate it without professional technical background, which greatly reduces the technical threshold for government data collection, provides high-quality government data support for the digital transformation of municipal government affairs, and promotes the dual improvement of government service efficiency and supervision capabilities.

[0076] The scope of protection of this invention is not limited to the specific embodiments and usage examples described above. Any equivalent substitutions or improvements made to the prior art based on the technical solutions of this invention fall within the scope of protection of this invention. Specifically, this includes: Improvements to the mapping model for heterogeneous government data include using deep learning algorithms to optimize mapping relationships or increasing the mapping adaptation for multimodal government data (images, audio). Improvements to lightweight compression algorithms for large language models include adopting more advanced lightweight technologies such as quantization-aware training and low-rank decomposition, or dynamically adjusting the compression ratio based on the computing power of different government servers. Improvements to differential privacy desensitization algorithms include replacing Laplace noise with Gaussian noise, or combining federated learning to achieve joint collection and desensitization of cross-departmental government data. Improvements to the government affairs semantic enhancement module include increasing the few-shot / zero-shot learning capability of the government affairs big language model, or combining reinforcement learning to achieve dynamic optimization of the collected results. Improvements to the system architecture include adopting a cloud-edge-device collaborative architecture to achieve distributed collection of government data, or adding blockchain technology to ensure that the collection operations are tamper-proof and traceable.

[0077] Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention shall be included within the scope of protection of this invention.

Claims

1. A method for collecting government data using an artificial intelligence large language model, wherein a data collection server is the executing entity, characterized in that, The method includes the following steps: S1. Access to government data sources and standardization of heterogeneous data: The server described in S1.1 obtains access information and data element information from cross-departmental government data sources; The server described in S1.2 constructs a heterogeneous government data mapping model based on government data metadata. Define heterogeneous data mapping functions For government raw data from different databases Standardization transformation is performed to obtain a standardized government data set. ,in ; The server described in S1.3 supports standardized government data sets. Standardize data formats and align fields to generate a standardized index table for government data. S2. Refined Cleaning and Quality Assessment of Government Data: The server described in S2.1 constructs a multi-dimensional cleansing rule base based on the characteristics of government data. ; The server instructs the cleaning rule base as described in S2.

2. Standardized government data sets Dirty data identification is performed to obtain a dirty data set. With the dataset to be cleaned ; The server described in S2.3 handles dirty data sets. The cleaning operation was performed, and missing fields were filled using a government affairs domain interpolation completion algorithm to obtain the cleaned government affairs dataset. ; The server described in S2.4 constructs a government data quality assessment model. Define the quality assessment function Calculate the cleaned government data set Quality rating ,in ; The server setting quality score threshold described in S2.5 ,like Then Determined as a qualified government data set ,like Then The sample is then sent back to step S2.2 for a second cleaning. S3, Lightweight Large Language Model Government Affairs Adaptation Training: The server described in S3.1 acquires the pre-trained large language model. A lightweight compression algorithm combining model pruning, quantization, and knowledge distillation is used to compress the data. Compression is performed to obtain a lightweight basic model. ; The server described in S3.2 constructs a labeled dataset for the government affairs sector. With government general knowledge graph ; The server described in S3.3 uses a small-sample fine-tuning algorithm to... and Inject lightweight base model Adaptive fine-tuning is performed in the field of government affairs to obtain a lightweight large language model for government affairs. ; S4. Government data collection under fine-grained compliance and security control: The server described in S4.1 constructs a fine-grained permission control model based on the government data permission system. Set the permission allocation function Based on the identity information of user U, assign corresponding government data collection permissions to them; The server described in S4.2 constructs a government sensitive information identification model. For qualified government data sets Sensitive information is identified to obtain a set of sensitive information. ; The server described in S4.3 uses a differential privacy de-identification algorithm to de-identify the sensitive information set. Data anonymization was performed by introducing Laplace noise to obtain a qualified de-identified government data set. ; The server described in S4.4 calls the government lightweight large language model. Based on the user's permission scope The task of collecting government data was carried out, and preliminary results of the data collection were obtained. ; The server described in S4.5 performs real-time auditing and logging of operations during the data collection process, generating a government data collection operation log. ; S5. Optimization and verification of data collection results with enhanced government semantics: The server described in S5.1 constructs a government affairs business rule base. Knowledge graphs of implicit business in government affairs ; The server described in S5.2 will and Injecting lightweight large language models into government affairs Set up a government affairs semantic optimization function Regarding the preliminary collection results The semantic correction and completion of government affairs data were performed to obtain optimized data collection results. ,in ; The server described in S5.3 constructs a government data collection and verification model. Set the verification function From the two dimensions of accuracy and compliance Verification was performed, and a verification score was obtained. ,in ; The server setting verification scoring threshold described in S5.4 ,like Then Confirmed as the final government data collection result ;like < Then Feedback is sent to the lightweight large language model for government affairs for secondary optimization; The server described in S5.5 will collect the final government data results. With the collection operation log The data is associated with the storage and then pushed to the corresponding government service terminals.

2. The method for collecting government data using an artificial intelligence large language model according to claim 1, characterized in that, In step S1.1, the access information includes database type, access address, and access permissions, and the data element information includes data structure, data type, field definition, and data encoding standard; In step S2.1, the cleaning rule base includes rules for cleaning duplicate data, rules for correcting format errors, and rules for completing missing fields; In step S3.2, the government affairs domain labeled dataset includes labeled information of government affairs long texts, government affairs terminology, and government affairs business data, and the government affairs general knowledge graph includes government affairs entities, entity attributes, and relationships between entities; In step S4.5, the operation log includes the user who collected the data, the time of collection, the range of data collected, and the collection operation behavior. In step S5.1, the implicit business knowledge graph of government affairs includes the causal relationship of approval process, material requirements, and discretionary standards.

3. The method for collecting government data using an artificial intelligence large language model according to claim 1, characterized in that, Its features are, In step S1.2, the government heterogeneous data mapping model The construction process is as follows: The server extracts features from government data metadata from different databases to obtain data element feature vectors. ,in For data structure features, For data type characteristics, Data encoding standard features; The server is configured with a standardized template for government data. This includes standardized data structures, standardized data types, standardized field definitions, and standardized data encoding; The server uses a cosine similarity algorithm to calculate the feature vector of data elements. With standardized template feature vector similarity Construct a heterogeneous data mapping table; The server sets a heterogeneous data mapping function based on a mapping table. ,in For the mapping weight matrix, As the bias term, it is applied using the gradient descent algorithm. and Training and optimization are conducted to transform raw government data into standardized data. In step S2.4, the government data quality assessment model Quality assessment function The calculation formula is: in: A data integrity score is assigned to characterize the completeness of government data fields. ; The data format standardization score represents the degree to which government data conforms to standardized templates. ; The data deduplication rate is scored to characterize the degree to which duplicate records are eliminated in government data. ; The data validity score represents the degree to which government data values ​​conform to the logic of government business operations. ; For the weighting coefficients, satisfying ,and It will be dynamically adjusted according to the needs of the government sector.

4. The method for collecting government data using an artificial intelligence large language model according to claim 1, characterized in that, In step S3.1, the execution process of the lightweight compression algorithm is as follows: Model pruning: The server prunes the pre-trained large language model Analyze the neural network weights and select those with weight values ​​less than the pruning threshold. Remove the connections to obtain the pruned model. ; Model quantization: The server will The 32-bit floating-point weights are quantized to 8-bit integers, and the quantization function is set. Inverse quantization function The quantized model is obtained. ; Knowledge distillation: The server will As a teacher model As a student model, the distillation loss function is defined. ,in For cross-entropy loss, Output for the teacher model, Output for the student model. Using the temperature coefficient, a lightweight base model is obtained by transferring knowledge from the teacher model to the student model through distillation training. .

5. The method for collecting government data using an artificial intelligence large language model according to claim 1, characterized in that, In step S4.3, the execution process of the differential privacy de-identification algorithm is as follows: The server determines the set of sensitive information. Privacy Budget Set a privacy budget allocation strategy and allocate funds accordingly. Assign to each sensitive field; The server sets the value of each sensitive field. Laplace noise was added to obtain the desensitized values. ,in Global sensitivity for query functions targeting sensitive fields. For noise that conforms to a Laplace distribution, its probability density function is: ; The server handles the anonymized values. Perform a reasonableness check, if If the data falls within the scope of government business logic, it is retained; otherwise, noise is added again for desensitization, ultimately resulting in a qualified desensitized government dataset. .

6. The method for collecting government data using an artificial intelligence large language model according to claim 1, characterized in that, In step S5.3, the government data collection and verification model verification function The calculation formula is: in: The accuracy of the data collection results is scored to characterize the degree of matching between the collected results and the original government data, as well as the accuracy of key information extraction. ; The compliance score for the collected results indicates the degree to which the collected results comply with government data security rules, business rules, and laws and regulations. ; To verify the weighting coefficients, the following conditions must be met: ,and The security level of government data will be dynamically adjusted according to the level of government data security. Sensitive government data The value should not be lower than 0.

6.

7. The method for collecting government data using an artificial intelligence large language model according to claim 1, characterized in that, The method also includes an adaptive migration step in the government sector: The server obtains a subset of domain-specific labels for the target government affairs domain. With domain knowledge graph ; The server will use a lightweight large language model for government affairs. The parameters of the general government affairs layer are fixed, and only the parameters of the domain adaptation layer are fine-tuned. The gradient descent algorithm is used to minimize the fine-tuning loss function. ; After the server completes fine-tuning, a lightweight large language model for the target government domain is obtained. This enables data collection and adaptation across government sectors.

8. A government data collection system based on an artificial intelligence large language model, characterized in that, The system includes a government data collection server, the server including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the government data collection method of the artificial intelligence large language model as described in any one of claims 1-7. The system also includes a communication connection with the government data collection server: Cross-departmental government data source terminal: used to store government raw data, provide government data source access information and data element information to the government data collection server, receive data collection requests from the server and feed back government raw data; Government security management terminal: used to configure government data security rules, fine-grained access control policies, and privacy budget parameters; receive sensitive information identification results and collection operation logs from government data collection servers; and realize security management and auditing of government data collection. Government business terminal: used to build government business rule base and government implicit business knowledge graph, receive the final collection results from government data collection server, manually verify the collection results and provide feedback on the verification opinions, and realize closed-loop optimization of collection results.

9. The government data collection system based on an artificial intelligence large language model according to claim 8, characterized in that, The government data collection server includes multiple functional modules, which work together to collect government data. The functional modules include: Heterogeneous data standardization module: This module is used to perform the steps of accessing government data sources and standardizing heterogeneous data, build a mapping model for heterogeneous government data, and realize the standardized transformation of raw government data. Government Data Cleaning and Assessment Module: Used to perform refined cleaning and quality assessment steps for government data, build a cleaning rule base and quality assessment model, and realize the cleaning and quality screening of dirty government data; Lightweight Model Training Module: This module is used to perform the government affairs adaptation training steps of the lightweight large language model. It compresses and fine-tunes the pre-trained large language model for the government affairs domain to obtain a lightweight large language model for government affairs. Compliance and security data collection module: used to execute government data collection steps under fine-grained compliance and security control, build access control model and sensitive information identification model, and realize de-identified data collection and operation auditing; Semantic Enhancement Optimization and Verification Module: This module is used to perform optimization and verification steps for the collection results of government semantic enhancement, injecting government business rules and implicit knowledge to achieve optimization and dual verification of the collection results. Domain Adaptive Migration Module: Used to execute adaptive migration steps for government domains, enabling rapid model adaptation across government domains; Data storage and push module: Used to associate and store the final collection results and operation logs, push the collection results to government business terminals, and receive manual verification feedback.

10. The government data collection system based on an artificial intelligence large language model according to claim 8, characterized in that, The cross-departmental government data source terminals include relational database terminals and non-relational database terminals. The relational database terminals include MySQL terminals and Oracle terminals, and the non-relational database terminals include MongoDB terminals. The government data collection server communicates with each terminal via the government extranet in encrypted form. The transmitted data is encrypted using the national cryptographic SM4 algorithm, and the identity authentication is performed using the national cryptographic SM2 algorithm.

Images