A Dynamic Interaction Method and System for Government Digital Humans Based on Multimodal Large Model

Abstract

This disclosure provides a method and system for dynamic interaction of digital human in government affairs based on a multimodal large-scale model. Applied to the field of intelligent government services, the method includes: acquiring policy announcement text, cleaning and structuring it to obtain a structured policy dataset; extracting old and new policy data, comparing differences and marking key change fields to generate policy change data; summarizing and extracting elements from the change data to form structured semantic fragments, and incrementally embedding them into a policy knowledge graph; generating question-answer pairs based on the updated knowledge graph, and performing self-supervised fine-tuning training on the multimodal large-scale model; and having the model generate and provide government responses when the user inputs multimodal data. This solution enables the multimodal large-scale model to continuously maintain the latest policy knowledge; it allows the model to generate accurate and context-consistent government responses while understanding multimodal inputs such as text, voice, and images, improving the timeliness, accuracy, and interactive experience of policy interpretation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of government affair intelligent service, in particular to a government digital human dynamic interaction method and system based on a multi-modal large model. BACKGROUND

[0002] With the deep integration of information technology and government governance, the public's demand for efficient, intelligent, and personalized government services is growing. Traditional government models are difficult to meet the service requirements of the digital era due to problems such as information silos, cumbersome processes, and delayed responses.

[0003] Current mainstream government technology solutions mainly use static knowledge bases to store unstructured policy files in policy processing, rely on manual input and OCR to extract text information, and use rule engines or early machine learning models to achieve shallow semantic matching. Policy updates need to be manually synchronized. At the service interaction level, government APPs and websites are mainly supported, with basic online handling and keyword search. Some systems integrate NLP technology to build intelligent customer service, and cutting-edge solutions attempt to introduce large model voice interaction and localized digital humans. Cross-departmental collaboration relies on government collaboration platforms to realize document circulation.

[0004] The policy explanations of the current mainstream government technology solutions are mostly pure text, and complex clauses (such as application processes) are difficult to quickly understand. Based on keyword matching or early NLP models, only surface information of policy text can be captured, and logical associations between clauses cannot be analyzed. The system may recommend policies that contain keywords but are not suitable for the industry, resulting in ineffective information. SUMMARY

[0005] To solve the problems of the prior art, the present disclosure provides a government digital human dynamic interaction method and system based on a multi-modal large model. The present disclosure solves the problem that the policy explanations of the current mainstream government technology solutions are mostly pure text, and complex clauses (such as application processes) are difficult to quickly understand. Based on keyword matching or early NLP models, only surface information of policy text can be captured, and logical associations between clauses cannot be analyzed. The system may recommend policies that contain keywords but are not suitable for the industry, resulting in ineffective information.

[0006] According to a first aspect of the present disclosure, a government digital human dynamic interaction method based on a multi-modal large model is provided, comprising: obtaining policy announcement text data, cleaning and structuring the policy announcement text data to obtain a structured policy data set;

[0007] According to the structured policy data set, old policy data and new policy data are extracted, and the old policy data and the new policy data are compared and analyzed to obtain policy change data marked with key change fields;

[0008] The policy change data is subjected to summary generation and policy element extraction to obtain a structured semantic segment containing policy elements and semantic relationships thereof;

[0009] The structured semantic segment is incrementally embedded into a preset policy knowledge graph, and a question and answer pair sample is generated based on the updated preset policy knowledge graph, and the preset multi-modal large model is subjected to self-supervised fine-tuning training according to the question and answer pair sample;

[0010] If multi-modal input data input by a user is received, the multi-modal input data is input into the preset multi-modal large model subjected to self-supervised fine-tuning training to obtain government affair response content, and the government affair response content is sent to the user.

[0011] According to a second aspect of the present disclosure, a government affair digital human dynamic interaction system based on a multi-modal large model is provided for executing the method as described in the first aspect, comprising: a data preprocessing module for obtaining policy announcement text data, cleaning and structuring the policy announcement text data to obtain a structured policy data set;

[0012] A difference analysis module is configured to extract old policy data and new policy data from the structured policy data set, and perform difference comparison analysis on the old policy data and the new policy data to obtain policy change data marked with a key change field.

[0013] A semantic processing module is configured to perform summary generation and policy element extraction on the policy change data to obtain a structured semantic segment containing policy elements and semantic relationships thereof.

[0014] A training module is configured to incrementally embed the structured semantic segment into a preset policy knowledge graph, and generate a question and answer pair sample based on the updated preset policy knowledge graph, and perform self-supervised fine-tuning training on the preset multi-modal large model according to the question and answer pair sample.

[0015] An answer generation module is configured to, if multi-modal input data input by a user is received, input the multi-modal input data into the preset multi-modal large model subjected to self-supervised fine-tuning training to obtain government affair response content, and send the government affair response content to the user.

[0016] According to a third aspect of the present disclosure, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method as described above.

[0017] In the multi-modal large model-based government digital human dynamic interaction method and system provided above, the feedback feature indicators reflecting user experience and needs are generated. Based on this, the parameter weights of the multi-modal large model are dynamically adjusted to realize adaptive optimization of the model, significantly improve the response accuracy and personalized service ability of the government digital human, and thus enhance user satisfaction and long-term stability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor.

[0019] Figure 1 A multi-modal large model-based government digital human dynamic interaction method flowchart is shown according to an embodiment of the present disclosure;

[0020] Figure 2 A multi-modal large model-based government digital human dynamic interaction method flowchart is shown according to an embodiment of the present disclosure;

[0021] Figure 3 A multi-modal large model-based government digital human dynamic interaction system schematic block diagram is shown according to an embodiment of the present disclosure;

[0022] Figure 4 A block diagram of an exemplary electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0024] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor indicate their logical order. It should also be understood that in the embodiments of the present disclosure, "multiple" can mean two or more, and "at least one" can mean one, two or more. It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, without explicit limitation or in the context of the preceding and following, it can be understood as one or more. In addition, the term "and / or" in the present disclosure only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects. It should also be understood that the description of various embodiments of the present disclosure emphasizes the differences between various embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, they will not be repeated.

[0025] At the same time, it should be understood that in order to facilitate the description, the size of each part shown in the drawings is not drawn in accordance with the actual proportional relationship. The following description of at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present disclosure and its application or use. The technology, methods and devices known to those skilled in the related art can not be discussed in detail, but in appropriate cases, the technology, methods and devices should be considered as part of the specification. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0026] In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0027] Figure 1 A flowchart of a government digital human dynamic interaction method based on a multi-modal large model is provided for the embodiments of the present disclosure. The method of the embodiments of the present disclosure aims to realize accurate detection of large targets and small targets of pictures.

[0028] In S101, policy announcement text data is obtained, and the policy announcement text data is cleaned and structured to obtain a structured policy data set.

[0029] The scheme is executed by a government digital person. The government digital person can be a virtual intelligent agent based on artificial intelligence technology (such as natural language processing, multi-modal large model, speech synthesis, virtual image rendering, etc.) for online provision of government-related consulting, answering, and service guidance.

[0030] The policy announcement text data can be original text data related to government affairs, policies, regulations, rules, guidelines, etc. published by government departments or official agencies.

[0031] The structured policy data set can be a policy information collection with clear field structure and computer directly processable after the original policy announcement text data is processed by cleaning, sentence segmentation, entity recognition, and element extraction.

[0032] The policy announcement text data in various formats such as text, PDF, HTML, and Word is scraped or downloaded from authoritative sources such as government department official websites, government affairs disclosure platforms, and policy and regulation databases, and is automatically collected regularly by using web crawling technology combined with data interface calling technology. During the collection process, different types of files are parsed by file format parsing technology. PDF files are converted into editable text by optical character recognition (OCR) technology, HTML web pages are parsed to extract the content of the text area using DOM structure, and Word documents are directly read for internal text data stream. During the cleaning of the obtained policy announcement text data, first, the text preprocessing technology is used to remove HTML tags, script codes, invalid spaces, and line breaks, and the regular expression matching and replacement technology is used to standardize the date, number, and unit format; then the language detection and coding unification technology is used to convert the text into UTF-8 encoding, and the segmentation and sentence segmentation algorithm is used to cut the content according to the natural paragraph and sentence boundary; then, the stop word filtering, special symbol removal, and text deduplication algorithm are used to eliminate meaningless words and repeated clauses, ensuring the neatness and consistency of the text data. In the structured processing stage, first, the named entity recognition (NER) technology is used to identify the policy name, issuing agency, issue date, policy clause number, and related policy entities (such as industry, region, and applicable population), and the rule matching algorithm and context semantic analysis technology are used to accurately locate and disambiguate the entities; then the dependency syntax analysis and semantic role labeling technology is used to extract the action, condition, and object of the policy clause, and establish the logical and semantic relationship between them; finally, the data structure mapping technology is used to map the above extraction results to the pre-defined policy data model, forming a structured policy data set with clear fields and explicit relationships.

[0033] S102, extracting old policy data and new policy data from the structured policy dataset, and performing difference comparison analysis on the old policy data and the new policy data to obtain policy change data marked with key change fields.

[0034] The old policy data can refer to a set of structured data of historical version policy clauses selected from the structured policy dataset, earlier in time than the current policy version, and under the same policy topic or the same policy clause number. The data usually includes fields such as issuing authority, clause number, clause content, applicable conditions, implementation time, and involved objects.

[0035] The new policy data can refer to a set of structured data of the current effective version policy clauses selected from the structured policy dataset, the latest in time, and corresponding to the old policy data. The data is comparable to the old policy data in terms of policy topic or clause number, and contains the latest clause content, applicable conditions, implementation time, and other information.

[0036] The key change field can refer to a field that is detected to have substantial differences or affect the scope of policy application, policy conditions, rights and obligations when comparing the old policy data and the new policy data field by field. For example, newly added or deleted legal obligation descriptions in the clause content field, adjusted industry or group range in the applicable object field, changes in the implementation time field, and modifications in the numerical indicator field.

[0037] The policy change data can refer to a set of structured data containing field name, old version value, new version value, difference type (add, delete, modify, etc.), and difference content formed after the old policy data and the new policy data are compared field by field and the key change fields are marked in the difference comparison analysis.

[0038] After obtaining the structured policy dataset, first, sort the dataset in time series according to the timestamp field, version number field or publication date field, so as to arrange the different version data under the same policy topic or the same clause number in chronological order. Then, extract the field set that can uniquely identify each policy record as the unique identifier for each policy record. The identifier can include but is not limited to structured fields such as clause number, policy number, chapter number and version number. In order to improve the robustness of matching, for the case of naming inconsistency or field missing, a text hash fingerprint (such as SHA-256 hash or SimHash method) of the policy clause full text is also generated, which can be used as a semantic consistency reference value. In the actual matching process, first, perform primary key matching based on the clause number or policy number, and aggregate different version records that may belong to the same policy into a group; if the primary key matching is ambiguous, calculate the similarity (such as Hamming distance or Jaccard similarity) of the text hash fingerprints in the aggregation result to determine whether the text body is the same, so as to finally determine the corresponding historical version and current version of each group. After confirming the version group, mark the earlier version as old policy data and the latest version as new policy data to prepare for subsequent difference comparison.

[0039] After extraction, perform difference comparison analysis on the corresponding fields of the old policy data and the new policy data. The analysis can be divided into two types of processing: structured field comparison: for numerical, date and enumeration type fields, use accurate matching algorithm or numerical difference calculation to determine whether the field value has changed, and mark the difference type as "modified", "added" or "deleted". Unstructured field comparison: for text type fields such as policy clause text and applicable condition description, first use word segmentation and part-of-speech tagging technology to decompose the text into semantic units, and then use semantic vectorization representation technology (such as BERT or RoBERTa word vector) to calculate the cosine similarity or semantic matching score between the old and new clauses. If the similarity is lower than the preset threshold, it is determined that there is a semantic difference in the field, and further difference highlighting algorithm (such as the method of combining longest common subsequence LCS and difference marking) is used to extract the specific change content.

[0040] In the difference identification stage, the field name of each changed field is matched with the semantic label to determine whether it belongs to the preset high-impact category, such as the scope of policy application (applicable population, applicable region), policy conditions (eligibility conditions, application threshold), rights and obligations (right to grant, obligation to require), numerical standards (subsidy amount, tax rate, index limit), implementation time (effective date, deadline), etc. For text-type changed fields, keyword matching and semantic similarity analysis are also needed to confirm whether the change is within the semantic domain of the above high-impact categories; for numerical-type changed fields, the change ratio (absolute change and relative change) is calculated and compared with the set numerical sensitivity threshold, and if it exceeds the threshold, it is determined as a key change. In addition, the system will comprehensively consider the hierarchical position of the field in the policy structure (such as first-level clause, second-level clause) and the impact range weight to weight the importance of the changed field, and finally filter out the fields with scores higher than the preset threshold and mark them as key changed fields. Finally, these key changed fields and their field values in the old and new versions, difference types, difference contents, difference positions, etc. are structured and stored to form policy change data.

[0041] S103, abstract generation and policy element extraction are performed on the policy change data to obtain structured semantic fragments containing policy elements and their semantic relationships.

[0042] Policy elements can be the smallest semantic units in government or policy texts that express the core content of the policy, including policy subjects (executing or applicable agencies, personnel), policy objects (affected objects or matters), applicable conditions (requirements needed to be met for taking effect), rights and obligations (granted rights, obligations to be fulfilled), execution standards (specific numerical values such as subsidy amount, index value, tax rate), time limit information (effective date, deadline, transition period arrangement), and exception clauses, etc.

[0043] Semantic relationships can refer to the logical and dependent associations between different policy elements in the context of policy texts, used to describe the action mode, constraint condition, and dependency structure between elements. Common semantic relationship types include but are not limited to condition-result relationship (such as "if condition A is met, then measure B is executed"), subject-action relationship (such as "government department X is responsible for executing measure Y"), numerical-constraint relationship (such as "the subsidy amount shall not exceed Z"), time-effect relationship (such as "the policy takes effect on date T"), exception-application relationship (such as "unless condition E is met, it does not apply"), etc.

[0044] A structured semantic fragment can be a structured information unit that organizes policy elements and their corresponding semantic relations according to a predefined data structure, each of which completely describes the core content of a certain clause or a certain policy logic in the policy text. For example, a structured semantic fragment can include "policy subject = financial bureau", "policy object = small and medium-sized enterprises", "applicable conditions = annual sales not more than 5 million yuan", "execution standard = subsidy ratio 20%", "effective date = January 1, 2025", and the semantic relations between them, such as condition-result, numerical-constraint, etc.

[0045] The input policy change data text can be processed. First, a rule-driven punctuation and sentence breaking algorithm is used in combination with a semantic similarity-based paragraph division method to divide the policy text according to natural paragraphs and clause numbers and other structures, ensuring that each text segment is semantically independent and retains the context boundary. Subsequently, regular expression matching technology and character-level anomaly detection methods are used to remove irrelevant markers, HTML tags, repeated spaces, and scan and identify error characters in the text, improving the quality of the text. Key content positioning is performed on the preprocessed text. The importance weight of the words in the text is calculated based on the self-attention mechanism of the Transformer architecture, and the TextRank keyword extraction algorithm is used to identify the core keywords and key phrases in the policy text. Subsequently, the BERT combined with the bidirectional long short-term memory network and the conditional random field joint model are used to perform sequence labeling on the policy text to mark the candidate sentences for the abstract. For the candidate sentence set, the cosine similarity deduplication algorithm is used to remove redundant sentences to form a candidate set of abstracts with high coverage and low information redundancy. Finally, the candidate abstract is input into the pre-training generation model based on the sequence-to-sequence architecture (such as BART or T5), and the Beam Search decoding strategy is used to generate a policy abstract text that is coherent in content and contains policy-specific terminology. The BERT-Span model is used for named entity recognition to accurately extract key policy elements such as policy subjects, policy objects, applicable conditions, rights and obligations, execution standards, and effective times. For numerical fields, time fields, and geographic range fields with high standardization, regular expression template matching technology is used for supplementary extraction to ensure complete coverage of key data information in the policy. The dependency syntax parsing technology based on the Universal Dependencies framework is used to parse the syntactic structure of the text and mark the dependency relationships of predicates, objects, and adverbials in the sentence. The BERT and multi-label classification technology are used to identify and extract the diverse semantic relationships between policy elements, including condition-result relationships, subject-action relationships, numerical-constraint relationships, and time-effect relationships, to construct a semantic relationship network between elements. The extracted policy elements and their corresponding semantic relationships are constructed into a structured data format, such as triples (subject-predicate-object) or nested JSON objects, to realize the standardized expression of policy content. For the generated structured semantic segments, a semantic conflict detection algorithm is applied to check the logical consistency between different segments and avoid conflicts in time or value. The training process of the pre-training generation model usually includes two stages of large-scale unsupervised pre-training and supervised fine-tuning. First, a large amount of text corpus is used to perform unsupervised pre-training of the model through language modeling tasks such as mask language model or conditional text generation, so that the model learns the context representation and generation ability of the language.Next, on downstream specific tasks such as policy text summary generation, a supervised fine-tuning is performed using a high-quality artificially annotated policy summary dataset to optimize the model's parameters to improve the accuracy and professionalism of the summary. During training, a sequence-to-sequence architecture is used, with the original text as input and the corresponding summary text as output. The error between the predicted and true summaries is calculated using the cross-entropy loss function, and the model weights are continuously adjusted through the backpropagation algorithm. In addition, decoding strategies such as BeamSearch are used to improve the coherence and diversity of the generated text, ensuring that the final output conforms to the professional terminology and expression specifications in the policy field.

[0046] S104, incrementally embedding the structured semantic segment into the preset policy knowledge graph, and generating question and answer pair samples based on the updated preset policy knowledge graph, and performing self-supervised fine-tuning training on the preset multi-modal large model according to the question and answer pair samples.

[0047] The preset policy knowledge graph can refer to a knowledge representation structure that has been established or initialized before system deployment, used to store and organize policy-related information. It usually exists in the form of a graph database, composed of nodes (representing policy elements, such as "applicable object", "subsidy ratio", "effective time") and edges (representing semantic relationships between elements, such as "applicable to", "effective on", "constraint condition"). This knowledge graph not only contains data of historical and current policies, but also may include information such as the association between policies, policy execution subjects, and relevant regulations.

[0048] The question and answer pair sample can refer to a pair of data composed of a question and an answer, used to train or fine-tune a question and answer model. It can be automatically generated based on the updated preset policy knowledge graph, such as: question: "What is the R&D expenditure subsidy ratio for small and medium-sized enterprises in 2025?" Answer: "From January 1, 2025, the R&D expenditure subsidy ratio for small and medium-sized enterprises is 20%."

[0049] The preset multi-modal large model can refer to a large-scale deep learning model pre-selected or deployed in the system, which can process multiple modal inputs (such as text, speech, images, etc.) and generate corresponding outputs. It usually has cross-modal alignment capability (such as converting chart content into text understanding, converting voice questions into semantic representation) and cross-modal reasoning capability (such as combining policy text and flowchart to answer user questions). This multi-modal large model has been pre-trained on large-scale general data and has certain language understanding and generation capabilities. The self-supervised fine-tuning step is to use the question and answer pair samples generated from the preset policy knowledge graph to make the model perform better in the policy field, especially to improve its accuracy and context understanding capability in cross-modal scenarios.

[0050] After generating the structured semantic fragments, first, the incremental embedding process is performed, which is as follows: for each policy element node and semantic relationship edge in the structured semantic fragments, a unique identifier mapping mechanism (such as a globally unique ID generated based on the combination of element type and text hash value) is used to perform node lookup and duplication judgment in the preset policy knowledge graph; for nodes that do not exist in the graph, a graph database writing interface (such as the CREATE statement of Neo4j or the RDF triple INSERT DATA statement) is called to write them as new nodes; for nodes that already exist but have changed attribute values, an attribute difference detection algorithm (based on field value hash comparison or vector cosine similarity calculation) is used to update the node attributes; for new or changed semantic relationships, they are inserted into the graph relationship library in the form of triples (subject-predicate-object), and relationship conflict detection (based on relationship type constraint rules to determine whether the new relationship conflicts with existing relationships) is performed to ensure the consistency and coherence of the updated knowledge graph data. Based on the updated preset policy knowledge graph, question and answer pair samples are generated, which is as follows: first, a graph pattern matching algorithm (such as Cypher query pattern matching or SPARQL template query) is used to filter out policy element combinations with complete context; then, based on the question generation template library (predefined multiple natural language questions for policy element combinations, such as "What is the subsidy ratio of a certain subject under certain time conditions?"), the filtered element combinations are filled into the templates to generate structured natural language questions; and finally, the corresponding policy element values and semantic relationships are combined to generate answer texts. In order to improve the diversity and naturalness of the generated question and answer pairs, the questions generated from the templates are rewritten (using a Transformer-based text rewriting model or a regularized synonym replacement technique), and the answer texts are processed for anaphora resolution (using a BERT-based anaphora resolution model to analyze pronouns and replace them with explicit entities) to ensure the availability and clarity of the question and answer pair samples in the multi-round dialogue scenario.The generated question and answer pair samples are used to perform self-supervised fine-tuning training on the preset multi-modal large model. The method is as follows: first, the question and answer pair samples are converted into a multi-modal input form acceptable to the model (if the question contains policy text, tables or diagrams, the text features, table structure features and image features are extracted respectively), and a cross-modal encoder-decoder structure (such as a CLIP text encoder combined with a visual encoder, or a BLIP architecture) is used to map the multi-modal features to a unified semantic space; then, the question is taken as the input and the answer is taken as the target output to construct a training task, a self-supervised loss function (such as cross-entropy loss, contrastive learning loss or multi-task loss) is calculated, and the model parameters are updated based on the gradient descent optimization algorithm (AdamW or LAMB); in the training process, a dynamic learning rate adjustment strategy (such as Cosine Annealing or Warmup+Decay) is used to improve the convergence speed and accuracy until the question and answer accuracy and generation fluency of the model on the validation set reach the preset threshold.

[0051] The preset policy knowledge graph construction steps include:

[0052] First, the multi-source heterogeneous policy data is preprocessed, including text normalization, sentence segmentation, and structured analysis; then, a pre-trained BERT-CRF model is used for named entity recognition, which is pre-trained on general corpus and fine-tuned on policy-specific data to accurately extract key elements such as policy entities and clauses; based on dependency syntax analysis and multi-label classification model, semantic relationships between entities are extracted to form entity-relation pairs; through semantic similarity calculation and rule-based disambiguation, entity unification is achieved, and a triple structure containing attribute information is constructed and stored in a Neo4j graph database; a time model is designed to encode policy timeliness, supporting version tracking and dynamic updating; finally, through automatic crawling and NLP analysis, the data timeliness is maintained, and combined with multi-round manual verification and automatic detection mechanism, the accuracy and reliability of the knowledge graph are ensured.

[0053] The construction process of the preset multi-modal large model includes:

[0054] A multi-modal corpus resource covering the field of government business is collected, including government text corpus (such as policy documents, laws and regulations, government announcements), structured data corpus (such as government statistical tables, business processing flow data), and visual corpus (such as policy interpretation diagrams, flowcharts, schematics, table screenshots, etc.), and the collected multi-modal corpus is subjected to format unification and annotation processing. The text corpus is subjected to natural language cleaning and word segmentation technology to remove redundant symbols, segment words, and perform named entity annotation. The table data is subjected to table structure parsing algorithm (based on cell coordinate mapping and table header inference rules) to extract field names and field values. The image data is subjected to image preprocessing and OCR text extraction technology (such as an OCR model based on a convolutional neural network) to identify the text information and layout structure therein. Subsequently, a multi-modal pre-training corpus alignment set is constructed. The method is as follows: the text and the corresponding table and image are mapped by a multi-modal alignment algorithm (such as a similarity matching method based on a cross-modal attention mechanism) to generate one-to-one or one-to-many multi-modal samples. Next, a cross-modal encoding-decoding framework is built. The text modality encoder adopts a Transformer structure (such as BERT or RoBERTa) to extract context semantic features, the visual modality encoder adopts a combination structure of a convolutional neural network (CNN) and a visual Transformer (ViT) to extract image global and local features, and the table modality encoder models the cell semantics and row-column relationship based on a table Transformer structure, and maps the features of different modalities to a unified high-dimensional semantic space through a multi-modal fusion layer (such as a feature interaction module based on self-attention). The decoding end adopts a conditional generation Transformer to decode the fused features into natural language output or structured prediction results. In order to enhance the understanding ability of the model for the policy field knowledge, a domain knowledge constraint mechanism is introduced in the multi-modal pre-training stage, that is, the attention weight distribution of the model is guided by the entities, attributes and relationships in the government knowledge graph, and semantic enhancement is realized through a knowledge comparison learning loss function (aligning the entity relationship vectors predicted by the model with the vector space of the knowledge graph). In the training process, a multi-task learning strategy is adopted to simultaneously optimize the cross-modal matching task (judging whether the text and the image or the table match), the cross-modal generation task (generating a policy text summary according to the image or the table), and the cross-modal question answering task (generating a policy reply according to the multi-modal input), and the AdamW optimizer is used in combination with the learning rate warm-up and cosine annealing decay strategy to accelerate convergence and prevent overfitting. After training, a preset multi-modal large model with government multi-modal understanding and generation capability is obtained, which provides a basic model support for subsequent self-supervised fine-tuning training.

[0055] On the basis of the above technical solutions, before incrementally embedding the structured semantic segment into the preset policy knowledge graph, the method further comprises:

[0056] extracting a policy entity name and a corresponding first effective time interval in the structured semantic segment, locating a same-name entity node in the preset policy knowledge graph according to the policy entity name, reading a second effective time interval of the same-name entity node;

[0057] if there is an overlapping time interval between the first effective time interval and the second effective time interval, determining a first policy clause content corresponding to the structured semantic segment, and determining a second policy clause content corresponding to the preset policy knowledge graph;

[0058] if there is no semantic difference between the first policy clause content and the second policy clause content, generating an allowed insertion instruction according to the structured semantic segment;

[0059] Correspondingly, incrementally embedding the structured semantic segment into the preset policy knowledge graph, comprising:

[0060] If the allowed insertion instruction is identified, the structured semantic segment is incrementally embedded into the preset policy knowledge graph.

[0061] In the scheme, the policy entity name can refer to a policy-related object name in the policy text that has a clear reference meaning and can be used as a unique node identifier in the policy knowledge graph, such as the name of a specific subsidy policy or a specific clause title. It is a key retrieval field that connects the structured semantic segment and the entity node in the knowledge graph.

[0062] The first effective time interval can refer to the effective start and end time period of the policy clause first specified in the structured semantic segment, for example, “from January 1, 2025 to December 31, 2025”.

[0063] The same-name entity node can refer to a node object in the preset policy knowledge graph whose entity name completely matches the policy entity name. This node stores all versioned policy clauses and their metadata related to the entity.

[0064] The second effective time interval can refer to the effective start and end time range recorded by a certain policy clause in the same-name entity node in the policy knowledge graph.

[0065] The overlapping time interval can refer to the part of the first effective time interval and the second effective time interval that has an intersection on the time axis, for example, the first interval is 2025 / 01 / 01-2025 / 12 / 31, and the second interval is 2025 / 06 / 01-2026 / 05 / 31. 2025 / 06 / 01-2025 / 12 / 31 is the overlapping time interval.

[0066] The content of the first policy clause can refer to the body of the policy clause corresponding to the first effective time interval in the structured semantic fragment, which has been cleaned and standardized, including the complete legal statements, numerical provisions, applicable conditions, etc. of the clause.

[0067] The content of the second policy clause can refer to the existing policy clause body corresponding to the second effective time interval within the same entity node in the policy knowledge graph, and its source is the clause data stored in the knowledge graph history.

[0068] The allow-insert instruction can refer to the control signal generated by the system when the comparison result shows that there is no semantic difference between the content of the first policy clause and the content of the second policy clause, indicating that the new structured semantic fragment can be safely inserted into the policy knowledge graph without causing conflicts or redundancies.

[0069] For the incremental embedding operation of the structured semantic fragment, first perform named entity recognition on the structured semantic fragment to accurately extract the policy entity name from the text. The NER process is based on the BERT-BiLSTM-CRF joint model fine-tuned with the corpus exclusive to the policy and regulation field. The specific steps are as follows: First, use the BERT encoding layer to generate context-related vector representations for the input character-level sequence, then capture the context dependence information through the BiLSTM bidirectional recurrent neural network, and finally use the CRF conditional random field layer to perform global annotation decoding on the entire sentence to ensure the global optimality of entity boundary recognition. This method can maintain a high accuracy rate when dealing with special forms such as abbreviations, ellipsis, or numerical numbers in the policy name.

[0070] At the same time, for the extraction of time elements, introduce a method that combines a regularized time parsing algorithm and semantic dependency analysis in the structured semantic fragment. First, use regular expressions to quickly match explicit date patterns such as "YYYY-MM-DD" and "from... to...", and then use time modifier analysis based on the dependency syntax tree to capture implicit expressions (such as "this quarter" and "within five years"), and convert them into the standard ISO 8601 time interval through rule mapping as the first effective time interval.

[0071] After extracting the policy entity name and the first effective time interval, the system accesses the preset policy knowledge graph, which is built based on RDF (Resource Description Framework) + OWL (Web Ontology Language) and stores policy entities, clause versions, time effectiveness information and associated relationships. To find the matching node in the graph, first perform exact string matching; if not hit, call the Word2Vec-based word vector similarity retrieval and edit distance calculation to obtain a set of node candidates close to the target policy entity name in semantics or character form, and combine the semantic word forest and policy and regulation naming specifications for weighted scoring to finally lock the same-named entity node.

[0072] After obtaining the same-named entity node, read the second effective time interval stored in the node, which has also been standardized to ISO 8601 format in the knowledge graph construction stage. Then, call the time interval overlap detection algorithm for comparison, which is based on the interval endpoint comparison principle: if start1<= end2 and start2<= end1, then it is determined that the intervals have an intersection, thus confirming the overlapping part of the two policy versions in terms of time effectiveness.

[0073] If the overlapping time interval is detected, enter the semantic comparison stage of the clause content. First, use the index structure of the knowledge graph (such as ElasticSearch + Neo4j hybrid index) to quickly locate the first policy clause content associated with the first effective time interval and the second policy clause content associated with the second effective time interval. Then, input these two pieces of clause content into the RoBERTa-wwm-ext-large model fine-tuned on legal and regulatory corpus to generate 768-dimensional semantic vector representation; use the cosine similarity calculation formula to calculate the closeness of the two clauses in the semantic space. If the similarity is higher than the preset threshold (e.g. 0.92), it is determined that there is no semantic difference between them, meaning that the clause content of the new semantic fragment is consistent with the clause semantics of the same time interval in the existing knowledge graph.

[0074] Under the condition of determining semantic consistency, the system generates an allowed insertion instruction based on the structured semantic fragment, which records the target node ID, insertion position identifier (such as append, merge), version number, update timestamp, and execution strategy parameters (such as overwrite or append) in JSON format. After receiving the allowed insertion instruction, the system calls the RDF triple batch write engine to convert the structured semantic fragment into a set of triples that comply with the RDF Schema and ontology definition. To avoid damaging the historical version chain of the knowledge graph, an incremental writing strategy can be used, that is, a version sub-node is added under the original node, and a version_of and valid_time relationship edge is established, while updating the latest_version index of the node to point to the new version, thereby realizing incremental embedding, preserving history and updating the present situation.

[0075] In this scheme, new information can be quickly integrated while ensuring the integrity of historical versions, avoiding repeated redundant data and semantic conflicts, thereby improving the timeliness, accuracy, and maintainability of the knowledge graph.

[0076] On the basis of the above technical scheme, optionally, after reading the second effective time interval of the same-name entity node, the method further comprises:

[0077] If there is no overlapping time interval between the first effective time interval and the second effective time interval, an allowed insertion instruction is generated according to the structured semantic fragment;

[0078] Correspondingly, incrementally embedding the structured semantic fragment into the preset policy knowledge graph comprises:

[0079] If the allowed insertion instruction is identified, the structured semantic fragment is incrementally embedded into the preset policy knowledge graph.

[0080] In this scheme, a logical judgment algorithm based on time interval overlap determination can be used to determine whether there is an intersection (overlapping time interval) between the two. If the determination result shows that there is no overlapping time interval between the first effective time interval and the second effective time interval, an "allowed insertion instruction" is generated according to the business rules, which is automatically triggered by the rule engine according to the time overlap-free condition, indicating that the system can safely insert the structured semantic fragment to update the knowledge graph, avoiding data confusion caused by time interval conflicts. Subsequently, the system receives the "allowed insertion instruction" and starts the incremental update mechanism, and writes the structured semantic fragment information into the preset policy knowledge graph in the form of new nodes and edges according to the data model of the graph database (such as based on RDF triples or property graph model), ensuring that the new information is correctly associated with the original knowledge structure.

[0081] In the scheme, the conflict and repetition of policy information are effectively avoided, the safety incremental embedding of structured semantic fragments is ensured, and the accuracy and timeliness of the policy knowledge graph are improved.

[0082] On the basis of the above technical scheme, optionally, after recognizing the permission to insert the instruction, the method further includes:

[0083] Obtaining historical government Q&A records, performing reverse retrieval on the historical government Q&A records based on the policy entity name corresponding to the structured semantic fragment, and locating the historical answer records containing the policy entity name;

[0084] Extracting the reply content of the historical answer records, performing semantic difference detection on the reply content and the latest policy provisions corresponding to the preset policy knowledge graph after incremental embedding, and obtaining a difference detection result;

[0085] If the difference detection result indicates that there is a semantic conflict, generating a policy update reminder corresponding to the historical answer record based on the difference detection result and the corresponding latest policy provisions in the preset policy knowledge graph after incremental embedding, and pushing the policy update reminder to the user according to a preset pushing strategy.

[0086] In the scheme, the historical government Q&A records can refer to all past interactive records between the system and the user in the past government digital people (or similar government service systems), including complete data sets of user questions, system generated answer content, interaction timestamps, and the like.

[0087] The historical answer records can refer to the specific answer entries containing the specific policy entity name related content in the historical government Q&A records, which are specific answer texts or structured answers to the user's historical questions.

[0088] The reply content can refer to the specific policy answer text given in the historical answer records for the user's question, including textual description, quoted policy provision explanation, etc., which is the reply content provided by the system to the user.

[0089] The difference detection result can refer to the conclusion obtained by comparing the historical answer content with the corresponding policy provisions in the current latest policy knowledge graph through semantic analysis and text comparison technology (such as BERT-based text similarity calculation or text difference detection algorithm), which reflects the judgment result of whether there is a conflict or inconsistency in policy meaning, execution details, rights and obligations, etc.

[0090] The latest policy clause can refer to specific policy clause information in the preset policy knowledge graph after being updated and containing the latest policy change content, which is the current effective and officially published policy provision content.

[0091] The policy update reminder can be a notification message automatically generated when it is found that there is a conflict between the historical answer content and the latest policy based on the difference detection result, reminding the user that the policy content of interest has changed and needs to pay attention to or confirm the latest policy information again.

[0092] The preset push strategy can refer to the rules and mechanisms predefined by the system to control the sending time, frequency, channel (such as SMS, APP notification, email, etc.), priority and user grouping management of the policy update reminder, to ensure that the reminder information reaches the target user accurately and efficiently, and avoid information overload or omission.

[0093] First, the historical government Q&A record is obtained from the storage medium, which contains the historical interaction information between all users and government digital people in the system, including user input question text, system generated answer content and interaction timestamp information. For the structured semantic segment to be processed, the policy entity name contained therein is extracted, which is a key entity identifier representing the policy subject or policy theme.

[0094] Subsequently, based on the extracted policy entity name, the historical government Q&A record is reversely searched using the inverted index retrieval technology to quickly locate all historical answer records containing the policy entity name. This retrieval process realizes accurate matching by analyzing the text field in the Q&A record and combining word segmentation technology and Boolean retrieval algorithm.

[0095] Next, the corresponding reply content, i.e. the policy answer text provided by the system to the user, is extracted from each historical answer record retrieved. To identify the differences between the historical reply and the current policy change, a double-tower model based on deep semantic understanding is used to calculate the semantic similarity between the reply content and the corresponding latest policy clause in the preset policy knowledge graph after incremental embedding, as the difference detection result. Specifically, the BERT pre-training model is used to encode the historical reply text and the latest policy clause text respectively to generate high-dimensional semantic vectors; then the cosine similarity of the two vectors is calculated, and a threshold determination mechanism is used to determine whether there is a significant semantic difference between the two. Difference detection can also introduce a discrimination mechanism based on natural language reasoning model to evaluate the conflict degree of the two in the logical inference level.

[0096] If the detection result determines that the historical reply content conflicts with the latest policy provisions in terms of semantics, the system automatically generates a policy update reminder for the historical reply based on the specific results of the difference detection and the text of the latest policy provisions. The reminder contains summary information of the policy change, a summary of the key content of the latest provisions, and an explanation of the reason for the change, ensuring that users can quickly understand the core points of the policy adjustment. Finally, according to the preset push strategy, the policy update reminder is sent to the relevant users. The BERT pre-training model is first unsupervised trained on a large-scale general text corpus, using the Masked Language Model (MLM) and Next Sentence Prediction (NSP) tasks. MLM randomly masks some words in the input sentence and trains the model to predict the masked words, thereby learning the context association and semantic representation of the words. The NSP task trains the model to determine whether two sentences are adjacent, enhancing the understanding of the logical relationship between sentences. During training, a multi-layer bidirectional Transformer encoder structure is used, and the parameters are optimized through backpropagation until the model can effectively capture the deep semantic and syntactic structure of language.

[0097] In this scheme, the accuracy and timeliness of policy information can be ensured; it helps to actively inform users of policy changes, improves the service quality and user satisfaction of digital government officials, and enhances the dynamic response capability of policy interpretation, achieving intelligent and precise policy management and user communication.

[0098] On the basis of the above technical scheme, optionally, after sending the policy update reminder to the user, the method further comprises:

[0099] Collecting feedback data from the user on the policy update reminder, performing multi-dimensional feature extraction and quantization processing on the feedback data to obtain a user response indicator;

[0100] Inputting the user response indicator into a preset user response model to obtain a user response utility value;

[0101] Based on the user response utility value, dynamically adjusting the sensitivity threshold of semantic difference detection, and the push frequency and time window in the preset push strategy.

[0102] In this scheme, feedback data can refer to various response information provided by users in response to the received policy update reminder, including click rate, reading time, reply content, likes or evaluations, forwarding behavior, etc.

[0103] The user response indicator can be the quantization result of the multi-dimensional features extracted from the feedback data, such as user activity score, satisfaction score, interaction frequency, response speed, etc.

[0104] The preset user response model can be a model constructed based on machine learning or statistical methods for inputting user response indicators and calculating comprehensive utility values. The model can use regression analysis, neural networks, or reinforcement learning techniques to evaluate the overall effectiveness and value of user responses.

[0105] The user response utility value can be a numerical value output by the user response model, reflecting the comprehensive quality and influence of user feedback. The higher the value, the better the user's acceptance and participation of the reminder.

[0106] The sensitivity threshold can refer to a threshold parameter for determining whether there is a significant difference in policy content during semantic difference detection. Adjusting this threshold can control the system's sensitivity to policy changes.

[0107] The push frequency can refer to the frequency of sending policy update reminders to users (such as the number of times sent per day or per week).

[0108] The time window can refer to a specific time range for sending (such as 9 am to 5 pm on weekdays) to optimize the sending strategy of reminders to avoid user fatigue and improve response rates.

[0109] Feedback data on the policy update reminders can be collected in real-time through an embedded front-end interaction log collection mechanism and a back-end behavior tracking system, including but not limited to click events, reading duration, user response content, likes and sharing behavior, bounce rate, and dwell time. For different feedback types, event encoding rules are applied to convert unstructured data into structured log records.

[0110] Subsequently, feature engineering techniques are used to extract multi-dimensional features from the collected feedback data. This process includes statistical features (such as the number of clicks and reading frequency), time series features (such as interaction time intervals and feedback response delays), text features (using TF-IDF and word vector embedding methods to extract semantic features from user response content), and behavior pattern features (identifying user behavior habits through clustering algorithms and behavior sequence analysis). After extraction, normalization and principal component analysis (PCA) techniques are used to quantify and reduce the dimensionality of the feature vectors.

[0111] Next, the obtained user response indicators are input into a pre-trained user response prediction model based on a deep neural network architecture, combining multi-layer perceptron (MLP) and attention mechanisms, and incorporating historical user feedback and contextual information. The model can output a comprehensive user response utility value. This utility value is a continuous numerical value that quantitatively reflects the user's attention, satisfaction, and interaction quality with policy update reminders.

[0112] The user response utility value is input into an adaptive threshold adjustment algorithm, which dynamically calculates the optimal sensitivity threshold adjustment amplitude based on historical response data and the current utility value using Bayesian optimization or gradient descent methods. Specifically, when the user response utility value is below the preset performance lower limit, the algorithm automatically reduces the sensitivity threshold of semantic difference detection, thereby enhancing the model's detection ability for subtle policy semantic changes; conversely, when the utility value is above the performance upper limit, the algorithm increases the threshold to avoid false alarms caused by excessive sensitivity, ensuring system stability and alarm accuracy.

[0113] At the same time, the system adopts a push strategy optimization framework based on deep reinforcement learning, considering the push frequency and time window as adjustable action space, and user feedback data and historical push effect as environment state input. By training the policy gradient method or deep Q network (DQN), the system learns the optimal push strategy, balancing the frequency and time distribution of push reminders while maximizing the user response utility value.

[0114] The training process of the preset user response model includes:

[0115] First, the historical user response data set is obtained from the historical data storage medium of the government digital human platform, which includes multi-dimensional user feedback indicators and corresponding user response utility values. Specifically, the multi-dimensional user feedback indicators include click rate, reading time, feedback type (such as likes, comments, complaints), response time, user behavior path, and multi-modal interaction features, etc. These data are collected in real time through user interaction records with policy update reminders and stored in a structured manner. The user response utility value is generated through expert annotation or a comprehensive scoring system based on rules, reflecting the user's satisfaction and effective response degree to policy update reminders.

[0116] Subsequently, the system performs feature engineering processing on the collected multi-dimensional feedback data, including feature normalization, missing value filling, outlier detection and removal, and feature dimension reduction based on principal component analysis (PCA) or autoencoder, to ensure the quality of input features and the stability of model training. The processed feature vector is used as the input of the model, and the user response utility value is used as the target label for supervised learning.

[0117] In the model construction phase, gradient boosting tree or deep neural network structure is used for training. During training, the system inputs the processed feedback feature vector into the model, combines mean square error (MSE) or cross-entropy loss function, and executes the backpropagation algorithm to adjust the model weight parameters, thereby improving the prediction accuracy by minimizing the error between the predicted response utility value and the actual label.

[0118] To prevent overfitting and improve generalization ability, the training process includes cross-validation, early stopping strategy, and regularization techniques (such as L2 regularization and Dropout). In addition, dynamic learning rate adjustment mechanisms (such as the Adam optimizer) are used to accelerate convergence and stabilize the training process. Finally, the trained user response model is tested offline and online through A / B experiments to ensure that the model can accurately predict the response utility value of users to policy update reminders.

[0119] In this scheme, the user response model is used to evaluate user satisfaction and response utility, enabling intelligent dynamic optimization of policy push strategies. This improves the accurate communication of policy information and user experience, effectively avoids information overload and push fatigue, and enhances the interaction efficiency and service quality of digital government.

[0120] S105, if the user inputs multi-modal input data, input the multi-modal input data into the pre-set multi-modal large model after self-supervised fine-tuning training, obtain the government response content, and send the government response content to the user.

[0121] Multi-modal input data can refer to a set of data containing two or more modal information input by the user during interaction with the digital government, including natural language text information (such as policy consultation questions, business handling application content), voice information (converted to text by voice recognition or directly input as audio signals), image information (such as uploaded application material photos, government form screenshots, and identification pictures), video information (such as business handling video materials), and structured data (such as form field values, identity information records, etc.). The data is processed by modal recognition, feature extraction, and feature alignment before being input into the multi-modal large model, so that the model can uniformly understand the content and semantics of different modalities.

[0122] Government response content can refer to the response information generated by the multi-modal large model based on government domain knowledge and semantic reasoning ability after receiving and understanding the multi-modal input data, which is intended for users. Its form can be natural language text answers (such as policy clause explanations, business handling process descriptions), structured data returns (such as required material lists, handling time limit tables), multi-modal outputs (such as explanatory documents with flowcharts and data visualization charts), etc.

[0123] If multi-modal input data input by a user is received, first, modal recognition processing is performed on the multi-modal input data, wherein, for text modal data, key information units are extracted using word segmentation, part-of-speech tagging, and named entity recognition technology; for speech modal data, an end-to-end speech recognition algorithm combining an acoustic model and a language model is used to convert audio signals into text sequences; for image modal data, convolutional neural network feature extraction and target detection technology are used to identify textual information (extracted by optical character recognition, OCR) and policy-related entities (such as certificate categories and seal identifiers) contained in the image; for video modal data, key frame extraction and temporal action recognition technology are used to obtain key information segments related to policy handling in the video, and dual analysis of images and speech is performed. Subsequently, feature vectorization and modal alignment processing are performed on the feature representations of different modalities, and a multi-modal feature fusion algorithm (such as a cross-modal attention mechanism based on Transformer) is used to map the modal features to a unified semantic representation space to form a fused multi-modal feature vector. Next, the multi-modal feature vector is input into a pre-set multi-modal large model that has been fine-tuned through self-supervised training. The large model uses its semantic understanding and generation capabilities in the government affairs knowledge field to generate government affairs response content that matches the semantics of the user input based on embedded pre-set policy knowledge graph information, historical question and answer corpus, and reasoning mechanisms. The response content can include natural language text explanations, structured data tables, process guide graphs, or comprehensive multi-modal output results. After generation is complete, output content formatting and rendering processing is performed, including natural language paragraph layout, chart visualization rendering, and picture-text layout generation, to ensure correct display on different user terminals (PC, mobile, and smart screen). Finally, the government affairs response content is sent to the user terminal through a reliable transmission mechanism of the network transmission protocol stack.

[0124] In the embodiments of the present application, the multi-modal large model continuously maintains the latest policy knowledge by automatically extracting policy changes and embedding them into the knowledge graph. Combined with self-supervised fine-tuning training, the model can understand multi-modal inputs such as text, speech, and images while generating accurate and contextually consistent government affairs responses, thereby improving the timeliness, accuracy, and interactive experience of policy interpretation, reducing manual maintenance costs, and enhancing user satisfaction.

[0125] Figure 2 A flowchart of a government affairs digital human dynamic interaction method based on a multi-modal large model is provided. The method can include the following steps:

[0126] S201, policy announcement text data is obtained, and the policy announcement text data is cleaned and structured to obtain a structured policy data set.

[0127] S202, extracting old policy data and new policy data from the structured policy dataset, and performing a difference comparison analysis on the old policy data and the new policy data to obtain policy change data marked with key change fields.

[0128] S203, performing summary generation and policy element extraction on the policy change data to obtain a structured semantic segment containing policy elements and semantic relationships thereof.

[0129] S204, incrementally embedding the structured semantic segment into a preset policy knowledge graph, generating question and answer pair samples based on the updated preset policy knowledge graph, and performing self-supervised fine-tuning training on the preset multi-modal large model according to the question and answer pair samples.

[0130] S205, if multi-modal input data input by a user is received, inputting the multi-modal input data into the self-supervised fine-tuned preset multi-modal large model to obtain government affair response content, and sending the government affair response content to the user.

[0131] S206, obtaining multi-dimensional feedback data of the user for the government affair response content in a current feedback collection period, and extracting interactive behavior features, content preference features, and time sequence change features of the multi-dimensional feedback data.

[0132] The current feedback collection period can be a time period in which the system collects user feedback data on government affair response content within a pre-set time interval (e.g., one day, one week, or one month). This period can be a fixed-length time window for statistical analysis of the continuity and trend of user behavior.

[0133] Multi-dimensional feedback data can refer to various types of data generated by users during the feedback process, including user ratings (e.g., satisfaction scores), click behavior (e.g., clicking on related links, expanding details, etc.), dwell time (user browsing time), user comments or opinions, voice or expression feedback (e.g., likes, emoticons, etc.), and revisit frequency (whether the user queries related questions multiple times).

[0134] Interactive behavior features can refer to features extracted from user operation behavior, such as click count, click position, page browsing depth, mouse hover time, swipe behavior, and question and answer interaction frequency.

[0135] Content preference features can be captured by analyzing user feedback on different types of content (e.g., policy categories, question topics, information detail levels, etc.) to capture user interest trends and preferences, such as whether the user is more interested in financial policy or environmental policy, and whether the user prefers concise answers or detailed interpretations.

[0136] The time-series change feature can refer to the change trend of the user feedback feature in a plurality of continuous feedback collection periods, including the change of feedback frequency, the fluctuation of satisfaction, the evolution of user behavior, etc., reflecting the dynamic change of user demand and attitude over time.

[0137] First, the current feedback collection period is set, which is a preset time window, such as 24 hours, 7 days or 30 days. The system extracts all user feedback data for government response content within the time window from the storage medium in batches, which covers multiple forms, including numerical rating of the user on the answer content, click log, browsing time record, text comment content, voice or expression feedback data, and user query history, etc. For the collected multi-dimensional feedback data, the system first uses distributed log processing technology and data cleaning algorithm to eliminate invalid or abnormal data and ensure data quality. Then, using feature engineering technology, three categories of key features are extracted from the feedback data:

[0138] Interaction behavior feature: based on user behavior log, the system calculates the number of clicks, click position distribution, page scroll depth and dwell time, and uses event sequence analysis method and time window statistical technology to quantify the interaction intensity and frequency of the user and the government response content.

[0139] Content preference feature: based on the user's feedback text data, the system uses natural language processing technology (NLP), encodes the comment text through BERT pre-training model, and combines topic model (such as LDA) to classify and cluster the text content, identifying the user's preferred policy field, problem type and information detail preference.

[0140] Time-series change feature: based on the feature data of a plurality of continuous feedback collection periods, the system uses time series analysis method, including sliding window statistics and autoregressive moving average model (ARMA), to capture the change trend and periodic fluctuation of user feedback behavior, reflecting the dynamic evolution of user demand and satisfaction over time.

[0141] S207, according to the interaction behavior feature, content preference feature and time-series change feature, a multi-dimensional feature vector is generated, and the feature vector is normalized and quantized to obtain a feedback feature index.

[0142] The multi-dimensional feature vector can be a high-dimensional vector composed of a plurality of feedback features of the user (including interaction behavior feature, content preference feature and time-series change feature) after numerical representation in a certain order.

[0143] The feedback feature index can be a set of standardized numerical indexes obtained after normalization (i.e. uniforming data of different dimensions to the same numerical range) and quantization (such as discretization or binning) of the above multi-dimensional feature vector.

[0144] The multi-dimensional features can be numerically expressed according to a pre-designed feature structure, and combined to form a one-dimensional real vector, i.e., the multi-dimensional feature vector. The vector dimension corresponds to the number of features, and ensures that each dimension of the feature is clear and independent of each other. In order to eliminate the dimensional differences between different feature indicators, the system normalizes the multi-dimensional feature vector. Specifically, the Min-Max normalization algorithm is used to linearly map each dimension of the feature value to the interval [0, 1]. After normalization, in order to facilitate the subsequent discretization of the model and improve the calculation efficiency, the system further uses the equidistant binning quantization technology to divide the continuous numerical features into a pre-set number of discrete levels, such as 5 levels or 10 levels, and converts them into corresponding integer coding values, thereby generating feedback feature indicators that are more suitable for machine learning algorithm processing.

[0145] S208, calculating a parameter weight adjustment amount of the pre-set multi-modal large model after self-supervised fine-tuning according to the feedback feature indicators, and correcting the parameter weight of the pre-set multi-modal large model after self-supervised fine-tuning according to the parameter weight adjustment amount.

[0146] The parameter weight can be a numerical parameter of the connection of each layer of neural network in the multi-modal large model, which determines how the model extracts features from the input data and generates output.

[0147] The parameter weight adjustment amount can be a specific numerical amount for increasing, decreasing or modifying the parameter weight calculated according to the feedback feature indicators. It represents the "knob rotation amplitude" that needs to be adjusted, and is used to correct the model to make it more consistent with the current user's needs and feedback performance.

[0148] After obtaining the feedback feature indicators, the system first inputs the feedback feature indicators into a weight adjustment calculation unit based on a gradient descent optimization algorithm. Specifically, the error back propagation algorithm is used to calculate the model prediction error in combination with the multi-dimensional feature vector of the user feedback and the current output result of the model. Based on the error, the Adam optimizer is used to adaptively adjust the gradient of each layer of the model parameter weight, to obtain the gradient update direction and amplitude of each parameter, and to form the corresponding parameter weight adjustment amount.

[0149] The weight adjustment amount specifically includes the numerical value that the parameter needs to increase or decrease in the current training round, which reflects the performance deviation and correction requirement of the model in the current feedback period. Subsequently, the system updates each element of the parameter weight of the pre-set multi-modal large model after self-supervised fine-tuning according to the calculated parameter weight adjustment amount, uses weight decay technology to prevent overfitting, and combines the momentum mechanism to smooth the parameter update process, thereby improving the stability and convergence speed of the training.

[0150] The entire weight correction process is executed in parallel in a GPU-accelerated environment, ensuring that the model parameters can be dynamically adjusted in real time, thereby continuously optimizing the accuracy of government response and user satisfaction.

[0151] In this embodiment, feedback feature indicators reflecting user experience and needs are generated. Based on this, the parameter weights of the multi-modal large model are dynamically adjusted to achieve adaptive optimization of the model, significantly improving the response accuracy and personalized service capability of the digital government officer, thereby enhancing user satisfaction and long-term stability of the system.

[0152] On the basis of the above technical solution, optionally, after adjusting the parameter weights of the pre-set multi-modal large model after self-supervised fine-tuning according to the parameter weight adjustment amount, the method further comprises:

[0153] Obtain multi-dimensional feedback data of users for the pre-set multi-modal large model after self-supervised fine-tuning in a plurality of continuous feedback collection periods, extract interactive behavior features, content preference features and time sequence change features of the multi-dimensional feedback data;

[0154] Generate a multi-dimensional feature vector according to the interactive behavior features, content preference features and time sequence change features, normalize and quantize the feature vector to obtain feedback feature indicators in a plurality of continuous feedback collection periods;

[0155] Generate a feedback feature indicator sequence according to the feedback feature indicators in a plurality of continuous feedback collection periods, and perform time sequence trend modeling and change rate calculation processing on the feedback feature indicator sequence to obtain a feedback trend analysis result;

[0156] Determine whether the feedback feature indicator sequence presents a continuous downward trend based on the feedback trend analysis result, and if so, obtain the current parameter weight of the pre-set multi-modal large model after self-supervised fine-tuning, perform deviation analysis processing on the current parameter weight and a pre-set performance benchmark to obtain a performance deviation amount;

[0157] Adjust the generation rule threshold of the reply generation strategy in the pre-set multi-modal large model after self-supervised fine-tuning according to the performance deviation amount, generate an artificial review instruction according to the adjusted generation rule threshold, and send the artificial review instruction to the control center.

[0158] In this solution, the feedback collection period can refer to the time interval in which the system periodically collects user feedback data for government response content, such as a fixed time period of daily, weekly or monthly.

[0159] The feedback feature index sequence can refer to a data sequence formed by arranging feedback feature indexes calculated in a plurality of continuous feedback collection periods in chronological order. The sequence reflects the dynamic changes of the feedback feature index over time and is used for trend analysis.

[0160] The feedback trend analysis result can be a conclusion obtained after time series modeling and change rate calculation based on the feedback feature index sequence, usually indicating whether the feedback feature index has an upward, downward or stable trend, thereby reflecting the change trend of user experience or model performance.

[0161] The current parameter weight can refer to a specific set of numerical values of the model internal parameters used by the preset multi-modal large model after the last self-supervised fine-tuning training, which determines the behavior and performance of the model.

[0162] The preset performance benchmark can be a pre-set model performance evaluation standard or threshold value, used to judge whether the current model performance meets the expected requirements, such as target values of accuracy, recall rate or user satisfaction.

[0163] The performance deviation amount can refer to the difference between the model performance corresponding to the current parameter weight and the preset performance benchmark, which quantitatively reflects the deviation degree of the model performance.

[0164] The reply generation strategy can refer to the rules and parameter configurations that control the multi-modal large model to generate government reply content, including strategies for diversity, accuracy and style of generated content.

[0165] The generation rule threshold value can be a threshold parameter in the reply generation strategy for adjusting the quality and range of generated content, such as numerical limits set for content confidence, generation diversity and other indicators.

[0166] The manual review instruction can refer to an operation instruction automatically triggered when the model performance decreases or is abnormal, requiring manual review of the model generated results to ensure the quality and accuracy of the reply.

[0167] The control center can refer to a background system or management platform responsible for receiving, managing and executing manual review instructions.

[0168] The multi-dimensional feedback data of the government affair response content generated by the user for the preset multi-modal large model after self-supervised fine-tuning training is acquired within a plurality of continuous feedback collection periods. The feedback data includes but is not limited to user click times, dwell time, sliding behavior, user rating, text comments, and question and answer interaction frequency, etc. The feedback data is processed by feature engineering technology respectively. Firstly, the sliding statistical method based on time window is applied to extract interaction behavior features, such as user activity, operation frequency, and interaction diversity. Secondly, the content classification and clustering algorithm is used to extract user preference features for different content types, reflecting the user preference trend, combined with the text sentiment analysis model. Thirdly, the time series analysis technology such as long short-term memory network (LSTM) or autoregressive integrated moving average model (ARIMA) is used to capture the time sequence change features of the feedback data from the time dimension, and identify the periodicity and mutation points in the feedback data.

[0169] The extracted interaction behavior features, content preference features and time sequence change features are integrated in the form of feature vectors to form multi-dimensional feature vectors with representativeness and discriminability. The Min-Max normalization or Z-score standardization technology is applied to the feature vectors to scale the feature values to a unified interval, eliminate the dimension difference, and use principal component analysis (PCA) or autoencoder method for feature dimension reduction and denoising to improve the effectiveness of feature expression. Subsequently, according to the pre-defined quantization rules, the normalized feature vectors are converted into numerical feedback feature indicators, which are used to quantify the comprehensive performance of user feedback.

[0170] For a plurality of continuous feedback collection periods, the corresponding feedback feature indicators are generated in sequence to construct a feedback feature indicator sequence in the form of time series. Based on the sequence, a time series prediction model (such as LSTM, GRU or variational autoencoder variant) is applied to model the trend, and the change rate is calculated by the sliding window difference method to extract the trend change signal, and finally the quantitative feedback trend analysis result is obtained. The result reveals the growth, stability or decline state of user feedback.

[0171] Based on the feedback trend analysis result, the threshold judgment algorithm is used to determine whether there is a sustained downward trend. If the feedback feature indicator sequence is determined to have a sustained downward trend, the model performance evaluation process is triggered, and the latest parameter weight of the current trained multi-modal large model is obtained, which is a weight matrix set of each layer neural network parameter in the model. The current parameter weight is compared with the pre-set performance benchmark (including the target values of model accuracy, response speed and user satisfaction, etc.), and the difference measurement algorithm (such as Euclidean distance, cosine similarity or Kullback-Leibler divergence) is applied to calculate the performance deviation, which quantitatively describes the deviation of the current model performance from the expected standard.

[0172] The generation rule threshold mainly includes the following types of parameters:

[0173] The content credibility threshold is used to control the lower limit of the credibility of the information in the generated reply. By adjusting this threshold, the system can limit the minimum confidence probability value allowed to be included in the generated text, ensuring that the output content meets the credibility standard. Specifically, the token-level confidence distribution calculated by the multi-modal large model during the generation process is used to dynamically set the minimum confidence threshold, and low-confidence content is removed, thereby reducing the generation of false information or irrelevant answers.

[0174] The diversity adjustment parameter controls the diversity and creativity of the generated text, usually corresponding to the temperature parameter in the sampling strategy or the Top-k, Top-p (nucleus sampling) sampling cutoff threshold. The system dynamically adjusts the sampling temperature according to the performance deviation: when the deviation is large, the temperature value is reduced to reduce randomness in generation, improving the accuracy and stability of the reply; when the deviation is small, the temperature is appropriately increased to increase the diversity of the content and the natural expression style.

[0175] The repeated content penalty factor is used to control the punishment intensity of repeated words or sentences in the generated text. By dynamically adjusting the penalty factor, the model is prevented from excessively repeating similar content, improving the richness and readability of the reply.

[0176] The adjustment process adopts a closed-loop feedback mechanism, taking the current performance deviation as input, and jointly optimizing the above-mentioned multiple threshold parameters through a multi-objective optimization algorithm (such as genetic algorithm or reinforcement learning strategy gradient method). The specific steps include: taking the performance deviation as the input of the objective function, defining the objective function as the weighted sum of the minimum deviation, while considering the diversity and credibility of the reply. Search for the optimal combination of generation rule threshold in the parameter space, and use efficient search strategies such as Bayesian optimization or grid search. After dynamically updating the generation rule threshold parameters, the adjustment results are applied to the decoding strategy of the multi-modal large model, affecting the generation process in real time. After adjustment, the system automatically generates artificial review instructions, including model performance anomaly alerts and answer content identification that needs to be reviewed by humans. The review instructions are sent to the control center.

[0177] In this scheme, the timing trend of the response quality of the digital government official can be accurately captured, and the risk of continuous decline in service performance can be timely identified. Through the dynamic adjustment mechanism driven by the performance deviation, the system actively optimizes the reply generation strategy, effectively improving the accuracy of the answer and user satisfaction.

[0178] Figure 3 A schematic block diagram of a dynamic interaction system for a digital government official based on a multi-modal large model is provided for the embodiments of the present disclosure. The system comprises:

[0179] The data preprocessing module 301 is configured to acquire policy announcement text data, clean and structure the policy announcement text data, and obtain a structured policy data set.

[0180] The difference analysis module 302 is configured to extract old policy data and new policy data from the structured policy data set, and perform difference comparison analysis on the old policy data and the new policy data to obtain policy change data marked with a key change field.

[0181] The semantic processing module 303 is configured to generate an abstract and extract a policy element from the policy change data to obtain a structured semantic segment containing the policy element and a semantic relationship thereof.

[0182] The training module 304 is configured to incrementally embed the structured semantic segment into a preset policy knowledge graph, generate a question and answer pair sample based on the updated preset policy knowledge graph, and perform self-supervised fine-tuning training on a preset multi-modal large model according to the question and answer pair sample.

[0183] The response generation module 305 is configured to, if receiving multi-modal input data input by a user, input the multi-modal input data into the self-supervised fine-tuned preset multi-modal large model to obtain government affair response content, and send the government affair response content to the user.

[0184] As shown in Figure 4 The electronic device 400 includes a processor 401, a memory 402, and a program or instruction stored in the memory 402 and executable on the processor 401. The program or instruction is executed by the processor 401 to implement each process of the encryption protection method of the deep learning large model, and achieve the same technical effect. To avoid repetition, details are not repeated here.

[0185] It should be noted that the electronic device in the present embodiment includes the mobile electronic device and the non-mobile electronic device described above.

[0186] The present embodiment also provides a readable storage medium having a program or instruction stored thereon. The program or instruction is executed by a processor to implement each process of the cable installation process based on the adaptive control system of the tension, and achieve the same technical effect. To avoid repetition, details are not repeated here.

[0187] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0188] It should be noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the methods and systems described in the present application are not limited to the order of execution of the steps of processes or methods, as described or illustrated herein, unless the order of execution is specifically stated or inherently necessitated by the disclosure.

[0189] Those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the method described in each embodiment of the present application.

[0190] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative, not restrictive, and those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

[0191] The above are only the preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions made by those skilled in the art will not deviate from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without deviating from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for dynamic interaction of government digital people based on a multi-modal large model, characterized in that, The method is executed by a digital official, and the method comprises: Obtaining policy announcement text data, cleaning and structuring the policy announcement text data to obtain a structured policy data set; Extracting old policy data and new policy data from the structured policy data set, and performing difference comparison and analysis on the old policy data and the new policy data to obtain policy change data marked with a key change field; Generating an abstract and extracting a policy element from the policy change data to obtain a structured semantic segment containing a policy element and a semantic relationship thereof; Extracting a policy entity name and a corresponding first effective time interval in the structured semantic segment, locating a same-name entity node in a preset policy knowledge graph according to the policy entity name, and reading a second effective time interval of the same-name entity node; If there is an overlapping time interval between the first effective time interval and the second effective time interval, determining a first policy clause content corresponding to the structured semantic segment, and determining a second policy clause content corresponding to the preset policy knowledge graph; If there is no semantic difference between the first policy clause content and the second policy clause content, generating an allowed insertion instruction according to the structured semantic segment; If the allowed insertion instruction is identified, incrementally embedding the structured semantic segment into the preset policy knowledge graph, generating a question and answer pair sample based on the updated preset policy knowledge graph, and performing self-supervised fine-tuning training on the preset multi-modal large model according to the question and answer pair sample; Obtaining historical government affairs question and answer records, performing reverse retrieval on the historical government affairs question and answer records based on the policy entity name corresponding to the structured semantic segment, and locating historical answer records containing the policy entity name; Extracting the reply content of the historical answer records, performing semantic difference detection on the reply content and the latest policy clause corresponding to the incrementally embedded preset policy knowledge graph, and obtaining a difference detection result; If the difference detection result indicates that there is a semantic conflict, generating a policy update reminder corresponding to the historical answer record based on the difference detection result and the corresponding latest policy clause in the incrementally embedded preset policy knowledge graph, and pushing the policy update reminder to the user according to a preset pushing strategy; If multi-modal input data input by the user is received, inputting the multi-modal input data into the self-supervised fine-tuned preset multi-modal large model to obtain government affairs reply content, and sending the government affairs reply content to the user.

2. The method of claim 1, wherein, Wherein, After reading the second effective time interval of the same-name entity node, the method further comprises: If there is no overlapping time interval between the first effective time interval and the second effective time interval, an allowed insertion instruction is generated according to the structured semantic segment; Correspondingly, incrementally embedding the structured semantic segment into the preset policy knowledge graph comprises: If the allowed insertion instruction is identified, the structured semantic segment is incrementally embedded into the preset policy knowledge graph.

3. The method of claim 1, wherein, Wherein, After sending the policy update reminder to the user, the method further comprises: Collect feedback data of the user on the policy update reminder, perform multi-dimensional feature extraction and quantization processing on the feedback data to obtain a user response index; input the user response index into a preset user response model to obtain a user response utility value; based on the user response utility value, dynamically adjust the sensitivity threshold of semantic difference detection, and the push frequency and time window in the preset push strategy.

4. The method of claim 1, wherein, wherein, after sending the government affair response content to the user, the method further comprises: obtaining multi-dimensional feedback data of the user on the government affair response content in a current feedback collection period, extracting interaction behavior features, content preference features and time sequence change features of the multi-dimensional feedback data; generating a multi-dimensional feature vector according to the interaction behavior features, content preference features and time sequence change features, performing normalization and quantization processing on the feature vector to obtain a feedback feature index; calculating a parameter weight adjustment amount of the preset multi-modal large model after self-supervised fine-tuning according to the feedback feature index, and correcting the parameter weight of the preset multi-modal large model after self-supervised fine-tuning according to the parameter weight adjustment amount.

5. The method of claim 4, wherein, wherein, after correcting the parameter weight of the preset multi-modal large model after self-supervised fine-tuning according to the parameter weight adjustment amount, the method further comprises: obtaining multi-dimensional feedback data of the user on the self-supervised fine-tuned preset multi-modal large model in a plurality of continuous feedback collection periods, extracting interaction behavior features, content preference features and time sequence change features of the multi-dimensional feedback data; generating a multi-dimensional feature vector according to the interaction behavior features, content preference features and time sequence change features, performing normalization and quantization processing on the feature vector to obtain a feedback feature index in a plurality of continuous feedback collection periods; generating a feedback feature index sequence according to the feedback feature indexes in a plurality of continuous feedback collection periods, performing time trend modeling and change rate calculation processing on the feedback feature index sequence to obtain a feedback trend analysis result; based on the feedback trend analysis result, determining whether the feedback feature index sequence presents a continuous downward trend, if so, obtaining a current parameter weight of the self-supervised fine-tuned preset multi-modal large model, performing deviation analysis processing on the current parameter weight and a preset performance benchmark to obtain a performance deviation amount; adjusting a generation rule threshold of a reply generation strategy in the self-supervised fine-tuned preset multi-modal large model according to the performance deviation amount, generating an artificial review instruction according to the adjusted generation rule threshold, and sending the artificial review instruction to a control center.

6. A government affair digital human dynamic interaction system based on a multi-modal large model, characterized in that, The system is configured in a government affair digital person, and the system comprises: a data preprocessing module configured to obtain policy announcement text data, clean and structure the policy announcement text data to obtain a structured policy data set; a difference analysis module configured to extract old policy data and new policy data from the structured policy data set, and perform difference comparison analysis on the old policy data and the new policy data to obtain policy change data marked with key change fields. The semantic processing module is configured to perform abstract generation and policy element extraction on the policy change data to obtain a structured semantic segment containing policy elements and semantic relationships thereof. The system is further configured to: extract a policy entity name and a corresponding first effective time interval in the structured semantic segment, locate a same-name entity node in a preset policy knowledge graph according to the policy entity name, and read a second effective time interval of the same-name entity node; if there is an overlapping time interval between the first effective time interval and the second effective time interval, determine a first policy clause content corresponding to the structured semantic segment, and determine a second policy clause content corresponding to the preset policy knowledge graph; if there is no semantic difference between the first policy clause content and the second policy clause content, generate an allowed insertion instruction according to the structured semantic segment; The training module is configured to, if the allowed insertion instruction is identified, incrementally embed the structured semantic segment into the preset policy knowledge graph, generate a question and answer pair sample based on the updated preset policy knowledge graph, and perform self-supervised fine-tuning training on the preset multi-modal large model according to the question and answer pair sample. The system is further configured to: obtain historical government affairs question and answer records, perform reverse retrieval on the historical government affairs question and answer records based on the policy entity name corresponding to the structured semantic segment, and locate historical answer records containing the policy entity name; extract the reply content of the historical answer records, perform semantic difference detection on the reply content and the latest policy clause corresponding to the preset policy knowledge graph after incremental embedding, and obtain a difference detection result; if the difference detection result indicates that there is a semantic conflict, generate a policy update reminder corresponding to the historical answer record based on the difference detection result and the latest policy clause corresponding to the preset policy knowledge graph after incremental embedding, and push the policy update reminder to the user according to a preset pushing strategy; The answer generation module is configured to, if multi-modal input data input by the user is received, input the multi-modal input data into the preset multi-modal large model after self-supervised fine-tuning training to obtain government affairs answer content, and send the government affairs answer content to the user.

7. An electronic device, comprising: A processor, a memory, and a program or instructions stored on the memory and executable on the processor are included, and the program or instructions are executed by the processor to implement the steps of the government affairs digital human dynamic interaction method based on the multi-modal large model according to any one of claims 1-5.

8. A readable storage medium, characterized by, A readable storage medium stores a program or instructions, and the program or instructions are executed by a processor to implement the steps of the government affairs digital human dynamic interaction method based on the multi-modal large model according to any one of claims 1-5.

Images