Government affair service content navigation method and system based on large language model

Abstract

The invention relates to the technical field of multi-modal large language models, in particular to a government affair service content navigation method and system based on a large language model, and the method comprises the following steps: receiving multi-modal information input by a user through texts, voices or pictures; the voice is converted into a text, and character information in the picture is analyzed by using an OCR (Optical Character Recognition) technology; fusing multi-modal data, and inputting the fused multi-modal data into a large language model for semantic understanding and context association analysis; matching items are retrieved in combination with a local government affair knowledge base, and an initial recommendation list is generated; the recommendation result is displayed through the intelligent assistant, and interaction optimization options are provided; the method has the beneficial effects that the current government affair service content navigation mode is optimized through the multi-modal capability, the retrieval enhancement capability and the content generation capability of the large language model, so that the navigation is more modal and more intelligent, and meanwhile, the privacy and authority of data reply are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal large language models, and specifically to a government service content navigation method and system based on a large language model. Background Art

[0002] Current status of large language model technology: Current large language models typically contain billions or even tens of billions of parameters. Through large amounts of training data and complex network structures, the model's understanding and generation capabilities are constantly improving. An increasing number of companies and research institutions are using these large-scale models to handle natural language tasks such as machine translation, text generation, sentiment analysis, and question-answering systems. Some large language models have cross-modal capabilities and can not only process text, but also understand multimodal data such as images and audio.

[0003] Problems with the application of large language model technology: Large language models are usually trained from large amounts of Internet text data, which may contain various biases, discrimination, race, gender and other negative content. In the field of government services, the processing of government data must strictly comply with relevant laws and policies to ensure that user information will not be abused or leaked, and must have high authority and credibility. Summary of the Invention

[0004] The purpose of the present invention is to provide a government service content navigation method and system based on a large language model to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a government service content navigation method based on a large language model, comprising the following steps:

[0006] Receive multimodal information input by users through text, voice or pictures;

[0007] Convert speech into text and use OCR technology to analyze text information in images;

[0008] Fuse multimodal data and input it into a large language model for semantic understanding and contextual analysis;

[0009] Combined with the local government knowledge base to search for matching items and generate an initial recommendation list;

[0010] The recommendation results are displayed through the intelligent assistant, and interactive optimization options are provided.

[0011] Preferably, multimodal information processing includes the following detailed steps:

[0012] When inputting voice, real-time noise reduction and dialect recognition technology are used to generate standard text;

[0013] When inputting images, the document type is automatically detected and key fields are extracted, and the pre-stored templates in the knowledge base are associated;

[0014] Perform alignment verification on multi-source heterogeneous data and filter conflicting information through confidence algorithms;

[0015] Generate structured query vectors in a unified format and input them into a large language model for inference.

[0016] Preferably, the method further includes the following user interaction steps:

[0017] After the initial recommendation, the user can select "More" to trigger a secondary search, and the model will expand the candidate set based on the new parameters;

[0018] When users add text / voice / picture information, the context-aware module is activated to adjust the recommendation weight;

[0019] After selecting the translation function, the recommended items will be automatically converted into multiple language versions and the format will be kept consistent;

[0020] After confirming the accurate matters, call the corresponding service platform interface and pre-fill the information provided by the user.

[0021] Preferably, knowledge base maintenance includes the following steps:

[0022] Regularly crawl the latest data on government service platforms and automatically mark entity relationships;

[0023] Mining high-frequency questions through user interaction logs to supplement missing nodes in the knowledge base;

[0024] Use graph database technology to achieve the associated storage of policy terms and handling procedures;

[0025] Establish a version control mechanism to ensure that knowledge base updates do not affect historical conversation records.

[0026] Preferably, multi-language support includes the following implementation steps:

[0027] Construct a bilingual dictionary in the government affairs field to optimize the accuracy of translation of professional terms;

[0028] When training a multilingual large language model, include a parallel corpus of policy texts;

[0029] When the user switches languages, the corresponding version interface is dynamically generated and the functional layout is kept consistent;

[0030] Confidence scores are assigned to multilingual query results, and manual intervention is prompted when the confidence score is below the threshold.

[0031] A system for navigating government service content based on a large language model, including four collaboratively operating system modules:

[0032] Intelligent interactive front-end: provides a multimodal input interface, integrating text input, voice input conversion, and image file parsing functions;

[0033] Large language model engine: It has the capabilities of natural language understanding, contextual analysis, and multi-round dialogue memory, and generates item recommendations through deep neural networks;

[0034] Local government knowledge graph: Build a structured database containing regional policy clauses, business process nodes, and department relationships, supporting private deployment and dynamic updates;

[0035] Cross-platform service routing: Enables multi-terminal business docking across government websites, mobile applications, and self-service terminals, and intelligently matches service entrances based on user identity and geographic location.

[0036] Preferably, the multimodal input processing module includes the following innovative features:

[0037] The voice input unit uses noise-resistant speech recognition technology, supports conversion between dialect and colloquial expressions, and has semantic error correction function;

[0038] The image input unit integrates a multilingual OCR engine that can recognize 20 types of government documents, such as ID cards and business licenses, automatically extract key fields, and associate them with knowledge base entries.

[0039] Trimodal data fusion algorithm: Dynamically assigns weight coefficients of text / speech / image features through the attention mechanism to solve the problem of cross-modal information conflict.

[0040] Optimally, the recommendation driven by the large language model follows the four-order decision logic:

[0041] Initial query phase: Generate the top 5 high-frequency item candidate sets based on user input;

[0042] Context understanding stage: Build user profiles based on conversation history and identify three-dimensional tags based on region, identity, and urgency.

[0043] Dynamic optimization stage: supports two interaction paths: "More - Precision" and "Supplement - Multimodality", and re-arranges the recommendation list after each interaction;

[0044] Execution routing stage: Generate a structured guidance card containing a QR code for the processing channel, a preview of the bill of materials, and an estimated waiting time.

[0045] Preferably, the local government knowledge base has a double closed-loop update architecture:

[0046] Inner ring update: Real-time acquisition of event change logs through the government service platform API, and automatic completion of knowledge graphs using graph neural networks;

[0047] Outer loop update: Construct feedback vectors based on user interaction data, apply reinforcement learning algorithms to optimize node association weights, and achieve "human-machine" collaborative knowledge evolution;

[0048] Security control: A federated learning framework is used to integrate cross-regional knowledge bases, ensuring that model parameter aggregation is completed without leaving the original data.

[0049] Preferably, it also includes a multi-language support system:

[0050] Build a dedicated machine translation model for government affairs, optimize terminology consistency for policy texts, and set up a whitelist for legal clause translation;

[0051] Developed a dynamic multilingual interface generation engine that automatically switches the service interface between 32 official languages based on the user input language;

[0052] Establish a cross-language intent alignment mechanism to resolve the ambiguous expression of the same issue in different language systems by comparing policy texts in different language versions.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The government service content navigation method and system based on the large language model proposed in the present invention optimize the current government service content navigation method through the multimodal capabilities, retrieval enhancement capabilities, and content generation capabilities of the large language model, making navigation more modal and intelligent while ensuring the privacy and authority of data responses. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a system architecture diagram of the present invention;

[0056] Figure 2 This is a rendering of the government service platform of the present invention. DETAILED DESCRIPTION

[0057] In order to clearly and completely describe the objectives and technical solutions of the present invention and make the advantages more clearly understood, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, not all of them, and are only used to explain the embodiments of the present invention, not to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] For example 1, please refer to Figure 1 The present invention provides a technical solution: The present invention is aimed at users who use the government service platform. In the past, when users search for items among many businesses, they use the following method:

[0059] Method 1: Page topic navigation, which requires users to have a good understanding of topic categories, otherwise they will repeatedly enter and exit the page;

[0060] Method 2: Keyword full-text search, which only supports one-time search and the keyword search results are fuzzy;

[0061] Method 3: Standard intelligent question answering cannot understand the context. For example, if you input "I am from city A and lost my ID card in city B", it cannot provide business suggestions.

[0062] The present invention is based on the capability of large language model and utilizes user input of multiple types of information for retrieval. The input module includes a text input module, a voice input module and a picture input module.

[0063] Module 1: Text input module, directly obtains text;

[0064] Module 2: Voice input module, which records short voice messages from users and converts them into text to obtain input information;

[0065] Module 3: The image input module is used by users to upload image files, and then uses OCR technology to switch text to obtain input information.

[0066] Based on the large language model capability, the present invention outputs recommended items after inputting information. Users can continue to refine questions and reply in the following ways:

[0067] Method 1: Click "More" to recommend more items based on the large language model.

[0068] Method 2: Add query content, including input through the aforementioned text, voice, and image methods, and update recommended items based on the large language model;

[0069] Method 3: Content translation: Enter the language to be translated, and update the content to the relevant language based on the large language model;

[0070] Method 4: Enter the precise matters. The government service platform will confirm the matters that need to be handled in the end and open the corresponding application based on the current handling platform.

[0071] The present invention has the following system structure:

[0072] 1. Intelligent assistant, providing front-end interaction with users;

[0073] 2. Large language model, providing content retrieval and generation capabilities;

[0074] 3. Local global government knowledge base, providing private, secure and controllable knowledge content;

[0075] 4. Government service platform, a platform that provides business applications, including the government service network, government APP, government mini-programs, self-service terminals and other service platforms.

[0076] Example 2, based on Example 1, refer to the attached Figure 2 As shown, an example is given to illustrate the effect of a government service content navigation method based on a large language model on a government service platform (taking a self-service terminal as an example).

[0077] The intelligent assistant based on the Large Language Model (LLM) provides multiple access methods. It can be accessed through the government service network, government apps, government applets, self-service terminals, and other systems. Third-party systems need to reference the intelligent assistant's JS-SDK. The opening methods support:

[0078] Open: open the pop-up window mode;

[0079] Rewrite: rewrite the current page path;

[0080] Embed: Embed the specified node on the page;

[0081] F1oat: Page floating layer mode.

[0082] On the left side of the figure is the smart assistant dialogue window embedded in the specified node of the page, and on the right side is the application function area for business processing. The smart assistant can be used to intelligently navigate the content and intelligently locate the business to be processed. If it is applied on the platform, it will be opened in the application function area. The implementation logic is as follows:

[0083] 1. The front-end of the intelligent assistant obtains the text, voice (speech-to-text conversion required) or image of the user's question and sends a request to the back-end API. The request content is as follows:

[0084]

[0085]

[0086] 2. The backend of the intelligent assistant combines a large language model with a local knowledge base, allowing the LLM to learn and extract information from the knowledge base to answer user questions;

[0087] 3. The front-end of the intelligent assistant obtains the answer content, desensitizes the data, and checks whether there is any content with URL:

[0088]

[0089]

[0090]

[0091]

[0092] 4. If there is URL content, the smart assistant sends the URL content to the self-service terminal. The self-service terminal checks whether the URL application exists and implements the route jump:

[0093]

[0094] Example 3. Based on Example 2, taking the government service network as an example, before using the government service content navigation method based on the large language model, users can only enter keywords once in the full-text search box for fuzzy search; or find the topic, find the department, find the major items, and finally find the precise items in the topic navigation; or ask questions in the intelligent question and answer only to list the item names without context association.

[0095] After using the government service content navigation method based on the large language model, users can intelligently locate service content according to the following process:

[0096] (1) The user clicks on the smart assistant on a mobile device, computer, self-service terminal, or other platform;

[0097] (2) Users can freely input text, or send voice or pictures;

[0098] (3) After the intelligent assistant responds, if the user determines that the content is inaccurate, they can continue to supplement it with text, voice, or pictures;

[0099] (4) After the intelligent assistant responds, if the user determines that the content is only close, they can enter words such as "more";

[0100] (5) After the intelligent assistant responds, if the user wishes to display the content in another language, enter "display in a certain language";

[0101] (6) After the intelligent assistant responds, if the user wishes to expand and view the details of the service, he or she can enter words such as "detailed introduction to a service";

[0102] (7) When the smart assistant detects that the displayed business can be handled on the platform (government service network), a "Process Now" button appears and the user can click it to open the corresponding business processing page.

[0103] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A government service content navigation method based on a large language model, characterized by: The following steps are involved: Receive multimodal information input by users through text, voice or pictures; Convert speech into text and use OCR technology to analyze text information in images; Fuse multimodal data and input it into a large language model for semantic understanding and contextual analysis; Combined with the local government knowledge base to search for matching items and generate an initial recommendation list; The recommendation results are displayed through the intelligent assistant, and interactive optimization options are provided.

2. A government service content navigation method based on a large language model according to claim 1, characterized in that: Multimodal information processing includes the following detailed steps: When inputting voice, real-time noise reduction and dialect recognition technology are used to generate standard text; When inputting images, the document type is automatically detected and key fields are extracted, and the pre-stored templates in the knowledge base are associated; Perform alignment verification on multi-source heterogeneous data and filter conflicting information through confidence algorithms; Generate structured query vectors in a unified format and input them into a large language model for inference.

3. A method for navigating government service content based on a large language model according to claim 2, characterized in that: The following user interaction steps are also included: After the initial recommendation, the user can select "More" to trigger a secondary search, and the model will expand the candidate set based on the new parameters; When users add text / voice / picture information, the context-aware module is activated to adjust the recommendation weight; After selecting the translation function, the recommended items will be automatically converted into multiple language versions and the format will be kept consistent; After confirming the accurate matters, call the corresponding service platform interface and pre-fill the information provided by the user.

4. A method for navigating government service content based on a large language model according to claim 3, characterized in that: Knowledge base maintenance includes the following steps: Regularly crawl the latest data on government service platforms and automatically mark entity relationships; Mining high-frequency questions through user interaction logs to supplement missing nodes in the knowledge base; Use graph database technology to achieve the associated storage of policy terms and handling procedures; Establish a version control mechanism to ensure that knowledge base updates do not affect historical conversation records.

5. A method for navigating government service content based on a large language model according to claim 4, characterized in that: Multilingual support includes the following implementation steps: Construct a bilingual dictionary in the government affairs field to optimize the accuracy of translation of professional terms; When training a multilingual large language model, include a parallel corpus of policy texts; When the user switches languages, the corresponding version interface is dynamically generated and the functional layout is kept consistent; Confidence scores are assigned to multilingual query results, and manual intervention is prompted when the confidence score is below the threshold.

6. A system for the government service content navigation method based on a large language model according to claim 5, characterized in that: It includes four major system modules that work together: Intelligent interactive front-end: provides a multimodal input interface, integrating text input, voice input conversion, and image file parsing functions; Large language model engine: It has the capabilities of natural language understanding, contextual analysis, and multi-round dialogue memory, and generates item recommendations through deep neural networks; Local government knowledge graph: Build a structured database containing regional policy clauses, business process nodes, and department relationships, supporting private deployment and dynamic updates; Cross-platform service routing: Enables multi-terminal business docking across government websites, mobile applications, and self-service terminals, and intelligently matches service entrances based on user identity and geographic location.

7. A system according to claim 6, characterized in that: The multimodal input processing module includes the following innovative features: The voice input unit uses noise-resistant speech recognition technology, supports conversion between dialect and colloquial expressions, and has semantic error correction function; The image input unit integrates a multilingual OCR engine that can recognize 20 types of government documents, such as ID cards and business licenses, automatically extract key fields, and associate them with knowledge base entries. Trimodal data fusion algorithm: Dynamically assigns weight coefficients of text / speech / image features through the attention mechanism to solve the problem of cross-modal information conflict.

8. A system according to claim 7, characterized in that: The recommendation driven by the large language model follows a four-level decision logic: Initial query phase: Generate the top 5 high-frequency item candidate sets based on user input; Context understanding stage: Build user profiles based on conversation history and identify three-dimensional tags based on region, identity, and urgency. Dynamic optimization stage: supports two interaction paths: "More - Precision" and "Supplement - Multimodality", and re-arranges the recommendation list after each interaction; Execution routing stage: Generate a structured guidance card containing a QR code for the processing channel, a preview of the bill of materials, and an estimated waiting time.

9. A system according to claim 8, characterized in that: The local government knowledge base has a dual closed-loop update architecture: Inner ring update: Real-time acquisition of event change logs through the government service platform API, and automatic completion of knowledge graphs using graph neural networks; Outer loop update: Construct feedback vectors based on user interaction data, apply reinforcement learning algorithms to optimize node association weights, and achieve "human-machine" collaborative knowledge evolution; Security control: A federated learning framework is used to integrate cross-regional knowledge bases, ensuring that model parameter aggregation is completed without leaving the original data.

10. A system according to claim 9, characterized in that: Also includes multi-language support system: Build a dedicated machine translation model for government affairs, optimize terminology consistency for policy texts, and set up a whitelist for legal clause translation; Developed a dynamic multilingual interface generation engine that automatically switches the service interface between 32 official languages based on the user input language; Establish a cross-language intent alignment mechanism to resolve the ambiguous expression of the same issue in different language systems by comparing policy texts in different language versions.

Images