Data linkage display method and system based on natural language interaction

Abstract

According to the data linkage display method and system based on natural language interaction, multi-modal natural language understanding, intelligent task matching and a front-end and back-end linkage control mechanism are fused, efficient, intelligent and low-delay interaction control between a mobile terminal and a data large screen is achieved, and the overall expandability and stability of the system are improved. Through a multi-modal natural language input mode, instruction understanding and tool selection are carried out in combination with a large language model, so that a non-professional user can also rapidly complete control operation on a data large screen through a natural language, the use threshold is remarkably reduced, after a rear end completes data processing, a control instruction is actively pushed to the front end of the large screen, and the user experience is improved. A user does not need to initiate a request again, the interface response speed is remarkably improved, meanwhile, operation tools are divided according to function categories, a dynamic loading mechanism is combined, related tool information is only input into a large language model when needed, the situation that context is overlong due to one-time input is avoided, and the adaptive capacity of a system in a complex service environment is improved.

Description

Technical Field

[0001] This disclosure relates to the field of data visualization technology, and more specifically, to a data linkage display method and system based on natural language interaction. Background Technology

[0002] With the development of information technology, data visualization tools have been widely used in various fields such as enterprise decision-making, operations management, and public services. Traditional data visualization platforms typically rely on complex user interfaces and menu-driven operation modes, which require users to have a certain level of professional knowledge to use these tools effectively. For example, when monitoring parking lot occupancy, users may need to navigate through multiple steps to the corresponding functional modules and manually set query conditions to obtain the required information.

[0003] On the one hand, current data dashboard platforms mostly rely on traditional interaction methods such as mouse clicks and touch controls, lacking natural language support. This makes it difficult to meet users' needs for efficient and intelligent human-computer interaction, especially in mobile scenarios or for non-professional users, where the operation threshold is high and efficiency is low. On the other hand, existing systems are mostly in a passive response mode, requiring users to actively initiate operation requests on the terminal before the dashboard backend executes the corresponding data processing and display processes. This client-driven mode has high response latency, making it difficult to meet the real-time requirements of some high-time-sensitivity business scenarios.

[0004] Furthermore, although some solutions have attempted to introduce large language models (LLMs) to assist in semantic understanding or task instruction generation, they are usually only used as interface-level tools and lack a systematic management mechanism. In particular, there is no effective organization strategy for tool information input, which can easily lead to problems such as model input overload, inference bias, and inaccurate task matching, thus limiting the practicality and scalability of large models in complex business systems. Summary of the Invention

[0005] This disclosure provides at least one data linkage display method and system based on natural language interaction. It integrates multimodal natural language understanding, intelligent task matching, and a front-end / back-end linkage control mechanism to achieve efficient, intelligent, and low-latency interactive control between mobile devices and a large data dashboard platform, while improving the overall scalability and stability of the system. By introducing multimodal natural language input methods such as voice and text, and combining them with a large language model for command understanding and tool selection, even non-professional users can quickly control the data dashboard using natural language, significantly lowering the barrier to entry. After completing data processing, the back-end proactively pushes control commands to the front-end of the dashboard without requiring users to make further requests, significantly improving interface response speed. This is particularly suitable for scenarios with high real-time requirements, such as parking space shortage warnings. Furthermore, by categorizing operation tools by function and combining them with a dynamic loading mechanism, relevant tool information is only input to the large language model when needed, avoiding excessively long contexts from one-time input and improving the system's adaptability in complex business environments.

[0006] This disclosure provides a data linkage display method based on natural language interaction, applied to an intelligent agent server in a data linkage display system. The data linkage display system further includes a mobile terminal, a large language model, a screen backend server, and a screen frontend server. The method includes:

[0007] The mobile terminal receives natural language commands input by the user, performs semantic analysis on the natural language commands, and determines the type of task indicated by the natural language commands.

[0008] Obtain a set of automated large-screen operation tools and target prompt words that match the task type, and input the set of automated large-screen operation tools, the natural language instructions, and the target prompt words into the large language model;

[0009] The system receives the target operation tool selected by the large language model in the large screen automatic operation tool set according to the natural language instructions and the target prompt words, as well as the input parameters corresponding to the target operation tool, and calls the service interface of the screen backend server corresponding to the target operation tool to generate target result data.

[0010] The target result data is sent to the mobile terminal for real-time rendering. At the same time, the screen backend server constructs visual control instructions based on the target result data and pushes them to the screen frontend server to perform interface update operations.

[0011] In one optional implementation, the mobile terminal receives natural language commands input by the user, performs semantic analysis on the natural language commands, and determines the task type indicated by the natural language commands, specifically including:

[0012] The mobile terminal receives natural language commands input by the user via voice or text, and receives the natural language commands through a preset communication interface.

[0013] Semantic analysis is performed on the natural language instruction to determine the user intent indicated by the natural language instruction and the task type corresponding to the user intent based on the semantic content corresponding to the natural language instruction.

[0014] The task types include at least parking lot operation, parking lot vehicle management, and parking lot equipment.

[0015] In one optional implementation, a set of automated large-screen operation tools and target prompts matching the task type are obtained, and the set of automated large-screen operation tools, the natural language instructions, and the target prompts are input into the large language model, specifically including:

[0016] Based on the semantic content and the user's intent, the matching target prompt word is extracted from the preset prompt word database;

[0017] Select a target tool set that matches the task type from a set of preset large-screen automatic operation tools, and load the large-screen automatic operation tools included in the target tool set;

[0018] The large screen automatic operation tool, the user intent, and the target prompt word are input into the large language model to trigger the large language model to select the target operation tool from the large screen automatic operation tools according to the user intent and the target prompt word.

[0019] In one optional implementation, the target operating tool includes at least parking lot operation tools, parking lot vehicle management tools, and parking lot equipment tools;

[0020] The parking lot management tools are at least used to query parking lot revenue data and order data, and to analyze revenue trends and user payment status.

[0021] The parking lot vehicle management tools are at least used to query newly added parking lot and parking space information, and to analyze the traffic flow of vehicles entering and exiting the parking lot, traffic flow trends, and vehicle attributes.

[0022] The parking lot equipment tools are at least used to count the number of online parking lots and equipment alarm information, and to query equipment maintenance records.

[0023] In one optional implementation, the target result data is generated by calling the service interface corresponding to the target operation tool in the screen backend server, specifically including:

[0024] The target operation tool is loaded by calling the service interface in the screen backend server.

[0025] Based on the user intent, a target business logic is formed, and the order of the target operation tools is arranged according to the target business logic;

[0026] The target operation tool is triggered to generate the target result data according to the target business logic.

[0027] In one optional implementation, the operation of the screen backend server includes at least the specific parameter values, data range, display mode, or analysis dimension required to execute the target operation tool;

[0028] The visual control instructions include at least icon type information, target display component identifier, visual parameters, and refresh action type.

[0029] This disclosure also provides a data linkage display system based on natural language interaction, including a mobile terminal, an intelligent agent server, a large language model, a screen backend server, and a screen frontend server.

[0030] The mobile terminal is used to receive natural language commands input by the user and send them to the intelligent agent server;

[0031] The intelligent agent server is used to perform semantic analysis on the natural language instructions to determine the task type indicated by the natural language instructions; obtain a set of large-screen automatic operation tools and target prompt words that match the task type; and input the set of large-screen automatic operation tools, the natural language instructions and the target prompt words into the large language model.

[0032] The large language model is used to select a target operation tool in the large screen automatic operation tool set according to the natural language instruction and the target prompt word, and generate a target operation to execute the natural language instruction;

[0033] The intelligent agent server is also used to call the service interface of the target operation tool in the screen backend server to generate target result data; and send the target result data to the mobile terminal for real-time rendering;

[0034] The screen backend server is used to construct visual control instructions based on the target result data and push them to the screen frontend server to perform interface update operations.

[0035] This disclosure also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the above-described data linkage display method based on natural language interaction, or any possible implementation of the above-described data linkage display method based on natural language interaction, are executed.

[0036] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described data linkage display method based on natural language interaction, or any possible implementation of the above-described data linkage display method based on natural language interaction.

[0037] This disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-described data linkage display method based on natural language interaction, or any possible implementation steps of the above-described data linkage display method based on natural language interaction.

[0038] This disclosure provides a data linkage display method and system based on natural language interaction. It integrates multimodal natural language understanding, intelligent task matching, and a front-end / back-end linkage control mechanism to achieve efficient, intelligent, and low-latency interactive control between mobile devices and a large data dashboard platform, while improving the overall scalability and stability of the system. By introducing multimodal natural language input methods such as voice and text, and combining them with a large language model for command understanding and tool selection, even non-professional users can quickly control the large data dashboard using natural language, significantly lowering the barrier to entry. After completing data processing, the back-end proactively pushes control commands to the front-end of the large screen, eliminating the need for users to initiate further requests and significantly improving interface response speed. This is particularly suitable for scenarios with high real-time requirements, such as parking space shortage warnings. Furthermore, by categorizing operation tools by function and combining them with a dynamic loading mechanism, relevant tool information is only input into the large language model when needed, avoiding excessively long contextual information from a single input and improving the system's adaptability in complex business environments.

[0039] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0041] Figure 1 This diagram illustrates a data linkage display system based on natural language interaction provided in an embodiment of the present disclosure;

[0042] Figure 2 A flowchart of a data linkage display method based on natural language interaction provided in an embodiment of this disclosure is shown;

[0043] Figure 3 A schematic diagram of an intelligent agent server provided in an embodiment of this disclosure is shown;

[0044] Figure 4 A schematic diagram of an electronic device provided in an embodiment of the present disclosure is shown. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0046] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0047] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0048] Research has revealed two main issues. First, current data dashboard platforms largely rely on traditional interaction methods such as mouse clicks and touch controls, lacking natural language support. This makes it difficult to meet users' demands for efficient and intelligent human-computer interaction, especially in mobile scenarios or for non-professional users, where the operational threshold is high and efficiency is low. Second, existing systems are mostly in a passive response mode, requiring users to actively initiate operation requests on their terminals before the dashboard backend executes the corresponding data processing and display. This client-driven model has high response latency, making it difficult to meet the real-time requirements of some high-time-sensitivity business scenarios. Furthermore, although some solutions have attempted to introduce Large Language Models (LLMs) to assist in semantic understanding or task instruction generation, these are typically only used as interface-level tools, lacking a systematic management mechanism. In particular, the lack of an effective organization strategy for tool information input can easily lead to problems such as model input overload, inference bias, and inaccurate task matching, limiting the practicality and scalability of large models in complex business systems.

[0049] Based on the above research, this disclosure provides a data linkage display method and system based on natural language interaction. It integrates multimodal natural language understanding, intelligent task matching, and a front-end / back-end linkage control mechanism to achieve efficient, intelligent, and low-latency interactive control between mobile devices and data dashboard platforms, while improving the overall scalability and stability of the system. By introducing multimodal natural language input methods such as voice and text, and combining them with a large language model for command understanding and tool selection, even non-professional users can quickly control the data dashboard using natural language, significantly lowering the barrier to entry. After completing data processing, the back-end proactively pushes control commands to the front-end of the dashboard, eliminating the need for users to initiate further requests and significantly improving interface response speed. This is particularly suitable for scenarios with high real-time requirements, such as parking space shortage warnings. Furthermore, by categorizing operation tools by function and combining them with a dynamic loading mechanism, relevant tool information is only input into the large language model when needed, avoiding excessively long contexts from one-time input and improving the system's adaptability in complex business environments.

[0050] To facilitate understanding of this embodiment, a detailed description of a data linkage display system based on natural language interaction disclosed in this disclosure will be provided first. (See also...) Figure 1The diagram shown is a schematic of a data linkage display system based on natural language interaction provided in an embodiment of this disclosure.

[0051] like Figure 1 As shown, the data linkage display system based on natural language interaction includes a mobile terminal, an intelligent agent server, a large language model, a screen backend server, and a screen frontend server.

[0052] Specifically, the mobile device receives natural language commands input by the user and sends them to the intelligent agent server. The intelligent agent server performs semantic analysis on the natural language commands to determine the task type indicated by the commands. It then acquires a set of automated operation tools for the large screen that matches the task type, along with target prompts, and inputs these tools, commands, and prompts into the large language model. The large language model selects the target operation tool from the set of automated operation tools based on the commands and prompts, and generates the target operation to execute the natural language commands. The intelligent agent server also calls the service interface of the corresponding target operation tool in the screen backend server to generate target result data. This target result data is then sent to the mobile device for real-time rendering. The screen backend server constructs visual control commands based on the target result data and pushes them to the screen frontend server to perform interface update operations.

[0053] Secondly, this disclosure first provides a detailed description of a data linkage display method based on natural language interaction, as disclosed in this embodiment. The execution entity of the data linkage display method based on natural language interaction provided in this disclosure is as follows: Figure 1 The intelligent agent server in the data linkage display system based on natural language interaction shown can be a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this data linkage display method based on natural language interaction can be implemented by a processor calling computer-readable instructions stored in memory.

[0054] See Figure 2 The diagram shows a flowchart of a data linkage display method based on natural language interaction provided in an embodiment of this disclosure. The method includes steps S101 to S104, wherein:

[0055] S101. Receive natural language instructions input by the user through the mobile terminal, perform semantic analysis on the natural language instructions, and determine the task type indicated by the natural language instructions.

[0056] In one embodiment of the present invention, the mobile terminal is used to receive natural language command input from the user, which may include voice or text. The user can issue natural language commands such as "Check today's parking lot revenue" or "Which parking lot has the most limited parking spaces" by clicking the voice button or input box in the mobile application.

[0057] In practice, after receiving the natural language instruction, the mobile terminal converts the speech content into a processable text instruction (when the input is in speech form) by calling the built-in or externally integrated speech-to-text module (ASR). Then, the natural language text instruction is sent to the intelligent agent server in the cloud for further processing through a preset API interface.

[0058] Here, after receiving the natural language instruction, the intelligent agent server performs semantic preprocessing on the instruction text, such as word segmentation, part-of-speech tagging, and entity recognition, based on the built-in semantic understanding engine (such as the NLU module) or by calling a large language model. Based on the task recognition model, it analyzes the semantic content of the instruction to determine the business objectives that the user is concerned about. Based on the semantic results, it extracts and determines the task type to which the instruction belongs.

[0059] For example, the instruction "View today's revenue ranking" is parsed as task type "parking lot operation", the instruction "Statistics on current parking space usage" is identified as task type "vehicle management", and the instruction "What is the status of equipment alarms" is identified as task type "equipment".

[0060] The task type identification result will serve as a prerequisite input for subsequent tool matching and service invocation, used to filter out large-screen automatic operation tools corresponding to the task type from the toolset, and assist in building further control strategies.

[0061] As one possible implementation, the mobile terminal receives natural language commands input by the user in the form of voice or text, and receives the natural language commands through a preset communication interface; semantic analysis is performed on the natural language commands, and the user intent indicated by the natural language commands and the task type corresponding to the user intent are determined based on the semantic content corresponding to the natural language commands; wherein, the task type includes at least parking lot operation, parking lot vehicle management and parking lot equipment.

[0062] In practice, the mobile terminal can be a mobile device with voice and text input capabilities, such as a smartphone or tablet. Users can issue natural language commands to the mobile terminal via voice (e.g., saying "Statistics on today's parking space occupancy rate") or text (e.g., typing "Query today's parking lot revenue").

[0063] Specifically, the mobile device receives natural language commands input by the user via voice or text and sends them to the cloud-based intelligent agent server through a pre-defined communication interface (such as a RESTful API under HTTPS). This communication interface supports structured data transmission, ensuring stable transmission of user commands in different network environments.

[0064] Here, after receiving the natural language instruction, the agent server calls the semantic parsing module to analyze it. The semantic analysis process includes steps commonly used in natural language processing (NLP) techniques, such as lexical analysis, syntactic analysis, named entity recognition (NER), keyword extraction, and intent recognition, aiming to extract the user's intent and related elements from the user's expression.

[0065] For example, the user intent of the instruction "Which parking lot has the highest revenue today?" is "to query the revenue ranking of parking lots"; the user intent of the instruction "How many devices are currently offline?" is "to check the online status of devices"; and the user intent of the instruction "to show the locations with the highest parking traffic" is "to analyze traffic flow trends".

[0066] Furthermore, based on the identified user intent, the system matches it with preset task classification rules to determine the task type to which the user intent belongs. Specifically, the system classifies task types into at least the following three categories: parking lot operation: such as parking lot revenue statistics, order analysis, payment method distribution, etc.; parking lot vehicle management: such as parking space status monitoring, vehicle inflow and outflow analysis, traffic flow trend display, etc.; parking lot equipment: such as equipment online rate statistics, equipment fault alarm analysis, equipment maintenance information viewing, etc. This task type will serve as one of the key inputs for subsequent toolset selection and large language model inference, thereby enabling the system to efficiently understand user instructions and automate the execution of business processes.

[0067] S102. Obtain a set of large-screen automatic operation tools and target prompt words that match the task type, and input the set of large-screen automatic operation tools, the natural language instructions and the target prompt words into the large language model.

[0068] In one embodiment of the present invention, after the intelligent agent server determines the task type corresponding to the user's natural language command, the system will retrieve a matching set of large-screen automated operation tools from a preset tool library based on the task type. The toolsets are organized and managed by category, such as: business toolsets, vehicle management toolsets, equipment toolsets, etc., with each toolset including multiple automated tools supporting specific operations.

[0069] For example, if the identified task type is "parking lot operation", the system will load a set of automated operation tools related to the operation task. The set of tools may include multiple automated operation tools such as "query revenue data", "generate revenue trend chart", and "TOP N parking lot revenue ranking". Each tool is equipped with meta-information such as function description, callable backend interface, and required parameter description.

[0070] Simultaneously, the system will also extract target prompt words matching the currently identified user intent from a pre-set prompt word template database. The design of these target prompt words aims to provide clearer contextual guidance for the large language model, improving the model's accuracy in understanding instructions and the efficiency of tool matching. These prompt words can be structured or semi-structured language templates, such as: "Please select the most suitable tool to display parking lot revenue trends based on the user's intent."

[0071] Subsequently, the system transmits the set of automated large-screen operation tools corresponding to the task type and the extracted target prompt words as input to the large language model. After receiving the input, the large language model, based on its own reasoning ability and combined with the semantic guidance information provided in the natural language instructions and prompt words, performs semantic matching and functional judgment on each tool in the toolset, automatically selects the target operation tool that best suits the user's needs, and generates structured instructions or calling parameters for performing the operation.

[0072] As one possible implementation, based on the semantic content and the user intent, a matching target prompt word is extracted from a preset prompt word database; a target tool set matching the task type is filtered from multiple preset large-screen automatic operation tool sets, and the large-screen automatic operation tools included in the target tool set are loaded; the large-screen automatic operation tool, the user intent, and the target prompt word are input into the large language model to trigger the large language model to select the target operation tool from the large-screen automatic operation tools according to the user intent and the target prompt word.

[0073] In practice, the system first retrieves target prompts highly relevant to the current task from a pre-set prompt database based on the parsed semantic content and user intent. These prompts can be structured task guides used to help the large language model accurately understand the correspondence between user intent and available tools. For example, if the user instruction is "View revenue trends for the past seven days," the matching prompt could be "Please select a chart tool from the following tools to display revenue trends."

[0074] Subsequently, the system filters the pre-set automated operation toolset library for large screens according to task type (e.g., operation, vehicle management, equipment monitoring) and extracts the target toolset that matches the current task type. Each toolset contains multiple automated operation tools, which have clear function definitions, API descriptions, and parameter templates. Taking "parking lot operation" as an example, the relevant toolset might include "query revenue data," "generate revenue trend charts," and "query order payment structure," etc.

[0075] The target operational tools include at least parking lot management tools, parking lot vehicle management tools, and parking lot equipment tools. Parking lot management tools should at least be used to query parking lot revenue data and order data, and analyze revenue trends and user payment status. Parking lot vehicle management tools should at least be used to query newly added parking lot and parking space information, and analyze entry vehicle flow, exit vehicle flow, traffic flow trends, and vehicle attributes. Parking lot equipment tools should at least be used to count the number of online parking lots and equipment alarm information, and query equipment maintenance records.

[0076] Here, after extracting and loading the target prompts and toolset, the system inputs the target prompts and all filtered information on the large-screen automated operation tools into the large language model. The input includes key metadata such as the name, function description, and calling format of each tool. Based on this input, the large language model performs semantic reasoning, combining the user intent expressed in the prompts, selecting the target operation tool from the provided toolset that best matches the current instruction's requirements, and outputting the related execution structure (such as tool identifier, calling parameters, and operation path).

[0077] S103. Receive the target operation tool selected by the large language model in the large screen automatic operation tool set according to the natural language instruction and the target prompt word, and the input parameters corresponding to the target operation tool generated for execution, and call the service interface of the corresponding target operation tool in the screen background server to generate target result data.

[0078] In this embodiment of the application, after loading the set of large-screen automatic operation tools corresponding to the target prompt words and the task type, the present invention further selects the operation tools and generates the input parameters corresponding to the target operation tools through a large language model.

[0079] Specifically, the intelligent agent server receives the processing results of the large language model. These results include: the target operation tool selected by the large language model from the loaded large-screen automatic operation toolkit based on the input target prompt words and available tool descriptions; and the specific target operation content generated to execute the user's natural language commands. This target operation may include a specific function call path, required parameters (such as query time range, parking lot identification, etc.), and desired output format.

[0080] Subsequently, the agent server extracts the identification information of the target operation tool based on the inference results of the large language model and calls the service interface corresponding to that operation tool in the background server. This service interface can adopt a RESTful API, gRPC, or other protocols that support structured calls, with clearly defined functional input parameter specifications and return result formats. During the call, the system fills the interface request with the parameter content generated from the large model output, thereby achieving accurate scheduling of the target operation tool.

[0081] Here, once the backend server receives the API call request, it can execute the corresponding data query, processing, or analysis tasks based on its internal business logic modules, thereby generating the final target result data. For example, if the target operation is "get the top 5 parking lots by revenue in the last seven days", the backend server will obtain the revenue details of the corresponding parking lot through the revenue database interface, calculate the ranking results, and generate structured data that can be used for frontend display.

[0082] The target result data can not only be used for front-end interface updates and display, but also be returned to mobile devices as system response information to achieve cross-terminal synchronous feedback and display, ensuring that users can seamlessly switch between mobile devices and data dashboards and obtain a consistent visualization experience.

[0083] It should be noted that the input parameters for executing the target operation tool include at least the specific parameter values, data range, display mode, or analysis dimension required to call the backend interface; the visualization control instructions include at least the icon type information, the target display component identifier, the visualization parameters, and the refresh action type.

[0084] As one possible implementation, the target operation tool is loaded by calling the service interface in the screen backend server; target business logic is formed according to the user intent; the order of the target operation tools is arranged according to the target business logic; and the target operation tool is triggered to generate the target result data according to the target business logic.

[0085] In practice, the intelligent agent server first calls the service interface corresponding to the target operation tool in the screen backend server to load the functional modules or services of the operation tool. The service interface can be an independently deployed microservice interface or a functional node under a unified scheduling framework. Through the transmission of interface parameters, the initialization and context configuration of the target tool are completed.

[0086] Here, after successfully loading the target operation tool, the system automatically constructs target business logic that matches the user intent obtained from the previous analysis. This target business logic guides the execution order, parameter passing rules, and data aggregation methods for multiple operation steps. For example, when the user intent is "analyze the traffic flow trend of vehicles entering and leaving the site," the system can generate a business process that includes "querying entry records → querying exit records → summarizing data → generating trend curves."

[0087] Subsequently, based on the target business logic, the intelligent agent server orchestrates the loaded target operation tools sequentially, that is, determines the invocation order and dependencies of each operation tool in the task flow. This orchestration process can employ a rule engine or a dynamic assembly mechanism based on process templates to achieve flexible and efficient operation flow construction.

[0088] Ultimately, the system triggers each target operation tool to execute specific tasks one by one according to the programmed business logic, including data querying, statistical calculation, data cleaning, chart generation, etc., until the entire task flow is completed. After execution, the outputs of each tool will be integrated to form structured target result data.

[0089] Here, the target result data will be returned as the system response result to the front-end display module or user operation terminal, realizing data linkage and intelligent display driven by natural language commands, which greatly improves the intuitiveness of user interaction and the level of intelligence of system response.

[0090] S104. The target result data is sent to the mobile terminal for real-time rendering. At the same time, the screen backend server constructs a visual control command based on the target result data and pushes it to the screen frontend server to perform an interface update operation.

[0091] In specific implementation, after obtaining the target result data, to achieve the real-time human-computer interaction and multi-terminal linked display, the agent server continues to execute the processing flow of result rendering and interface control. Specifically, the agent server first sends the generated target result data to the mobile device used by the user for operation. After receiving the target result data, the mobile device performs real-time parsing and graphical display of the data according to the preset rendering rules. This rendering process can be implemented based on technical frameworks such as HTML5, Vue, React Native, etc., ensuring that users can intuitively view the analysis results, data charts or text feedback content on the mobile device, thereby improving the usability and response efficiency of the operation.

[0092] Meanwhile, the agent server also synchronously transmits the target result data to the screen background server. The screen background server constructs a visual control instruction according to the received target result data, and the control instruction includes but is not limited to chart type, layer structure, refresh mode, interaction method, and interface component mapping information, etc.

[0093] Here, after the visual control instruction is constructed, the screen background server pushes the control instruction to the screen front-end server through the network. After receiving the control instruction, the screen front-end server triggers the large-screen display system to perform interface update operations, including data layer loading, visual component refreshing, animation switching, etc., so as to synchronously display the data view associated with the user's natural language instruction on the large screen.

[0094] In this way, through the above method, the present invention realizes the whole-process data linkage from natural language input to multi-terminal display, significantly improves the intelligence of human-computer interaction and the intuitiveness of visualization, and meets the visualization operation requirements in various scenarios such as parking lot management, equipment operation and maintenance, and business analysis.

[0095] This disclosure provides a data linkage display method based on natural language interaction, integrating multimodal natural language understanding, intelligent task matching, and a front-end and back-end linkage control mechanism to achieve efficient, intelligent, and low-latency interactive control between mobile terminals and data dashboard platforms, while improving the overall scalability and stability of the system. By introducing multimodal natural language input methods such as voice and text, and combining them with a large language model for command understanding and tool selection, even non-professional users can quickly complete control operations on the data dashboard using natural language, significantly lowering the barrier to entry. After completing data processing, the back-end proactively pushes control commands to the front-end of the dashboard, eliminating the need for users to initiate further requests and significantly improving interface response speed. This is particularly suitable for scenarios with high real-time requirements, such as parking space shortage warnings. Furthermore, by categorizing operation tools by function and combining them with a dynamic loading mechanism, relevant tool information is only input into the large language model when needed, avoiding excessively long contexts from one-time input and improving the system's adaptability in complex business environments.

[0096] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0097] Based on the same inventive concept, this disclosure also provides an intelligent agent server, which is the execution subject corresponding to the data linkage display method based on natural language interaction. Since the principle of the intelligent agent server in this disclosure is similar to the above-mentioned data linkage display method based on natural language interaction in this disclosure, the implementation of the intelligent agent server can refer to the implementation of the method, and the repeated parts will not be described again.

[0098] Please see Figure 3 , Figure 3 This is a schematic diagram of an intelligent agent server provided in an embodiment of this disclosure. Figure 3 As shown in the embodiments of this disclosure, the intelligent agent server 300 includes:

[0099] The instruction acquisition module 310 is used to receive natural language instructions input by the user through the mobile terminal, perform semantic analysis on the natural language instructions, and determine the task type indicated by the natural language instructions.

[0100] The data analysis module 320 is used to acquire a set of large-screen automatic operation tools and target prompt words that match the task type, and input the set of large-screen automatic operation tools, the natural language instructions and the target prompt words into the large language model.

[0101] The operation execution module 330 is used to receive the target operation tool selected by the large language model in the large screen automatic operation tool set according to the natural language instruction and the target prompt word, as well as the input parameters corresponding to the target operation tool, and call the service interface of the screen backend server corresponding to the target operation tool to generate target result data.

[0102] The linkage display module 340 is used to send the target result data to the mobile terminal for real-time rendering, and at the same time, the screen backend server constructs visual control instructions based on the target result data and pushes them to the screen frontend server to perform interface update operations.

[0103] The processing flow of each module in the intelligent agent server and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0104] Corresponding to Figure 1 In addition to the data linkage display method based on natural language interaction, this disclosure also provides an electronic device 400, such as... Figure 4 The diagram shown is a structural schematic of an electronic device 400 provided in an embodiment of this disclosure, including:

[0105] Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including main memory 421 and external memory 422; the main memory 421, also called internal memory, is used to temporarily store the computational data in processor 41, as well as the data exchanged with external memory 422 such as hard disk. Processor 41 exchanges data with external memory 422 through main memory 421. When the electronic device 400 is running, processor 41 and memory 42 communicate through bus 43, enabling processor 41 to execute... Figure 1 The steps of the data linkage display method based on natural language interaction.

[0106] This disclosure also provides a computer-readable storage medium storing a computer program. When a processor executes the computer program, it performs the steps of the data linkage display method based on natural language interaction described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0107] This disclosure also provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they can perform the steps of the data linkage display method based on natural language interaction described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0108] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0112] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data linkage display method based on natural language interaction, characterized in that, An intelligent agent server is applied to a data-linked display system, which further includes a mobile terminal, a large language model, a screen backend server, and a screen frontend server. The method includes: The mobile terminal receives natural language commands input by the user, performs semantic analysis on the natural language commands, and determines the type of task indicated by the natural language commands. Obtain a set of automated large-screen operation tools and target prompt words that match the task type, and input the set of automated large-screen operation tools, the natural language instructions, and the target prompt words into the large language model; The system receives the target operation tool selected by the large language model in the large screen automatic operation tool set according to the natural language instructions and the target prompt words, as well as the input parameters corresponding to the target operation tool, and calls the service interface of the screen backend server corresponding to the target operation tool to generate target result data. The target result data is sent to the mobile terminal for real-time rendering. At the same time, the screen backend server constructs visual control instructions based on the target result data and pushes them to the screen frontend server to perform interface update operations.

2. The method according to claim 1, characterized in that, The mobile terminal receives natural language commands input by the user, performs semantic analysis on the natural language commands, and determines the task type indicated by the natural language commands, specifically including: The mobile terminal receives natural language commands input by the user via voice or text, and receives the natural language commands through a preset communication interface. Semantic analysis is performed on the natural language instruction to determine the user intent indicated by the natural language instruction and the task type corresponding to the user intent based on the semantic content corresponding to the natural language instruction. The task types include at least parking lot operation, parking lot vehicle management, and parking lot equipment.

3. The method according to claim 2, characterized in that, Obtain a set of automated large-screen operation tools and target prompts that match the task type, and input the set of automated large-screen operation tools, the natural language instructions, and the target prompts into the large language model, specifically including: Based on the semantic content and the user's intent, the matching target prompt word is extracted from the preset prompt word database; Select a target tool set that matches the task type from a set of preset large-screen automatic operation tools, and load the large-screen automatic operation tools included in the target tool set; The large screen automatic operation tool, the user intent, and the target prompt word are input into the large language model to trigger the large language model to select the target operation tool from the large screen automatic operation tools according to the user intent and the target prompt word.

4. The method according to claim 1, characterized in that, The target operating tools include at least parking lot operation tools, parking lot vehicle management tools, and parking lot equipment tools; The parking lot management tools are at least used to query parking lot revenue data and order data, and to analyze revenue trends and user payment status. The parking lot vehicle management tools are at least used to query newly added parking lot and parking space information, and to analyze the traffic flow of vehicles entering and exiting the parking lot, traffic flow trends, and vehicle attributes. The parking lot equipment tools are at least used to count the number of online parking lots and equipment alarm information, and to query equipment maintenance records.

5. The method according to claim 2, characterized in that, The process of generating target result data by calling the service interface of the corresponding target operation tool in the screen backend server specifically includes: The target operation tool is loaded by calling the service interface in the screen backend server. Based on the user intent, a target business logic is formed, and the order of the target operation tools is arranged according to the target business logic; The target operation tool is triggered to generate the target result data according to the target business logic.

6. The method according to claim 1, characterized in that: The operation of the screen backend server includes at least the specific parameter values, data range, display mode, or analysis dimension required to execute the target operation tool; The visual control instructions include at least icon type information, target display component identifier, visual parameters, and refresh action type.

7. A data linkage display system based on natural language interaction, characterized in that, This includes mobile devices, intelligent agent servers, large language models, screen backend servers, and screen frontend servers. The mobile terminal is used to receive natural language commands input by the user and send them to the intelligent agent server; The intelligent agent server is used to perform semantic analysis on the natural language instructions to determine the task type indicated by the natural language instructions; Obtain a set of automated large-screen operation tools and target prompt words that match the task type, and input the set of automated large-screen operation tools, the natural language instructions, and the target prompt words into the large language model; The large language model is used to select a target operation tool in the large screen automatic operation tool set according to the natural language instruction and the target prompt word, and generate a target operation to execute the natural language instruction; The intelligent agent server is also used to call the service interface of the target operation tool in the screen backend server to generate target result data; The target result data is sent to the mobile device for real-time rendering. The screen backend server is used to construct visual control instructions based on the target result data and push them to the screen frontend server to perform interface update operations.

8. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the data linkage display method based on natural language interaction as described in any one of claims 1 to 6 are performed.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the data linkage display method based on natural language interaction as described in any one of claims 1 to 6.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the data linkage display method based on natural language interaction as described in any one of claims 1 to 6.

Images