Natural language driven situation awareness control method, system and medium based on task intent

Abstract

The application discloses a natural language driven situation awareness control method and system based on task intention, and a medium; relates to the field of artificial intelligence and man-machine interaction technology; generates a natural language template library and a task intention field table based on three-table knowledge; and constructs an interface-data double-mapping knowledge graph by taking the three-table knowledge as a construction basis, the natural language template library and the task intention field table as retrieval and rearrangement tools, then performs semantic retrieval and reference resolution to generate structured intention data; under the premise of not depending on a fixed layout, parses user instructions into task intentions, realizes stable anchoring of interface objects by adopting semantic positioning, and automatically compiles the task intentions into system API calls or interface event sequences; meanwhile, the availability and compliance of execution are guaranteed through a trusted execution scoring and a closed-loop mechanism of minimizing clarification, so that the operation complexity is effectively reduced, and the response efficiency and intelligent interaction level of the situation awareness system are improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and human-computer interaction technology, specifically to a natural language-driven situational awareness control method, system, and medium based on task intent. Background Technology

[0002] Situational awareness systems are widely used in urban management, emergency response, and command and dispatch scenarios. Their core functions include real-time monitoring, data analysis, and integrated command and dispatch to help decision-makers quickly understand and respond to complex and dynamic business environments. Existing systems typically integrate multiple topics, panels, and numerous controls. The interface contains hundreds or even thousands of buttons, switches, dropdown filters, and interactive cards, with interconnected relationships between different panels. When performing continuous operations such as "query—locate—link—export / dispatch," staff need to frequently switch between multiple topics and interface objects, search, and set parameters item by item. This results in high learning and usage costs, and the error rate and delay risks are particularly prominent in high-pressure duty scenarios.

[0003] However, the existing interactive methods still have obvious shortcomings: (1) Most solutions are based on static mapping of "controls / components" (such as relying on fixed positions, DOM paths or control IDs), which are prone to failure when the interface is redesigned or the layout is changed; (2) There is a lack of unified expression for "tasks", which cannot reliably decompose a user's natural language into an executable operation micro-flow (macro), resulting in the need for manual connection of cross-panel and cross-topic linkage steps; (3) There is a lack of semantic alignment and evidence-based positioning mechanism between interface objects and business indicators / standards, making it difficult to ensure that "the right object is found"; (4) The lack of quantifiable assessment and gating of data freshness QoS and role or permission compliance makes it difficult to ensure execution reliability in emergency scenarios; (5) Existing large language model-driven configuration / visualization control ideas usually focus on directly mapping natural language into component control commands or configuration files to create or modify charts and interface attributes, but pay insufficient attention to issues such as "semantic positioning of interface objects, arrangement of cross-panel operation micro-processes, and reliable execution under data freshness QoS and permission constraints" in multi-topic situational systems, making it difficult to meet the requirements of low learning cost, low latency and high stability in practical scenarios. Summary of the Invention

[0004] The technical problem this invention aims to solve is as follows: With the development of artificial intelligence and natural language processing technologies, large language models possess the ability to understand user intent and decompose tasks. However, in multi-topic situational awareness systems, there is still a lack of a universal method that can stably convert natural language into executable operation sequences across panels and controls, and reliably operate under data freshness and permission constraints. Existing methods rely on fixed layouts or single component mappings, which are prone to failure when the interface is redesigned, parameter definitions differ, or data is temporarily unavailable, making it difficult to meet the requirements of high reliability and low latency human-computer collaboration. The purpose of this invention is to provide a natural language-driven situational awareness control method, system, and medium based on task intent. Without relying on a fixed layout, it parses user instructions into task intents that include topics, targets, actions, and parameters. Semantic localization is used to achieve stable anchoring of interface objects, and the task intents are automatically compiled into system API calls or interface event sequences. At the same time, a closed-loop mechanism of trusted execution scoring and minimal clarification ensures the availability and compliance of execution, thereby effectively reducing operational complexity and improving the response efficiency and intelligent interaction level of the situational awareness system.

[0005] This invention is achieved through the following technical solution:

[0006] This solution provides a natural language-driven situational awareness control method based on task intent, including:

[0007] Acquire sensing data and extract three tables of knowledge from the sensing data, including: an object table, a capability table, and a parameter table;

[0008] Based on the knowledge of the three tables, a natural language template library and a task intent field table are generated; and using the knowledge of the three tables as the foundation, and the natural language template library and the task intent field table as retrieval and rearrangement tools, an interface-data dual mapping knowledge graph is constructed.

[0009] In response to natural language instructions, semantic retrieval and referential resolution are performed based on a natural language template library, a task intent field table, and an interface-data dual-mapping knowledge graph table to generate structured intent data; the structured intent data includes task intent and a location candidate set;

[0010] The task intents in the structured intent data are compiled into operation micro-flows, and an operation instruction format list is generated;

[0011] Execute the instructions step by step according to the list of operation instructions, and return a summary of the execution results;

[0012] A reliable execution score is then applied to the aggregated execution results.

[0013] A further optimized solution is that the perceived data includes: front-end structure definition document, front-end interface description, front-end build artifacts, and interface operation telemetry logs;

[0014] The front-end build artifacts include: DOM tree, component metadata, accessibility semantic attributes, and stable selector identifiers.

[0015] A further optimization scheme is that the object table is the target master data of the interface, including: name, alias, tag text, indicator name, unit of measurement, topic, and visibility conditions;

[0016] The capability table is a list of executable actions, including: action name, target type, invocation method, parameter mode, preconditions, postconditions, linkage range, and influence domain;

[0017] The parameter table is a parameter specification, including: type, value range, default value, verification rules, and data caliber identifier.

[0018] A further optimization scheme is that the task intent field table defines a set of fields, which includes: topic, goal, action, parameters, data freshness, role or permission, and interaction constraints.

[0019] A further optimization scheme is that the task intent definition includes: topic, objective, action, parameters, data freshness, role or permission, and interaction constraints;

[0020] The localization candidate set is defined as including: target object identifier, localization confidence, and semantic localization evidence vector.

[0021] A further optimized solution is that the method for generating the structured intent data includes:

[0022] Obtain natural language instructions and perform layered parsing of the text of the natural language instructions:

[0023] Based on the natural language template library and the task intent field table, candidate values ​​for the topics, objectives, actions, and parameters are first extracted from the natural language instructions.

[0024] By combining the interface-data dual-mapping knowledge graph for semantic retrieval and reference resolution, user terms in natural language instructions are mapped to a controlled vocabulary and parameter enumeration, and interface objects consistent with the target semantics are retrieved from the object table to generate semantic localization evidence vectors.

[0025] A further optimized solution involves compiling the intents in the structured intent data into operational micro-flows and generating a list of operational instruction formats; including the following method:

[0026] The execution plan engine compiles the task intent into operation micro-flows and generates a list of operation instruction formats to be executed in sequence. Each operation instruction format list contains at least the following fields: target object identifier, action identifier, parameters, preconditions, post-validation, rollback strategy, timeout, and idempotent token.

[0027] The execution plan engine selects the optimal execution channel based on the capability table. The selection rules include: prioritizing the assembly of API requests through the native interface adapter; and automatically switching to the interface event adapter when the target only supports UI interaction.

[0028] A further optimized solution involves executing the operation instructions step-by-step according to a list of formatted instructions and returning a summary of the execution results; including:

[0029] The execution plan engine uses the trusted execution score of the scheduling plan as a list of gating execution operation instruction formats;

[0030] Before each action begins, the execution plan engine checks the pre parameters and automatically fills in any missing parameters; the pre parameters include object visibility, required role / permission, and data freshness QoS.

[0031] After the corresponding adapter is invoked to complete the action, the action result is validated in real time using the post parameters; the post parameters include: changes in the state of the interface object, the structure and quantity of the returned data, and matching of the prompt message;

[0032] If the verification fails or times out, the fallback path will be initiated.

[0033] After all actions are completed, return a summary of the execution results.

[0034] A further optimization is that the method for obtaining the trusted execution score includes:

[0035] Trusted execution scoring is achieved by nonlinearly fusing semantic matching, interface location, data availability, and permission compliance based on a trusted execution scoring gating mechanism.

[0036] This solution also provides a task intent-based natural language-driven situational awareness control system to implement the aforementioned task intent-based natural language-driven situational awareness control method; the system includes:

[0037] The preprocessing module is used to acquire the perception data and extract three tables of knowledge from the perception data, which include: an object table, a capability table, and a parameter table.

[0038] The knowledge generation module is used to generate a natural language template library and a task intent field table based on the knowledge of the three tables; and to construct an interface-data dual mapping knowledge graph using the knowledge of the three tables as the foundation and the natural language template library and the task intent field table as retrieval and rearrangement tools.

[0039] The retrieval and resolution module is used to respond to natural language commands by performing semantic retrieval and referential resolution based on a natural language template library, task intent fields, and an interface-data dual-mapping knowledge graph table to generate structured intent data; the structured intent data includes task intent and a location candidate set;

[0040] The compilation module is used to compile the task intents in the structured intent data into operation micro-flows and generate a list of operation instruction formats;

[0041] The execution module is used to execute the operation instructions step by step according to the list of operation instructions and return a summary of the execution results.

[0042] This solution also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, can implement the task intent-based natural language-driven situational awareness control method described above.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] 1. The natural language-driven situational awareness control method, system, and medium based on task intent provided by this invention, without relying on a fixed layout, parses user instructions into task intents containing topics, targets, actions, and parameters, uses semantic localization to achieve stable anchoring of interface objects, and automatically compiles task intents into system API calls or interface event sequences; at the same time, it ensures the availability and compliance of execution through a closed-loop mechanism of trusted execution scoring and minimal clarification, thereby effectively reducing operational complexity and improving the response efficiency and intelligent interaction level of the situational awareness system.

[0045] 2. The task intent-based natural language-driven situational awareness control method, system and medium provided by this invention are different from the existing approach of "directly mapping natural language into single component commands". Taking task intent as the core, it combines interface-data dual mapping knowledge graph and semantic localization evidence vector to achieve layout-independent target localization, and completes continuous operation orchestration by operation micro-flows, which significantly reduces the learning and usage costs caused by interface complexity.

[0046] 3. The natural language-driven situational awareness control method, system, and medium based on task intent provided by this invention compiles the task intent into the shortest executable path through an execution plan engine, prioritizes the use of native interface adapters, and automatically switches the interface event adapter while maintaining idempotent token control when missing. With the addition of parameter normalization and default value completion, the total latency from "expressing requirements" to "completing linkage, exporting, and dispatching" can be significantly shortened, reducing the risk of manual searching across panels and erroneous operations.

[0047] 4. The task intent-based natural language-driven situational awareness control method, system, and medium provided by this invention rely on a three-table system of object table, capability table, and parameter table, as well as a natural language template library. New topics or new systems can be reused simply by importing control capabilities and parameter specifications. At the same time, it provides unified constraints on data freshness and role permissions, and is compatible with multiple interface types such as city operation screens, duty seats, and Web, possessing versatility and scalability.

[0048] 5. The natural language-driven situational awareness control method, system and medium based on task intent provided by this invention, through trusted execution scoring and a gated-clarification-execution closed loop of minimizing clarification, automatically provides alternative paths or security degradation and records operation logs when there is positioning ambiguity, data unavailability or restricted permissions, thereby achieving high robustness and full-link auditability.

[0049] 6. The natural language-driven situational awareness control method, system, and medium based on task intent provided by this invention, based on a semantic adaptation optimization mechanism, incrementally learns the natural language template library, interface-data dual mapping knowledge graph, and measurement parameters, and precipitates high-frequency success processes into reusable operation macros that can be directly called in similar instructions; it automatically evolves with business updates, reduces maintenance costs, and continuously improves recognition accuracy and interaction stability. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0051] Figure 1 This is a schematic diagram of a natural language-driven situational awareness control method based on task intent.

[0052] Figure 2 A schematic diagram illustrating the process of generating a knowledge graph that maps three tables of knowledge to the interface and data.

[0053] Figure 3This is a schematic diagram of the structured intent data generation process;

[0054] Figure 4 This is a diagram illustrating the execution process of the operation instruction format list;

[0055] Figure 5 This is a schematic diagram of the structure of a natural language-driven situational awareness control system based on task intent. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0057] With the development of artificial intelligence and natural language processing technologies, large language models have the ability to understand user intent and decompose tasks. However, in multi-topic situational awareness systems, there is still a lack of a general method that can stably convert natural language into executable operation sequences across panels and controls, and reliably run under data freshness and permission constraints. In view of this, this solution provides the following embodiments to solve the above-mentioned technical problems:

[0058] Example 1: This example provides a natural language-driven situational awareness control method based on task intent, such as... Figure 1 As shown, it includes:

[0059] Step 1, as follows Figure 2 As shown, the system acquires perception data and extracts three tables of knowledge from the perception data, which include: an object table, a capability table, and a parameter table.

[0060] The sensing data includes: front-end structure definition document, front-end interface description, front-end build artifacts and interface operation telemetry logs;

[0061] The front-end build artifacts include: DOM tree, component metadata, accessibility semantic attributes, and stable selector identifiers.

[0062] Step 2: Generate a natural language template library and a task intent field table based on the knowledge of the three tables; and construct an interface-data dual mapping knowledge graph using the knowledge of the three tables as the foundation and the natural language template library and the task intent field table as retrieval and rearrangement tools.

[0063] The object table OTB contains the interface target master data, including: name, alias, label text, indicator name, unit of measurement, topic, and visibility conditions;

[0064] The Capability Table (ACT) is a list of executable actions, including: action name, target type, invocation method (e.g., API / script / shortcut key), parameter mode, preconditions, postconditions, linkage scope, and affected domain.

[0065] The Parameter Table (PRM) is a parameter specification that includes: type, enumeration / value range, default value, validation rules, and data caliber identifier.

[0066] For each interface object, a layout-independent stable identifier (view identifier VID, object identifier OID) and structural fingerprint are generated to construct an interface-data dual mapping knowledge graph UDKG, which serves as the foundational knowledge for mapping natural language and system functions.

[0067] Based on the knowledge of three tables—OTB (Object Table), ACT (Capability Table), and PRM (Parameter Table)—the Natural Language Template Library (NLT) and the Task Intent Field Table (TTI-F) are generated offline.

[0068] The Task Intent Fields (TTI-F) table defines the following set of fields: Topic, Target, Action, Params, QoS (Quality of Service), Role or Permission, and Constraints.

[0069] The enumerations, value ranges, and default values ​​from the parameter table are written into placeholders in the natural language template library to form constrained and populated template instances. Simultaneously, a semantic localization evidence vector (SEV) is generated for each target for subsequent semantic localization and confidence calculation. This results in a bidirectional mapping library of "example-template-parameter," which is then compiled into a micro-process for subsequent operations. Optionally, the natural language template library and the thesaurus are continuously updated based on online interaction logs.

[0070] Step 3: In response to natural language instructions, semantic retrieval and referential resolution are performed based on the natural language template library, task intent field table, and interface-data dual mapping knowledge graph table to generate structured intent data; the structured intent data includes task intent and location candidate set;

[0071] The Task Intent Field Table (TTI-F) defines a set of fields, including: Topic, Target, Action, Parameter, Data Freshness, Role or Permission, and Interaction Constraints. Structured intent data serves as the intermediate decision-making basis for function calls, stably mapping natural language to the correct interface objects and their executable capabilities without relying on a fixed page layout. This facilitates the subsequent compilation of Task Intent TTIs into operational micro-flows, enabling automatic scheduling and precise triggering across panels and topics. Example: The Task Intent TTI for "Display the top ten most congested traffic items this week and export to Excel" can be parsed as follows: Topic = Traffic Operation, Target = Congestion Ranking Table, Action = Filter + Sort + Export, Data Freshness QoS = Online, Role or Permission = Dispatch Desk, Interaction Constraints = Shortest Interaction Path.

[0072] like Figure 3 As shown, after receiving the natural language command input by the user, the text of the natural language command is parsed hierarchically, and a task intent (TTI) and a target location candidate set are generated according to the process of "template matching - semantic retrieval - parameter normalization - target localization". Specifically, based on the natural language template library (NLT) and the task intent field table (TTI-F), candidate values ​​for fields such as topic, target, action, and parameters are first extracted from the command; then, semantic retrieval and referential resolution are performed using the interface-data dual mapping knowledge graph (UDKG), mapping user language (including aliases and colloquial expressions) to a controlled vocabulary and parameter enumeration; subsequently, the normalization of parameter units / ranges and the completion of default values ​​are completed, and interface objects with semantic consistency with "target" are retrieved from the object table (OTB), generating semantic localization evidence vectors (SEVs) (such as text similarity, unit of measurement matching, stable selector identifier matching, structural fingerprint consistency, and historical click prior) and their confidence scores, which are used for subsequent executability evaluation and minimization clarification.

[0073] Step four, as Figure 4 As shown, the intents in the structured intent data are compiled into operation micro-flows, and an operation instruction format list is generated;

[0074] The task intent definition includes: topic, objective, action, parameters, data freshness, role or permission, and interaction constraints;

[0075] The localization candidate set is defined as including: target object identifier, localization confidence, and semantic localization evidence vector.

[0076] The method for generating the structured intent data includes:

[0077] Obtain natural language instructions and perform layered parsing of the text of the natural language instructions:

[0078] Based on the natural language template library and the task intent field table, candidate values ​​for the topics, objectives, actions, and parameters are first extracted from the natural language instructions.

[0079] By combining the interface-data dual-mapping knowledge graph for semantic retrieval and referential resolution, user terms (including aliases and colloquial expressions) in natural language instructions are mapped to a controlled vocabulary and parameter enumeration. Interface objects that are consistent with the target semantics are retrieved from the object table to generate semantic localization evidence vectors.

[0080] The step of compiling the intents in the structured intent data into operation micro-flows and generating a list of operation instruction formats includes the following methods:

[0081] The execution plan engine compiles the task intent into operation micro-flows and generates a list of operation instruction formats to be executed in sequence. Each operation instruction format list contains at least the following fields: target object identifier, action identifier, parameters, preconditions, post-validation, rollback strategy, timeout, and idempotent token.

[0082] The execution plan engine selects the optimal execution channel based on the capability table. The selection rules include: prioritizing the assembly of API requests through the native interface adapter; and automatically switching to the interface event adapter (such as atomic event sequences like focus, expand, select, input, and click) when the target only supports interface interaction.

[0083] Step 5: Follow the operation instruction format list step by step, and return a summary of the execution results; this step specifically includes:

[0084] The execution plan engine uses the trusted execution score of the scheduling plan as a list of gating execution operation instruction formats;

[0085] Before each action begins, the execution plan engine checks the pre parameters and automatically fills in any missing parameters; the pre parameters include object visibility, required roles or permissions, and data freshness QoS.

[0086] After the corresponding adapter is invoked to complete the action, the action result is validated in real time using the post parameters; the post parameters include: changes in the state of the UI object, the returned data structure and quantity, and matching of the prompt message;

[0087] If the verification fails or times out, a fallback path is initiated (e.g., switching to a similar target candidate, downgrading to the default time window, or using a read-only interface) to ensure the process is robust.

[0088] After all actions are completed, a summary of execution results is returned. The summary includes: overall status, step-by-step status codes, key information extraction (such as exported file identifier, alarm handling order number), and interface change summary; and "original instruction, task intent TTI, selected target object identifier OID, semantic location evidence vector SEV, operation instruction format OCF sequence, and execution log" are written to the operation log OpLog; through the above micro-process orchestration and gating verification, stable automatic scheduling and precise triggering across panels and topics are achieved from natural language.

[0089] Both native interface adapters and UI event adapters are presented externally as a unified list of operation instructions. To ensure executability, the Execution Plan Engine (EPE) uploads the scheduling plan for trusted execution scoring before scheduling, using this score as a gating mechanism. When the score meets the threshold, step-by-step scheduling begins; otherwise, minimization clarification or candidate replacement is triggered. This process does not rely on fixed layouts or coordinates, utilizing a UI-data dual-mapping knowledge graph (UDKG) and semantic localization evidence vectors (SEVs) to ensure stable location of the target object and its executable capabilities even after UI redesigns.

[0090] The method for obtaining the trusted execution score includes:

[0091] The trusted execution score is obtained by nonlinearly fusing semantic matching, interface location, data availability and permission compliance based on the trusted execution score gating mechanism, which summarizes the current execution results.

[0092] Specifically, to improve the executability of natural language commands in multi-topic, multi-panel environments, this solution introduces a Trusted Execution Score (CES) gating mechanism; the Trusted Execution Score nonlinearly fuses four types of signals: semantic matching, interface location, data availability, and permission compliance. [0, 1] 4 ;in, These represent the retrieval or re-ranking score of the semantic matching score (derived from the Natural Language Template Knowledge (NLT) and the Interface-Data Dual Mapping Knowledge Graph (UDKG)), the locator confidence score (derived from the Semantic Localization Evidence Vector (SEV)), the data freshness QoS score, and the permission consistency score, respectively; the components are arranged in descending order as follows: ,remember Let be the set of "the first i indicators", where The overall score uses CI points:

[0093]

[0094] in, The trusted execution score is represented by μ; μ represents the fuzzy measure FM defined on {semantic matching index, locator confidence index, data freshness index, and permission consistency index}, which satisfies monotonicity and normalization; optionally, μ({semantic matching index, locator confidence index})>μ({semantic matching index})+μ({locator confidence index}) is set to reflect the synergistic gain of "semantic + location";

[0095] The specific gating rules of the execution plan engine include:

[0096] The preset first threshold τ1 < the second threshold τ2;

[0097] When the trustworthy execution score S CES When the value is ≥τ2, execute directly;

[0098] When τ1 ≤ Trusted Execution S CES When <τ2, trigger the minimization of the clarified MCS and only supplement key fields (such as target objects or key parameters).

[0099] When the trustworthy execution score S CES When <τ1, a list of candidate intents (including target object identifier OID, evidence summary, and expected action) is presented for the user to select;

[0100] If a permission conflict or data unavailability is detected, a compliant alternative path or security downgrade solution will be provided first.

[0101] This solution also continuously records information such as "original instructions, task intent (TTI), candidate and selected target object identifiers (OID), semantic localization evidence vectors (SEV), operation instruction format (OCF) sequence, step-by-step verification results, and final feedback," forming an operation log (OpLog). The operation log (OpLog) is used for semantic adaptation optimization (SAO): iteratively updating the template weights and thesaurus of the Natural Language Template Library (NLT) based on real interaction samples, fine-tuning the entity aliases and calibrated mappings of the Interface-Data Dual Mapping Knowledge Graph (UDKG), calibrating the measurement parameters and reordering model of the Fuzzy Measure (FM), and generating a regression test set for offline acceptance. At the same time, high-frequency successful micro-processes are precipitated as reusable operation macros (Macro), which can be directly reused in subsequent similar instructions to reduce latency.

[0102] This solution achieves robust execution, auditable traceability, and continuous optimization driven by natural language through a closed loop of Trusted Execution Score (CES) gating mechanism, Minimized Clarification Score (MCS), and Semantic Adaptation Optimization Learning (SAO).

[0103] Example 2

[0104] This embodiment provides a task intent-based natural language-driven situational awareness control system to implement the task intent-based natural language-driven situational awareness control method described in Embodiment 1; such as Figure 5 As shown, the system includes:

[0105] The preprocessing module is used to acquire the perception data and extract three tables of knowledge from the perception data, which include: an object table, a capability table, and a parameter table.

[0106] The knowledge generation module is used to generate a natural language template library and a task intent field table based on the knowledge of the three tables; and to construct an interface-data dual mapping knowledge graph using the knowledge of the three tables as the foundation and the natural language template library and the task intent field table as retrieval and rearrangement tools.

[0107] The retrieval and resolution module is used to respond to natural language commands by performing semantic retrieval and referential resolution based on a natural language template library, task intent fields, and an interface-data dual-mapping knowledge graph table to generate structured intent data; the structured intent data includes task intent and a location candidate set;

[0108] The compilation module is used to compile the intents in the structured intent data into operation micro-flows and generate a list of operation instruction formats;

[0109] The execution module executes the operation instructions step by step according to the list of format and returns a summary of the execution results.

[0110] Example 3

[0111] This embodiment provides a computer-readable medium storing a computer program. The computer program, when executed by a processor, can implement the task intent-based natural language-driven situational awareness control method as described in Embodiment 1; specifically, it performs the following steps:

[0112] Step 1: Acquire the perception data and extract three tables of knowledge from the perception data. The three tables of knowledge include: object table, capability table and parameter table.

[0113] Step 2: Generate a natural language template library and a task intent field table based on the knowledge of the three tables; and construct an interface-data dual mapping knowledge graph using the knowledge of the three tables as the foundation and the natural language template library and the task intent field table as retrieval and rearrangement tools.

[0114] Step 3: In response to natural language instructions, semantic retrieval and referential resolution are performed based on the natural language template library, task intent field table, and interface-data dual mapping knowledge graph table to generate structured intent data; the structured intent data includes task intent and location candidate set;

[0115] Step 4: Compile the intents in the structured intent data into operation micro-flows and generate a list of operation instruction formats;

[0116] Step 5: Follow the operation instruction format list step by step and return the summary of execution results.

[0117] Specifically, this embodiment uses the situational awareness screen of the city operation management center as a typical scenario for illustration as follows:

[0118] Constructing a three-table knowledge and interface-data dual-mapping knowledge graph:

[0119] The large screen in the city operation management center includes multiple topics such as social governance, traffic management, culture and tourism, and environmental protection, and the interface integrates thousands of target objects and controls.

[0120] During the initialization phase, based on the front-end structure definition files and interface descriptions of each module, the object table OTB, capability table ACT, and parameter table PRM are extracted, and view identifier VID and object identifier OID are generated.

[0121] Taking the "Monthly Air Quality Report" as an example, the object table OTB records: Name = Air Quality Report, Alias ​​= Air Quality Trend, Report; Topic = Environmental Protection; Visibility Condition = Displayed after entering the Environmental Protection topic. The capability table ACT records: action_id = {Filter, Time Switch, Sort, Export}; Calling Method = {API, Interface Event}; Preconditions = {Target Visible, Read-Only Role Available}; Post-Validation = {Chart Rendering Completed, Data Points ≥ N}. The parameter table PRM records: Period ∈ {Current Day, Last 7 Days, Last 30 Days, Current Month, Last Month}; Format ∈ {png,xlsx}; Default Period = Current Month. Based on this, the system constructs a dual-mapping knowledge graph UDKG (interface-data), and generates a Natural Language Template Library (NLT) and a Task Intent Field Table (TTI-F), forming a bidirectional mapping library of "Example—Template—Parameter".

[0122] Semantic retrieval and referential resolution generate structured intent data: The command center operator issues the instruction via microphone: "Please retrieve the monthly air quality report from the environmental protection topic." The system converts the speech to text, extracts fields based on the Natural Language Template Library (NLT) and TTI-F, and retrieves the target from the Interface-Data Dual Mapping Knowledge Graph (UDKG) to obtain the task intent TTI: Topic=Environmental Protection, Target=Air Quality Report, Action=View, Params={period=Current Month}, QoS=Online, Role=Read-Only Access, Constraints=Shortest Interaction Path. Simultaneously, a location candidate set L={〈OID_a, Confidence 0.92, Evidence=[Name Match, Unit Match, Structural Fingerprint Consistency]〉, 〈OID_b, 0.61,…〉} is generated. The system performs a trusted execution score based on "semantic matching, location confidence, data freshness, and permission consistency"; in this scheme, S... CES If the threshold τ2 is greater than or equal to the threshold value, the system enters the execution phase without further clarification.

[0123] Generate a list of operation instruction formats and execute them sequentially: The Execution Plan Engine (EPE) compiles the Task Intent (TTI) into an Operation Microflow (OpFlow), generating a list of operation instruction formats (OCFs) to be executed in sequence. Example:

[0124] 1) OCF-1: target_oid=OID_a, action_id=Focus on topic, params={topic=Environmental Protection}, pre={Target Visible}, post={Topic Page Activation}, fallback={Direct Navigation to Topic Route}.

[0125] 2) OCF-2: target_oid=OID_a, action_id=view, params={}, pre={chart container visible}, post={rendering start}, fallback={refresh container}.

[0126] 3) OCF-3: target_oid=OID_a, action_id=time switch, params={period=current month}, pre={existing period enumeration}, post={chart data points ≥ N}, fallback={period=last 30 days}.

[0127] The Execution Plan Engine (EPE) prioritizes fulfilling native API requests; if a specific action lacks an API, it automatically switches to a UI event sequence (focus, expand, select). Post-validation is performed after each step; if a failure or timeout occurs, a fallback strategy is executed. After the process concludes, the system returns a summary of execution results and writes it to the Operation Log (OpLog), which includes the selected object identifier (OID), the operation instruction format (OCF) sequence, and key outputs (such as the exported file identifier).

[0128] The specific gating rules of the execution plan engine include:

[0129] If semantic retrieval yields multiple similar targets, or if it detects that period=month is unavailable in the current data source, then S CES Falling into τ1≤S CES In the interval <τ2, the system triggers a minimal clarification MCS, providing only necessary options, such as "Please select report granularity: Monthly / Annual / Current Status". After the user selects "Monthly Report", execution continues, and the "Original Instruction, Task Intent (TTI), Candidate and Selected Targets, Evidence Summary, Operation Instruction Format Sequence, and Verification Result" are written to the Operation Log (OpLog). The system optimizes the SAO mechanism based on semantic adaptation, using the Operation Log (OpLog) to update the synonym weights of the Natural Language Template Library (NLT) and the alias mappings of the Interface-Data Dual Mapping Knowledge Graph (UDKG), thus accumulating this instruction as a reusable operation macro for direct invocation in subsequent similar requests, reducing latency.

[0130] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A natural language-driven situational awareness control method based on task intent, characterized in that, include: Acquire sensing data and extract three tables of knowledge from the sensing data, including: an object table, a capability table, and a parameter table; Based on the knowledge of the three tables, a natural language template library and a task intent field table are generated; and using the knowledge of the three tables as the foundation, and the natural language template library and the task intent field table as retrieval and rearrangement tools, an interface-data dual mapping knowledge graph is constructed. In response to natural language instructions, semantic retrieval and referential resolution are performed based on a natural language template library, a task intent field table, and an interface-data dual-mapping knowledge graph table to generate structured intent data; the structured intent data includes task intent and a location candidate set; The task intents in the structured intent data are compiled into operation micro-flows, and an operation instruction format list is generated; The operation instructions are executed step by step according to the list of instruction formats, and the summary of execution results is returned.

2. The task intent-based natural language-driven situational awareness control method according to claim 1, characterized in that, The sensing data includes: front-end structure definition document, front-end interface description, front-end build artifacts and interface operation telemetry logs; The front-end build artifacts include: DOM tree, component metadata, accessibility semantic attributes, and stable selector identifiers.

3. The natural language-driven situational awareness control method based on task intent according to claim 1, characterized in that, The object table is the main data of the interface target, including: name, alias, tag text, indicator name, unit of measurement, topic, and visibility conditions; The capability table is a list of executable actions, including: action name, target type, invocation method, parameter mode, preconditions, postconditions, linkage range, and influence domain; The parameter table is a parameter specification, including: type, value range, default value, verification rules, and data caliber identifier.

4. The task intent-based natural language-driven situational awareness control method according to claim 1, characterized in that, The task intent field table defines a set of fields, which includes: topic, goal, action, parameters, data freshness, role or permission, and interaction constraints.

5. The task intent-based natural language-driven situational awareness control method according to claim 4, characterized in that, The task intent in the structured intent data includes: topic, target, action, parameters, data freshness, role or permission, and interaction constraints; the location candidate set in the structured intent data includes: target object identifier, location confidence, and semantic location evidence vector. The method for generating the structured intent data includes: Obtain natural language instructions and perform layered parsing of the text of the natural language instructions: Based on the natural language template library and the task intent field table, candidate values ​​for the topics, objectives, actions, and parameters are first extracted from the natural language instructions. By combining the interface-data dual-mapping knowledge graph for semantic retrieval and reference resolution, user terms in natural language instructions are mapped to a controlled vocabulary and parameter enumeration, and interface objects consistent with the target semantics are retrieved from the object table to generate semantic localization evidence vectors.

6. The task intent-based natural language-driven situational awareness control method according to claim 5, characterized in that, The task intents in the structured intent data are compiled into operation micro-flows, and an operation instruction format list is generated; Including methods: The task intent is compiled into operation micro-flows based on the execution plan engine, and a list of operation instruction formats to be executed in sequence is generated. Each operation instruction format list must contain at least the following fields: target object identifier, action identifier, parameters, preconditions, post-validation, rollback strategy, timeout, and idempotent token; The execution plan engine selects the optimal execution channel based on the capability table. The selection rules include: prioritizing the assembly of API requests through the native interface adapter. When the target only supports UI interaction, automatically switch to the UI event adapter.

7. The task intent-based natural language-driven situational awareness control method according to claim 6, characterized in that, Execute the instructions step by step according to the list of operation instructions, and return a summary of the execution results; Including methods: The execution plan engine uses the trusted execution score of the scheduling plan as a list of gating execution operation instruction formats; Before each action begins, the execution plan engine checks the pre parameters and automatically fills in any missing parameters; the pre parameters include object visibility, required roles or permissions, and data freshness QoS. After the corresponding adapter is invoked to complete the action, the action result is validated in real time using the post parameter; The post parameters include: changes in the state of the interface object, the structure and quantity of the returned data, and matching of the prompt message; If the verification fails or times out, the fallback path will be initiated. After all actions are completed, return a summary of the execution results.

8. The natural language-driven situational awareness control method based on task intent according to claim 7, characterized in that, The method for obtaining the trusted execution score includes: Trusted execution scoring is achieved by nonlinearly fusing semantic matching, interface location, data availability, and permission compliance based on a trusted execution scoring gating mechanism.

9. A task intent-based natural language-driven situational awareness control system, characterized in that, Used to implement the task intent-based natural language-driven situational awareness control method according to any one of claims 1-8; The system includes: The preprocessing module is used to acquire the perception data and extract three tables of knowledge from the perception data, which include: an object table, a capability table, and a parameter table. The knowledge generation module is used to generate a natural language template library and a task intent field table based on the knowledge of the three tables; and to construct an interface-data dual mapping knowledge graph using the knowledge of the three tables as the foundation and the natural language template library and the task intent field table as retrieval and rearrangement tools. The retrieval and resolution module is used to respond to natural language commands by performing semantic retrieval and referential resolution based on a natural language template library, task intent fields, and an interface-data dual-mapping knowledge graph table to generate structured intent data; the structured intent data includes task intent and a location candidate set; The compilation module is used to compile the task intents in the structured intent data into operation micro-flows and generate a list of operation instruction formats; The execution module executes the instructions step by step according to the list of operation instructions and returns a summary of the execution results. The scoring module is used to perform a reliable execution score on the summary of the execution results.

10. A computer-readable medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, can implement the task intent-based natural language-driven situational awareness control method as described in any one of claims 1-8.

Images