Interactive sand table demonstration system based on artificial intelligence

Abstract

The invention discloses an interactive sand table demonstration system based on artificial intelligence, and relates to the technical field of intelligent interactive sand table demonstration. Comprising a multi-mode perception module, a semantic intention recognition module, a three-dimensional simulation module, a decision feedback module and a visualization module. Accurate recognition and semantic analysis of a user instruction are realized by fusing various input modes such as voice, gestures and images; based on a knowledge graph and a deep learning model, user demands can be intelligently understood, and a three-dimensional sand table is driven to carry out dynamic interactive demonstration. The situation adaptation module automatically adjusts the sand table scene according to the environment and the demonstration progress, and the reality sense and immersion sense of demonstration are improved; the decision feedback module generates demonstration results and strategy suggestions in real time, and supports multi-dimensional and intelligent interactive experience; the system is widely applied to the fields of emergency drilling, military deduction, urban planning, educational training and the like, and the intelligent level and the operation convenience of sand table demonstration are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent interactive sandbox demonstration, in particular to an interactive sandbox demonstration system based on artificial intelligence. Background Art

[0002] The interactive sandbox demonstration system is a comprehensive platform integrating multimedia display, 3D modeling, sensor control, information fusion, and human-computer interaction technologies. It is widely used in scenarios such as urban planning, military exercises, geographic information display, and emergency management. This system typically uses technologies such as laser projection, real-world modeling, touch controls, voice recognition, and motion capture to achieve dynamic visualization and immersive interactive operation of terrain, buildings, transportation, and resources, enhancing the intuitiveness, flexibility, and engagement of demonstrations. Furthermore, it can integrate backend databases, simulation engines, and AI analysis modules to perform real-time simulation, deduction, and decision support for complex scenarios.

[0003] However, existing interactive sandbox demonstration systems have problems with insufficient ability to understand user intentions and weak situational adaptability. First, they can only recognize simple, preset voice or touch commands, and find it difficult to accurately parse the user's true intentions in multimodal interactions such as natural language expression, body movements, and gestures. This leads to inaccurate system responses to complex commands and limited interactive experience. Second, due to the lack of in-depth understanding of the interaction context and event logic, it is impossible to effectively judge the rationality and legality of user operations, which easily causes the simulation process to be disconnected from the actual scenario, affecting the accuracy of the deduction results and the reliability of decision-making assistance, and limiting the practical value of the system in complex application scenarios such as emergency drills and urban governance. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems mentioned in the above background technology and to propose an interactive sandbox demonstration system based on artificial intelligence.

[0005] The purpose of the present invention can be achieved by the following technical solutions: comprising: a semantic intent recognition module, a three-dimensional simulation module and a decision feedback module, wherein the semantic intent recognition module performs semantic intent recognition after receiving a unified user operation vector: pre-decoding the operation vector, extracting potential semantic fragments, entering the preliminary intent analysis channel to obtain a reconstructed structured user intent expression sequence; performing a semantic decomposition and matching process according to the obtained user intent expression sequence, extracting and classifying components from it to obtain a component combination sequence; executing an intent classifier according to the component combination sequence, classifying the semantic structure into a standard intent type to obtain a standard intent label; then performing a logical judgment on each standard intent type to obtain a confidence level, if the confidence level ≥ a threshold, it is determined to be a valid intent; if the confidence level is lower than a lower threshold, it is directly eliminated; if the confidence level is between the lower threshold and the threshold, triggering an intent secondary analysis mechanism, and calling the context map and event dependency logic again for re-judgment; if there is an intention that is eliminated or lacks confidence, the state is marked as requiring interactive confirmation, and a candidate semantic guidance prompt is automatically generated; feedback is sent to the visualization module for display, waiting for the multimodal perception module to obtain different types of user input information again to form a unified user operation vector;

[0006] After receiving the clear structured standard intent parsed by the semantic intent recognition module, the 3D simulation module performs corresponding 3D simulation modeling according to the different types of intent;

[0007] The decision feedback module responds to the "response call class" and "status assessment class" intention types in the output of the semantic intention recognition module. Based on the dynamic evolution results of the three-dimensional simulation module, it completes the emergency response strategy path, execution response strategy and optional strategy from the simulation situation analysis.

[0008] Compared with the prior art, the present invention has the following beneficial effects:

[0009] 1. The multimodal perception module collects user voice, body movements, gestures, and sandbox changes in real time, integrating them into a user action vector. The semantic intent recognition module then performs semantic analysis, intent classification, fuzzy reference identification, and correction to generate a clear and structured user intent. The system accurately understands complex user interaction commands and, combining contextual graphs and event-dependent logic, determines the legitimacy of intent, providing an accurate semantic foundation for subsequent 3D simulation and decision-making feedback.

[0010] 2. The present invention has stronger user interaction understanding ability and situational adaptability. It can generate a three-dimensional simulation process that conforms to the scene semantics based on the user's natural language and dynamic behavior, and deduce event evolution, assess risk status, and recommend response strategies based on this. Finally, it performs graphical display and interactive feedback through the visualization module, which greatly improves the intelligence, practicality and decision-making support capabilities of the sandbox demonstration, and is particularly suitable for complex scenarios such as emergency drills, urban governance, and disaster management. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0012] Figure 1 Schematic diagram of module connection of the present invention. DETAILED DESCRIPTION

[0013] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0014] See also Figure 1 As shown in the figure, the interactive sandbox demonstration system based on artificial intelligence includes: multimodal perception module, semantic intention recognition module, 3D simulation module, decision feedback module and visualization module;

[0015] After the sandbox demonstration system is started, the multimodal perception module is activated to collect different types of user input information in real time; it receives user natural language commands through the microphone array, monitors user position, expression, movement, etc. through the camera, records them as user body language commands, and obtains hand movements such as waving, clicking, and sliding through recognition, which are mapped into interactive operations; recorded as interactive operation commands, and uses sensors to detect the movement of physical props on the sandbox, changes in lighting, and other states, which are recorded as sandbox transformation commands; the various commands input by the multimodal perception module are integrated with multi-source input to form a unified user operation vector, which is then passed to the semantic intent recognition module.

[0016] After receiving the unified user operation vector, the semantic intent recognition module uses natural language processing and multi-round dialogue technology to perform semantic intent recognition on the user operation vector:

[0017] Perform initial decoding, pre-decode the operation vector, extract potential semantic fragments (tokens, labels, coordinate information, etc.), and enter the preliminary intent analysis channel. In the preliminary intent analysis channel: first, pre-decode the input operation vector to extract semantic fragments including instruction keywords, spatial scope, time conditions, event types, and variable descriptions. Input the semantic fragments into the pre-trained language model that has been fine-tuned for the domain, and combine named entity recognition and direction recognition mechanisms to identify and classify key intent elements. Based on the label information and location index output by the model, reconstruct a structured user intent expression sequence. For example, please show the load changes of the evacuation channel in this area during a nighttime earthquake.

[0018] Based on the obtained user intention expression sequence, a semantic decomposition and matching process is performed to extract and classify components to obtain a sub-combination sequence, including: action intentions: display, simulation, call, analysis, switching, etc.; operation targets: area, event, layer, object, etc.; semantic constraints: time period, environmental status, previous scene, etc.; output requirements: static image, dynamic trajectory, thermal distribution, etc.; fuzzy or undefined items: such as "over there" and "nearest", etc., are recorded as "unconfirmed entities";

[0019] Based on the sequence of component combinations, the intent classifier is executed to classify the semantic structure into a standard intent type: after obtaining the component combination, rule matching and feature extraction are performed based on the combination relationship between the semantic components (such as the matching of operation type and variables, time and event constraints, etc.), and a semantic combination feature vector for classification is extracted; the semantic combination feature vector is input into the intent classifier, and by comparing it with the intent templates defined in the existing standard intent type library, its category (such as "disaster simulation query", "historical data backtracking", "real-time monitoring display", etc.) is determined, and the corresponding standard intent label is finally output; standard intent types include: information visualization intent, scenario deduction intent, emergency response call intent, state analysis intent, and instruction correction / reference intent;

[0020] Perform a logical decision for each standard intent type identified; specifically:

[0021] Perform an intent integrity check and analyze the structured semantic intent to determine whether it contains actions (such as "display" and "analysis"), goals (such as objects or variables such as "evacuation channel load"), and output requirements (such as "change situation" and "comparison results") at the same time. If all three are present, the structure is considered complete; if any element is missing, the intent is marked as incomplete and fed back to the upstream module for completion or interactive correction to ensure that subsequent processing has a clear and executable semantic basis.

[0022] The context graph and event dependency logic are then used to verify the legality of the intent. After the intent structure is identified, the context graph and event dependency logic rules are called to perform constraint verification on the relationship between time, events, variables, and operations in the current intent. For example, the verification checks whether the requested variables have valid data support within the current region and time range, whether the events involved are consistent with the context logic (e.g., whether "nighttime earthquake" occurs in the selected time period), and whether the operation type is applicable to the event scenario. If the verification passes, the intent is deemed legal. Otherwise, the intent is marked as illegal or conflicting, and the user is prompted to reconfirm the input content.

[0023] Perform fuzzy calculations. When fuzzy expressions or unclear references (such as "this area," "what happened just now," "its changes," etc.) are detected in the user's intent, the fuzzy tracing sub-channel is automatically started. The position and type of the fuzzy reference item in the semantic structure are identified, and historical operation records (such as the last executed instruction), current layer status (such as the user's selected area, layer visibility), and user interaction focus (such as mouse hover, click area, view center, etc.) are called in sequence as sources of context clues. The confidence level is obtained using fuzzy inference rules and confidence assessment mechanisms. The relevance scores of various sources (correlation analysis) are integrated to complete or clarify the fuzzy items. Finally, a revised clear intent structure is generated. If the completion is successful, the intent confirmation process is entered. If the completion fails, the state is transferred to "pending clarification" and the user prompt is returned.

[0024] The corrected clear intent structure is then judged according to the obtained confidence level. If the confidence level is greater than or equal to a threshold (e.g., 0.85), it is considered a valid intent. If the confidence level is lower than the lower threshold (e.g., <0.5), it is directly eliminated.

[0025] If the confidence level is between 0.5 and 0.85, the secondary intent resolution mechanism is triggered, and the context graph and event dependency logic are called again for re-judgment.

[0026] If there is an intention that is rejected or lacks confidence, the system marks the status as requiring interactive confirmation and automatically generates a candidate semantic guidance prompt: "Which area do you mean?" Feedback is sent to the visualization module for display, waiting for the multimodal perception module to obtain different types of user input information again to form a unified user operation vector.

[0027] After receiving the clear structured standard intent parsed by the semantic intent recognition module, the 3D simulation module performs the corresponding 3D simulation modeling process for different intent types (such as disaster scenario simulation, emergency evacuation drill, personnel flow prediction, etc.), specifically:

[0028] First, based on the location information contained in the intent, the corresponding 3D scene model is initialized. This 3D scene model includes elements such as building structure, site layout, and geographical environment, and incorporates a hybrid indoor and outdoor modeling mechanism as needed. During the initialization process, basic environmental data for the area, such as lighting, accessible paths, entrance and exit layout, and access control status, is simultaneously loaded to provide foundational support for subsequent event evolution.

[0029] Based on the event type field, the corresponding event-driven model is automatically matched and embedded, such as fire, earthquake, and flood. Each event type includes complete physical evolution rules and influencing mechanisms, such as the spread speed of fire, the range of smoke diffusion, the magnitude of earthquakes, and the response behavior of buildings. Furthermore, based on the simulation time context (such as nighttime and holidays), the distribution of personnel and response resource capabilities are further adjusted to ensure the real-world correspondence of the simulation scenario.

[0030] After the scenarios and events are set, the distribution pattern of people and resources within the region is constructed. Personnel deployment can be generated based on historical behavioral data or rule-based simulation, including personnel numbers, locations, identity characteristics, and individual behavior patterns. Resources include emergency facilities, evacuation routes, safety guidance devices, etc., all of which have state attributes and interactive capabilities. At this stage, behavioral driving parameters are also injected into each individual, such as risk perception ability, path preference, and movement speed changes, providing a parameter basis for individual behavior evolution.

[0031] Next, the built-in behavioral rule library is called upon to map the "perception-judgment-action" chain behavior model to each individual based on the event type and spatial state. This model supports the evolution of complex behaviors such as dynamic path selection, avoidance behavior, and coordinated evacuation. It also updates individual decision-making strategies in real time through a path replanning mechanism to adapt to dynamic changing situations such as channel blockage and fire spread, achieving realism and fine-grained control of multi-agent simulation.

[0032] After completing the behavioral mapping, the simulation engine starts to promote the dynamic evolution of the overall scenario at a set time step. During the dynamic evolution process, key indicators such as personnel movement trajectories, node congestion index, and environmental state changes are continuously calculated to achieve a continuous expression of the overall event evolution situation. If the event state undergoes a secondary change during the process (such as fire expansion or building collapse), the scene state is automatically adjusted and the relevant chain response is triggered.

[0033] Finally, structured result data is generated for the simulation process based on the user's output requirements. The output content includes but is not limited to: trajectory data of individuals or groups, spatial heat maps, evacuation completion rate statistics, high-risk node identification, path efficiency evaluation, and a visual demonstration frame sequence of the complete simulation process. The results are synchronously transmitted to the visualization module and the decision feedback module to assist in situation assessment, emergency drills, or command decision-making.

[0034] The decision feedback module responds to the "response call class" and "state assessment class" intent types output by the semantic intent recognition module. Based on the dynamic evolution results of the 3D simulation module, it completes the entire process from simulation situation analysis to intelligent strategy generation and feedback recommendation. Specifically:

[0035] The dynamic evolution data transmitted by the simulation module is semantically deconstructed and indicators are extracted. By calculating and summarizing multi-dimensional key parameters such as evacuation time, personnel density distribution, node congestion level, and regional completion rate, a structured computational graph of the current scenario is quickly established. The structured computational graph not only reflects the results characteristics of the simulation process, but also retains the development trend of the event and clues to abnormal changes.

[0036] After the simulation data structured computation graph is completed, semantic matching is automatically performed between the current event type and the spatial state, and the built-in emergency rule library and response knowledge graph are called. Through condition matching and contextual reasoning, the system is guided to identify feasible emergency response strategy paths in the current situation. For example, in a fire simulation, it can identify "main channel obstruction and dense personnel distribution in the middle area" and infer a joint response strategy of "activating backup channels and broadcasting to guide diversion." This stage further combines the event development stage (initial, diffusion, out of control) and spatial resource status (available facilities, access control status, etc.) to enhance reasoning accuracy.

[0037] Then, based on preliminary reasoning, historical behavior data, resource distribution information and strategy effect models are integrated to dynamically generate execution response strategies. The execution response strategy process not only includes the logical combination of strategies (such as broadcasting first and then blocking, and simultaneous multi-path diversion), but also introduces a strategy effect prediction mechanism to quantitatively evaluate and prioritize multi-dimensional parameters such as coverage improvement, risk suppression capability, and response timeliness after the implementation of each plan. If some strategies are limited by the current facility status or insufficient resources, the priority plans in the priority ranking will be automatically eliminated to improve the actual adaptability and feasibility of the strategy output.

[0038] Finally, when it is detected that there are multiple optional strategies with similar scores or different execution methods, an interactive feedback mechanism will be entered to display key parameters such as the execution path, resource dependencies, and simulation effect predictions of each strategy to users or decision makers, and the user will be guided to confirm or adjust through a visual interface; after the user confirms, the optional strategies will be pushed to the visualization module for dynamic presentation.

[0039] The visualization module uniformly renders candidate semantic guidance prompts, structured result data, and optional strategies in the form of graphics, animations, charts, etc.

[0040] Based on the structured result data output by the 3D simulation module, an interactive 3D visual scene is constructed to display dynamic processes such as personnel distribution, event evolution, and evacuation routes;

[0041] Graphically compare the execution paths, coverage, resource allocation, etc. of various optional strategies to assist users in making choices and interventions.

[0042] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An interactive sandbox demonstration system based on artificial intelligence, including: The semantic intention recognition module, the three-dimensional simulation module and the decision feedback module are characterized by: After receiving the unified user operation vector, the semantic intent recognition module performs semantic intent recognition: pre-decodes the operation vector, extracts potential semantic fragments, enters the preliminary intent analysis channel to obtain a reconstructed structured user intent expression sequence; performs semantic decomposition and matching according to the obtained user intent expression sequence, extracts and classifies components from it to obtain a component combination sequence; according to the component combination sequence, executes the intent classifier, classifies the semantic structure into a standard intent type to obtain a standard intent label; then performs logical judgment on each standard intent type to obtain confidence, if the confidence ≥ threshold, it is judged as a valid intent; if the confidence is lower than the lower threshold, it is directly eliminated; if the confidence is between the lower threshold and the threshold, triggers the intent secondary analysis mechanism, and calls the context map and event dependency logic again for re-judgment; if there is an intention that is eliminated or lacks confidence, the state is marked as requiring interactive confirmation, and a candidate semantic guidance prompt is automatically generated; feedback is sent to the visualization module for display, waiting for the multimodal perception module to obtain different types of user input information again to form a unified user operation vector; After receiving the clear structured standard intent parsed by the semantic intent recognition module, the 3D simulation module performs corresponding 3D simulation modeling according to the different types of intent; The decision feedback module responds to the "response call class" and "state assessment class" intention types output by the semantic intent recognition module. Based on the dynamic evolution results of the three-dimensional simulation module, it completes the emergency response strategy path, execution response strategy and optional strategies from the simulation situation analysis.

2. The interactive sandbox demonstration system based on artificial intelligence according to claim 1 is characterized in that: Also includes: Multimodal perception module and visualization module; The multimodal perception module collects different types of user input information in real time; Fuse multi-source input; Form a unified user operation vector and pass it to the semantic intent recognition module; The visualization module uniformly renders candidate semantic guidance prompts, structured result data, and optional strategies; based on the structured result data output by the 3D simulation module, it constructs an interactive 3D visual scene to display the dynamic process of personnel distribution, event evolution, and evacuation path; Graphically compare the execution paths, coverage, and resource allocation of multiple optional strategies to assist users in making choices and interventions.

3. The interactive sandbox demonstration system based on artificial intelligence according to claim 1 is characterized in that: The specific process of entering the preliminary intent analysis channel and reconstructing a structured user intent expression sequence is as follows: The input operation vector is pre-decoded to extract semantic fragments including instruction keywords, spatial scope, time conditions, event types and variable descriptions; the semantic fragments are input into the pre-trained language model that has been fine-tuned in the domain, and the key intent elements are identified and classified by combining named entity recognition and pointing recognition mechanisms; based on the label information and position index output by the model, a structured user intent expression sequence is reconstructed.

4. The interactive sandbox demonstration system based on artificial intelligence according to claim 3 is characterized in that: The specific process of classifying semantic structures into standard intent types and obtaining standard intent labels is as follows: After obtaining the component combination, rule matching and feature extraction are performed based on the combination relationship between the semantic components to extract the semantic combination feature vector for classification; the semantic combination feature vector is input into the intent classifier, and by comparing it with the intent template defined in the existing standard intent type library, its category is determined, and finally the corresponding standard intent label is output.

5. The interactive sandbox demonstration system based on artificial intelligence according to claim 4 is characterized in that: The specific process of performing logical judgment on each standard intent type to obtain confidence is as follows: Perform an intent integrity check, analyze the structured semantic intent, and determine whether it contains actions, goals, and output requirements. If all three are present, the structure is considered complete. If any element is missing, the intent is marked as incomplete and fed back to the upstream module for completion or interactive correction. Context graphs and event dependency logic are then used to verify the legitimacy of the intent. After completing intent structure identification, the context graph and event dependency logic rules are invoked to perform constraint verification on the relationships between time, events, variables, and operations in the current intent. If the verification passes, the intent is considered legitimate; otherwise, the intent is marked as illegal or conflicting, and the user is prompted to reconfirm the input content. Perform fuzzy calculations and automatically start the fuzzy tracing sub-channel when fuzzy expressions or unclear references are detected in the user's intentions; Identify the position and type of ambiguous referents in the semantic structure, and use historical operation records, current layer status, and user interaction focus as context clues. Utilize fuzzy inference rules and confidence assessment mechanisms to obtain confidence, integrate the relevance scores of various sources, and complete or clarify the fuzzy terms; ultimately, generate a revised clear intent structure; If the completion is successful, the intention confirmation process will begin; If the completion fails, it will enter the "pending clarification" state and return to the user prompt.

6. The interactive sandbox demonstration system based on artificial intelligence according to claim 5 is characterized in that: The specific process of performing the corresponding 3D simulation modeling is as follows: Initialize the corresponding 3D spatial scene model based on the location information carried in the intent. During the initialization process, synchronize the basic environmental data of the modeling area. Automatically match and embed the corresponding event-driven model based on the event type field. Furthermore, further adjust the distribution of personnel and response resource capabilities based on the time context of the simulation. After the scene and event settings are completed, construct the distribution pattern of people and resources within the area. The population is based on historical behavioral data, including the number of people, location, identity characteristics, and individual behavior patterns, while resources include emergency facilities, evacuation routes, and safety guidance devices. Simultaneously, behavioral driving parameters are injected into each individual. Next, call the built-in behavioral rule library and map the "perception-judgment-action" chain behavior model to each individual based on the event type and spatial status. After completing the behavior mapping, the simulation engine starts to advance the dynamic evolution of the overall scenario at a set time step; If the event state changes twice during the dynamic evolution process, the scene state will be automatically adjusted and the relevant chain response will be triggered; finally, structured result data will be generated for the simulation process based on the user's output requirements.

7. The interactive sandbox demonstration system based on artificial intelligence according to claim 6 is characterized in that: The specific process of obtaining the emergency response strategy path from the simulation situation analysis, executing the response strategy and the optional strategy is as follows: The dynamic evolution data transmitted by the simulation module is semantically deconstructed and indicators are extracted; by calculating and summarizing the multi-dimensional key parameters of evacuation time, personnel density distribution, node congestion level, and regional completion rate, a structured calculation graph of the current scenario is quickly established; after the structured calculation graph of the simulation data is completed, the current event type and spatial status are automatically semantically matched, and the built-in emergency rule library and response knowledge graph are called; through condition matching and context reasoning, the system is guided to identify feasible emergency response strategy paths in the current situation; based on the preliminary reasoning, historical behavior data, resource distribution information and strategy effect model are integrated to dynamically generate and execute response strategies; a strategy effect prediction mechanism is introduced to quantify the multi-dimensional parameters after the implementation of each plan and prioritize them; if some strategies are limited by the current facility status or insufficient resources, the priority plans in the priority ranking are automatically eliminated; finally, when multiple optional strategies with similar scores are detected, an interactive feedback mechanism is entered to display the key parameters of each strategy to the user, and the user is guided to confirm or adjust through a visual interface; After user confirmation, the optional strategies are pushed to the visualization module for dynamic presentation.

Images