Immersive teaching situation intelligent construction method and system based on generative AI and industrial big data

Abstract

The invention relates to the field of teaching situation construction, in particular to an immersive teaching situation intelligent construction method and system based on generative AI and industrial big data. According to the method, firstly, multi-source heterogeneous data from an industrial site is subjected to standardization processing, a corresponding industrial knowledge graph is constructed, then a received teaching target is analyzed and decomposed into structured knowledge points, skill points and literacy points, a teaching logic sequence is constructed, and on the basis of the teaching logic sequence, an industrial knowledge graph is constructed. And dynamically querying and instantiating the industrial knowledge graph to generate a situation logic script, generating multi-modal content according to the situation logic script in combination with a generative AI model drive, and finally outputting an interactive teaching situation application package. According to the invention, teaching elements can be accurately identified, extracted and structured from multi-modal industrial big data, so that the generated teaching situation not only accords with industrial reality, but also can closely surround a preset teaching target.

Description

Technical Field

[0001] This invention relates to the field of teaching context construction, specifically to an intelligent construction method and system for immersive teaching contexts based on generative AI and industrial big data. Background Technology

[0002] In traditional practical teaching, the construction of real-world industry scenarios has long faced bottlenecks such as high costs, difficulty in replication, and lagging updates. Teaching scenarios that rely on artificial design are often static and fixed, making it difficult to dynamically reflect the rapidly changing industry realities, resulting in a disconnect between talent cultivation and job requirements.

[0003] The integration of generative AI and industrial big data offers a breakthrough solution to this problem. Industrial big data aggregates massive amounts of real-world information, including technological evolution, process flows, and market cases, providing a dynamic and objective foundation for scenario construction. Generative AI, based on this data, can intelligently generate highly realistic task scenarios, interactive elements, and complex cases, achieving low-cost, high-efficiency, and customizable scenario construction. The immersive learning environment built by this method not only enhances learners' sense of presence and participation but also strengthens their ability to solve complex problems by simulating real-world industry challenges. This effectively bridges the gap between theory and practice, promoting deep integration of industry and education and precise talent cultivation.

[0004] Existing methods for generating immersive teaching scenarios often face challenges when using industrial big data to construct contexts. These challenges include high data heterogeneity, difficulty in extracting teaching elements, and a disconnect between the generated context and the teaching objectives. Specifically, industrial big data (such as equipment logs, operation videos, design drawings, and process documents) is multi-source and heterogeneous. Directly using it to generate teaching scenarios results in information redundancy and the burying of key teaching knowledge points (such as safety regulations, operational difficulties, and fault logic). Furthermore, general generative AI (such as large language models and text-based video models) lacks an understanding of industry-specific knowledge and teaching logic, easily generating scenarios that are correct in content but lack teaching relevance, have logical jumps, or do not conform to training procedures, thus making it difficult to guarantee teaching effectiveness. Summary of the Invention

[0005] To address the technical challenge of intelligently identifying, extracting, and structuring teaching elements from multimodal industrial big data, and driving generative AI to construct immersive training scenarios that align with industry realities and closely adhere to pre-defined teaching objectives, this invention aims to provide a method and system for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data. The specific technical solution adopted is as follows: This invention proposes a method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data, the method comprising: Standardize the multi-source heterogeneous data from the industry site, extract entities and relationships based on a predefined teaching element classification system, and construct an industry knowledge graph containing multi-dimensional information on equipment, operation, faults, and safety. The teaching objectives described in natural language are received, analyzed and broken down into structured knowledge points, skill points and literacy points, and a logical sequence of teaching is constructed based on educational theory. Based on the teaching logic sequence, specific equipment, components, parameters and fault cases are dynamically queried and instantiated from the industry knowledge graph to generate a situational logic script that includes scene settings, role tasks, event sequences and interactive feedback rules. Based on the scenario logic script, structured instructions are generated for different generative AI models, driving the generation and synchronous synthesis of multimodal content, and finally packaged and output as an interactive immersive teaching scenario application package.

[0006] Furthermore, the extraction of entities and relationships based on the predefined teaching element classification system includes: Based on the entity types in the teaching element classification system, operation steps, safety warnings and process parameters are extracted from text data based on rule and pattern matching. The object detection results in the visual data are matched and associated with the equipment component entities in the industry knowledge graph; By aligning the feature events in time series data with the operation actions described in the text with timestamps, a causal relationship between parameter changes and operation behavior can be established.

[0007] Furthermore, the construction of the teaching logic sequence includes: The decomposed knowledge points, skill points, and literacy points are mapped to a teaching knowledge graph based on cognitive development levels; Based on skill dependencies and teaching principles, the mapped teaching elements are sorted to form a progressive logical sequence from theoretical understanding to practical training, and from normal procedures to troubleshooting. This sequence serves as the teaching logical sequence, and a difficulty adjustment factor is configured for the teaching logical sequence. The difficulty adjustment factor includes information completeness, time pressure, and interference factor level.

[0008] Furthermore, the dynamic query and instantiation are performed through a dual-graph mapping engine, which performs the following operations: The abstract tasks in the teaching logic sequence are parsed to generate a joint query statement for the industry knowledge graph and the teaching knowledge graph; The decision-making process for query results is based on teaching suitability rules, which include priority given to teaching frequency, typicality of operation, richness of data, and difficulty suitability. Create an instance configuration file for the selected industry knowledge graph entity. The configuration file defines the initial state of the instance in the context, interactive attributes, and operational constraints based on real data.

[0009] Furthermore, the method for generating the contextual logic script includes: The instantiated specific tasks are arranged in a dual-track system, integrating the teaching logic track and the narrative logic track, thus giving the teaching tasks a narrative background and plot challenges. At key nodes in the teaching logic sequence, based on historical fault probability data in the industry knowledge graph, fault or interference events that meet the teaching objectives are dynamically injected, and corresponding branch logic is generated. Finally, a machine-readable, structured script file is generated as a situational logic script, which specifies the object states, event triggering conditions, character behavior logic, and teaching assessment points in the virtual environment.

[0010] Furthermore, the generation of structured instructions for different generative AI models includes: Based on the modal type of the elements in the scenario logic script, call the corresponding instruction template and fill in the script parameters; The instruction template for generating 3D or visual scenes is required to be described in a structured format of "subject object - precise attributes - spatial relationships - environmental context - style constraints"; The instruction template for generating role dialogue strictly defines the speaker's role, knowledge scope, teaching intention, and tone constraints.

[0011] Furthermore, the driving generation and synchronous synthesis of multimodal content includes: The structured instructions are sent in parallel to the corresponding generative AI service; For the generated key equipment assets, standard models are retrieved first from the benchmark digital asset library associated with the industry knowledge graph, and the AI ​​is instructed to modify the parameterized state based on them. The spatiotemporal registration engine calibrates and synchronizes all generated 3D models, animations, audio, and UI elements in a unified virtual coordinate system and timeline, ensuring consistency of multimodal feedback during interaction.

[0012] Furthermore, the packaged output also includes: An automatic scenario list file is generated. The list file records at least the teaching metadata of this scenario, the traceability information of the industry knowledge graph nodes and teaching logic sequence on which it depends, the generated version, and the target running platform.

[0013] This invention also proposes an intelligent construction system for immersive teaching scenarios based on generative AI and industrial big data. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any one of the steps of an intelligent construction method for immersive teaching scenarios based on generative AI and industrial big data.

[0014] The present invention also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any one of the steps of an intelligent construction method for immersive teaching scenarios based on generative AI and industrial big data.

[0015] The present invention has the following beneficial effects: 1. Significantly improved teaching adaptability: Through the "teaching logic sequence" and "contextual logic middleware", abstract teaching objectives are transformed into specific and executable control instructions, ensuring that each generated contextual fragment directly serves a specific knowledge point or skill point, overcoming the problems of scattered content and deviation from objectives in general generative AI.

[0016] 2. Traceable Industry Authenticity: All scenario details (equipment, tools, fault chains) are instantiated from real industry knowledge graphs, rather than AI-generated, ensuring a high degree of consistency between training content and actual job requirements, thus solving the problem of accuracy in "data-scenario" conversion.

[0017] 3. The generation process is highly controllable and interpretable: The core of the solution lies in rules, graphs and structured assembly, rather than a black box model. Generative AI acts only as an "executor". The quality and direction of its generated content are strictly controlled by the front-end logic. The process is transparent and the results are stable.

[0018] 4. Achieves low-cost personalization and dynamism: By adjusting the teaching logic sequence and mapping rules, personalized scenarios can be quickly generated for different learners (newbies / experts) and different teaching objectives (process familiarization / troubleshooting) without remaking materials, thereby achieving dynamic optimization of the scenarios.

[0019] 5. Solved the problem of consistency in multimodal content: Since text, visual, and audio content all originate from the same contextual logic script, the consistency and coordination of different modal information in terms of teaching logic, timeline, and spatial relationships are guaranteed. Attached Figure Description

[0020] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an intelligent construction method for immersive teaching scenarios based on generative AI and industrial big data, provided as an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an intelligent construction system for immersive teaching scenarios based on generative AI and industrial big data, provided as an embodiment of the present invention. Detailed Implementation

[0022] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an intelligent construction method and system for immersive teaching scenarios based on generative AI and industrial big data proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] The following description, in conjunction with the accompanying drawings, details the specific solution of the intelligent construction method and system for immersive teaching scenarios based on generative AI and industrial big data provided by this invention.

[0025] Please see Figure 1 The diagram illustrates a flowchart of an intelligent construction method for immersive teaching scenarios based on generative AI and industrial big data, according to an embodiment of the present invention. The method includes: Step S1: Standardize the multi-source heterogeneous data from the industry site, extract entities and relationships based on the predefined teaching element classification system, and construct an industry knowledge graph containing multi-dimensional information on equipment, operation, faults, and safety.

[0026] Since industry data is usually multi-source, heterogeneous, and unstructured, and is seriously out of touch with actual teaching needs, this embodiment of the invention first transforms the raw, messy data into an industry knowledge graph rich in semantic relationships that can be used to drive context generation, thereby laying a data foundation for subsequent steps.

[0027] Because industry data comes from diverse sources, such as operation manuals (PDF / Word), design drawings (CAD), surveillance videos, sensor time-series data, and maintenance work orders (text), these data have vastly different formats. Direct processing would lead to a surge in algorithm complexity and poor results. Therefore, standardization is essential to extract core information such as text, numerical values, keyframe images, and metadata. A unified index with timestamps and device IDs should be established to unify the data entry point. This transforms raw data from different formats and sources into standardized semi-structured or structured data that can be recognized by subsequent processing procedures, laying the foundation for subsequent correlation analysis.

[0028] Multi-source heterogeneous data in industrial settings typically include textual data (such as operation manuals and safety procedures), drawings and image data (such as CAD drawings), time-series data (such as SCADA sensor data), and process data (such as operation videos).

[0029] For text-based data, optical character recognition and natural language processing technologies can be used to not only extract the text but also parse its structure (such as chapters, lists, and tables), identifying step-by-step, warning, and parametric text paragraphs. For drawing and image data, the hierarchical structure of CAD files can be parsed to extract equipment component lists, assembly relationships, and dimensional parameters. Object detection and classification can be performed on key frames of on-site photos or videos to identify specific tools, instruments, and equipment status. For time-series data, filtering, noise reduction, and feature extraction (such as peak values, mean values, and trends) can be performed to transform it into an event sequence representing changes in equipment status (such as "pressure over-limit alarm" or "temperature steadily rising"). For process-related data, it can be aligned with the corresponding operation manual steps and sensor timelines to form a triplet association of "operation action - equipment status change - time point".

[0030] This completes the standardization of industry data, forming data units centered on "events" or "entities" and linked to multimodal evidence, breaking down data silos and laying the foundation for building a comprehensive and three-dimensional knowledge graph.

[0031] Considering that general knowledge graph construction pursues comprehensiveness, but teaching requires focus, this invention predefines a set of "teaching element classification system" (such as: equipment entities, operation actions, process parameters, failure modes, safety elements, tools and materials, etc.) for skills training. This system guides the information extraction process, accurately identifying and extracting entities, attributes and relationships that are directly valuable for constructing teaching scenarios from standardized data, avoiding information overload and ensuring that the extracted knowledge is highly focused on teaching applications.

[0032] Preferably, in one embodiment of the present invention, the entity and relation extraction method specifically includes: Based on the entity types in the teaching element classification system, operation steps, safety warnings, and process parameters are extracted from text data using rules and pattern matching. Specifically, structured information can be extracted from the text based on predefined entity dictionaries (such as equipment model, component name, tool name) and pattern rules (such as <action> + <object> + <parameter>). For example, from the sentence "Use a torque wrench to tighten the sealing gland bolt with a torque of 5±0.5N·m", the following can be extracted: operation action = tighten, tool = torque wrench, object = sealing gland bolt, parameter = torque: 5±0.5N·m.

[0033] The object detection results in visual data are matched and associated with equipment component entities in the industry knowledge graph. Specifically, objects detected in images / videos can be associated with entities extracted from text. For example, if a "pressure gauge" is detected in a video frame, its reading (identified by OCR or instrument) can be associated with the parameter standard described in the text, "the outlet pressure should be maintained at 0.8-1.0MPa", forming a visual-text association knowledge of "pressure gauge-display-outlet pressure".

[0034] By aligning the feature events in time series data with the operational actions described in the text using timestamps, a causal relationship between parameter changes and operational behaviors can be established. For example, the "pressure drop" event detected by the sensor can be aligned with the "closing the inlet valve" action in the operation video and the "sealing failure" record in the maintenance work order on the timeline, thereby establishing a causal and diagnostic relationship of "closing the valve → may lead to → pressure drop" and "pressure drop → may indicate → sealing failure".

[0035] Because graph structures can most naturally represent complex multi-dimensional relationships between entities (such as composition, causality, order, and dependency), graph-based systems enable efficient association queries, path discovery, and reasoning. This is crucial for generating logically coherent teaching scenarios. Therefore, extracted teaching elements and their relationships can be stored and managed in the form of graph structures, forming a queryable, reasonable, and scalable "industry knowledge network," i.e., an industry knowledge graph. Subsequently, all factual evidence and logical relationships required for constructing scenarios can be accurately obtained from the industry knowledge graph, ensuring the authenticity and accuracy of the generated content.

[0036] First, nodes are created, and the extracted entities (such as "CH100 centrifugal pump", "mechanical seal", "torque wrench", "seal leakage fault") are used as nodes in the knowledge graph, along with all their attributes (model, specifications, image links, parameter standards, etc.).

[0037] Then, relationships are established by creating edges between nodes based on the semantics contained in the data source. For example: (CH100 centrifugal pump) - [hasPart] -> (mechanical seal), (mechanical seal) - [requiresTool] -> (torque wrench), (improper installation) - [mayCause] -> (seal leakage), (seal leakage) - [hasSymptom] -> (pressure drop).

[0038] A diagram segment constructed using "centrifugal pump seal replacement" as an example may include: a seal node with attributes such as model=KR-501, installation torque=5N·m, and common fault=wear / aging. This node is connected to the mechanical seal node via the [partOf] relationship, and then to the CH100 centrifugal pump node. At the same time, it is connected to operation nodes such as cleaning the sealing surface and applying grease via the [requiresAction] relationship, and to the leakage fault node via the [improperMayCause] relationship.

[0039] This completes the construction of the industry knowledge graph.

[0040] Step S2: Receive the teaching objectives described in natural language, parse and decompose them into structured knowledge points, skill points and literacy points, and construct a teaching logic sequence based on educational theory.

[0041] Considering that vague objectives cannot guide content generation, they must be broken down into atomic units such as "knowledge points" (what needs to be known), "skill points" (what needs to be done), and "competency points" (what awareness needs to be possessed). At the same time, the required depth of mastery for each unit (such as memorization, understanding, and application) should be clearly defined. Therefore, the teaching objectives described in natural language can be parsed and decomposed into structured knowledge points, skill points, and competency points. This transforms the user-input natural language training needs (such as "training new employees to master pump seal replacement") into a series of specific and measurable teaching objective components, making abstract training needs concrete, measurable, and linked to real industry knowledge.

[0042] The system first receives the teaching objectives described in natural language and identifies key dimensions through semantic analysis. For example, for the teaching objective of “training junior maintenance workers to independently replace the mechanical seal ring of a CH100 centrifugal pump and handle simple faults,” the key dimensions identified are: audience = junior maintenance worker, equipment = CH100 centrifugal pump, core task = mechanical seal ring replacement, extended task = simple fault handling, and ability level = independent completion.

[0043] Then, based on the pre-built skill task library and industry knowledge graph, the core tasks are automatically decomposed into structured knowledge points, skill points and literacy points. For example, "sealing ring replacement" can be decomposed into: knowledge points: sealing ring model identification, pump body structure cognition, safety risk points; skill points: use of special tools (puller, torque wrench), sealing surface cleaning, installation and alignment of new sealing ring, pressure test operation; literacy points: lock and tag procedure, clean up the site after work.

[0044] Furthermore, each point is associated with evidence nodes from the industry knowledge graph. For example, the skill point "Using a torque wrench" is associated with the usage specifications of the torque wrench node and the torque parameters of the sealing gland bolt node in the industry knowledge graph.

[0045] Considering that the transmission of knowledge and the cultivation of skills need to follow a certain order (such as theory before practice, normal before abnormal), this invention encodes teaching theories (such as Bloom's Taxonomy of Cognitive Objectives and the principle of simple to complex) into algorithmic rules for automatically arranging learning paths. Therefore, the decomposed teaching objectives can be further arranged into an orderly and progressive learning path, i.e., a teaching logic sequence, according to educational psychology and the laws of skill acquisition. This produces a scientific and personalized "teaching script" framework, which specifies the rhythm, order, and challenge of the teaching content presentation, ensuring that the generated teaching situation conforms to the laws of learning and can adapt to the differences of different learners.

[0046] Preferably, in one embodiment of the present invention, the method for constructing the teaching logic sequence specifically includes: First, the decomposed knowledge points, skill points, and competency points are mapped onto a teaching knowledge graph based on cognitive development levels. Specifically, the "knowledge points" are ordered according to the cognitive level of "memory → understanding → application." For example, first learn "sealing ring model and structure" (memory), then understand "its sealing principle" (understanding), and finally apply it in "selection judgment." The dependencies between "skill points" are analyzed to form a skill graph. For example, "installing a new sealing ring" depends on "successfully disassembling the old sealing ring" and "completing the cleaning of the sealing surface." The system generates a skill training sequence based on this. Following a progressive stage of "explanation and demonstration → guided practice → independent operation → fault introduction → comprehensive assessment," knowledge and skill points are allocated to different stages. For example, the first stage explains the structure through 3D animation; the second stage practices disassembly with virtual tutor guidance; and the third stage involves independently completing disassembly and handling a preset fault of "a bolt rusted shut."

[0047] Then, based on skill dependencies and teaching principles, the mapped teaching elements are sorted to form a progressive logical sequence from theoretical understanding to practical training, and from normal procedures to troubleshooting. This sequence serves as the teaching logic sequence, and difficulty adjustment factors are configured for it. These factors include information completeness (e.g., providing complete drawings or only partial information), time pressure (e.g., no restrictions or time limits) and interference level (e.g., no interference or noise interference). Finally, a structured teaching logic sequence file is generated, which defines the learning stages, the objectives of each stage, the tasks included, the difficulty configuration of the tasks, and the progression conditions between stages.

[0048] This completes the construction of the teaching logic sequence, providing a teaching outline for subsequent scenario generation, and generating corresponding scenario logic scripts based on this outline.

[0049] Step S3: Based on the teaching logic sequence, dynamically query and instantiate specific equipment, components, parameters, and fault cases from the industry knowledge graph to generate a situational logic script that includes scene settings, role tasks, event sequences, and interactive feedback rules.

[0050] In the teaching logic sequence, the tasks are abstract (e.g., "practice handling sealing leaks"), while the knowledge in the industry knowledge graph is concrete and instantiated (e.g., "On May 7, 2023, Pump No. 3 experienced a media leak due to aging of the sealing ring"). This process, through "mapping" and "instantiation," finds the most relevant and valuable real-world background and details for the abstract tasks, thereby concretizing the abstract tasks in the teaching logic sequence into real and usable entities and cases in the industry knowledge graph. This solves the problem of "fictitious" generated content. Through this process, each teaching task is anchored to real industry data and cases, ensuring the authenticity and fidelity of the generated context, and providing a solid creative basis for generative AI.

[0051] Preferably, in one embodiment of the present invention, dynamic querying and instantiation are performed through a dual-graph mapping engine, wherein the dual-graph mapping engine performs the following operations: First, the abstract tasks in the teaching logic sequence are parsed, for example, a task that "under guidance, completes the preliminary diagnosis and treatment of a sealing leak", and a joint query statement is generated for the industry knowledge graph and the teaching knowledge graph.

[0052] Then, decisions are made on the query results based on teaching suitability rules, which include priority based on teaching frequency, typicality of operation, richness of data, and difficulty suitability.

[0053] When the middleware's "dual-graph mapping engine" queries both the industry knowledge graph and the teaching knowledge graph, it does not randomly select a leakage case. Instead, it makes intelligent decisions based on "teaching suitability rules." For example, it prioritizes cases with the highest frequency of occurrence in historical fault records, such as "seal ring aging leakage," rather than rare cases like "shaft sleeve breakage leakage," based on typicality. Or it prioritizes cases with clear fault scene photos, complete processing video recordings, and detailed work order records based on data richness. Or it prioritizes cases with single causes and standard processing steps for "junior workers," while for "advanced workers," it can select cases with multiple concurrent causes and complex judgments.

[0054] Then, an instance configuration file is created for the selected industry knowledge graph entity. The configuration file defines the initial state, interactive attributes and operational constraints based on real data of the instance in the context. The configuration file describes in detail the "scenario materials" that will be used in this teaching task.

[0055] Considering that simply simulating operational procedures can easily bore learners, gamification and narrative design concepts are adopted to package teaching tasks into meaningful "challenges" or "stories," which can significantly improve engagement. The core is the organic integration of "teaching" and "immersion." Therefore, a situational logic script can be further generated, including scene settings, role tasks, event sequences, and interactive feedback rules. This transforms linear, potentially tedious teaching steps into a narrative experience with background, conflict, and interaction, enhancing learners' sense of involvement and learning motivation. The resulting "situational logic script" not only includes the teaching logic of "what to do," but also defines the narrative and interactive logic of "in what context," "what will be encountered," and "what will different choices lead to." This upgrades the final generated scenario from an "operation simulator" to an "immersive learning experience," greatly enhancing the attractiveness and lasting impact of the training.

[0056] Preferably, in one embodiment of the present invention, the method for generating a contextual logic script specifically includes: First, the specific tasks are arranged in a dual-track system, integrating the teaching logic and the narrative logic. This gives the teaching tasks a narrative background and plot challenges. Specifically, for the narrative background, a unified background story can be set for the entire teaching unit. For example, instead of "performing a seal replacement exercise," it could be "You are a night shift maintenance worker who receives a notification from the central control system that 'the pressure of the B-line feed pump has dropped abnormally, suspected of leaking,' and you need to go there immediately to handle the situation and restore production." For the plot challenges, the specific teaching tasks are transformed into plots within the story. For example, the teaching task of "identifying the leak point" is transformed into the plot of "Entering a virtual workshop, accompanied by the noise of the pump running, you need to locate the faulty pump through visual observation (looking for oil stains) and instrument checks (pressure drop)."

[0057] Then, at key nodes in the teaching sequence, based on historical fault probability data in the industry knowledge graph, fault or interference events that meet the teaching objectives are dynamically injected, and corresponding branch logic is generated. For example, when trainees complete multiple steps quickly and continuously, a interference event such as "tool falling" or "receiving a urging phone call" can be randomly inserted to examine their compliance with procedures under pressure. The probability and type of these events can be configured based on historical event data in the industry knowledge graph.

[0058] Finally, all plots, rules, and branches are integrated to generate a machine-readable, structured script file as a situational logic script. This script specifies the object states, event triggering conditions, character behavior logic, and teaching assessment points in the virtual environment. The situational logic script clearly defines the scene (e.g., time, place), characters (e.g., student characters), event sequence (e.g., triggering conditions, execution content, and results of main tasks and side events), and interaction logic (e.g., actions that students can perform, action objects, the legality and consequences of actions).

[0059] This completes the generation of the situational logic script.

[0060] Step S4: Based on the contextual logic script, generate structured instructions for different generative AI models, drive the generation and synchronous synthesis of multimodal content, and finally package and output it as an interactive immersive teaching context application package.

[0061] If an AI large language model is directly given a script-like natural language description, the generated multimodal content is prone to errors in detail, logical contradictions, or stylistic inconsistencies. This invention provides "precision guidance" for AI creation by designing highly structured instruction templates, ensuring that its output strictly conforms to industry standards and teaching requirements. Therefore, it can first generate structured instructions for different generative AI models based on the contextual logic script, thereby transforming the descriptive and logical content in the contextual logic script into precise and executable instructions for different modal generation models. This solves the problems of comprehension bias and uncontrollable output in generative AI. Through structured instructions, the randomness and error rate of generated content are greatly reduced, ensuring that the generated environment, objects, and dialogues are accurate in technical details, consistent in style, and clear in teaching intent.

[0062] Preferably, in one embodiment of the present invention, the method for generating structured instructions for different generative AI models specifically includes: Based on the modal type of the elements in the contextual logic script, the corresponding instruction template is invoked and the script parameters are filled in. Among them, the instruction template for generating 3D or visual scenes is required to be described in a structured format of "subject object - precise attributes - spatial relationship - environmental context - style constraints". The instruction template for generating character dialogue strictly specifies the speaker's role identity, knowledge scope, teaching intention and tone constraints.

[0063] Meanwhile, considering that immersive experiences require a high degree of consistency across multiple channels such as vision, hearing, and interaction, for example, when a wrench is tightening a bolt in the scene, there should be a sound of tightening, and the feel (such as force feedback effect) should change. This requires coordinating different generation services and rendering engines. Therefore, it is also necessary to further drive the generation and synchronous synthesis of multimodal content, and finally package and output it as an interactive immersive teaching scenario application package, thereby forming a complete, consistent, and interactive virtual environment.

[0064] Preferably, in one embodiment of the present invention, the method for driving the generation and synchronous synthesis of multimodal content specifically includes: Structured instructions are sent in parallel to the corresponding generative AI services, which include, for example, text-based 3D, text-based image, and text-based speech.

[0065] For the generated key equipment assets, the system prioritizes retrieving standard models from the benchmark digital asset library associated with the industry knowledge graph, and instructs the AI ​​to modify the parameterized state based on these models. Specifically, for key industry assets (such as specific models of centrifugal pumps and special tools), the system does not generate them entirely from scratch. Instead, it prioritizes retrieving high-precision basic models that have been verified by engineering from the benchmark digital asset library associated with the industry knowledge graph. The task of the generative AI is to modify the state or enhance the details based on these benchmark models, as instructed. For example, the instruction becomes: Add a 3mm long scratch to the mechanical seal mounting position on the CH100 pump benchmark model and generate a wet oil stain area on the ground below the pump body.

[0066] Through a spatiotemporal registration engine, all generated 3D models, animations, audio, and UI elements are calibrated and synchronized in a unified virtual coordinate system and timeline to ensure consistency in multimodal feedback during interaction. Specifically, this includes spatial registration: all 3D models are placed in a unified virtual coordinate system, ensuring that bolts can be precisely screwed into their holes and tools can be grasped correctly by the virtual hand. This relies on precise model origin and collider settings; timeline synchronization: a master timeline is established, binding all temporal events, such as perfectly synchronizing the audio file of the "virtual tutor uttering warning lines," the tutor's 3D lip-sync animation, and the timing of the warning text popping up on the UI; and interaction logic binding: the interaction rules defined in the script (such as clicking the pressure gauge to magnify the reading, and the wrench not being able to tighten the bolt when the torque is insufficient) are written into scripts executable by the game engine and bound to the corresponding 3D objects.

[0067] Finally, all generated models, textures, audio, animations, and script files are optimized and packaged according to the specifications of the target platform (such as PC-VR, all-in-one VR), outputting a standalone or integrated application package. Simultaneously, a scenario manifest file is automatically generated, serving as a digital identity for the scenario. The final product is a ready-to-use, high-fidelity, and highly interactive immersive learning scenario application. Students can freely explore and operate within this environment, receiving immediate feedback that conforms to real physical laws and instructional design. The manifest file ensures the traceability and manageability of the scenario.

[0068] Preferably, in one embodiment of the present invention, the manifest file records at least the teaching metadata of the current scenario, the traceability information of the industry knowledge graph nodes and teaching logic sequences on which it depends, the generation version and the target running platform. Specifically, it includes: scenario ID, name, teaching objective, estimated duration, industry knowledge graph node ID on which it depends, teaching logic sequence ID used, generation time, version number, applicable hardware, etc.

[0069] One embodiment of the present invention provides an intelligent construction system for immersive teaching scenarios based on generative AI and industrial big data. Please refer to [link / reference]. Figure 2 The diagram shows a schematic of the structure of an immersive teaching scenario intelligent construction system based on generative AI and industrial big data according to an embodiment of the present invention. The system includes a memory 201, a processor 202 and a computer program. The memory 201 is used to store the corresponding computer program, and the processor 202 is used to run the corresponding computer program. When the computer program runs in the processor 202, it can implement the methods described in steps S1 to S4.

[0070] Furthermore, the electronic device also includes a communication interface 203 for communication between the memory 201 and the processor 202.

[0071] The memory 201 may include high-speed RAM memory, and may also include nonvolatile memory, such as at least one disk storage.

[0072] If the memory 201, processor 202, and communication interface 203 are implemented independently, then the communication interface 203, memory 201, and processor 202 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 The bus is represented by a single line, but this does not mean that there is only one bus or one type of bus.

[0073] Optionally, in a specific implementation, if the memory 201, processor 202, and communication interface 203 are integrated on a single chip, then the memory 201, processor 202, and communication interface 203 can communicate with each other through an internal interface.

[0074] The processor 202 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0075] One embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the methods described in steps S1 to S4 when the computer program is running in the processor.

[0076] In summary, this invention first standardizes multi-source heterogeneous data from industry sites, extracts entities and relationships based on a predefined teaching element classification system, and constructs an industry knowledge graph containing multi-dimensional information on equipment, operation, faults, and safety. This transforms the raw, disorganized data into a semantically rich industry knowledge graph that can drive context generation, providing a data foundation for subsequent teaching contexts. Then, it receives teaching objectives described in natural language, parses and decomposes them into structured knowledge points, skill points, and competency points, and constructs a teaching logic sequence based on educational theory. This transforms human training needs into a machine-executable, structured teaching logic sequence, providing a large-scale teaching framework for context generation. The outline is then used to dynamically query and instantiate specific equipment, components, parameters, and fault cases from the industry knowledge graph based on the teaching logic sequence. This generates a contextual logic script that includes scene settings, role tasks, event sequences, and interactive feedback rules. This effectively integrates the generated industry knowledge graph and the teaching logic sequence to generate an immersive and teaching-targeted contextual logic script. Finally, based on the contextual logic script, structured instructions for different generative AI models are generated to drive the generation and synchronous synthesis of multimodal content. The final output is packaged into an interactive immersive teaching context application package, ensuring that the final teaching context not only conforms to industry realities but also closely aligns with the preset teaching objectives, thus guaranteeing teaching effectiveness.

[0078] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data, characterized in that, The method includes: Standardize the multi-source heterogeneous data from the industry site, extract entities and relationships based on a predefined teaching element classification system, and construct an industry knowledge graph containing multi-dimensional information on equipment, operation, faults, and safety. The teaching objectives described in natural language are received, analyzed and broken down into structured knowledge points, skill points and literacy points, and a logical sequence of teaching is constructed based on educational theory. Based on the teaching logic sequence, specific equipment, components, parameters and fault cases are dynamically queried and instantiated from the industry knowledge graph to generate a situational logic script that includes scene settings, role tasks, event sequences and interactive feedback rules. Based on the scenario logic script, structured instructions are generated for different generative AI models, driving the generation and synchronous synthesis of multimodal content, and finally packaged and output as an interactive immersive teaching scenario application package.

2. The method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data according to claim 1, characterized in that, The entity and relation extraction based on the predefined teaching element classification system includes: Based on the entity types in the teaching element classification system, operation steps, safety warnings and process parameters are extracted from text data based on rule and pattern matching. The object detection results in the visual data are matched and associated with the equipment component entities in the industry knowledge graph; By aligning the feature events in time series data with the operation actions described in the text with timestamps, a causal relationship between parameter changes and operation behavior can be established.

3. The method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data according to claim 1, characterized in that, The constructed teaching logic sequence includes: The decomposed knowledge points, skill points, and literacy points are mapped to a teaching knowledge graph based on cognitive development levels; Based on skill dependencies and teaching principles, the mapped teaching elements are sorted to form a progressive logical sequence from theoretical understanding to practical training, and from normal procedures to troubleshooting. This sequence serves as the teaching logical sequence, and a difficulty adjustment factor is configured for the teaching logical sequence. The difficulty adjustment factor includes information completeness, time pressure, and interference factor level.

4. The method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data according to claim 1, characterized in that, The dynamic query and instantiation are performed through a dual-graph mapping engine, which performs the following operations: The abstract tasks in the teaching logic sequence are parsed to generate a joint query statement for the industry knowledge graph and the teaching knowledge graph; The decision-making process for query results is based on teaching suitability rules, which include priority given to teaching frequency, typicality of operation, richness of data, and difficulty suitability. Create an instance configuration file for the selected industry knowledge graph entity. The configuration file defines the initial state of the instance in the context, interactive attributes, and operational constraints based on real data.

5. The method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data according to claim 4, characterized in that, The method for generating the contextual logic script includes: The instantiated specific tasks are arranged in a dual-track system, integrating the teaching logic track and the narrative logic track, thus giving the teaching tasks a narrative background and plot challenges. At key nodes in the teaching logic sequence, based on historical fault probability data in the industry knowledge graph, fault or interference events that meet the teaching objectives are dynamically injected, and corresponding branch logic is generated. Finally, a machine-readable, structured script file is generated as a situational logic script, which specifies the object states, event triggering conditions, character behavior logic, and teaching assessment points in the virtual environment.

6. The method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data according to claim 1, characterized in that, The structured instructions for generating different generative AI models include: Based on the modal type of the elements in the scenario logic script, call the corresponding instruction template and fill in the script parameters; The instruction template for generating 3D or visual scenes is required to be described in a structured format of "subject object - precise attributes - spatial relationships - environmental context - style constraints"; The instruction template for generating role dialogue strictly defines the speaker's role, knowledge scope, teaching intention, and tone constraints.

7. The method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data according to claim 6, characterized in that, The driving generation and synchronous synthesis of multimodal content includes: The structured instructions are sent in parallel to the corresponding generative AI service; For the generated key equipment assets, standard models are retrieved first from the benchmark digital asset library associated with the industry knowledge graph, and the AI ​​is instructed to modify the parameterized state based on them. The spatiotemporal registration engine calibrates and synchronizes all generated 3D models, animations, audio, and UI elements in a unified virtual coordinate system and timeline, ensuring consistency of multimodal feedback during interaction.

8. The method for intelligently constructing immersive teaching scenarios based on generative AI and industrial big data according to claim 1, characterized in that, The packaged output also includes: An automatic scenario list file is generated. The list file records at least the teaching metadata of this scenario, the traceability information of the industry knowledge graph nodes and teaching logic sequence on which it depends, the generated version, and the target running platform.

9. An intelligent construction system for immersive teaching scenarios based on generative AI and industrial big data, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.

Images