Digital factory operation virtual simulation teaching method and system

Abstract

The invention provides a digital factory operation virtual simulation teaching method and system, and relates to the technical field of smart factories, and the method comprises the steps: collecting factory operation data, carrying out the standardization processing and correlation analysis, constructing a multi-dimensional data model, carrying out the feature extraction and semantic mining of the hierarchical data of each dimension, and constructing a multi-level digital twin factory model. Carrying out dynamic modeling, constructing a personalized knowledge graph, generating a learning demand list, matching the learning demand list with a digital twin factory model, and generating a multi-dimensional simulation environment; in the multi-dimensional simulation environment, performing analysis and intention recognition on an operation instruction of the learner, converting the operation instruction into a virtual operation, calculating a normative score of the virtual operation, generating corresponding feedback information based on the normative score and the personalized knowledge graph, updating model parameters based on a response of the learner to the feedback information, and performing simulation on the learner according to the updated model parameters. And completing iteration of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of smart factories, and in particular to a digital factory operation virtual simulation teaching method and system. Background Art

[0002] With the rapid development of intelligent manufacturing, digital factories have become an important development direction of modern manufacturing. Digital factories integrate advanced information technology, automation technology and manufacturing technology to achieve comprehensive digitalization, networking and intelligence of the production process, thereby improving production efficiency, reducing costs and improving product quality. However, the complexity and high integration of digital factories also bring great challenges to the training of relevant talents.

[0003] Traditional factory operation teaching methods mainly rely on theoretical explanations and limited field visits, which have many disadvantages. Due to the complexity of the actual factory environment and safety considerations, there is a lack of practical experience, and a large amount of money needs to be invested in the construction of training factories; field teaching has potential safety hazards; it is difficult to simulate emergency scenarios, which limits comprehensive learning; in order to solve these problems, computer simulation technology can be used to assist factory operation teaching.

[0004] To sum up, there is an urgent need for a new type of virtual simulation teaching method for digital factory operation to comprehensively improve the quality of training of digital factory-related talents. It can provide an immersive learning experience, cover all aspects of factory operations, support multiple scenario simulations and collaborative learning, and have flexible customization and expansion capabilities. Through the virtual simulation teaching method, the challenges currently faced by digital factory talent training can be effectively solved. The present invention can solve the problems in the prior art. Summary of the invention

[0005] The embodiment of the present invention provides a digital factory operation virtual simulation teaching method and system, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] A digital factory operation virtual simulation teaching method is provided, including:

[0008] The factory operation data is collected in real time through the IoT sensor network, and the factory operation data is standardized and analyzed for relevance based on the multi-source heterogeneous data fusion algorithm of the graph neural network to build a multidimensional data model. Based on the multidimensional data model, a deep learning method combining transfer learning and federated learning is used to extract features and mine semantics of data at each dimensional level to build a multi-level digital twin factory model;

[0009] Based on the multi-level digital twin factory model, a multi-agent reinforcement learning algorithm is used to dynamically model learners and build a personalized knowledge graph. Based on the personalized knowledge graph, the learner's knowledge blind spots and learning preferences are identified, and a list of learning needs is generated. The list of learning needs is matched with the digital twin factory model, and a multi-dimensional simulation environment is automatically generated through programmatic generation technology and virtual reality scene reconstruction algorithm.

[0010] In the multi-dimensional simulation environment, the learner's operation instructions are accepted, the operation instructions are parsed and the intention is recognized, the recognition results are converted into virtual operations in the multi-dimensional simulation environment, the virtual operations are executed and the corresponding state of the multi-dimensional simulation environment is updated, and the captured data is determined by capturing the learner's operation posture and expression data, and the captured data is compared with the preset standard action model, and the normative score of the virtual operation is calculated. Based on the normative score and the personalized knowledge graph, corresponding feedback information is generated, and based on the learner's response to the feedback information, the difficulty parameters of the multi-dimensional simulation environment are adjusted. At the same time, the data generated during the learning process, including operation records, learning trajectories and difficulty adjustment records, are input into the digital twin factory model, the model parameters are updated, and the model iteration is completed.

[0011] In an optional embodiment,

[0012] The multi-source heterogeneous data fusion algorithm based on graph neural network is used to standardize and analyze the correlation of the factory operation data, and to build a multidimensional data model including:

[0013] Standardize the factory operation data and convert the corresponding structured data, semi-structured data and unstructured data into a graph structure, where the data items are used as nodes of the graph and the relationships between the data items are used as edges of the graph, to obtain the initial factory operation data graph;

[0014] Initialize a meta-path generator, wherein the meta-path generator uses a recurrent neural network as a core structure, define the action space of the meta-path generator as all node type conversions in the initial plant operation data graph, train the meta-path generator using a policy gradient method, and use the trained meta-path generator to generate a set of candidate meta-paths;

[0015] Based on the candidate meta-path set, a hierarchical attention mechanism is constructed, including node-level attention, meta-path-level attention, and semantic-level attention, and features of nodes in the initial plant operation data graph are aggregated through the hierarchical attention mechanism to obtain a preliminary node representation;

[0016] Based on the preliminary node representation, the graph structure is learned, the intrinsic relationship of the plant operation data is adapted, the original adjacency matrix is ​​determined, a dynamic adjacency matrix of the initial plant operation data graph is calculated, and the dynamic adjacency matrix is ​​combined with the original adjacency matrix to obtain an updated adjacency matrix, and the updated adjacency matrix is ​​used to pass messages between nodes, and the node representation is updated to obtain an enhanced node representation;

[0017] Use corresponding encoders to encode structured data, semi-structured data, and unstructured data to obtain feature representations of each type of data;

[0018] The feature importance of different types of data features is calculated using the self-attention mechanism, and based on the calculated feature importance, the features of different types of data are weighted and fused to obtain fused features, which are then combined with the enhanced node representation as the input node representation of the graph neural network;

[0019] Two different views are generated for each input node representation, and the graph attention network model is used to process the two views separately to obtain the node contrast representation. The contrast loss function of the node contrast representation is defined, and the contrast loss function is added to the loss of the construction task corresponding to the factory multidimensional data model to obtain the total loss function. Through end-to-end training, the updated adjacency matrix, graph attention network model parameters and total loss function are optimized at the same time to obtain the optimized adjacency matrix, model parameters and final node representation to construct the factory multidimensional data model.

[0020] In an optional embodiment,

[0021] Initialize the meta-path generator, the meta-path generator uses a recurrent neural network as the core structure, defines the action space of the meta-path generator as all node type conversions in the initial plant operation data graph, trains the meta-path generator through a policy gradient method, and uses the trained meta-path generator to generate a set of candidate meta-paths including:

[0022] Constructing a recurrent neural network based on a long short-term memory network and determining the core structure of a meta-path generator, wherein the long short-term memory network includes a forget gate, an input gate, an output gate, and a unit state;

[0023] Determine the input dimension, hidden layer dimension and output dimension of the recurrent neural network, wherein the input dimension is set to the one-hot encoding length of the node type, the hidden layer dimension is set to a preset value, and the output dimension is set to the number of node type conversions;

[0024] Initializing the parameters of the recurrent neural network by using a weight initialization method based on uniform distribution, wherein the range of the uniform distribution is determined based on the number of input and output neurons;

[0025] Determine the action space based on the node types in the initial plant operation data graph, encode each action into an integer, and obtain an encoded action set;

[0026] Constructing a composite reward function, the composite reward function includes a path length reward, a type diversity reward, and a semantic relevance reward, wherein the path length reward is calculated based on the difference between the generated path length and the predefined optimal length, the type diversity reward is calculated based on the ratio of the number of unique types to the total number of types, and the semantic relevance reward is calculated based on the cosine similarity of the semantic vectors of the path start node and the end node; updating the policy parameters by calculating the product of the logarithmic probability of the parameterized strategy, the action taken, and the discounted return, and summing them, implementing a policy gradient-based reinforcement learning algorithm, setting the training parameters of the policy gradient method, and determining the learning rate attenuation path according to the preset attenuation rule;

[0027] Based on the current training parameters, multiple meta-paths are generated, the reward value of each meta-path is calculated using the composite reward function, and based on the reinforcement learning algorithm, the reward value is used to update the strategy parameters, and the cycle is iterated in sequence until a preset number of training rounds is reached to obtain a trained recurrent neural network;

[0028] Based on the trained recurrent neural network and the preset validation set, the average reward value is calculated, and the learning rate is adjusted according to the average reward value. When the preset reward value early stopping threshold is met, the early stopping mechanism is triggered to determine the final meta-path generator;

[0029] Based on the final meta-path generator, a meta-path set is generated. Based on the composite reward function, the meta-path set is scored. Based on the scoring results, the corresponding number of meta-paths are selected according to the preset preferential ratio to determine the candidate meta-path set.

[0030] In an optional embodiment,

[0031] Based on the multi-level digital twin factory model, a multi-agent reinforcement learning algorithm is used to dynamically model learners and build a personalized knowledge graph including:

[0032] Implement a multi-agent reinforcement learning algorithm in the model layer of the multi-level digital twin factory model, where each agent corresponds to a learner, and construct a state representation module based on a graph neural network to capture the learner's interactive behavior; determine the action space, including knowledge point learning sequence selection, learning resource recommendation, and learning difficulty adjustment; determine a hierarchical reward function, including short-term learning effect rewards, long-term knowledge structure optimization rewards, and factory operation efficiency improvement rewards; process the action space through a strategy network and a value network based on an attention mechanism;

[0033] The asynchronous advantage actor-critic algorithm is used to train multi-agents. Learners are dynamically modeled based on state representation modules, action space, reward function, policy network, and value network. According to the dynamic modeling results, a personalized knowledge graph is constructed for each learner. The personalized knowledge graph includes knowledge point nodes, multi-dimensional relationship edges, and time series attributes.

[0034] In an optional embodiment,

[0035] Based on the personalized knowledge graph, learners’ knowledge blind spots and learning preferences are identified, and a list of learning needs is generated, including:

[0036] Through the graph neural network model including the spatial-temporal convolution layer and the graph attention layer, the personalized knowledge graph is processed to generate the knowledge point embedding vector of each knowledge point. Through the density-based anomaly detection algorithm, the knowledge point embedding vector is used to calculate the density information of the knowledge point, including the local density and relative density. Through the meta-learning-based knowledge tracking model, the learner's mastery of each knowledge point is dynamically evaluated and the mastery value is determined. Based on the density information of the knowledge point and the mastery value of the knowledge point, the knowledge blind spots in the personalized knowledge graph are identified and a list of knowledge blind spots is generated.

[0037] Based on learners, knowledge points and learning resources, a heterogeneous information network is constructed, and the heterogeneous information network is input into a collaborative filtering algorithm based on a graph convolutional network, and the input is processed through a multi-head graph attention mechanism to generate a learner-knowledge point association matrix;

[0038] Based on the learner-knowledge point association matrix and combined with the density information of knowledge points, the learner's personalized learning preference is predicted. Based on reinforcement learning, a learning demand generator is set up. The knowledge blind spot list, personalized learning preferences and the density information of knowledge points are used as input to determine the state space, action space and reward function of the learning demand generator. The learning demand generator is trained to output a personalized learning demand list.

[0039] In an optional embodiment,

[0040] Parsing and identifying the intention of the operation instruction and converting the identification result into a virtual operation in a multi-dimensional simulation environment includes:

[0041] The operation instruction includes a natural language instruction, receiving the natural language instruction, converting the natural language instruction into a structured abstract syntax tree through a neural semantic parser, encoding the abstract syntax tree into a node sequence, encoding the node sequence using a bidirectional long short-term memory network, obtaining a hidden state representation of each node, calculating a correlation score between a query vector and a hidden state of each node using an attention mechanism, selecting the most relevant node as an operation parameter based on the correlation score, obtaining a selected node, and decoding the node information corresponding to the selected node into an operation parameter value;

[0042] Based on a pre-built plant operation knowledge graph, use a graph attention network to reason on the plant operation knowledge graph to obtain a knowledge enhancement vector, and fuse the knowledge enhancement vector with the abstract syntax tree to generate a fused representation;

[0043] By using knowledge distillation technology, the factory operation knowledge graph is compressed into a lightweight neural network, the fusion representation is input into the lightweight neural network, and the initial conversion result of the virtual operation is determined;

[0044] Modeling the mapping process of the initial conversion result of the virtual operation as a Markov decision process, setting a state space, an action space and a reward function, wherein the state in the state space includes the semantic representation of the current natural language instruction, the executed operation sequence and the current state of the simulation environment, and the action space includes all candidate virtual operations;

[0045] Based on the multi-head self-attention layer, feedforward neural network layer and output layer, a policy network is constructed, the state value function is estimated, and the critic network is determined. The parameters of the policy network and the critic network are updated alternately through the dual-network reinforcement learning algorithm to obtain the trained policy network.

[0046] Based on the training completion strategy network, the final conversion result of the virtual operation is determined.

[0047] In an optional embodiment,

[0048] The parameters of the policy network and the critic network are updated alternately through the dual-network reinforcement learning algorithm. The trained policy network includes:

[0049] Input the current state of the simulation environment into the policy network to obtain the action probability distribution, select a virtual operation based on a preset probability threshold and the action probability distribution, execute the virtual operation in the simulation environment, record the feedback of the simulation environment, and calculate the reward value;

[0050] Determine the value of the current state using the estimated state-value function in the critic network, calculate the difference between the reward value and the value, determine a performance metric, update the parameters of the policy network based on the performance metric, and update the parameters of the critic network using the temporal difference error;

[0051] Repeat the iteration until the policy network and the critic network meet the preset convergence conditions, and determine the trained policy network.

[0052] According to a second aspect of the embodiments of the present invention,

[0053] A digital factory operation virtual simulation teaching system is provided, comprising:

[0054] The first unit is used to collect factory operation data in real time through the Internet of Things sensor network, perform standardization processing and correlation analysis on the factory operation data based on the multi-source heterogeneous data fusion algorithm of the graph neural network, build a multi-dimensional data model, and based on the multi-dimensional data model, perform feature extraction and semantic mining on the data of each dimensional level through a deep learning method combining transfer learning and federated learning to build a multi-level digital twin factory model;

[0055] The second unit is used to dynamically model learners based on a multi-level digital twin factory model using a multi-agent reinforcement learning algorithm, build a personalized knowledge graph, identify learners' knowledge blind spots and learning preferences based on the personalized knowledge graph, generate a list of learning needs, match the list of learning needs with the digital twin factory model, and automatically generate a multi-dimensional simulation environment through programmatic generation technology and virtual reality scene reconstruction algorithm;

[0056] The second unit is used to accept the learner's operation instructions in the multi-dimensional simulation environment, parse and recognize the intention of the operation instructions, convert the recognition results into virtual operations in the multi-dimensional simulation environment, execute the virtual operations and update the corresponding state of the multi-dimensional simulation environment, determine the captured data by capturing the learner's operation posture and expression data, compare the captured data with the preset standard action model, calculate the normative score of the virtual operation, generate corresponding feedback information based on the normative score and the personalized knowledge graph, adjust the difficulty parameters of the multi-dimensional simulation environment based on the learner's response to the feedback information, and at the same time input the data generated in the learning process, including operation records, learning trajectories and difficulty adjustment records, into the digital twin factory model, update the model parameters, and complete the iteration of the model.

[0057] According to a third aspect of the embodiments of the present invention,

[0058] An electronic device is provided, comprising:

[0059] processor;

[0060] a memory for storing processor-executable instructions;

[0061] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0062] A fourth aspect of the embodiments of the present invention is:

[0063] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0064] In the embodiment of the present invention, through the Internet of Things sensor network, key operating data such as temperature, humidity, and pressure are collected in real time to improve the factory's data perception and monitoring capabilities; different types of data are integrated using graph neural networks to capture the complex relationships between data sources and enhance data correlation analysis; through transfer learning and federated learning, key features of factory data in various dimensions are effectively extracted to ensure privacy protection while improving model performance and helping to predict equipment failures or product quality problems; multi-agent reinforcement learning is used to tailor learning strategies for each learner to improve learning efficiency and effectiveness; the learner's knowledge map is updated in real time through the graph neural network to accurately identify learning blind spots and mastery levels and optimize learning content; and the learner's performance and knowledge map are used to predict the learning content. It generates a personalized list of learning needs, provides targeted learning suggestions and estimated duration; uses virtual reality technology to automatically generate multi-dimensional learning scenarios, enhances the immersion and targeting of the learning experience, and meets the needs of different learning preferences; through natural language processing and deep learning, it accurately analyzes the learner's operating instructions to ensure the smooth progress of simulation operations; highly realistic virtual environments render learners' operating feedback in real time to enhance immersion and help learners better master practical skills; by capturing learners' postures, expressions and other data, combined with standard models, it evaluates the standardization of operations in real time, improves operational quality and safety awareness; provides personalized text, visual, and voice feedback based on learners' operating performance and knowledge structure to promote learning effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A flowchart of a virtual simulation teaching method for digital factory operation according to an embodiment of the present invention;

[0066] Figure 2 It is a structural schematic diagram of a digital factory operation virtual simulation teaching system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0069] Figure 1 FIG. 1 is a flow chart of a virtual simulation teaching method for digital factory operation according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0070] S101. Collect factory operation data in real time through the IoT sensor network, perform standardization processing and correlation analysis on the factory operation data based on the multi-source heterogeneous data fusion algorithm of the graph neural network, build a multidimensional data model, and based on the multidimensional data model, perform feature extraction and semantic mining on the data of each dimensional level through a deep learning method combining transfer learning and federated learning to build a multi-level digital twin factory model;

[0071] First, deploy an IoT sensor network in the factory to collect factory operation data in real time. The network includes various types of sensors such as temperature sensors, humidity sensors, pressure sensors, flow sensors, vibration sensors, etc., which are distributed in key areas such as production lines, warehouses, and logistics systems. Each sensor collects data at a fixed time interval (such as every 5 seconds) and transmits the data to a central data processing system through a low-power wide area network (such as LoRaWAN). For example, the data sequence that a temperature sensor on a production line may collect is: [25.3℃, 25.5℃, 25.4℃, 25.6℃, 25.5℃].

[0072] Next, a multi-source heterogeneous data fusion algorithm based on graph neural network is used to standardize and analyze the correlation of the collected factory operation data. First, different types of data are standardized and all data are mapped to the range of 0-1. For example, the temperature data is standardized as: [(25.3-25.0) / (26.0-25.0), (25.5-25.0) / (26.0-25.0), ...]. Then, a graph structure is constructed in which nodes represent different data sources (such as different sensors) and edges represent the association between data sources. This graph structure is processed using a graph convolutional network to capture the complex relationships between different data sources. Through multi-layer graph convolution operations, a feature representation that integrates multi-source data information is finally obtained.

[0073] Based on the fusion features obtained in the previous step, a multidimensional data model is constructed. The model includes time dimension, space dimension, equipment dimension and product dimension. The time dimension records the timestamp information of the data; the space dimension indicates the specific location of data collection; the equipment dimension contains information related to the equipment, such as equipment type, operating status, etc.; the product dimension includes product type, batch number and other information. For example, a fused data may be represented as: [2023-05-01 10:30:00, production line A, injection molding machine 1, product X, 0.75, 0.6, 0.8], where the last three values ​​represent the standardized temperature, pressure and vibration data respectively.

[0074] Then, based on the constructed multidimensional data model, feature extraction and semantic mining are performed on the data at each dimensional level through a deep learning method that combines transfer learning and federated learning. First, a pre-trained deep neural network (such as ResNet) is used as the basis of the feature extractor, and the network is adapted to the specific field of factory data through transfer learning. Specifically, the parameters of the first few layers of the pre-trained network are retained, and only the subsequent layers are fine-tuned to quickly adapt to the characteristics of the factory data. Exemplarily, the pre-trained image recognition network can be migrated to identify product defects on the production line.

[0075] At the same time, considering that data from different factories or production lines may have privacy protection requirements, federated learning is used for model training. Each participant (such as a different production line or factory) trains the model on local data and then uploads only the model parameters to the central server. The central server aggregates these parameters, updates the global model, and then distributes the updated global model to each participant. Through multiple rounds of iterations, a model that performs well on data from all parties is finally obtained without directly sharing the original data.

[0076] Based on feature extraction, semantic mining is performed. A method combining attention mechanism and long short-term memory network (LSTM) is used to capture long-term dependencies and key information in the data. For example, it can be identified that certain combinations of temperature and pressure may cause product quality problems, or the association between equipment vibration patterns and impending failures can be found.

[0077] Finally, based on the extracted features and mined semantic information, a multi-level digital twin factory model is constructed, including the physical layer, data layer, analysis layer, and application layer. The physical layer represents the actual factory facilities and equipment; the data layer stores and manages the raw data collected from the IoT sensor network; the analysis layer includes the various models and algorithms built in the previous steps; and the application layer provides various application functions based on the digital twin model, such as production optimization, predictive maintenance, quality control, etc.

[0078] In this embodiment, key operational data such as temperature, humidity, and pressure are collected in real time through the IoT sensor network to improve the factory's data perception and monitoring capabilities; different types of data are integrated using graph neural networks to capture the complex relationships between data sources and enhance data correlation analysis; through transfer learning and federated learning, key features of factory data in various dimensions are effectively extracted to ensure privacy protection while improving model performance and helping to predict equipment failures or product quality issues; a multi-level digital twin factory model is constructed to achieve comprehensive management of the factory's physical layer, data layer, and analysis layer, supporting advanced application functions such as production optimization, predictive maintenance, and quality control.

[0079] In an optional embodiment, a multi-source heterogeneous data fusion algorithm based on a graph neural network performs standardization processing and correlation analysis on the factory operation data, and constructing a multidimensional data model includes:

[0080] Standardize the factory operation data and convert the corresponding structured data, semi-structured data and unstructured data into a graph structure, where the data items are used as nodes of the graph and the relationships between the data items are used as edges of the graph, to obtain the initial factory operation data graph;

[0081] Initialize a meta-path generator, wherein the meta-path generator uses a recurrent neural network as a core structure, define the action space of the meta-path generator as all node type conversions in the initial plant operation data graph, train the meta-path generator using a policy gradient method, and use the trained meta-path generator to generate a set of candidate meta-paths;

[0082] Based on the candidate meta-path set, a hierarchical attention mechanism is constructed, including node-level attention, meta-path-level attention, and semantic-level attention, and features of nodes in the initial plant operation data graph are aggregated through the hierarchical attention mechanism to obtain a preliminary node representation;

[0083] Based on the preliminary node representation, the graph structure is learned, the intrinsic relationship of the plant operation data is adapted, the original adjacency matrix is ​​determined, a dynamic adjacency matrix of the initial plant operation data graph is calculated, and the dynamic adjacency matrix is ​​combined with the original adjacency matrix to obtain an updated adjacency matrix, and the updated adjacency matrix is ​​used to pass messages between nodes, and the node representation is updated to obtain an enhanced node representation;

[0084] Use corresponding encoders to encode structured data, semi-structured data, and unstructured data to obtain feature representations of each type of data;

[0085] The feature importance of different types of data features is calculated using the self-attention mechanism, and based on the calculated feature importance, the features of different types of data are weighted and fused to obtain fused features, which are then combined with the enhanced node representation as the input node representation of the graph neural network;

[0086] Two different views are generated for each input node representation, and the graph attention network model is used to process the two views separately to obtain the node contrast representation. The contrast loss function of the node contrast representation is defined, and the contrast loss function is added to the loss of the construction task corresponding to the factory multidimensional data model to obtain the total loss function. Through end-to-end training, the updated adjacency matrix, graph attention network model parameters and total loss function are optimized at the same time to obtain the optimized adjacency matrix, model parameters and final node representation to construct the factory multidimensional data model.

[0087] First, the factory operation data is standardized and different types of data are converted into a unified graph structure. For structured data, such as equipment operation parameters, production plans, etc., each data item is directly used as a node of the graph. For example, the equipment ID "Machine001" and its temperature value "25.5℃" are respectively used as two nodes, and the relationship "has_temperature" between them is used as the edge connecting the two nodes. For semi-structured data, such as sensor data in JSON format, it is disassembled into multiple key-value pairs, each key-value pair is used as a node, and the hierarchical relationship between the key-value pairs is used as an edge. For unstructured data, such as equipment maintenance logs, natural language processing technology is first used to extract key entities and relationships, and then the entities are used as nodes and the relationships are used as edges. In this way, an initial factory operation data graph containing all types of data is obtained.

[0088] Next, initialize the meta-path generator. A meta-path refers to a path pattern that connects nodes of different types in a heterogeneous graph. Exemplarily, in a factory data graph, there may be a meta-path such as "equipment-parameter-abnormality", which represents a path from the equipment node through the parameter node to the abnormal node. The meta-path generator uses a recurrent neural network (RNN) as the core structure, whose input is the current node type and the output is the next possible node type. The action space is defined as all possible node type conversions in the initial factory operation data graph, exemplarily from the "equipment" type to the "parameter" type. The meta-path generator is trained using the policy gradient method, and the reward function is designed to be the frequency and importance of the generated meta-path in the data graph. After training, the generator is used to generate a series of candidate meta-paths, such as ["equipment-parameter-abnormality", "equipment-maintenance record-failure", "product-quality indicator-production line"].

[0089] Based on the generated set of candidate meta-paths, a hierarchical attention mechanism is constructed. The mechanism includes three levels: node-level attention, meta-path-level attention, and semantic-level attention. Node-level attention calculates the importance of each node in its neighbors. For example, in the "equipment-parameter" relationship, the temperature parameter may be more important than the humidity parameter. Meta-path-level attention evaluates the importance of different meta-paths, such as the "equipment-parameter-abnormality" path may better reflect the equipment status than the "equipment-maintenance record-fault" path. Semantic-level attention considers the overall importance of different types of data (structured, semi-structured, unstructured). Through these three levels of attention mechanisms, the nodes in the initial plant operation data graph are feature aggregated to obtain a preliminary representation of each node.

[0090] After obtaining the preliminary node representation, the graph structure is further learned to adapt to the intrinsic relationship of the plant operation data. First, the similarity between nodes is calculated based on the preliminary node representation to obtain the original adjacency matrix. Then, considering the dynamic nature of the plant data, the dynamic adjacency matrix is ​​calculated. For example, if the temperature change patterns of two devices are found to be highly correlated, even if they are not directly connected in the original data, a connection will be established in the dynamic adjacency matrix. The dynamic adjacency matrix is ​​combined with the original adjacency matrix to obtain an updated adjacency matrix. This updated adjacency matrix is ​​used for message passing between nodes to update the representation of each node and obtain an enhanced node representation.

[0091] Different types of data are encoded using corresponding encoders. Structured data is encoded using a multi-layer perceptron; semi-structured data, such as sensor data in JSON format, uses a recursive neural network to process its hierarchical structure; unstructured data, such as text logs, is encoded using a pre-trained language model (such as BERT). Through these encoders, feature representations of various types of data are obtained.

[0092] Next, the importance of different types of data features is calculated using the self-attention mechanism. The self-attention mechanism allows the model to adaptively determine which features are more important. For example, when predicting equipment failure, features from sensor data may be more important than features from maintenance logs. Based on the calculated feature importance, different types of data features are weighted and fused to obtain a fused feature. This fused feature is combined with the previously obtained enhanced node representation as the input node representation of the graph neural network.

[0093] In order to further improve the generalization ability and robustness of the model, a contrastive learning mechanism is introduced. Two different views are generated for each input node representation, which can be achieved by adding random noise or using different data augmentation techniques. For example, for temperature data, a small random perturbation can be added as one view, and a slight translation of the time series can be used as another view. The two views are processed separately using the graph attention network model to obtain a contrastive representation of the node. The contrastive loss function is defined so that the representation similarity of the two views from the same node is maximized, while the representation similarity with other nodes is minimized.

[0094] The contrastive loss function is added to the main loss of the factory multidimensional data model building task to obtain the total loss function. Through end-to-end training, the updated adjacency matrix, graph attention network model parameters and total loss function are optimized simultaneously. The optimization process uses the back propagation algorithm and the stochastic gradient descent method. After multiple rounds of iterations, the optimized adjacency matrix, model parameters and the final representation of each node are finally obtained.

[0095] Based on these optimized results, a multidimensional data model of the factory is constructed. This model contains complex relationships and deep features between data, and can be used for various downstream tasks, such as anomaly detection, predictive maintenance, production optimization, etc. For example, the model can be used to predict the probability of equipment failure or analyze the impact of different production parameters on product quality.

[0096] In this embodiment, structured, semi-structured and unstructured data are integrated into a unified graph structure to break down data silos and improve the integrity of data analysis. The meta-path generator is used to automatically discover potential associations between data, revealing key paths in factory operations, such as the relationship between equipment parameters and anomalies, which is helpful for deep data mining. Through the node, meta-path and semantic-level attention mechanism, the model focuses on important nodes, paths and data types, so that the model can more accurately capture the key features and information in the factory data. By dynamically adjusting the connections between data nodes, the model's adaptability to changing relationships in factory operations is improved, and the accuracy of prediction and analysis is enhanced. Different types of data are feature encoded, and the self-attention mechanism is used to fuse features to ensure that various types of data play a full role in the fusion process and improve the expressiveness of the model. The generalization ability of the model is enhanced through the contrastive learning mechanism, making it more stable and accurate when processing diverse and noisy factory data. The finally optimized factory multidimensional data model can be applied to tasks such as anomaly detection, predictive maintenance and production optimization, improving the factory's operational efficiency, equipment reliability and product quality.

[0097] In an optional embodiment, a meta-path generator is initialized, the meta-path generator uses a recurrent neural network as a core structure, an action space of the meta-path generator is defined as all node type conversions in an initial plant operation data graph, the meta-path generator is trained by a policy gradient method, and a candidate meta-path set is generated using the trained meta-path generator, including:

[0098] Constructing a recurrent neural network based on a long short-term memory network and determining the core structure of a meta-path generator, wherein the long short-term memory network includes a forget gate, an input gate, an output gate, and a unit state;

[0099] Determine the input dimension, hidden layer dimension and output dimension of the recurrent neural network, wherein the input dimension is set to the one-hot encoding length of the node type, the hidden layer dimension is set to a preset value, and the output dimension is set to the number of node type conversions;

[0100] Initializing the parameters of the recurrent neural network by using a weight initialization method based on uniform distribution, wherein the range of the uniform distribution is determined based on the number of input and output neurons;

[0101] Determine the action space based on the node types in the initial plant operation data graph, encode each action into an integer, and obtain an encoded action set;

[0102] Constructing a composite reward function, the composite reward function includes a path length reward, a type diversity reward, and a semantic relevance reward, wherein the path length reward is calculated based on the difference between the generated path length and the predefined optimal length, the type diversity reward is calculated based on the ratio of the number of unique types to the total number of types, and the semantic relevance reward is calculated based on the cosine similarity of the semantic vectors of the path start node and the end node; updating the policy parameters by calculating the product of the logarithmic probability of the parameterized strategy, the action taken, and the discounted return, and summing them, implementing a policy gradient-based reinforcement learning algorithm, setting the training parameters of the policy gradient method, and determining the learning rate attenuation path according to the preset attenuation rule;

[0103] Based on the current training parameters, multiple meta-paths are generated, the reward value of each meta-path is calculated using the composite reward function, and based on the reinforcement learning algorithm, the reward value is used to update the strategy parameters, and the cycle is iterated in sequence until a preset number of training rounds is reached to obtain a trained recurrent neural network;

[0104] Based on the trained recurrent neural network and the preset validation set, the average reward value is calculated, and the learning rate is adjusted according to the average reward value. When the preset reward value early stopping threshold is met, the early stopping mechanism is triggered to determine the final meta-path generator;

[0105] Based on the final meta-path generator, a meta-path set is generated. Based on the composite reward function, the meta-path set is scored. Based on the scoring results, the corresponding number of meta-paths are selected according to the preset preferential ratio to determine the candidate meta-path set.

[0106] First, a recurrent neural network based on the long short-term memory network (LSTM) is constructed as the core structure of the meta-path generator. The LSTM network consists of a forget gate, an input gate, an output gate, and a cell state. The forget gate determines the information that needs to be discarded from the cell state, the input gate determines the new information that needs to be added to the cell state, and the output gate controls the output information. This structure can effectively capture long-term dependencies and is suitable for processing sequence data, such as meta-path generation tasks.

[0107] Determine the input dimension, hidden layer dimension, and output dimension of the recurrent neural network. The input dimension is set to the one-hot encoding length of the node type. For example, if there are 5 node types in the factory data graph (equipment, parameters, anomalies, maintenance records, production lines), the input dimension is 5. The hidden layer dimension is set to a preset value, such as 128 or 256, which needs to be adjusted according to the complexity of the specific task. The output dimension is set to the number of node type transitions, that is, the number of possible next node types. In this case, the output dimension is also 5.

[0108] Initialize the parameters of the recurrent neural network. Use a weight initialization method based on uniform distribution to ensure that the initial weight distribution is within a reasonable range. The range of the uniform distribution is determined based on the number of input and output neurons. For example, a uniform distribution in the range of [-1 / sqrt(fan_in), 1 / sqrt(fan_in)] can be used, where fan_in is the number of input neurons. This initialization method helps prevent the gradient vanishing or exploding problem.

[0109] The action space is determined based on the node types in the initial plant operation data graph. Each action represents a transition from one node type to another. Each action is encoded as an integer to obtain a set of encoded actions. For example, "equipment → parameter" can be encoded as 0, "parameter → abnormality" can be encoded as 1, and so on. This encoding method facilitates model processing and learning.

[0110] Construct a composite reward function to evaluate the quality of the generated meta-path. The composite reward function includes three components: path length reward, type diversity reward, and semantic relevance reward. The path length reward is calculated based on the difference between the generated path length and the predefined optimal length. For example, if the predefined optimal length is 3 and the generated path length is 4, a smaller penalty can be given. The type diversity reward is calculated based on the ratio of the number of unique types to the total number of types, and encourages the generation of paths containing a variety of different node types. The semantic relevance reward is calculated based on the cosine similarity of the semantic vectors of the start node and the end node of the path to ensure that the generated path is semantically meaningful.

[0111] Implement a policy gradient-based reinforcement learning algorithm. Update the policy parameters by calculating the product of the logarithmic probability of the parameterized policy, the action taken, and the discounted reward, and summing them. Set the training parameters of the policy gradient method, such as the initial value of the learning rate (exemplarily 0.001), the discount factor (exemplarily 0.99), the number of training rounds (exemplarily 10,000 rounds), etc. Determine the learning rate decay path according to the preset decay rule, and exemplarily decay the learning rate to 0.9 times the original every 1,000 rounds.

[0112] Based on the current training parameters, multiple meta-paths are generated. For example, 100 meta-paths are generated in each round. The reward value of each meta-path is calculated using a composite reward function. Based on the reinforcement learning algorithm, these reward values ​​are used to update the policy parameters. The cycle is repeated until the preset number of training rounds is reached to obtain a trained recurrent neural network.

[0113] After the training is completed, the average reward value is calculated based on the trained recurrent neural network and the preset validation set. According to the average reward value, the learning rate is adjusted. Exemplarily, if the average reward value does not improve for 5 consecutive rounds, the learning rate is halved. When the preset reward value early stopping threshold is met, the early stopping mechanism is triggered. Exemplarily, if the relative improvement of the average reward value for 10 consecutive rounds is less than 0.1%, the training is stopped. In this way, the final meta-path generator is determined.

[0114] Use the final meta-path generator to generate a large number of meta-paths to form a meta-path set. For example, 10,000 meta-paths are generated. Based on the previously defined composite reward function, each path in the meta-path set is scored. The meta-paths are sorted according to the scoring results, from high to low. According to the preset preferential ratio, the top 20% of the meta-paths are selected for example to determine the candidate meta-path set.

[0115] In this embodiment, the memory gate mechanism of LSTM is used to effectively capture long-term dependencies in data sequences, which is suitable for generating complex meta-paths. The node type conversion is encoded as an integer, making the model more efficient in processing node conversions. The compound reward function comprehensively considers the path length, type diversity, and semantic relevance to ensure that the generated meta-path is both reasonable and diverse, and has practical significance. The policy gradient method of reinforcement learning is used to iteratively update the model parameters to ensure that the meta-path generator is continuously optimized to generate high-quality paths. By dynamically adjusting the learning rate and early stopping mechanism, the model is prevented from overfitting or the training time is too long, thereby improving the training efficiency. The generated results are sorted according to the score of the meta-path, and the best meta-path is screened out to improve the quality of subsequent data analysis and modeling.

[0116] S102. Based on the multi-level digital twin factory model, a multi-agent reinforcement learning algorithm is used to dynamically model learners, build a personalized knowledge graph, identify learners' knowledge blind spots and learning preferences based on the personalized knowledge graph, generate a list of learning needs, match the list of learning needs with the digital twin factory model, and automatically generate a multi-dimensional simulation environment through programmatic generation technology and virtual reality scene reconstruction algorithm;

[0117] First, build a multi-level digital twin factory model. The model includes a physical layer, a data layer, a model layer, and an application layer. The physical layer contains the physical equipment, production lines, and personnel information in the factory; the data layer collects and stores various data from the physical layer; the model layer builds various mathematical models and simulation models based on the data layer; and the application layer provides various model-based application functions. For example, for an automobile manufacturing plant, the physical layer may include assembly lines, robots, workers, etc.; the data layer includes production data, quality data, equipment status data, etc.; the model layer may include production process models, equipment predictive maintenance models, etc.; the application layer may include production scheduling optimization, quality control, and other functions.

[0118] Next, a multi-agent reinforcement learning algorithm is used to dynamically model the learners. Each learner is regarded as an agent, and its status includes knowledge level, learning progress, learning preference, etc. The action space includes selecting learning content, adjusting learning strategies, etc. The reward function is designed based on learning effect, learning efficiency, and learning satisfaction. The deep Q network (DQN) is used as the core learning algorithm for each agent, and the learning stability is improved through technologies such as experience replay and target network. For example, for a worker who is learning welding skills, his status may include the level of welding theoretical knowledge, practical operation proficiency, preference for different welding methods, etc.; actions may include choosing to learn welding theory, conducting virtual welding exercises, adjusting learning difficulty, etc.; rewards may be calculated based on theoretical test scores, virtual welding quality scores, learning time, etc.

[0119] Based on the results of multi-agent reinforcement learning, a personalized knowledge graph is constructed for each learner. The knowledge graph includes nodes (representing knowledge points) and edges (representing the relationship between knowledge points). The knowledge graph is processed and updated using a graph neural network (GNN) to capture the complex relationships between knowledge points. For example, a welder's knowledge graph may include nodes such as "welding principles", "welding materials", and "welding processes", as well as the relationships between them such as preconditions and associations. The mastery of the nodes and the weights of the edges are continuously updated through the performance in the learning process.

[0120] Based on the personalized knowledge graph, learners’ knowledge blind spots and learning preferences are identified. Graph analysis algorithms, such as PageRank variants, are used to identify nodes in the knowledge graph with low mastery as knowledge blind spots. Learning preferences are identified by analyzing learners’ learning behaviors and performance on different types of knowledge points. For example, it may be found that a learner has obvious deficiencies in “welding materials” and prefers to learn through practice rather than theory.

[0121] Based on the identified knowledge blind spots and learning preferences, a learning needs list is generated. The list includes information such as knowledge points that need to be strengthened, recommended learning methods, and estimated learning time. For example, for the aforementioned welder, the learning needs list may include: strengthening "welding material" knowledge, it is recommended to learn through interactive virtual experiments, and the estimated learning time is 2 hours; consolidating "weld inspection" skills, it is recommended to practice through virtual reality simulation, and the estimated learning time is 1.5 hours.

[0122] Match the learning requirements list with the digital twin factory model. Through the semantic matching algorithm, the knowledge points and skills in the learning requirements are associated with the relevant components and processes in the digital twin factory model. For example, the "welding material" knowledge may be associated with the material library, welding station and other components in the model; the "weld inspection" skill may be associated with the quality control process.

[0123] Finally, a multi-dimensional simulation environment is automatically generated through programmatic generation technology and virtual reality scene reconstruction algorithm. Programmatic generation technology generates a variety of learning scenarios and tasks based on predefined rules and templates combined with randomness. The virtual reality scene reconstruction algorithm uses the 3D model and parameters in the digital twin factory model to build a highly realistic virtual environment. Exemplarily, the simulation environment generated for welders may include: an interactive virtual material laboratory where learners can conduct virtual experiments with various welding materials; a VR welding training scene that simulates a real factory environment, including different types of welding tasks and real environmental factors such as noise and light; an augmented reality (AR) application for weld quality inspection, where learners can observe and analyze various weld cases through AR devices.

[0124] The simulation environment is dynamically generated and adjusted according to the learners' personalized needs to ensure the pertinence of the learning content and the immersion of the learning experience. For example, for learners who prefer practical learning, the system will generate more virtual practice scenes; for learners with weak theoretical foundations, the system will add more theoretical knowledge point prompts and explanations in the virtual environment.

[0125] In this embodiment, a full range of digital factory simulation from physical equipment to application functions is provided to support complex production process management, equipment maintenance and quality control; multi-agent reinforcement learning is used to tailor learning strategies for each learner to improve learning efficiency and effectiveness; the learner's knowledge graph is updated in real time through a graph neural network to accurately identify learning blind spots and mastery levels and optimize learning content; a personalized learning needs list is generated based on the learner's performance and knowledge graph, and targeted learning suggestions and estimated duration are provided; virtual reality technology is used to automatically generate multi-dimensional learning scenarios to enhance the immersion and targeting of the learning experience and meet the needs of different learning preferences.

[0126] In an optional embodiment, based on a multi-level digital twin factory model, a multi-agent reinforcement learning algorithm is used to dynamically model learners, and building a personalized knowledge graph includes:

[0127] Implement a multi-agent reinforcement learning algorithm in the model layer of the multi-level digital twin factory model, where each agent corresponds to a learner, and construct a state representation module based on a graph neural network to capture the learner's interactive behavior; determine the action space, including knowledge point learning sequence selection, learning resource recommendation, and learning difficulty adjustment; determine a hierarchical reward function, including short-term learning effect rewards, long-term knowledge structure optimization rewards, and factory operation efficiency improvement rewards; process the action space through a strategy network and a value network based on an attention mechanism;

[0128] The asynchronous advantage actor-critic algorithm is used to train multi-agents. Learners are dynamically modeled based on state representation modules, action space, reward function, policy network, and value network. According to the dynamic modeling results, a personalized knowledge graph is constructed for each learner. The personalized knowledge graph includes knowledge point nodes, multi-dimensional relationship edges, and time series attributes.

[0129] First, a multi-agent reinforcement learning algorithm is implemented in the model layer of the multi-level digital twin factory model. The digital twin factory model includes the physical layer, data layer, model layer, and application layer. In the model layer, each agent corresponds to a learner, which is used to simulate and predict the learner's behavior. For example, for a smart manufacturing factory, there may be dozens to hundreds of learners, each of which is modeled as an independent agent.

[0130] A state representation module based on graph neural network is constructed to capture the interactive behavior of learners. This module uses graph convolutional network (GCN) to process the interaction data between learners and knowledge points and learning resources. The state representation includes information such as the learner's knowledge mastery, learning history, and interaction mode. For example, for a worker who is learning CNC machine tool operation, his state may include: node features: the mastery level of each knowledge point (such as "tool selection", "program writing", "quality control") (a value between 0-1); edge features: learning sequence, interaction frequency, learning time, etc.; global features: overall learning progress, learning efficiency score, etc.

[0131] Determine the action space, including knowledge point learning sequence selection, learning resource recommendation and learning difficulty adjustment. Knowledge point learning sequence selection refers to selecting the next knowledge point to learn; learning resource recommendation includes selecting suitable learning materials, such as videos, documents or virtual simulation courses; learning difficulty adjustment dynamically adjusts the content difficulty according to the learner's performance. For example, for CNC machine tool operation training, the action space may include: choosing to learn "tool wear detection" or "processing parameter optimization"; recommending "CNC programming advanced skills" video courses or "virtual CNC operation" simulation exercises; adjusting the current task difficulty from "medium" to "difficult".

[0132] Determine a hierarchical reward function, including short-term learning effect rewards, long-term knowledge structure optimization rewards, and factory operation efficiency improvement rewards. Short-term learning effect rewards are based on the performance of a single learning activity, such as test scores; long-term knowledge structure optimization rewards consider the integrity and balance of the knowledge graph; and factory operation efficiency improvement rewards are related to the application effect of the learning content in actual work. For example: short-term rewards, completing the "tool selection" module test, scoring 90 points, reward 0.9; long-term rewards, successfully linking the "tool selection", "processing parameters" and "quality control" knowledge points, reward 0.5; operational efficiency rewards, applying the learned knowledge to increase CNC processing efficiency by 5%, reward 1.0.

[0133] The action space is processed through the policy network and value network based on the attention mechanism. The policy network is used to generate the action probability distribution, and the value network estimates the state value. The attention mechanism allows the network to focus on the most relevant information. For example, when recommending learning resources, the attention mechanism may pay more attention to the learner's recent learning behavior and current knowledge blind spots.

[0134] The asynchronous advantage actor-critic (A3C) algorithm is used to train multiple agents. The A3C algorithm allows multiple agents to learn in parallel, improving training efficiency. Each agent maintains its own policy network and value network, and synchronizes with the global network regularly. During training, the agent collects experience by interacting with the environment and updates the network parameters. For example, 50 parallel working processes can be set, each of which simulates the learning process of a learner and synchronizes with the global network every 100 steps.

[0135] Based on the state representation module, action space, reward function, policy network, and value network, the learner is dynamically modeled. The dynamic modeling process continuously evaluates the learner's state, predicts the optimal action, and adjusts the model based on actual feedback. For example, the system may observe that a learner is making slow progress in "CNC programming" but performs well in "machine tool operation", and adjust the recommended learning path and resources accordingly.

[0136] Based on the dynamic modeling results, a personalized knowledge graph is constructed for each learner. The knowledge graph includes knowledge point nodes, multidimensional relationship edges, and time series attributes. Knowledge point nodes represent specific knowledge or skills; multidimensional relationship edges describe the relationship between knowledge points, such as pre-requisites, correlations, similarities, etc.; time series attributes record the time information during the learning process. For example, a personalized knowledge graph for a CNC operator may contain:

[0137] Nodes: "Tool selection" (Mastery: 0.8, last learning time: 2023-05-15); "Processing parameter optimization" (Mastery: 0.6, last learning time: 2023-05-20); "Quality control" (Mastery: 0.7, last learning time: 2023-05-18);

[0138] Edge: "Tool selection"->"Processing parameter optimization" (Relationship type: Predecessor, Weight: 0.9); "Processing parameter optimization"<->"Quality control" (Relationship type: Mutual influence, Weight: 0.8);

[0139] Sequence properties: Learning sequence: "Tool selection" -> "Quality control" -> "Processing parameter optimization"

[0140] Duration of studies: "Tool selection" (5 days), "Quality control" (3 days), "Processing parameters optimization" (in progress).

[0141] In this embodiment, the interactive behavior between learners and knowledge points is captured based on a graph neural network, and the learning path is adjusted in real time to provide an accurate personalized learning experience. The most suitable learning resources and learning difficulty are customized for each learner through dynamic adjustment of the action space (knowledge point selection, learning resource recommendation, and learning difficulty adjustment). Short-term and long-term reward functions are used to motivate learners to efficiently master knowledge and apply it to actual factory operations to improve factory operating efficiency. The learner's learning progress is predicted through a multi-agent reinforcement learning model, and the learning strategy is dynamically adjusted to improve the learning effect. A personalized knowledge graph is built for each learner to track learning progress and knowledge mastery, and to optimize the learning path and resource allocation.

[0142] In an optional embodiment, based on the personalized knowledge graph, identifying the learner's knowledge blind spots and learning preferences, and generating a learning needs list includes:

[0143] Through the graph neural network model including the spatial-temporal convolution layer and the graph attention layer, the personalized knowledge graph is processed to generate the knowledge point embedding vector of each knowledge point. Through the density-based anomaly detection algorithm, the knowledge point embedding vector is used to calculate the density information of the knowledge point, including the local density and relative density. Through the meta-learning-based knowledge tracking model, the learner's mastery of each knowledge point is dynamically evaluated and the mastery value is determined. Based on the density information of the knowledge point and the mastery value of the knowledge point, the knowledge blind spots in the personalized knowledge graph are identified and a list of knowledge blind spots is generated.

[0144] Based on learners, knowledge points and learning resources, a heterogeneous information network is constructed, and the heterogeneous information network is input into a collaborative filtering algorithm based on a graph convolutional network, and the input is processed through a multi-head graph attention mechanism to generate a learner-knowledge point association matrix;

[0145] Based on the learner-knowledge point association matrix and combined with the density information of knowledge points, the learner's personalized learning preference is predicted. Based on reinforcement learning, a learning demand generator is set up. The knowledge blind spot list, personalized learning preferences and the density information of knowledge points are used as input to determine the state space, action space and reward function of the learning demand generator. The learning demand generator is trained to output a personalized learning demand list.

[0146] First, the personalized knowledge graph is processed through a graph neural network model that includes a spatial-temporal convolution layer and a graph attention layer. This model can simultaneously capture the spatial relationship between knowledge points and the temporal characteristics of the learning process. The spatial-temporal convolution layer uses a three-dimensional convolution operation while considering the topological structure and time dimension of the knowledge graph. The graph attention layer highlights important knowledge associations by calculating the attention weights between nodes.

[0147] For example, for a personalized knowledge graph containing 100 knowledge points, the convolution kernel size of the spatial-temporal convolution layer can be set to 3x3x3 (spatial dimension 2, temporal dimension 1), the step size is 1, and the padding is 1. The graph attention layer can be set to 8 attention heads, and the output dimension of each head is 64. After this processing, each knowledge point will generate a fixed-dimensional knowledge point embedding vector, such as 512 dimensions.

[0148] Next, the density information of knowledge points is calculated using the knowledge point embedding vector through a density-based anomaly detection algorithm. A variant of the local outlier factor (LOF) algorithm is used here to calculate the local density and relative density of each knowledge point. The local density reflects the compactness of the knowledge point in its neighborhood, while the relative density compares the local density of the knowledge point with its neighbors.

[0149] For example, for the knowledge point "CNC programming", the local density may be calculated to be 0.8 and the relative density may be 1.2. This indicates that "CNC programming" is in a relatively dense area in the knowledge graph, but slightly sparser than its neighbors.

[0150] Then, the learner's mastery of each knowledge point is dynamically evaluated through a knowledge tracking model based on meta-learning. The model uses a meta-learning strategy and can quickly adapt to the learning patterns of different learners. The model input includes the learner's historical learning data, knowledge point characteristics, and current learning status, and the output is the mastery value of each knowledge point.

[0151] For example, for a learner, the model may give the following mastery assessment results: "CNC programming basics": 0.85, "tool selection": 0.92, "machining parameter optimization": 0.73, "quality control": 0.68.

[0152] Based on the density information and mastery value of knowledge points, the knowledge blind spots in the personalized knowledge graph are identified. The criteria for determining knowledge blind spots can be set as: knowledge points with a relative density lower than a threshold (such as 0.8) and a mastery value lower than a threshold (such as 0.7). In this way, those knowledge points that are important in the knowledge structure but not mastered by learners can be found.

[0153] The resulting list of knowledge blind spots might be as follows: "Processing parameter optimization" (relative density: 0.75, mastery: 0.73); "Quality control" (relative density: 0.68, mastery: 0.68).

[0154] Next, a heterogeneous information network is constructed based on learners, knowledge points and learning resources. This network contains three types of nodes: learners, knowledge points and learning resources, as well as multiple relationship edges between them. For example, the learner-knowledge point edge represents the learning history, and the knowledge point-learning resource edge represents the knowledge scope covered by the resource.

[0155] The constructed heterogeneous information network is input into the collaborative filtering algorithm based on graph convolutional network. The algorithm uses graph convolution layer to extract the structural features of the network, and then processes the input through the multi-head graph attention mechanism. Multi-head attention allows the model to learn the relationship between nodes from different angles, improving the model's expressiveness.

[0156] For example, 3 graph convolution layers can be set, each with an output dimension of 64, and then an 8-head attention mechanism is used, with an output dimension of 32 for each head. Finally, a learner-knowledge point association matrix is ​​generated through a fully connected layer.

[0157] Based on the learner-knowledge point association matrix and combined with the density information of knowledge points, the learner's personalized learning preference is predicted. This step uses the idea of ​​collaborative filtering, but also takes into account the characteristics of knowledge structure. For example, if a learner performs well in a high-density knowledge point area, he may be more suitable for a systematic learning method.

[0158] Predicted personalized learning preferences may include: learning style: systematic learning, optimal learning time: 40-60 minutes per session, and preferred resource types: video tutorials, interactive simulations.

[0159] Finally, a learning demand generator is set based on reinforcement learning. The knowledge blind spot list, personalized learning preferences, and density information of knowledge points are used as input to determine the state space, action space, and reward function of the learning demand generator.

[0160] The state space includes: current knowledge blind spots, learning progress, time budget, etc.

[0161] The action space includes: selecting learning topics, determining learning duration, recommending learning resources, etc.

[0162] The reward function considers: knowledge blind spot coverage, learning efficiency improvement, personalized preference matching, etc.

[0163] The training learning demand generator uses the deep Q network (DQN) algorithm. For example, the experience replay buffer size can be set to 10,000, the target network update frequency can be set to 100 steps, and the discount factor can be set to 0.99.

[0164] After the training is completed, the learning needs generator can output a personalized learning needs list, exemplarily;

[0165] Learning topic: "Processing parameter optimization", recommended duration: 45 minutes, recommended resources: interactive virtual CNC machine simulation software;

[0166] Learning topic: "Basics of Quality Control", recommended duration: 30 minutes, recommended resources: micro-course video "Key Points of CNC Machining Quality Control";

[0167] Learning topic: "Relationship between processing parameters and quality control", recommended duration: 60 minutes, recommended resources: case study discussion.

[0168] In this embodiment, the graph neural network is combined with temporal and spatial features to dynamically evaluate the learner's mastery of knowledge points, effectively identify knowledge blind spots, and provide personalized learning paths; based on density and mastery, the learner's weak points in the knowledge structure are accurately located, which is conducive to key breakthroughs; through heterogeneous information networks and collaborative filtering algorithms, learning behavior and knowledge density information are combined to accurately predict learning style, duration and preferred resource types to improve learning efficiency; using a reinforcement learning learning demand generator, personalized recommendations on learning topics, duration and resources are made to ensure that the learning path is optimized to meet individual needs; combined with dynamic evaluation and learning preferences, the learner's learning experience and effect are optimized, and the comprehensiveness and depth of knowledge mastery are improved.

[0169] S103. In the multi-dimensional simulation environment, accept the learner's operation instructions, parse and identify the intention of the operation instructions, convert the identification results into virtual operations in the multi-dimensional simulation environment, execute the virtual operations and update the corresponding state of the multi-dimensional simulation environment, determine the captured data by capturing the learner's operation posture and expression data, compare the captured data with the preset standard action model, calculate the normative score of the virtual operation, generate corresponding feedback information based on the normative score and the personalized knowledge graph, adjust the difficulty parameters of the multi-dimensional simulation environment based on the learner's response to the feedback information, and at the same time input the data generated during the learning process, including operation records, learning trajectories and difficulty adjustment records, into the digital twin factory model, update the model parameters, and complete the iteration of the model.

[0170] First, learners' operating instructions are received in a multi-dimensional simulation environment. The simulation environment is a highly realistic virtual factory scene that includes various industrial equipment, tools and materials. Learners can input operating instructions through virtual reality devices (such as VR headsets and handles) or interactive devices such as touch screens. For example, learners may input the instruction "start the CNC machine tool".

[0171] Next, the received operation instructions are parsed and the intention is recognized. This step uses natural language processing technology to accurately understand the learner's intention by combining contextual information and the learner's operation history. The system uses an intention recognition model based on deep learning. The model has been trained with a large amount of industrial operation instruction data and can recognize various complex operation intentions with high accuracy.

[0172] For example, for the instruction "start CNC machine tool", the system may recognize the following intentions:

[0173] Main Intent: Start the device;

[0174] Target equipment: CNC machine tools;

[0175] Operation type: power on;

[0176] Operational urgency: Routine;

[0177] Then, the recognized intention is converted into specific virtual operations in the multi-dimensional simulation environment. The system maintains a detailed operation mapping table to convert high-level intentions into specific low-level operation sequences. For the example of "starting the CNC machine tool", the converted virtual operation sequence may include: checking the safety status around the machine tool, turning on the main power switch, waiting for the system self-check to complete, initializing the machine tool control panel, and confirming the machine tool ready status.

[0178] The system performs these virtual operations and updates the state of the multi-dimensional simulation environment in real time. Each operation triggers a series of environmental changes, such as device state changes, sound effects, visual feedback, etc. These changes are accurately calculated and rendered by the simulation engine to ensure that learners can get a realistic operation experience.

[0179] At the same time, the system captures the learner's operating posture and expression data through various sensors. This includes using a depth camera to capture the learner's body posture, a facial recognition camera to capture expression changes, and various sensors built into the VR device to collect hand movement data. For example, the system may capture the following data:

[0180] Body posture: standing position, upper body tilt angle, arm extension degree;

[0181] Facial expressions: concentration, tension, satisfaction;

[0182] Hand movements: key accuracy, operation fluency, and force control;

[0183] Next, the captured data is compared with the preset standard action model. The standard action model is constructed from the operation data of industry experts and senior workers, representing best practices and safety specifications. The system uses a dynamic time warping algorithm to calculate the similarity between the learner's operation trajectory and the standard model, thereby deriving a normative score for the virtual operation.

[0184] For example, for the operation of "starting a CNC machine tool", the system may calculate the following normative scores:

[0185] Operation sequence correctness: 95%;

[0186] Movement smoothness: 88%;

[0187] Security awareness: 92%

[0188] Overall normativeness: 91%;

[0189] Based on the normative score and personalized knowledge graph, the system generates corresponding feedback information. The personalized knowledge graph contains information such as the learner's knowledge structure, learning history, and preferences. The feedback generator comprehensively considers the operational normativeness, knowledge mastery, and learning style to generate the most suitable feedback for the current learner.

[0190] Feedback may include:

[0191] Text prompts: concise and clear operating suggestions, such as "next time when checking the safety conditions around the machine tool, please observe the ground conditions more carefully";

[0192] Visual guidance: Using augmented reality technology to highlight areas that require attention or correct operation paths in the simulation environment;

[0193] Voice guidance: Provide real-time operation guidance and encouragement through AI voice assistant;

[0194] The system closely monitors learners’ responses to feedback information, including the degree of improvement in subsequent operations, changes in learning enthusiasm, etc. Based on these responses, the system dynamically adjusts the difficulty parameters of the multi-dimensional simulation environment. The difficulty parameters involve multiple dimensions, such as operation complexity, time pressure, environmental interference, etc.

[0195] For example, if the learner improves the operational norms quickly, the system may make the following adjustments:

[0196] Increase the complexity of the operating steps, such as introducing more safety check items; shorten the allowed operating time to simulate the time pressure in actual production; increase environmental interference, such as simulating the operating noise of other equipment.

[0197] At the same time, the system inputs various data generated during the learning process into the digital twin factory model. These data include:

[0198] Operation records: detailed operation sequence, timestamp, operation parameters, etc.;

[0199] Learning track: knowledge point mastering progress, skill improvement curve, common error types, etc.

[0200] Difficulty adjustment records: difficulty parameter change history, adjustment trigger conditions, adjustment effect evaluation, etc.;

[0201] The digital twin factory model uses this data to update its parameters and complete the iterative optimization of the model. The update process uses an online learning algorithm that can adjust the model structure and parameters in real time to make it more accurately reflect the operating status of the real factory and the skill level of the learner.

[0202] For example, the model may make the following updates based on the learning data: adjust the standard execution time of certain operations, update the probability distribution of equipment failures, optimize the evaluation algorithm of skill mastery, and improve the difficulty adjustment strategy.

[0203] Through this continuous data collection and model updating, the digital twin factory continues to evolve into a dynamic and accurate virtual environment that can provide the most appropriate learning experience for learners of different levels.

[0204] In this embodiment, through natural language processing and deep learning, the learner's operating instructions are accurately parsed to ensure the smooth progress of the simulation operation; the highly realistic virtual environment renders the learner's operation feedback in real time to enhance the sense of immersion and help learners better master practical skills; by capturing the learner's posture, expression and other data, combined with the standard model, the operation standardization is evaluated in real time to improve the operation quality and safety awareness; according to the learner's operation performance and knowledge structure, personalized text, visual, and voice feedback are provided to promote learning effects; according to the learner's progress, the task complexity, time pressure and environmental interference are automatically adjusted to ensure moderate challenges and enhance learning motivation; by continuously collecting operation and learning data, the digital twin model is updated online to dynamically reflect the real factory operation status and provide learners with more accurate learning support.

[0205] In an optional embodiment, parsing and identifying the intention of the operation instruction, and converting the identification result into a virtual operation in a multi-dimensional simulation environment includes:

[0206] The operation instruction includes a natural language instruction, receiving the natural language instruction, converting the natural language instruction into a structured abstract syntax tree through a neural semantic parser, encoding the abstract syntax tree into a node sequence, encoding the node sequence using a bidirectional long short-term memory network, obtaining a hidden state representation of each node, calculating a correlation score between a query vector and a hidden state of each node using an attention mechanism, selecting the most relevant node as an operation parameter based on the correlation score, obtaining a selected node, and decoding the node information corresponding to the selected node into an operation parameter value;

[0207] Based on a pre-built plant operation knowledge graph, use a graph attention network to reason on the plant operation knowledge graph to obtain a knowledge enhancement vector, and fuse the knowledge enhancement vector with the abstract syntax tree to generate a fused representation;

[0208] By using knowledge distillation technology, the factory operation knowledge graph is compressed into a lightweight neural network, the fusion representation is input into the lightweight neural network, and the initial conversion result of the virtual operation is determined;

[0209] Modeling the mapping process of the initial conversion result of the virtual operation as a Markov decision process, setting a state space, an action space and a reward function, wherein the state in the state space includes the semantic representation of the current natural language instruction, the executed operation sequence and the current state of the simulation environment, and the action space includes all candidate virtual operations;

[0210] Based on the multi-head self-attention layer, feedforward neural network layer and output layer, a policy network is constructed, the state value function is estimated, and the critic network is determined. The parameters of the policy network and the critic network are updated alternately through the dual-network reinforcement learning algorithm to obtain the trained policy network.

[0211] Based on the training completion strategy network, the final conversion result of the virtual operation is determined.

[0212] First, the system receives natural language operation instructions input by the learner. These instructions may be verbal voice input or text form. For example, the learner may input "Please start the injection molding machine on production line 3 and set the mold temperature to 180 degrees."

[0213] Next, the system uses a neural semantic parser to convert natural language instructions into a structured abstract syntax tree. This process involves multiple steps: first, the input text is converted into a dense vector representation through a pre-trained word embedding model; then, an encoder-decoder model based on the Transformer architecture is used to generate an abstract syntax tree. In this example, the generated abstract syntax tree may contain the following nodes:

[0214] Root node: start the device;

[0215] Subnode 1: Equipment type (injection molding machine);

[0216] Child node 2: location (production line 3);

[0217] Subnode 3: Set parameters (mold temperature);

[0218] Subnode 4: parameter value (180 degrees).

[0219] The system then encodes the abstract syntax tree into a sequence of nodes. This process uses a depth-first search algorithm to traverse the abstract syntax tree and convert each node and its attributes into a vector representation. For the above example, the node sequence may be as follows:

[0220] [Start equipment, equipment type, injection molding machine, location, production line 3, set parameters, mold temperature, parameter value, 180 degrees].

[0221] Next, the system uses a bidirectional long short-term memory network (Bi-LSTM) to encode the node sequence. The Bi-LSTM network contains two LSTM layers, forward and backward, which can capture long-term dependencies and contextual information in the sequence. After Bi-LSTM encoding, each node obtains a hidden state representation that combines the information of the node itself and its contextual information in the entire instruction.

[0222] The system then uses the attention mechanism to calculate the correlation score between the query vector and the hidden state of each node. The query vector is a learnable parameter that represents the focus of the system. The attention mechanism calculates the dot product of the query vector and the hidden state of each node, and then normalizes it with softmax to obtain the attention weight of each node.

[0223] Based on the calculated relevance scores, the system selects the most relevant nodes as operating parameters. In this example, the system may select the nodes "Injection Molding Machine", "Production Line 3", "Mold Temperature", and "180 degrees". The system decodes the node information corresponding to these selected nodes into specific operating parameter values, such as equipment ID, location coordinates, parameter type, and value.

[0224] At the same time, the system uses a graph attention network (GAT) for knowledge reasoning based on a pre-built factory operation knowledge graph. The factory operation knowledge graph contains rich information such as the relationship between equipment, operation procedures, parameter limits, etc. GAT can effectively extract knowledge related to the current operation by weighting the nodes and edges in the knowledge graph. For example, it may extract information such as the safe operating temperature range and startup sequence of the injection molding machine.

[0225] The knowledge enhancement vector obtained by reasoning is fused with the previously generated abstract syntax tree to generate a fused representation. This fusion process is implemented using a multi-layer perceptron, which concatenates the knowledge vector and the syntax tree vector and inputs them into the multi-layer perceptron to obtain a fused high-dimensional vector representation.

[0226] Through knowledge distillation, the huge factory operation knowledge graph is compressed into a lightweight neural network. This process involves designing a teacher network and a student network. The teacher network is a large graph neural network that can directly reason on the knowledge graph; the student network is a small feedforward neural network. By having the student network imitate the output of the teacher network, a lightweight model that can quickly reason about knowledge is finally obtained.

[0227] The system inputs the fused representation into the lightweight neural network after knowledge distillation to obtain the initial conversion result of the virtual operation. This initial result contains the basic operation sequence and parameter settings, but may not be accurate or complete enough.

[0228] In order to further optimize the conversion results, the system models the mapping process of virtual operations as a Markov decision process. In this framework, the system defines the state space, action space and reward function:

[0229] The state space consists of three main components: the semantic representation of the current natural language instruction, which is a feature vector extracted using a pre-trained language model such as BERT; the sequence of operations that have been executed, which uses a fixed-length queue to store the most recent N operations executed; and the current state of the simulation environment, including the working status of each device, parameter settings, production line status, etc.

[0230] The action space contains all possible virtual operations, for example: starting the device, stopping the device, adjusting parameters, switching modes, and performing quality inspection;

[0231] The reward function design takes into account multiple factors:

[0232] Correctness of operation: similarity to the standard operation sequence defined by experts.

[0233] Efficiency: The number of steps required to complete an instruction.

[0234] Safety: Whether there is any violation of safety operating procedures.

[0235] Production effects: the impact of virtual operations on product quality and production capacity.

[0236] Next, the system built a policy network and a critic network based on the Transformer architecture. The policy network consists of a multi-head self-attention layer, a feedforward neural network layer, and an output layer to generate action probability distributions. The critic network has a similar structure, but the output layer is a scalar that estimates the value of the current state.

[0237] The system uses a dual-network reinforcement learning algorithm (such as TD3 or SAC) to alternately update the parameters of the policy network and the critic network. During the training process, the system simulates a large number of operation scenarios, allowing the policy network to continuously try different operation sequences and adjust the network parameters based on the rewards obtained. For example, for the scenario of starting an injection molding machine, the system may learn the following strategy: first check the safety conditions around the equipment, confirm that the mold has been correctly installed, turn on the main power, wait for the system self-check to complete, set the mold temperature (within a safe range), start the injection molding machine, and monitor the initial operating status;

[0238] Through a large number of iterations, the policy network gradually learned how to accurately and efficiently convert natural language instructions into optimal virtual operation sequences.

[0239] Finally, the system uses the trained policy network to optimize the initial conversion results and determine the final conversion results of the virtual operation. This final result not only includes detailed operation steps, but also takes into account safety, efficiency and specific environmental factors.

[0240] In this embodiment, through knowledge distillation, the complex factory operation knowledge graph is compressed into a lightweight neural network, which improves the reasoning efficiency and enables the model to quickly perform knowledge reasoning in a low-resource environment; the conversion of virtual operations is modeled as a Markov decision process, combined with natural language processing and operation status to ensure accurate semantic parsing of instructions and optimize the operation sequence; through repeated training with reinforcement learning algorithms (such as TD3 or SAC), the system gradually learns efficient and safe operation strategies to ensure that the simulated operation is consistent with the actual factory operation; the policy network continuously optimizes the conversion results of virtual operations, comprehensively considers safety, operational correctness and production efficiency, and generates a more intelligent and accurate operation sequence; the policy network learns autonomously in a simulation environment, and improves its ability to respond to complex factory operation scenarios through continuous iteration to achieve intelligent operation optimization.

[0241] In an optional embodiment, the parameters of the policy network and the critic network are updated alternately through a dual-network reinforcement learning algorithm, and the trained policy network includes:

[0242] Input the current state of the simulation environment into the policy network to obtain the action probability distribution, select a virtual operation based on a preset probability threshold and the action probability distribution, execute the virtual operation in the simulation environment, record the feedback of the simulation environment, and calculate the reward value;

[0243] Determine the value of the current state using the estimated state-value function in the critic network, calculate the difference between the reward value and the value, determine a performance metric, update the parameters of the policy network based on the performance metric, and update the parameters of the critic network using the temporal difference error;

[0244] Repeat the iteration until the policy network and the critic network meet the preset convergence conditions, and determine the trained policy network.

[0245] First, the current state of the simulation environment is input into the policy network. The state of the simulation environment includes information in multiple dimensions, such as equipment working status, parameter settings, production line status, etc. This information is encoded into a high-dimensional vector as input to the policy network. The policy network is a neural network based on a deep learning architecture that contains multiple processing layers.

[0246] After receiving the state vector, the policy network passes through a series of neuron transmissions and activation function operations, and finally generates an action probability distribution in the output layer. This probability distribution represents the probability of executing each possible action in the current state.

[0247] Next, based on the preset probability threshold, the virtual operation is selected in combination with the action probability distribution. The system compares the probability of each action with the preset threshold and selects the action with the highest probability that exceeds the threshold as the execution object. If no action probability exceeds the threshold, the action with the highest probability may be selected or an action may be randomly selected, depending on the system's exploration strategy.

[0248] After selecting an action, the system executes the virtual operation in the simulation environment. The simulation environment simulates the execution process and updates the environment state based on the current state and the selected action. This process involves complex physical models and logical rules to simulate the actual operating environment as realistically as possible.

[0249] The system will record the feedback of the simulation environment on this operation. The feedback usually includes whether the operation is successfully executed, execution time, energy consumption, impact on other devices, etc. This feedback information is used to evaluate the effect of the operation.

[0250] Based on the feedback from the environment, the system calculates a reward value. The calculation of the reward value takes into account multiple factors, such as the correctness, efficiency, safety and impact on production of the operation. The design of the reward function directly affects the effect of reinforcement learning and needs to be carefully adjusted according to the specific application scenario.

[0251] At the same time, the system uses the estimated state value function in the critic network to determine the value of the current state. The structure of the critic network is similar to that of the policy network, but the output layer has only one node, which is used to estimate the value of the state. The critic network receives the current state as input, and after a series of neuron calculations, it outputs a scalar value as the estimated value of the state.

[0252] The system calculates the difference between the reward value and the state value estimated by the critic network to determine the performance metric. This difference reflects the relationship between the actual reward obtained and the expected value, and is used to evaluate the performance of the selected action.

[0253] Based on this performance metric, the system updates the parameters of the policy network. The goal of the update is to increase the probability of actions that produce high performance metrics. Specifically, the system uses a policy gradient method to adjust the network parameters in the direction that improves the performance metric. This process involves complex gradient calculations and parameter optimization algorithms.

[0254] At the same time, the system uses the temporal difference error to update the parameters of the critic network. The temporal difference error is the difference between the actual observed reward and the value predicted by the critic network. The system uses this error to adjust the parameters of the critic network to make its predictions more accurate. This process also involves gradient calculation and parameter optimization.

[0255] The system will iterate the above process repeatedly to continuously optimize the policy network and the critic network. In each iteration, the system will select an action and execute it, observe the environmental feedback and calculate the reward, and then update the parameters of the two networks.

[0256] This training process will continue until the policy network and the critic network meet the preset convergence conditions. Convergence conditions may include the average reward value reaching a certain threshold, the network output change amplitude is less than a certain threshold, the prediction error is less than a certain threshold, or the training round reaches a preset value. When these conditions are met, the system considers the training to be complete and determines the final policy network.

[0257] For example: training a smart factory operating system. In a certain round of iteration:

[0258] The current state of the simulation environment is: the injection molding machine of production line 3 is on standby, the mold temperature is 25°C, the raw material warehouse is 80% full, the conveyor belt is stopped, and the quality inspection station is idle. This state is encoded as a vector [0, 25, 0.8, 0, 1] and input into the policy network.

[0259] The strategy network outputs the probability distribution of actions: start the injection molding machine 0.7, set the mold temperature 0.2, open the conveyor belt 0.05, check the raw material inventory 0.03, and other actions 0.02.

[0260] Assuming the preset probability threshold is 0.6, the system selects the action of "starting the injection molding machine".

[0261] The system executes the operation of starting the injection molding machine in the simulation environment, simulating the entire startup process.

[0262] Feedback from the simulation environment: The injection molding machine started successfully, the startup time was 30 seconds, the energy consumption was 0.5kWh, and there were no safety issues.

[0263] Based on the feedback, the system calculates a reward value of +10.

[0264] The critic network estimates the current state value to be 8.

[0265] Computational performance metric: 10-8=2.

[0266] The system uses this positive performance measure to update the policy network, increasing the probability of choosing "Start injection molding machine" in similar states.

[0267] At the same time, the system uses the temporal difference error 2 to update the critic network to improve the accuracy of its state value estimation.

[0268] The system continues to the next round of iteration until the convergence conditions are met, such as the average reward value remains above 9.5 for 100 consecutive rounds and the change in the policy network output distribution is less than 0.01.

[0269] In this way, the system gradually learns the optimal operation strategy and can eventually accurately and efficiently convert operation instructions into specific virtual operation sequences.

[0270] In this embodiment, through the in-depth analysis of the state of the simulation environment by the policy network, the system can generate a probability distribution and select the optimal virtual operation, greatly improving the accuracy and efficiency of the operation; based on the feedback of the simulation environment, the system can instantly evaluate the operation effect, calculate the reward value, and use reinforcement learning to continuously optimize the operation strategy, gradually improving the execution efficiency and safety; through the value estimation and temporal difference error calculation of the critic network, the system can dynamically adjust the parameters of the policy network and the critic network to ensure the adaptability and accuracy of the model in different scenarios; the reward function design integrates factors such as operation correctness, efficiency and safety, so that the system can generate efficient and safe operation sequences that are more in line with production requirements; through multiple rounds of iterations and repeated training, the system continuously improves its decision-making ability in complex simulation environments, and can eventually converge to the optimal operation strategy, significantly improving the quality and productivity of industrial automation operations.

[0271] The present invention also provides an optional embodiment, comprising:

[0272] The implementation of the digital factory virtual simulation training system construction plan began with an in-depth analysis of the specific talent needs of the industrial Internet and intelligent manufacturing industries. According to the specific needs, the target school type and professional direction are determined, and detailed capacity training goals are formulated, including consulting, implementation, use and operation and maintenance of factory digital systems. At the same time, the expected learning outcomes and evaluation standards of the system are set to provide clear guidance for subsequent development and implementation.

[0273] In the overall system planning stage, design a system architecture that includes hardware, software, data, and application layers. Develop a comprehensive list of system function modules to ensure that all aspects of the digital factory are covered. At the same time, consider the scalability of the system, reserve interfaces for future upgrades and function additions, and formulate strict system security policies to ensure data security and user privacy.

[0274] The hardware environment is the basis for realizing the system functions. Select and purchase high-performance servers whose configurations must meet the requirements of 3D rendering and big data processing. Install multi-touch interactive screens and 3D visualization smart screens to ensure good display effects and interactive experience. Configure mobile devices such as workstation PADs, information magic cards, and barcode guns to simulate a real factory environment. Build a high-speed, stable wireless WiFi network to support multiple users operating online at the same time. Considering future development, VR / AR devices can be appropriately added to provide a more immersive experience.

[0275] The development and integration of software systems is the core of the entire solution. First, develop or purchase MES (manufacturing execution system) software, including modules such as production management, quality management, and equipment management. Then develop an identification resolution system to achieve product life cycle tracking. Design and implement a dynamic simulation system for production and operation data simulation to simulate the enterprise operation process. Develop an artificial intelligence-based student evaluation system to achieve automated and personalized learning evaluation. Finally, integrate these systems to ensure data flow and functional coordination.

[0276] This system uses wireless Wifi wiring and consists of hardware equipment such as multi-touch interactive screen, 3D visualization smart screen, server, display (keyboard), workstation PAD, information magic card and barcode gun. It is combined with MES system supporting software, identification resolution system supporting software based on automobile engine assembly and supply chain process management, production and operation data simulation dynamic simulation and artificial intelligence evaluation system to build a virtual simulation training teaching environment for digital factories that combines virtual and real.

[0277] Network connection: wireless Wifi;

[0278] Server: A high-performance server that carries the operation of the system, electronic sandbox, and 3D models;

[0279] Smart screen: Displays 3D projection of the twin intelligent production line and 3D supply chain process, as well as a full tour of the production line equipment and supply chain;

[0280] Computer display screen (keyboard): used for digital factory virtual simulation training system administrator operation.

[0281] PAD: divided into workstation PAD and lineside PAD, executes the digital factory virtual simulation training system to view, receive and send materials, execute orders and processing, and record the production information of the current workstation of the order;

[0282] Barcode gun: used to scan personnel barcodes, material barcodes, work-in-progress pallet barcodes and other information in the digital factory virtual simulation training system to realize electronic and convenient data traceability entry;

[0283] Virtual production line 2D electronic sandbox: presents the overall layout of the production site, executes the entire product processing and material flow process between workstations. Displays all data information and processing dynamics in the production process, intuitively displays the dashboard information in the digital factory virtual simulation training system, introduces role variables through human-computer interactive operations, and enables students to master the role of the digital factory virtual simulation training system in the production process and understand and actually operate the functions of the digital factory virtual simulation training system in scenario-based and entertaining teaching;

[0284] Magic Card: Complete equipment layout, production line setting operations, 3D supply chain navigation, execute DEMO software, display dynamic information on the dashboard, explain the operation process of the display station, and perform multi-view browsing of 3D smart production lines and supply chain department function navigation.

[0285] The digital factory virtual simulation training system creates a nearly realistic virtual workshop production environment for teachers and students, which can simulate various production scenarios of discrete manufacturing and actual supply chain flow scenarios. It can realize the simulation of processing scene actions of actual production lines and the application of identification resolution based on supply chain flow management in the production process. By running digital systems (MES system, identification resolution system, dynamic simulation of production and operation data simulation, and student evaluation system based on artificial intelligence), the interaction between the digital factory system and the virtual production line is realized.

[0286] Students play different roles in the production workshop, participate in the production process of the workshop and the management of various units and departments in the supply chain, operate MES system software, identification resolution system software, supply chain process management software and modern production management and operation data simulation dynamic simulation, and cooperate to complete various production order tasks. This allows students to intuitively experience the important role of factory digitalization in the production execution process, understand the core position of factory digitalization in the intelligent manufacturing and industrial Internet industries, and consolidate the learning content of various factory digitalization courses.

[0287] The construction of a virtual production environment is the key to providing students with a realistic experience. First, design and develop a comprehensive 2D electronic sandbox, including a lineside warehouse management module, production process simulation of 8 independent workstations, interaction between the MES system and the identification resolution system of the processing station, stereoscopic warehouse storage management, and AGV material distribution path planning and scheduling. Then create a high-precision 3D digital twin model to model the various components of the intelligent production line, such as robots, conveyor belts, etc., and build a complete supply chain 3D scene, covering the entire process from raw material suppliers to end customers. Realize the linkage between 3D models and real-time data to reflect the real production status. Finally, develop a dynamic simulation engine to achieve real-time simulation of the production process.

[0288] The design and development of interactive interfaces directly affect the user experience. Design intuitive and easy-to-use user interfaces to meet the needs of different roles such as students, teachers, and administrators. Develop modules that support the playback and management of high-definition teaching videos. Design and implement mobile APPs to support functions such as equipment inspection and product sampling. Develop AR / VR interactive modules to provide immersive equipment operation and maintenance training. Implement multi-person collaboration functions, support role-playing and team collaboration training, and make the learning process more vivid and interesting.

[0289] The identity resolution virtual simulation training system focuses on providing in-depth professional training on identity resolution technology in the industrial Internet. The training content of the system revolves around the core principles, technical implementation, system integration, and application in supply chain management, product traceability, etc. The core functions of the system are divided into two main parts: module training and comprehensive training.

[0290] At the module training level, a series of training modules are designed, each focusing on the key aspects of identity resolution technology. Through these modules, students can gradually build up an understanding of the entire process of identity resolution, including identity encoding, carrier selection, identity registration and resolution. This step-by-step learning process helps students systematically master the basic knowledge of identity resolution and lay a solid foundation for entering comprehensive training.

[0291] The comprehensive training aims to realize the identification registration, identification resolution, identification query, identification application and identification service of industrial Internet connection objects and links such as raw materials, production equipment, production information data, packaging, logistics, etc. in a virtual simulation environment, and learn the integrated innovative applications of industrial Internet based on identification, such as production traceability and supply chain collaboration. Allow students to apply the knowledge of identification resolution they have learned in a real and comprehensive situation. In this environment, students will play different roles in the supply chain and production workshop, participate in supply chain activities and the production process of the workshop, cooperate to complete production tasks and supply chain collaboration tasks, and intuitively feel the key role of identification resolution technology in optimizing supply chain processes, improving production efficiency and ensuring product quality, so as to deepen their understanding of the importance of identification resolution technology.

[0292] In order to ensure the scalability of the system, it is necessary to develop a complete API interface. Design a RESTful API architecture and develop a data interface to support integration with other teaching systems. Provide SDK and detailed development documents to facilitate third-party developers to expand functions so that the system can continuously adapt to new teaching needs.

[0293] The design of practical training course content is the core of system application. The MES basic and application training project includes the design of practical tasks for the establishment of MES basic data, the development of simulation scenarios for product production processes and order establishment, the creation of simulation environments for equipment management and maintenance, the design of cases and operation procedures for quality management and traceability, and the development of practical projects for the production and embedding of information dashboards. The identification resolution technology application training project covers the creation of practice modules for identification coding design, the development of an operating platform for identification registration and resolution, the design of production traceability scenarios, and the construction of supply chain collaboration scenarios. The enterprise production and operation data simulation dynamic simulation project includes the development of an enterprise business decision-making simulation system, the design of cross-enterprise collaboration scenarios in the industrial Internet environment, the creation of role-playing modules for different positions, and the development of data analysis and visualization tools to assist decision-making.

[0294] The physical environment configuration of the training room also needs to be carefully planned. Design an ergonomic workstation layout and configure an appropriate number of computer workstations and mobile devices. Install large-screen display devices for centralized display and explanation. Plan network cabling to ensure network coverage and performance. Consider adding layouts that simulate factory environments, such as production line models, to enhance the realism of training.

[0295] The development of teaching resources is an important part of ensuring teaching quality. Compile core textbooks and produce supporting multimedia teaching resources, such as video tutorials, animation demonstrations, etc. Develop a rich case library, including digital transformation cases of enterprises in different industries and sizes. Create an online assessment question bank covering theoretical knowledge and practical skills. Design project-based teaching materials to support problem-oriented and task-driven teaching models to improve students' learning interest and effectiveness.

[0296] In order to ensure that teachers can effectively use the system for teaching, it is necessary to establish a complete teacher training system. Develop system operation training courses to ensure that teachers are familiar with all functions. Organize special training on digital factory related theory and practice. Establish a "dual-qualified" teacher training mechanism and arrange teachers to participate in enterprise practice. Organize teaching method seminars to explore the best practices of virtual simulation teaching and continuously improve teaching quality.

[0297] Before full deployment, trial operation and optimization are required. Select a pilot class to test the system and collect feedback from students and teachers. Perform performance tests, including indicators such as the number of concurrent users and response time. Based on the feedback, perform system tuning and functional improvement to ensure that the system is stable and reliable.

[0298] In the comprehensive deployment and operation phase, formulate a school-wide promotion plan and implement it in relevant majors in batches. Establish a daily operation and maintenance and technical support team, develop a user manual and FAQ document. Establish a regular training and update mechanism to ensure that the system can be continuously improved and adapt to the ever-changing teaching needs.

[0299] In order to evaluate the implementation effect of the system and continuously improve it, it is necessary to design a comprehensive evaluation indicator system, including learning effect, satisfaction, employment quality, etc. Regularly collect feedback from teachers and students and usage data, conduct horizontal comparisons, and understand the implementation status of similar institutions. Organize expert reviews to obtain opinions from the industry and academia. According to the evaluation results, formulate a system iteration upgrade plan to continuously improve system performance and teaching effects.

[0300] Finally, promoting industry-university-research cooperation is an important way to maintain the advancement and practicality of the system. Establish cooperative relationships with leading companies in the industry and introduce real cases and data. Carry out scientific research projects based on virtual simulation systems to promote technological innovation. Organize students to participate in corporate internships and combine virtual training with actual work. Hold system-based skill competitions to improve student participation and practical ability. Through these measures, ensure that the system can keep up with industry development and cultivate high-quality talents that meet market demand.

[0301] Figure 2 FIG. 1 is a structural diagram of a digital factory operation virtual simulation teaching system according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0302] The first unit is used to collect factory operation data in real time through the Internet of Things sensor network, perform standardization processing and correlation analysis on the factory operation data based on the multi-source heterogeneous data fusion algorithm of the graph neural network, build a multi-dimensional data model, and based on the multi-dimensional data model, perform feature extraction and semantic mining on the data of each dimensional level through a deep learning method combining transfer learning and federated learning to build a multi-level digital twin factory model;

[0303] The second unit is used to dynamically model learners based on a multi-level digital twin factory model using a multi-agent reinforcement learning algorithm, build a personalized knowledge graph, identify learners' knowledge blind spots and learning preferences based on the personalized knowledge graph, generate a list of learning needs, match the list of learning needs with the digital twin factory model, and automatically generate a multi-dimensional simulation environment through programmatic generation technology and virtual reality scene reconstruction algorithm;

[0304] The second unit is used to accept the learner's operation instructions in the multi-dimensional simulation environment, parse and recognize the intention of the operation instructions, convert the recognition results into virtual operations in the multi-dimensional simulation environment, execute the virtual operations and update the corresponding state of the multi-dimensional simulation environment, determine the captured data by capturing the learner's operation posture and expression data, compare the captured data with the preset standard action model, calculate the normative score of the virtual operation, generate corresponding feedback information based on the normative score and the personalized knowledge graph, adjust the difficulty parameters of the multi-dimensional simulation environment based on the learner's response to the feedback information, and at the same time input the data generated in the learning process, including operation records, learning trajectories and difficulty adjustment records, into the digital twin factory model, update the model parameters, and complete the iteration of the model.

[0305] According to a third aspect of the embodiments of the present invention,

[0306] An electronic device is provided, comprising:

[0307] processor;

[0308] a memory for storing processor-executable instructions;

[0309] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0310] A fourth aspect of the embodiments of the present invention is:

[0311] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0312] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0313] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A digital factory operation virtual simulation teaching method, characterized in that: include: The factory operation data is collected in real time through the IoT sensor network, and the factory operation data is standardized and analyzed for relevance based on the multi-source heterogeneous data fusion algorithm of the graph neural network to build a multidimensional data model. Based on the multidimensional data model, a deep learning method combining transfer learning and federated learning is used to extract features and mine semantics of data at each dimensional level to build a multi-level digital twin factory model; Based on the multi-level digital twin factory model, a multi-agent reinforcement learning algorithm is used to dynamically model learners and build a personalized knowledge graph. Based on the personalized knowledge graph, the learner's knowledge blind spots and learning preferences are identified, and a list of learning needs is generated. The list of learning needs is matched with the digital twin factory model, and a multi-dimensional simulation environment is automatically generated through programmatic generation technology and virtual reality scene reconstruction algorithm. In the multi-dimensional simulation environment, the learner's operation instructions are accepted, the operation instructions are parsed and the intention is recognized, the recognition results are converted into virtual operations in the multi-dimensional simulation environment, the virtual operations are executed and the corresponding state of the multi-dimensional simulation environment is updated, and the captured data is determined by capturing the learner's operation posture and expression data, and the captured data is compared with the preset standard action model, and the normative score of the virtual operation is calculated. Based on the normative score and the personalized knowledge graph, corresponding feedback information is generated, and based on the learner's response to the feedback information, the difficulty parameters of the multi-dimensional simulation environment are adjusted. At the same time, the data generated during the learning process, including operation records, learning trajectories and difficulty adjustment records, are input into the digital twin factory model, the model parameters are updated, and the model iteration is completed.

2. The method according to claim 1, characterized in that The multi-source heterogeneous data fusion algorithm based on graph neural network is used to standardize and analyze the correlation of the factory operation data, and to build a multidimensional data model including: Standardize the factory operation data and convert the corresponding structured data, semi-structured data and unstructured data into a graph structure, where the data items are used as nodes of the graph and the relationships between the data items are used as edges of the graph, to obtain the initial factory operation data graph; Initialize a meta-path generator, wherein the meta-path generator uses a recurrent neural network as a core structure, define the action space of the meta-path generator as all node type conversions in the initial plant operation data graph, train the meta-path generator using a policy gradient method, and use the trained meta-path generator to generate a set of candidate meta-paths; Based on the candidate meta-path set, a hierarchical attention mechanism is constructed, including node-level attention, meta-path-level attention, and semantic-level attention, and features of nodes in the initial plant operation data graph are aggregated through the hierarchical attention mechanism to obtain a preliminary node representation; Based on the preliminary node representation, the graph structure is learned, the intrinsic relationship of the plant operation data is adapted, the original adjacency matrix is ​​determined, a dynamic adjacency matrix of the initial plant operation data graph is calculated, and the dynamic adjacency matrix is ​​combined with the original adjacency matrix to obtain an updated adjacency matrix, and the updated adjacency matrix is ​​used to pass messages between nodes, and the node representation is updated to obtain an enhanced node representation; Use corresponding encoders to encode structured data, semi-structured data, and unstructured data to obtain feature representations of each type of data; The feature importance of different types of data features is calculated using the self-attention mechanism, and based on the calculated feature importance, the features of different types of data are weighted and fused to obtain fused features, which are then combined with the enhanced node representation as the input node representation of the graph neural network; Two different views are generated for each input node representation, and the graph attention network model is used to process the two views separately to obtain the node contrast representation. The contrast loss function of the node contrast representation is defined, and the contrast loss function is added to the loss of the construction task corresponding to the factory multidimensional data model to obtain the total loss function. Through end-to-end training, the updated adjacency matrix, graph attention network model parameters and total loss function are optimized at the same time to obtain the optimized adjacency matrix, model parameters and final node representation to construct the factory multidimensional data model.

3. The method according to claim 2, characterized in that Initialize the meta-path generator, the meta-path generator uses a recurrent neural network as the core structure, defines the action space of the meta-path generator as all node type conversions in the initial plant operation data graph, trains the meta-path generator through a policy gradient method, and uses the trained meta-path generator to generate a set of candidate meta-paths including: Constructing a recurrent neural network based on a long short-term memory network and determining the core structure of a meta-path generator, wherein the long short-term memory network includes a forget gate, an input gate, an output gate, and a unit state; Determine the input dimension, hidden layer dimension and output dimension of the recurrent neural network, wherein the input dimension is set to the one-hot encoding length of the node type, the hidden layer dimension is set to a preset value, and the output dimension is set to the number of node type conversions; Initializing the parameters of the recurrent neural network by using a weight initialization method based on uniform distribution, wherein the range of the uniform distribution is determined based on the number of input and output neurons; Determine the action space based on the node types in the initial plant operation data graph, encode each action into an integer, and obtain an encoded action set; Constructing a composite reward function, the composite reward function includes a path length reward, a type diversity reward, and a semantic relevance reward, wherein the path length reward is calculated based on the difference between the generated path length and the predefined optimal length, the type diversity reward is calculated based on the ratio of the number of unique types to the total number of types, and the semantic relevance reward is calculated based on the cosine similarity of the semantic vectors of the path start node and the end node; updating the policy parameters by calculating the product of the logarithmic probability of the parameterized strategy, the action taken, and the discounted return, and summing them, implementing a policy gradient-based reinforcement learning algorithm, setting the training parameters of the policy gradient method, and determining the learning rate attenuation path according to the preset attenuation rule; Based on the current training parameters, multiple meta-paths are generated, the reward value of each meta-path is calculated using the composite reward function, and based on the reinforcement learning algorithm, the reward value is used to update the strategy parameters, and the cycle is iterated in sequence until a preset number of training rounds is reached to obtain a trained recurrent neural network; Based on the trained recurrent neural network and the preset validation set, the average reward value is calculated, and the learning rate is adjusted according to the average reward value. When the preset reward value early stopping threshold is met, the early stopping mechanism is triggered to determine the final meta-path generator; Based on the final meta-path generator, a meta-path set is generated. Based on the composite reward function, the meta-path set is scored. Based on the scoring results, the corresponding number of meta-paths are selected according to the preset preferential ratio to determine the candidate meta-path set.

4. The method according to claim 1, characterized in that: Based on the multi-level digital twin factory model, a multi-agent reinforcement learning algorithm is used to dynamically model learners and build a personalized knowledge graph including: Implement a multi-agent reinforcement learning algorithm in the model layer of the multi-level digital twin factory model, where each agent corresponds to a learner, and construct a state representation module based on a graph neural network to capture the learner's interactive behavior; determine the action space, including knowledge point learning sequence selection, learning resource recommendation, and learning difficulty adjustment; determine a hierarchical reward function, including short-term learning effect rewards, long-term knowledge structure optimization rewards, and factory operation efficiency improvement rewards; process the action space through a strategy network and a value network based on an attention mechanism; The asynchronous advantage actor-critic algorithm is used to train multi-agents. Learners are dynamically modeled based on state representation modules, action space, reward function, policy network, and value network. According to the dynamic modeling results, a personalized knowledge graph is constructed for each learner. The personalized knowledge graph includes knowledge point nodes, multi-dimensional relationship edges, and time series attributes.

5. The method according to claim 4, characterized in that Based on the personalized knowledge graph, learners’ knowledge blind spots and learning preferences are identified, and a list of learning needs is generated, including: Through the graph neural network model including the spatial-temporal convolution layer and the graph attention layer, the personalized knowledge graph is processed to generate the knowledge point embedding vector of each knowledge point. Through the density-based anomaly detection algorithm, the knowledge point embedding vector is used to calculate the density information of the knowledge point, including the local density and relative density. Through the meta-learning-based knowledge tracking model, the learner's mastery of each knowledge point is dynamically evaluated and the mastery value is determined. Based on the density information of the knowledge point and the mastery value of the knowledge point, the knowledge blind spots in the personalized knowledge graph are identified and a list of knowledge blind spots is generated. Based on learners, knowledge points and learning resources, a heterogeneous information network is constructed, and the heterogeneous information network is input into a collaborative filtering algorithm based on a graph convolutional network, and the input is processed through a multi-head graph attention mechanism to generate a learner-knowledge point association matrix; Based on the learner-knowledge point association matrix and combined with the density information of knowledge points, the learner's personalized learning preference is predicted. Based on reinforcement learning, a learning demand generator is set up. The knowledge blind spot list, personalized learning preferences and the density information of knowledge points are used as input to determine the state space, action space and reward function of the learning demand generator. The learning demand generator is trained to output a personalized learning demand list.

6. The method according to claim 1, characterized in that Parsing and identifying the intention of the operation instruction and converting the identification result into a virtual operation in a multi-dimensional simulation environment includes: The operation instruction includes a natural language instruction, receiving the natural language instruction, converting the natural language instruction into a structured abstract syntax tree through a neural semantic parser, encoding the abstract syntax tree into a node sequence, encoding the node sequence using a bidirectional long short-term memory network, obtaining a hidden state representation of each node, calculating a correlation score between a query vector and a hidden state of each node using an attention mechanism, selecting the most relevant node as an operation parameter based on the correlation score, obtaining a selected node, and decoding the node information corresponding to the selected node into an operation parameter value; Based on a pre-built plant operation knowledge graph, use a graph attention network to reason on the plant operation knowledge graph to obtain a knowledge enhancement vector, and fuse the knowledge enhancement vector with the abstract syntax tree to generate a fused representation; By using knowledge distillation technology, the factory operation knowledge graph is compressed into a lightweight neural network, the fusion representation is input into the lightweight neural network, and the initial conversion result of the virtual operation is determined; Modeling the mapping process of the initial conversion result of the virtual operation as a Markov decision process, setting a state space, an action space and a reward function, wherein the state in the state space includes the semantic representation of the current natural language instruction, the executed operation sequence and the current state of the simulation environment, and the action space includes all candidate virtual operations; Based on the multi-head self-attention layer, feedforward neural network layer and output layer, a policy network is constructed, the state value function is estimated, and the critic network is determined. The parameters of the policy network and the critic network are updated alternately through the dual-network reinforcement learning algorithm to obtain the trained policy network. Based on the training completion strategy network, the final conversion result of the virtual operation is determined.

7. The method according to claim 6, characterized in that The parameters of the policy network and the critic network are updated alternately through the dual-network reinforcement learning algorithm. The trained policy network includes: Input the current state of the simulation environment into the policy network to obtain the action probability distribution, select a virtual operation based on a preset probability threshold and the action probability distribution, execute the virtual operation in the simulation environment, record the feedback of the simulation environment, and calculate the reward value; Determine the value of the current state using the estimated state-value function in the critic network, calculate the difference between the reward value and the value, determine a performance metric, update the parameters of the policy network based on the performance metric, and update the parameters of the critic network using the temporal difference error; Repeat the iteration until the policy network and the critic network meet the preset convergence conditions, and determine the trained policy network.

8. A digital factory operation virtual simulation teaching system, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect factory operation data in real time through the Internet of Things sensor network, perform standardization processing and correlation analysis on the factory operation data based on the multi-source heterogeneous data fusion algorithm of the graph neural network, build a multi-dimensional data model, and based on the multi-dimensional data model, perform feature extraction and semantic mining on the data of each dimensional level through a deep learning method combining transfer learning and federated learning to build a multi-level digital twin factory model; The second unit is used to dynamically model learners based on a multi-level digital twin factory model using a multi-agent reinforcement learning algorithm, build a personalized knowledge graph, identify learners' knowledge blind spots and learning preferences based on the personalized knowledge graph, generate a list of learning needs, match the list of learning needs with the digital twin factory model, and automatically generate a multi-dimensional simulation environment through programmatic generation technology and virtual reality scene reconstruction algorithm; The second unit is used to accept the learner's operation instructions in the multi-dimensional simulation environment, parse and recognize the intention of the operation instructions, convert the recognition results into virtual operations in the multi-dimensional simulation environment, execute the virtual operations and update the corresponding state of the multi-dimensional simulation environment, determine the captured data by capturing the learner's operation posture and expression data, compare the captured data with the preset standard action model, calculate the normative score of the virtual operation, generate corresponding feedback information based on the normative score and the personalized knowledge graph, adjust the difficulty parameters of the multi-dimensional simulation environment based on the learner's response to the feedback information, and at the same time input the data generated in the learning process, including operation records, learning trajectories and difficulty adjustment records, into the digital twin factory model, update the model parameters, and complete the iteration of the model.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images