Digital human configuration method and system based on industrial metaverse digital twinning

Abstract

The application provides a digital human configuration method and system based on industrial meta-universe digital twinning, comprising the following steps: S1, constructing a digital human model, and training the constructed digital human model based on a configured data set to obtain a trained digital human model; wherein the configured data set comprises a standard operation action data set in a manufacturing process in an industrial scene, a voice data set in an industrial manufacturing scene, and an industrial manufacturing knowledge base; S2, obtaining the running state of a production line equipment based on real-time monitoring of the production line equipment, using the trained digital human model to perform anomaly detection based on the obtained running state of the production line equipment, and when an anomaly is detected, performing multi-modal early warning, fault positioning, and providing a repair scheme through the digital human model; S3, obtaining detection data of a target finished product based on a detection device, and the digital human model analyzing and counting the obtained detection data; and optimizing the corresponding production line based on the analysis and counting results.

Description

Technical Field

[0001] This invention relates to the field of digital twin and artificial intelligence fusion technology, specifically to a digital human configuration method and system based on industrial metaverse digital twins. Background Technology

[0002] With the vigorous promotion of the construction of intelligent industrial systems and the development of digital twin applications combined with artificial intelligence, industrial production efficiency can be significantly improved, achieving cost reduction and efficiency enhancement. As enterprises focus on building metaverse applications, how to enhance industrial productivity, reduce human resource costs, and improve product quality has become an urgent issue to be addressed.

[0003] Currently, existing solutions for improving the efficiency of digital human configuration involve pre-producing digital humans or dynamically generating them according to needs, enabling rapid and diversified configuration to ultimately improve the accuracy of digital human models and allow for the free replacement of various functional configurations. However, in factories across different industries, processes vary greatly, and generalized solutions contain too much redundancy, making them difficult to integrate with specific industries. Furthermore, the lack of targeted intelligent processing during development means that the costs of intelligent model retrieval and learning remain high, resulting in limited efficiency improvements and difficulties in reusing intelligent modules. Therefore, reducing the costs of intelligent model learning and retrieval, and increasing the efficiency of reusable configuration, has become a key research focus in this field.

[0004] Patent document CN119917951A (application number: 202411784849.2) discloses a digital twin system based on multimodal recognition. By integrating equipment, a digital twin, and an information transmission medium, it achieves multimodal information acquisition, processing, fusion analysis, and feedback control. The system uses a multimodal signal acquisition device to collect various modal information, and performs data fusion analysis and scenario simulation through a simulator and a large multimodal model to guide the equipment execution subject to perform precise operations. It possesses intelligent learning and adaptability, and can update its professional knowledge base through manual teaching and filter uncertainties using a knowledge graph. In this technical solution, the digital human module is a general-purpose design, without optimization for appearance configurations of special industrial tooling (such as safety helmets and protective gloves), resulting in insufficient model adaptability. The motion library mainly consists of standard industrial robot motions, lacking refined motion data for special working conditions such as high-temperature smelting and heavy machinery operation, resulting in incomplete coverage of industrial scenarios. It does not achieve dynamic binding between the digital human's behavioral logic and real-time production data, and cannot automatically adjust the interaction strategy according to the equipment status.

[0005] Patent document CN119884388A (application number: 202411902560.6) discloses a digital human interaction method, system, medium, and product program based on knowledge graphs. It constructs a multi-layered knowledge structure to achieve digital human reasoning interaction, optimizing the limitations of cross-domain knowledge processing, and realizing multi-turn dialogue through hierarchical association of the knowledge graph. However, in this technical solution, knowledge representation is not integrated with 3D virtual scenes, making it impossible to visualize the internal structure of equipment and the location of faults. It does not integrate speech synthesis and action generation modules, limiting the interaction method to text-based question and answer. Furthermore, it does not consider industrial environmental noise interference, resulting in a significant improvement in speech recognition accuracy within industrial manufacturing scenarios.

[0006] Patent document CN120259500A (application number: 202510325920.9) discloses a method and system for digital human generation and streaming. Based on the ER-NERF algorithm, it generates and streams a 2.5D digital human model through depth estimation and light radiation field reconstruction of multi-view image data. During data transmission, model parameters and texture information are pushed to the user terminal in streaming media format. The terminal decodes and reconstructs the model in real time and performs dynamic rendering. However, this technical solution suffers from insufficient stereoscopic depth in the 2.5D modeling, failing to meet the spatial positioning accuracy requirements of complex industrial scenarios. While the streaming latency is optimized to 150ms, it still exceeds the user's patent requirement of 100ms for industrial-grade performance, indicating room for improvement in real-time performance. Furthermore, the lack of an integrated industrial knowledge graph means the digital human lacks professional decision-making capabilities such as equipment fault diagnosis, resulting in limited interaction depth. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a digital human configuration method and system based on industrial metaverse digital twins.

[0008] A digital human configuration method based on an industrial metaverse digital twin, provided by the present invention, includes:

[0009] Step S1: Construct a digital human model and train the constructed digital human model based on the configured dataset to obtain the trained digital human model; wherein, the configured dataset includes: a standard operation action dataset in the manufacturing process in an industrial scenario, a voice dataset in an industrial manufacturing scenario, and an industrial manufacturing knowledge base.

[0010] Step S2: Monitor the production line equipment in real time to obtain the operating status of the production line equipment. Based on the obtained operating status of the production line equipment, use the trained digital human model to detect anomalies. When an anomaly is detected, the digital human model will be used to perform multimodal early warning, fault location and provide repair solutions.

[0011] Step S3: Based on the testing equipment, obtain the test data of the target finished product. The digital human model analyzes and statistically analyzes the obtained test data, and optimizes the corresponding production line based on the analysis and statistical results.

[0012] Preferably, step S1 includes:

[0013] Step S1.1: Collect standard operating actions in the manufacturing process within an industrial setting using motion capture equipment, including equipment maintenance and parts assembly; use a motion recognition model to identify the collected standard operating actions and build a motion library for digital humans;

[0014] Step S1.2: Acquire voice data in the industrial manufacturing scenario, including: equipment operating sounds and operator commands; preprocess the acquired voice data to obtain preprocessed voice data; use a voice recognition model to recognize the preprocessed voice data, thereby constructing a voice database for digital humans;

[0015] Step S1.3: Integrate industrial manufacturing process knowledge, equipment maintenance manuals, and fault diagnosis cases to construct a knowledge graph; use natural language processing technology to transform the knowledge graph into a mapping relationship of equipment fault phenomena, fault causes, and solutions, thereby constructing a digital human industrial manufacturing knowledge base;

[0016] Step S1.4: Train the digital human model based on the digital human's action library, digital human's voice library, and digital human's industrial manufacturing knowledge base to obtain the trained digital human model.

[0017] Preferably, step S2 includes:

[0018] Step S2.1: Real-time acquisition of the operating status of the production line equipment, including vibration, temperature and current information of the production line equipment;

[0019] Step S2.2: Utilize the trained digital human model to detect anomalies based on the collected data on the operating status of the production line equipment; when an anomaly is detected, a multimodal warning is issued, including visual warnings and voice warnings.

[0020] Step S2.3: The digital human model is a reasoning model built based on a knowledge graph. It locates faults and provides corresponding repair solutions based on the collected operating status of production line equipment.

[0021] Step S2.4: Provide interactive guidance based on the digital human model of the repair scheme, including: augmented reality annotation, step-by-step voice guidance, and knowledge Q&A.

[0022] Preferably, step S3 includes:

[0023] Step S3.1: Obtain test data by testing the target finished product using testing equipment;

[0024] Step S3.2: The digital human model judges the inspection data based on the standard finished product requirements in the knowledge graph. When the inspection data does not meet the preset requirements, the defect location is displayed through a visual interface; and corresponding processing suggestions are provided based on the inspection data.

[0025] Step S3.3: The digital human model performs real-time statistical analysis on the detection data, generating quality reports and trend charts;

[0026] Step S3.4: Analyze potential quality problems in the production process based on the generated quality reports and trend charts, and optimize the production process and procedures based on the potential quality problems found in the analysis.

[0027] Preferably, the digital human model supports remote collaboration;

[0028] The digital human model supports remote collaboration by: constructing a virtual 3D model of on-site industrial equipment, using the virtual 3D model to reflect the operating status of the on-site industrial equipment in real time, thereby constructing a metaverse space;

[0029] The system acquires the operator's physical movements, postures, and position information, and converts these information into the state of the on-site operator's digital model in the metaverse space in real time. It then converts the state of the on-site operator's digital model into three-dimensional space operation commands, and realizes real-time response of the virtual three-dimensional model based on these commands. Finally, it transmits the real-time state of the on-site operator's digital model and the real-time response of the virtual three-dimensional model to the head-mounted display device of a remote expert, thereby achieving an intuitive display of the on-site operation.

[0030] Remote experts obtain physical movements, postures, and position information of experts through human body sensors, and convert the obtained physical movements, postures, and positions of experts into the state of the remote expert digital model in the metaverse space in real time; convert the state of the remote expert digital model into three-dimensional space operation commands, and realize the real-time response of the virtual three-dimensional model based on the three-dimensional space operation commands; transmit the real-time state of the remote expert digital model and the real-time response of the virtual three-dimensional model to the head-mounted display device of the on-site operator, thereby realizing the remote operation demonstration of the expert.

[0031] Preferably, the method further includes:

[0032] Based on the production tasks of the industrial production line, and using multiple historical planning simulation schemes, a comprehensive optimal solution is generated by scoring based on a multi-objective decision-making algorithm.

[0033] Based on the generated comprehensive optimal solution, instructions are issued through edge computing nodes. When production fluctuations occur in the production line equipment, production fluctuations are responded to through real-time rescheduling or resource coordination.

[0034] The real-time rescheduling includes: responding to production fluctuations by invoking process compensation rules in the knowledge graph;

[0035] The resource coordination includes: responding to production fluctuations by reallocating paths through a distributed task scheduling algorithm.

[0036] Preferably, the method further includes: performing a three-dimensional dynamic demonstration of the standard operating procedures of industrial production equipment using a digital human model, thereby achieving safety training; simultaneously, during the three-dimensional dynamic demonstration, displaying the internal status of the equipment and text prompts for key operating points through a visual interface;

[0037] During the hands-on practice, the motion capture system collects the trainees' movements in real time. The digital human model compares and analyzes the captured movements with a standard motion library. Based on the comparison and analysis results, it performs operations including real-time correction, motion decomposition, and stress testing.

[0038] A digital human configuration system based on an industrial metaverse digital twin, provided by the present invention, includes:

[0039] Module M1: Constructs a digital human model and trains the constructed digital human model based on a configured dataset to obtain a trained digital human model; wherein, the configured dataset includes: a dataset of standard operating actions in the manufacturing process within an industrial scenario, a voice dataset within an industrial manufacturing scenario, and an industrial manufacturing knowledge base.

[0040] Module M2: Real-time monitoring of production line equipment to obtain the operating status of the production line equipment. Based on the obtained operating status of the production line equipment, anomaly detection is performed using a trained digital human model. When an anomaly is detected, the digital human model is used to perform multimodal early warning, fault location and provide repair solutions.

[0041] Module M3: Based on the testing equipment, the digital human model acquires the test data of the target finished product, analyzes and statistically processes the acquired test data, and optimizes the corresponding production line based on the analysis and statistical results.

[0042] Preferably, the module M1 includes:

[0043] Module M1.1: Collects standard operating actions in the manufacturing process within an industrial setting using motion capture equipment, including equipment maintenance and parts assembly; uses a motion recognition model to identify the collected standard operating actions and builds a motion library for digital humans;

[0044] Module M1.2: Acquires voice data in industrial manufacturing scenarios, including equipment operating sounds and operator commands; preprocesses the acquired voice data to obtain preprocessed voice data; and uses a voice recognition model to recognize the preprocessed voice data, thereby constructing a voice library for digital humans.

[0045] Module M1.3: Integrates industrial manufacturing process knowledge, equipment maintenance manuals, and fault diagnosis cases to construct a knowledge graph; uses natural language processing technology to transform the knowledge graph into a mapping relationship of equipment fault phenomena, fault causes, and solutions, thereby constructing a digital human industrial manufacturing knowledge base;

[0046] Module M1.4: Train the digital human model based on the digital human's motion library, digital human's voice library, and digital human's industrial manufacturing knowledge base to obtain the trained digital human model;

[0047] The module M2 includes:

[0048] Module M2.1: Real-time acquisition of the operating status of production line equipment, including vibration, temperature and current information of the production line equipment;

[0049] Module M2.2: Utilizes the trained digital human model to detect anomalies based on the collected data on the operating status of production line equipment; when an anomaly is detected, it provides multimodal warnings, including visual and voice warnings.

[0050] Module M2.3: The digital human model is a reasoning model built on a knowledge graph, which locates faults and provides corresponding repair solutions based on the collected operating status of production line equipment;

[0051] Module M2.4: Provides interactive guidance based on the repair solution digital human model, including: augmented reality annotation, step-by-step voice guidance, and knowledge Q&A;

[0052] The module M3 includes:

[0053] Module M3.1: Obtains test data by testing the target finished product using testing equipment;

[0054] Module M3.2: The digital human model judges the inspection data based on the standard finished product requirements in the knowledge graph. When the inspection data does not meet the preset requirements, the defect location is displayed through a visual interface; and corresponding processing suggestions are provided based on the inspection data.

[0055] Module M3.3: The digital human model performs real-time statistics and analysis on the detection data, generating quality reports and trend charts;

[0056] Module M3.4: Analyzes potential quality problems in the production process based on the generated quality reports and trend charts, and optimizes production processes and procedures based on the potential quality problems found in the analysis.

[0057] Preferably, the digital human model supports remote collaboration;

[0058] The digital human model supports remote collaboration by: constructing a virtual 3D model of on-site industrial equipment, using the virtual 3D model to reflect the operating status of the on-site industrial equipment in real time, thereby constructing a metaverse space;

[0059] The system acquires the operator's physical movements, postures, and position information, and converts these information into the state of the on-site operator's digital model in the metaverse space in real time. It then converts the state of the on-site operator's digital model into three-dimensional space operation commands, and realizes real-time response of the virtual three-dimensional model based on these commands. Finally, it transmits the real-time state of the on-site operator's digital model and the real-time response of the virtual three-dimensional model to the head-mounted display device of a remote expert, thereby achieving an intuitive display of the on-site operation.

[0060] Remote experts obtain physical movements, postures, and position information of experts through human body sensors, and convert the obtained physical movements, postures, and positions of experts into the state of the remote expert digital model in the metaverse space in real time; convert the state of the remote expert digital model into three-dimensional space operation commands, and realize the real-time response of the virtual three-dimensional model based on the three-dimensional space operation commands; transmit the real-time state of the remote expert digital model and the real-time response of the virtual three-dimensional model to the head-mounted display device of the on-site operator, thereby realizing the remote operation demonstration of the expert.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. To address the issues of fragmented knowledge and low retrieval efficiency in the industrial manufacturing sector, a knowledge graph of all industrial manufacturing sectors is constructed, including all process rules and fault cases within historical models. This significantly improves the accuracy of equipment fault diagnosis and supports digital humans in pushing precise maintenance solutions to operators using natural language.

[0063] 2. Based on the production tasks of the industrial production line, this invention generates a comprehensive optimal solution based on multiple historical plan simulation schemes and a multi-objective decision-making algorithm. When orders change, the digital human can generate high-quality solutions in a short time, greatly improving the agility of production scheduling. Moreover, the longer the metaverse system runs, the richer the historical solution database becomes, and the higher the accuracy of the matching and generated solutions. The digital human will then have the ability to self-evolve.

[0064] 3. This invention is formed by the integration of digital twin and artificial intelligence technologies, enabling the rapid construction and dynamic optimization of digital human behavior logic in industrial scenarios, thereby improving production efficiency and product quality;

[0065] 4. In the industrial manufacturing sector, the efficient and stable operation of production lines is of paramount importance. This invention enables intelligent operation and maintenance of production line equipment by configuring digital humans, thereby improving production efficiency and quality. Attached Figure Description

[0066] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0067] Figure 1 A flowchart for configuring digital humans based on industrial metaverse digital twins.

[0068] Figure 2 A schematic diagram of a digital human configuration system based on an industrial metaverse digital twin. Detailed Implementation

[0069] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0070] Example 1

[0071] According to the present invention, a digital human configuration method based on an industrial metaverse digital twin is provided, such as... Figure 1 As shown, it includes:

[0072] Step S1: Construct a digital human model and train the constructed digital human model based on the configured dataset to obtain the trained digital human model; wherein, the configured dataset includes: a standard operation action dataset in the manufacturing process in an industrial scenario, a voice dataset in an industrial manufacturing scenario, and an industrial manufacturing knowledge base.

[0073] Step S2: Monitor the production line equipment in real time to obtain the operating status of the production line equipment. Based on the obtained operating status of the production line equipment, use the trained digital human model to detect anomalies. When an anomaly is detected, the digital human model will be used to perform multimodal early warning, fault location and provide repair solutions.

[0074] Step S3: Based on the testing equipment, obtain the test data of the target finished product. The digital human model analyzes and statistically analyzes the obtained test data, and optimizes the corresponding production line based on the analysis and statistical results.

[0075] Specifically, step S1 includes:

[0076] Step S1.1: Collect standard operating actions in the manufacturing process within an industrial setting using motion capture equipment, including equipment maintenance and parts assembly; use a motion recognition model to identify the collected standard operating actions and build a motion library for digital humans;

[0077] Step S1.2: Acquire voice data in the industrial manufacturing scenario, including: equipment operating sounds and operator commands; preprocess the acquired voice data to obtain preprocessed voice data; use a voice recognition model to recognize the preprocessed voice data, thereby constructing a voice database for digital humans;

[0078] Step S1.3: Integrate industrial manufacturing process knowledge, equipment maintenance manuals, and fault diagnosis cases to construct a knowledge graph; use natural language processing technology to transform the knowledge graph into a mapping relationship of equipment fault phenomena, fault causes, and solutions, thereby constructing a digital human industrial manufacturing knowledge base;

[0079] Step S1.4: Train the digital human model based on the digital human's action library, digital human's voice library, and digital human's industrial manufacturing knowledge base to obtain the trained digital human model.

[0080] Specifically, step S2 includes:

[0081] Step S2.1: Real-time acquisition of the operating status of the production line equipment, including vibration, temperature and current information of the production line equipment;

[0082] Step S2.2: Utilize the trained digital human model to detect anomalies based on the collected data on the operating status of the production line equipment; when an anomaly is detected, a multimodal warning is issued, including visual warnings and voice warnings.

[0083] Step S2.3: The digital human model is a reasoning model built based on a knowledge graph. It locates faults and provides corresponding repair solutions based on the collected operating status of production line equipment.

[0084] Step S2.4: Provide interactive guidance based on the digital human model of the repair scheme, including: augmented reality annotation, step-by-step voice guidance, and knowledge Q&A.

[0085] Specifically, step S3 includes:

[0086] Step S3.1: Obtain test data by testing the target finished product using testing equipment;

[0087] Step S3.2: The digital human model judges the inspection data based on the standard finished product requirements in the knowledge graph. When the inspection data does not meet the preset requirements, the defect location is displayed through a visual interface; and corresponding processing suggestions are provided based on the inspection data.

[0088] Step S3.3: The digital human model performs real-time statistical analysis on the detection data, generating quality reports and trend charts;

[0089] Step S3.4: Analyze potential quality problems in the production process based on the generated quality reports and trend charts, and optimize the production process and procedures based on the potential quality problems found in the analysis.

[0090] Specifically, the digital human model supports remote collaboration;

[0091] The digital human model supports remote collaboration by: constructing a virtual 3D model of on-site industrial equipment, using the virtual 3D model to reflect the operating status of the on-site industrial equipment in real time, thereby constructing a metaverse space;

[0092] The system acquires the operator's physical movements, postures, and position information, and converts these information into the state of the on-site operator's digital model in the metaverse space in real time. It then converts the state of the on-site operator's digital model into three-dimensional space operation commands, and realizes real-time response of the virtual three-dimensional model based on these commands. Finally, it transmits the real-time state of the on-site operator's digital model and the real-time response of the virtual three-dimensional model to the head-mounted display device of a remote expert, thereby achieving an intuitive display of the on-site operation.

[0093] Remote experts obtain physical movements, postures, and position information of experts through human body sensors, and convert the obtained physical movements, postures, and positions of experts into the state of the remote expert digital model in the metaverse space in real time; convert the state of the remote expert digital model into three-dimensional space operation commands, and realize the real-time response of the virtual three-dimensional model based on the three-dimensional space operation commands; transmit the real-time state of the remote expert digital model and the real-time response of the virtual three-dimensional model to the head-mounted display device of the on-site operator, thereby realizing the remote operation demonstration of the expert.

[0094] Specifically, the method further includes:

[0095] Based on the production tasks of the industrial production line, and using multiple historical planning simulation schemes, a comprehensive optimal solution is generated by scoring based on a multi-objective decision-making algorithm.

[0096] Based on the generated comprehensive optimal solution, instructions are issued through edge computing nodes. When production fluctuations occur in the production line equipment, production fluctuations are responded to through real-time rescheduling or resource coordination.

[0097] The real-time rescheduling includes: responding to production fluctuations by invoking process compensation rules in the knowledge graph;

[0098] The resource coordination includes: responding to production fluctuations by reallocating paths through a distributed task scheduling algorithm.

[0099] Specifically, the method further includes: using a digital human model to perform a three-dimensional dynamic demonstration of the standard operating procedures of industrial production equipment, thereby achieving safety training; at the same time, during the three-dimensional dynamic demonstration, a visual interface is used to display the internal status of the equipment and text prompts for key operating points.

[0100] During the hands-on practice, the motion capture system collects the trainees' movements in real time. The digital human model compares and analyzes the captured movements with a standard motion library. Based on the comparison and analysis results, it performs operations including real-time correction, motion decomposition, and stress testing.

[0101] The present invention also provides a digital human configuration system based on an industrial metaverse digital twin. The digital human configuration system based on an industrial metaverse digital twin can be implemented by executing the process steps of the digital human configuration method based on an industrial metaverse digital twin. That is, those skilled in the art can understand the digital human configuration method based on an industrial metaverse digital twin as a preferred embodiment of the digital human configuration system based on an industrial metaverse digital twin.

[0102] Example 2

[0103] Example 2 is a preferred example of Example 1.

[0104] According to the present invention, a digital human configuration method based on an industrial metaverse digital twin is provided, such as... Figure 2 As shown, it includes:

[0105] Step 1: Digital Human Model Construction and Training;

[0106] Specifically, step 1 includes:

[0107] Step 1.1: Appearance Configuration. Using voice, text, and images, a highly realistic digital human model is generated based on the image of an industrial manufacturing worker. The current model has rich details, including facial expressions and body movements. At the same time, various work clothes and tools, such as safety helmets and wrenches, are added to the digital human to make it conform to the industrial production scenario.

[0108] Step 1.2: Behavior Configuration. Collect a large amount of standard operating actions in the manufacturing process in industrial scenarios, such as equipment maintenance and parts assembly, and accurately record them using motion capture equipment. Use deep learning algorithms to train these action data and build a motion library for the digital human. For example, use a recurrent neural network (RNN) combined with a long short-term memory network (LSTM) to learn and optimize the action sequences, enabling the digital human to perform various operating actions accurately and naturally, with a significant improvement in action execution accuracy.

[0109] Step 1.3: Voice Configuration. Acquire voice data from the industrial manufacturing scenario, including equipment operating sounds and operator commands. Preprocess the acquired voice data to efficiently remove background noise, resulting in preprocessed voice. Use a Transformer-based speech recognition model to recognize the preprocessed voice, building a voice library for the digital human. Simultaneously, configure multiple voice modes for the digital human, such as normal operation mode and emergency alarm mode, to meet the voice interaction needs in different scenarios. For example, in the event of equipment failure, the digital human can issue warning messages with a rapid voice and specific alarm tone, reminding operators to handle the situation promptly.

[0110] Step 1.4: Knowledge base configuration. Integrate industrial manufacturing process knowledge, equipment maintenance manuals, fault diagnosis cases, and other information to construct a knowledge graph. Through natural language processing technology, transform this knowledge into a form that the digital human can understand and use. For example, associate equipment fault phenomena with possible causes and solutions. When the digital human receives abnormal equipment information, it can quickly perform fault diagnosis and recommend solutions based on the knowledge graph.

[0111] Step 1.5: Train the digital human model using the digital human action library, digital human speech library, and knowledge graph to obtain the trained digital human model.

[0112] Step 2: Implementation of intelligent operation and maintenance functions for digital humans;

[0113] Specifically, step 2 includes:

[0114] Step 2.1: Real-time monitoring and early warning. The digital human connects to the equipment data acquisition system to monitor the operating status of the production line equipment in real time, for example, with a sampling frequency of 1kHz. The system inputs the collected parameters such as vibration, temperature, and current into the digital human model for anomaly detection. When an abnormal pattern is detected, a multimodal early warning is immediately triggered.

[0115] The multimodal early warning includes:

[0116] Visual warning: The digital human flashes a red light and points to the faulty part in the twin scene;

[0117] Voice alert: Employs an emergency alarm voice mode, for example: the speech rate is significantly increased compared to normal, the tone is raised several times, and the message "Industrial production line, vibration frequency exceeds the threshold, temperature is abnormal" is broadcast.

[0118] Data push: Automatically retrieves several historical similar cases associated with the knowledge graph, with an alert response time of less than 1 second.

[0119] Step 2.2: Fault Diagnosis and Repair Guidance: Construct a knowledge graph-driven inference engine to achieve intelligent processing of the entire fault handling process.

[0120] Fault location: After receiving abnormal data from the equipment, the fault characteristics are matched in the knowledge graph using the SPARQL Protocol and RDF QueryLanguage; for example, "abnormal increase in cutting torch current → unstable arc → nozzle blockage";

[0121] Solution generation: Based on entity relationships in a knowledge graph, such as "nozzle blockage → solution = replace nozzle + clean channel", a repair solution is automatically generated, which includes 3D animation, tool list, and safety precautions.

[0122] Interactive guidance: The digital human assists in maintenance in the following ways:

[0123] Augmented reality annotation: Overlaying AR arrows onto physical devices to indicate the disassembly sequence;

[0124] The voice prompts step by step: "Step 1: Close the gas supply valve of the cutting torch. Step 2: Use a special wrench to remove the nozzle..."

[0125] Knowledge Q&A: Provides real-time answers to operators' questions and retrieves a list of required operation descriptions from the knowledge graph.

[0126] Effect Verification: After the repair is completed, the digital human automatically verifies the repair effect through sensor data and acceptance criteria in the knowledge graph.

[0127] Step 2.3: Remote Collaboration: In the process of handling complex faults, the digital human supports remote collaboration.

[0128] On-site operators use head-mounted displays and on-site cameras to transmit digital models of industrial equipment and personnel to remote experts via AR / VR, 3D projection, and other devices. Experts, using sensors, can manipulate the on-site models with gestures, performing various auxiliary functions such as 3D spatial movement, zooming, burst splitting, and projection of 3D holographic digital avatars of on-site personnel. They can guide on-site personnel in repairs by observing faulty models and through voice communication, and can also provide timely corrections by observing the actions of the on-site personnel's holographic digital avatars.

[0129] On-site operators can use AR / VR head-mounted displays and voice connection devices to listen to explanations and demonstrations from remote experts, effectively aiding communication and operational demonstrations. For example, for high-altitude or internal structural equipment, live video cannot provide aerial guidance or simulate the explosion and disassembly of real-world equipment; therefore, digital humans and digital models become particularly important, significantly improving the efficiency of remote collaboration.

[0130] Step 3: Digital Human Scheduling, Execution, and Monitoring;

[0131] Step 3.1: Industrial Metaverse Pre-scheduling: The digital human performs virtual scheduling simulations in the industrial metaverse twin space, optimizing production instructions through the following process:

[0132] Multi-scheme simulation: For production tasks on industrial production lines, based on multiple historical simulation schemes, a comprehensive optimal solution is generated using a multi-objective decision-making algorithm. This provides a pre-selected optimal solution for industrial production and refining, characterized by short processing time, high capacity, low personnel requirements, and low energy consumption. The digital robot explains this pre-selected solution to the operators and provides a comparison score with other pre-selected solutions.

[0133] Efficiency First: Maximize equipment utilization by using digital robots to calculate vehicle paths and industrial production line transfer sequences.

[0134] Energy consumption optimization: Through knowledge graphs, digital humans call process parameters to simulate energy consumption under different parameter combinations.

[0135] Solution optimization: The digital human calls a multi-objective decision-making algorithm for scoring and automatically recommends the best comprehensive solution, with a pre-test time of less than 5 minutes.

[0136] The process of generating a comprehensive optimal solution for the production tasks of an industrial production line, based on multiple historical planning simulation schemes and a multi-objective decision-making algorithm, includes:

[0137] The production tasks of the industrial production line are obtained, and the obtained production tasks are processed including edge transformation and feature extraction to obtain the processed production tasks.

[0138] Based on the processed production tasks, similar solutions are selected from the historical database and a candidate set is generated;

[0139] The candidate solutions are evaluated for multiple objectives, and the optimal solution is selected through a scoring function.

[0140] Specifically, the optimization objective function is defined, including: maximizing production efficiency, minimizing configuration cost, minimizing response latency, and maximizing resource utilization balance.

[0141]

[0142] in, The weights are determined by experts based on historical scenarios and the current situation; This indicates maximizing production efficiency; This indicates that the configuration cost is minimized; This indicates that the response latency is minimized; This indicates that the balance of resource utilization is maximized.

[0143] Step 3.2: Virtual-Real Interaction Execution and Control: The digital human acts as the central hub for virtual-real interaction, enabling precise coordination between the physical system and the virtual space.

[0144] Command issuance: The pre-optimized scheduling command is issued through the edge computing node (ECS).

[0145] Dynamic adjustment: Digital humans respond to production fluctuations through the following mechanisms:

[0146] Real-time rescheduling: When the raw material composition fluctuates, the digital human calls the process compensation rules in the knowledge graph to adjust the oxygen supply intensity of the converter.

[0147] Resource coordination: When a driving conflict is detected, the digital human visualizes the conflict point in the twin scene and reallocates the path through a distributed task scheduling algorithm.

[0148] If a conflict occurs in the operation of multiple trains, the algorithm reads all subsequent routes of all trains, automatically recalculates all new planned routes for all trains, assigns new routes, and finally displays the new planned route configuration scheme.

[0149] Process visualization: The digital human presents its execution status in three modes within a twin scenario:

[0150] Global dashboard: Real-time updates on the progress of each process.

[0151] Local drilling: Click on the equipment model to display detailed parameters.

[0152] Historical review: Supports replaying the scheduling process at any time period.

[0153] Step 4: Implement the digital human quality detection function;

[0154] Step 4.1: Real-time Inspection: On the production line of industrial equipment, the digital human performs real-time quality inspection of products by working collaboratively with inspection equipment. When the equipment passes by the inspection equipment, the digital human system immediately acquires the inspection data and analyzes and judges it using a quality inspection model.

[0155] Step 4.2: Defect Marking and Feedback: Once a defect is detected in the industrially refined product, the digital robot precisely marks the defect location on the product image through a visual interface and feeds the defect information back to the production operators. Simultaneously, the digital robot provides corresponding handling suggestions based on the defect type and severity, such as rework or scrapping. For minor welding defects, the digital robot recommends manual repair by the operators; for severe functional defects, the digital robot instructs that the product be scrapped directly.

[0156] Step 4.3: Quality Statistics and Analysis: The digital human system performs real-time statistics and analysis on the inspection data, generating quality reports and trend charts. Through in-depth analysis of quality data, it helps companies identify potential quality problems in the production process and optimize production techniques and procedures. By analyzing quality data over a period of time, if a high defect rate is found in a particular production stage, the company can make targeted improvements and training to the equipment, processes, or operators in that stage, thereby improving overall product quality.

[0157] Step 5: Safety Training for Operating Digital Human Devices

[0158] Step 5.1: Equipment Operation Demonstration: The digital human provides a 3D dynamic demonstration of the standard operating procedures for key industrial production equipment. In the virtual scene, the operating procedures for key industrial production equipment are demonstrated dynamically in 3D by the digital human. The demonstration includes pre-start checks, parameter settings, normal operating procedures, and shutdown protocols (e.g., operating in the order of "stop feeding first → then cooling → finally shutting off gas"). During the demonstration, the internal status of the equipment is visualized and key operating points are displayed simultaneously. The demonstration video has a frame rate ≥120fps, and the motion reproduction accuracy error is ≤2°.

[0159] Step 5.2: Motion Guidance: The trainee's movements are captured in real-time using a motion capture system (sampling frequency 120Hz). The digital human then compares and analyzes these movements against a standard motion library (comparison latency ≤200ms). Three levels of guidance are provided for deviations:

[0160] Real-time correction: When a dangerous action is detected (such as starting the equipment without closing the safety door), vibration feedback is immediately triggered and a voice prompt is given: "Danger! Please perform a safety check first."

[0161] Motion decomposition: Complex operations (such as ladle turntable positioning) are broken down into 8 basic motion units, which are demonstrated step by step by a digital human and compared in real time with joint angles.

[0162] Stress test: Simulate emergency response training under extreme conditions (such as sudden power outages or sensor malfunctions) to assess the trainees' accuracy in completing safe procedures within 30 seconds.

[0163] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0164] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A digital human configuration method based on industrial meta-universe digital twinning, characterized in that, The method comprises the following steps: Step S1: constructing a digital human model and training the constructed digital human model based on a configured data set to obtain a trained digital human model; wherein the configured data set comprises a standard operation action data set in a manufacturing process in an industrial scene, a voice data set in an industrial manufacturing scene, and an industrial manufacturing knowledge base; Step S2: monitoring a production line equipment in real time to obtain the running state of the production line equipment, and based on the obtained running state of the production line equipment, using the trained digital human model to perform anomaly detection, when an anomaly is detected, then using the digital human model to perform multi-modal early warning, fault positioning, and providing a repair scheme; Step S3: based on the detection equipment, obtaining detection data of a target finished product, the digital human model analyzes and statistics the obtained detection data, and optimizes the corresponding production line based on the analysis and statistics results; The step S2 comprises: Step S2.1: real-time acquisition of the running state of the production line equipment, including vibration, temperature and current information of the production line equipment; Step S2.2: using the trained digital human model to perform anomaly detection according to the collected running state of the production line equipment; when an anomaly is detected, then multi-modal early warning is performed, including visual early warning and voice early warning; Step S2.3: the digital human model performs fault positioning and provides a corresponding repair scheme according to the collected running state of the production line equipment based on the reasoning model constructed by the knowledge graph; Step S2.4: based on the repair scheme, the digital human model provides interactive guidance, including augmented reality labeling, voice step-by-step guidance, and knowledge question and answer; Wherein, the augmented reality labeling is to superimpose AR arrow on the physical equipment to indicate the disassembly sequence.

2. The digital human configuration method based on industrial meta-universe digital twinning according to claim 1, characterized in that, The step S1 comprises: Step S1.1: acquiring standard operation actions in a manufacturing process in an industrial scene by using an action capture device, including equipment maintenance and part assembly; using an action recognition model to recognize the acquired standard operation actions to construct an action library of the digital human; Step S1.2: acquiring voice data in an industrial manufacturing scene, including equipment running sound and operator instructions; pre-processing the acquired voice data to obtain pre-processed voice data; using a voice recognition model to recognize the pre-processed voice data to construct a voice library of the digital human; Step S1.3: integrating industrial manufacturing process knowledge, equipment maintenance manual and fault diagnosis cases to construct a knowledge graph; using natural language processing technology to convert the knowledge graph into a mapping relationship of equipment fault phenomenon, fault reason and solution method, thereby constructing an industrial manufacturing knowledge base of the digital human; Step S1.4: training the digital human model based on the action library of the digital human, the voice library of the digital human, and the industrial manufacturing knowledge base of the digital human to obtain the trained digital human model.

3. The digital human configuration method based on industrial meta-universe digital twinning according to claim 1, characterized in that, The step S3 comprises: Step S3.1: detecting the target finished product by a detection equipment to obtain detection data; Step S3.2: The digital human model judges the detection data based on the standard finished product requirements in the knowledge graph. When the detection data does not meet the preset requirements, the digital human model displays the defect location through a visual interface and provides corresponding processing suggestions based on the detection data; Step S3.3: The digital human model performs real-time statistics and analysis on the detection data to generate quality reports and trend charts; Step S3.4: Based on the generated quality reports and trend charts, potential quality problems in the production process are analyzed, and the production process and flow are optimized based on the potential quality problems in the production process obtained through analysis.

4. The digital human configuration method based on industrial meta-universe digital twinning according to claim 1, characterized in that, The digital human model supports remote collaboration; The digital human model supports remote collaboration, which includes constructing a virtual three-dimensional model of the on-site industrial equipment, using the virtual three-dimensional model to reflect the running state of the on-site industrial equipment in real time, and thus constructing a meta-universe space; Physical actions, postures, and position information of the operator are obtained, and the physical actions, postures, and positions of the operator are converted into the state of the on-site operator digital model in the meta-universe space in real time; the state of the on-site operator digital model is converted into three-dimensional space operation instructions, and the virtual three-dimensional model is realized in real time based on the three-dimensional space operation instructions; the real-time state of the on-site operator digital model and the real-time response of the virtual three-dimensional model are transmitted to the head-mounted display device of the remote expert, thereby realizing the intuitive display of the on-site operation; The remote expert obtains the physical actions, postures, and position information of the remote expert through a human body sensor, and converts the obtained physical actions, postures, and positions of the remote expert into the state of the remote expert digital model in the meta-universe space in real time; the state of the remote expert digital model is converted into three-dimensional space operation instructions, and the virtual three-dimensional model is realized in real time based on the three-dimensional space operation instructions; the real-time state of the remote expert digital model and the real-time response of the virtual three-dimensional model are transmitted to the head-mounted display device of the on-site operator, thereby realizing the remote operation display of the remote expert.

5. The digital human configuration method based on industrial meta-universe digital twinning according to claim 1, characterized in that, The method further comprises: According to the production task of the industrial production line, based on multiple historical planning simulation schemes, a comprehensive optimal scheme is generated according to a multi-objective decision algorithm score; Based on the generated comprehensive optimal scheme, instructions are issued through an edge computing node, and when production fluctuations occur in the production line equipment, real-time rescheduling or resource coordination is performed for production fluctuation response; The real-time rescheduling includes calling the process compensation rules in the knowledge graph for production fluctuation response; The resource coordination includes reassigning the path through a distributed task scheduling algorithm for production fluctuation response.

6. The digital human configuration method based on industrial meta-universe digital twinning according to claim 1, characterized in that, The method further comprises: performing three-dimensional dynamic demonstration of the standard operation process of the industrial production equipment through the digital human model, thereby realizing safety training; meanwhile, during the three-dimensional dynamic demonstration, the internal state of the equipment and the operation key point text prompts are displayed through a visual interface; During the student's actual operation, the student's operation actions are collected in real time through a motion capture system, and the digital human model compares and analyzes the captured operation actions with the standard action library, and performs real-time correction, action decomposition, and stress test operations according to the comparison and analysis results.

7. A digital human configuration system based on industrial metaverse digital twinning, characterized in that, It includes: Module M1: construct a digital human model and train the constructed digital human model based on a configured data set to obtain a trained digital human model; wherein the configured data set includes: a standard operation action data set in a manufacturing process in an industrial scene, a voice data set in an industrial manufacturing scene, and an industrial manufacturing knowledge base; Module M2: real-time monitoring of production line equipment to obtain the running state of the production line equipment, based on the obtained running state of the production line equipment, using the trained digital human model to detect abnormalities, when an abnormality is detected, then through the digital human model to include multi-modal early warning, fault positioning and provide repair scheme; Module M3: based on the detection equipment to obtain the detection data of the target finished product, the digital human model analyzes and statistics the obtained detection data, and optimizes the corresponding production line based on the analysis and statistics results; The module M2 includes: Module M2.1: real-time acquisition of the running state of the production line equipment, including the vibration, temperature and current information of the production line equipment; Module M2.2: using the trained digital human model to detect abnormalities according to the collected running state of the production line equipment; when an abnormality is detected, then multi-modal early warning is performed, including: visual early warning and voice early warning; Module M2.3: the digital human model based on the reasoning model constructed by the knowledge graph, according to the collected running state of the production line equipment to locate the fault and provide the corresponding repair scheme; Module M2.4: based on the repair scheme, the digital human model provides interactive guidance, including: augmented reality labeling, voice step-by-step guidance, and knowledge question and answer; Wherein, the augmented reality labeling is to superimpose AR arrow on the physical equipment to indicate the disassembly sequence.

8. The digital human configuration system based on industrial meta-universe digital twinning according to claim 7, characterized in that, The module M1 includes: Module M1.1: collect standard operation actions in a manufacturing process in an industrial scene through an action capture device, including: equipment maintenance and part assembly; use an action recognition model to recognize the collected standard operation actions to construct an action library of the digital human; Module M1.2: obtain voice data in an industrial manufacturing scene, including: equipment running sound and operator instructions; pre-process the collected voice data to obtain pre-processed voice data; use a voice recognition model to recognize the pre-processed voice data to construct a voice library of the digital human; Module M1.3: integrate industrial manufacturing process knowledge, equipment maintenance manual and fault diagnosis cases to construct a knowledge graph; use natural language processing technology to convert the knowledge graph into a mapping relationship of equipment fault phenomenon, fault reason and solution method, thereby constructing an industrial manufacturing knowledge base of the digital human; Module M1.4: train the digital human model based on the action library of the digital human, the voice library of the digital human and the industrial manufacturing knowledge base of the digital human to obtain the trained digital human model; The module M3 includes: Module M3.1: detect the target finished product through the detection equipment to obtain the detection data; The digital human model judges the detection data based on standard finished product requirements in the knowledge graph, and when the detection data does not meet the preset requirements, the digital human model displays the defect position through a visual interface, and provides corresponding processing suggestions according to the detection data; The digital human model performs real-time statistics and analysis on the detection data, and generates a quality report and a trend chart; The digital human model analyzes potential quality problems in the production process based on the generated quality report and trend chart, and optimizes the production process and flow based on the potential quality problems in the production process obtained through analysis.

9. The digital human configuration system based on industrial meta-universe digital twinning according to claim 7, characterized in that, The digital human model supports remote collaboration; The digital human model supports remote collaboration, including constructing a virtual three-dimensional model of the on-site industrial equipment, reflecting the running state of the on-site industrial equipment in real time by using the virtual three-dimensional model, and thereby constructing a meta universe space; Physical actions, postures and position information of the operator are obtained, and the physical actions, postures and positions of the operator are converted into the state of the on-site operator digital model in the meta universe space in real time; the state of the on-site operator digital model is converted into a three-dimensional space operation instruction, and the virtual three-dimensional model is realized in real time based on the three-dimensional space operation instruction; the real-time state of the on-site operator digital model and the real-time response of the virtual three-dimensional model are transmitted to the head-mounted display device of the remote expert, so as to realize intuitive display of on-site operation; The remote expert obtains physical actions, postures and position information of the remote expert through a human body sensor, and converts the obtained physical actions, postures and positions of the remote expert into the state of the remote expert digital model in the meta universe space in real time; the state of the remote expert digital model is converted into a three-dimensional space operation instruction, and the virtual three-dimensional model is realized in real time based on the three-dimensional space operation instruction; the real-time state of the remote expert digital model and the real-time response of the virtual three-dimensional model are transmitted to the head-mounted display device of the on-site operator, so as to realize remote operation display of the remote expert.

Images