AR-based teaching and practical training system for building, debugging and maintenance of unmanned factory

Abstract

The invention relates to the technical field of teaching practical training, and discloses an AR-based teaching practical training system for unmanned factory building, debugging and maintenance, which comprises a practical training operation module, a fault generation and assessment module, a collaborative interaction module, a personalized teaching module and a school-enterprise collaborative management module. Through AR virtual-real fusion, quadruple positioning and force feedback technologies, the whole-process training of construction, debugging and maintenance of the unmanned factory can be covered, the cost of physical equipment is greatly reduced, parallel operation of multiple persons is supported, tactile feedback and equipment data visualization are incorporated, and the training experience is closer to a real industrial scene; in addition, through a fault dynamic evolution + AI personalized learning technology, adaptation of a differentiated fault scene and a precise learning path is realized, the limitation of traditional single practical training and homogeneous teaching is broken, the skills of trainees are enabled to be matched with compound demands of enterprises, and the precision of talent training is improved.

Description

Technical Field

[0001] This invention relates to the field of teaching and training technology, and more specifically discloses an AR-based unmanned factory construction, debugging and maintenance teaching and training system. Background Technology

[0002] Unmanned factories refer to advanced manufacturing models that are highly automated and rely on industrial robots and intelligent sensing and control systems to achieve unmanned operation of the entire production process. Their construction, debugging, and maintenance involve composite skills from multiple fields such as mechanics, electrical engineering, robotics, and PLC programming. However, traditional practical training relies on physical industrial equipment, facing problems such as high costs, lagging equipment updates, and limited training scale. Furthermore, it is difficult to simulate the collaborative processes and dynamic fault scenarios of real production lines, leading to a disconnect between talent cultivation and industry needs. Therefore, an AR-based teaching and training system is needed for practical training.

[0003] The prior art patent document with authorization announcement number CN110264816A discloses a "Simulation Teaching System and Method for Intelligent Manufacturing Factory Based on 3D Virtual Technology", which includes: an intelligent manufacturing virtual simulation scene terminal connected via the Internet of Things, used to establish a three-dimensional simulation system for intelligent manufacturing factory and display the three-dimensional scene of intelligent manufacturing factory; and a student operation terminal, including VR glasses, operation tools and storage devices. The VR glasses display the three-dimensional scene, the operation tools input data, and the storage device saves the three-dimensional scene experienced by the student and the input data as the student's internship task.

[0004] The patent document with authorization announcement number CN114023126A discloses "a simulation teaching factory for aniline production", which includes a semi-physical simulation factory and a virtual reality simulation factory. The semi-physical simulation factory includes a dynamic process simulation system, an aniline physical object device and a control system. The dynamic process simulation system provides real-time calculation of process data for the aniline physical object device and the control system. The physical object device and the control system can realize hands-on training.

[0005] While existing technologies can break through the limitations of time and space through virtual simulation or a combination of virtual and semi-physical methods, allowing students to repeatedly practice to familiarize themselves with relevant production processes and equipment operations, and improve teaching and training effectiveness through teacher evaluation or automated assessment, and can also recreate factory production scenarios to a certain extent and ensure training safety, existing technologies lack precise alignment between virtual and real environments and tactile feedback experiences. They fail to achieve dynamic fault evolution based on students' skill levels and personalized teaching path adaptation, have insufficient cross-job collaborative training functions, and have not formed a deep linkage between real enterprise projects and teaching and training, nor an authoritative evidence storage and employment matching mechanism for training results, making it difficult to meet the comprehensive needs of cultivating compound skilled talents. Summary of the Invention

[0006] The main technical problem solved by this invention is to provide a teaching and training system for building, debugging and maintaining unmanned factories based on AR, which can solve the problems mentioned in the background art.

[0007] To address the aforementioned technical problems, according to one aspect of the present invention, more specifically, a teaching and training system for the construction, debugging, and maintenance of unmanned factories based on AR, comprising: a training operation module, a fault generation and assessment module, a collaborative interaction module, a personalized teaching module, and a school-enterprise collaborative management module. The training operation module constructs a virtual training environment for unmanned factories based on AR technology, supporting immersive hands-on simulation of unmanned factory construction, equipment debugging, and fault repair. The fault generation and assessment module builds basic data support based on common equipment and production line fault types in unmanned factories, generates fault scenarios, and formulates corresponding training assessment standards and evaluation rules. The collaborative interaction module supports multi-terminal access adaptation, enabling real-time transmission and interaction of training data between different users, and also has operation permission divisions for different roles. The personalized teaching module collects user operation data and skill performance during training, provides customized teaching content and training scenarios based on this data, and pushes suitable training courses. The school-enterprise collaborative management module realizes the transformation and connection between actual enterprise production resources and teaching and training resources, establishes a channel for school-enterprise joint training projects, records training results, and provides employment resource matching services.

[0008] Furthermore, the training operation module includes: a precise positioning module, a force feedback and visualization module, and a virtual production line debugging module;

[0009] Precise positioning module: It adopts infrared markers + spatial anchor points + SLAM real-time modeling + dynamic compensation algorithm to achieve precise spatial positioning between virtual devices and physical training platforms;

[0010] Force feedback and visualization module: The physical training platform is equipped with pressure sensors and torque sensors to simulate operating resistance, and AR glasses display the internal structure and data of the equipment in real time;

[0011] Virtual production line debugging module: Supports the construction of virtual production lines in unmanned factories, equipment wiring simulation, and PLC programming linkage debugging operations.

[0012] Furthermore, the fault generation and assessment module includes: a fault knowledge base module, a fault generation module, and a practical training assessment and evaluation module;

[0013] Fault Knowledge Base Module: Collects related data on common fault types in unmanned factories, their occurrence probability, inducing factors, chain reactions, standard maintenance time, and tool requirements, forming a structured fault knowledge base that includes 3D fault models, AR disassembly animations, troubleshooting flowcharts, standard maintenance procedures, and safety precautions.

[0014] Fault generation module: Based on AI learning and analysis of learners' skill levels, it generates differentiated fault scenarios at the beginner, intermediate, and expert levels;

[0015] Practical training assessment and evaluation module: Real-time recording of trainees' fault identification time and troubleshooting steps, and generating skills assessment reports and improvement suggestions based on the difficulty of the fault.

[0016] Furthermore, the collaborative interaction module includes: a terminal adaptation module, a data transmission module, and a permission allocation module;

[0017] Terminal adaptation module: Adapts to AR smart glasses, tablets, and PC terminals;

[0018] Data transmission module: adopts a 5G+edge computing architecture and a UDP+TCP hybrid transmission protocol;

[0019] Permission allocation module: Based on the training stage and the student's role, dynamic permissions such as scene editing, equipment operation, data viewing, and remote assistance are assigned.

[0020] Furthermore, the personalized teaching module includes: a skills assessment module, a scenario-based teaching module, and a course delivery module;

[0021] Skills assessment module: Analyze trainees' basic skills deficiencies in equipment knowledge and operation through practical training data;

[0022] Scenario-based teaching module: Based on real enterprise projects, it designs full-process scenario-based training tasks that include task decomposition and industry standard requirements;

[0023] Course delivery module: Based on the skills assessment results, push differentiated AR training courses, including beginner-level basic operation, intermediate-level complex debugging, and expert-level complex fault diagnosis.

[0024] Furthermore, the school-enterprise collaborative management module includes: a resource transformation module, a project training docking module, a blockchain evidence storage module, and an employment docking module;

[0025] Resource transformation module: Transforms enterprise production line layout and technical standard document resources into standardized training scenarios;

[0026] Project training integration module: Establishes a virtual training task channel based on real enterprise orders, allowing enterprise technical personnel to provide real-time guidance to trainees through a cloud platform;

[0027] Blockchain-based evidence storage module: Encrypts and stores student training reports, project results, and skill assessment levels;

[0028] Employment Matching Module: Excellent training results are synchronized to the school-enterprise collaborative cloud platform to provide enterprises with talent selection references and realize employment recommendation and matching.

[0029] Furthermore, the infrared markers of the precise positioning module are set at the physical training platform equipment installation interface, operation node, and fault detection point, and each marker is assigned a unique ID code.

[0030] Furthermore, the blockchain evidence storage module uses the SHA-256 hash algorithm to encrypt the evidence storage data.

[0031] The beneficial effects of this AR-based teaching and training system for the construction, debugging, and maintenance of unmanned factories are as follows: Through AR virtual-real fusion, quadruple positioning, and force feedback technology, it can cover the entire process of unmanned factory construction, debugging, and maintenance training. This not only significantly reduces the cost of physical equipment and supports multi-person parallel operation, but also incorporates tactile feedback and equipment data visualization, making the training experience closer to real industrial scenarios. Furthermore, through dynamic fault evolution and AI personalized learning technology, it achieves adaptation between differentiated fault scenarios and precise learning paths, breaking the limitations of traditional single-skill training and homogeneous teaching. This allows students' skills to match the complex needs of enterprises, improving the accuracy of talent cultivation. Simultaneously, through 5G edge computing and blockchain-based school-enterprise collaborative cloud platform technology, it provides a carrier for multi-terminal low-latency collaboration and real-time resource sharing, enabling synchronization of the latest enterprise technical standards and real projects. Furthermore, it facilitates employment matching based on the evidence of training results, greatly improving the efficiency of industry-education integration and the industry adaptability of unmanned factory training. Attached Figure Description

[0032] The present invention will now be described in further detail with reference to the accompanying drawings and specific implementation methods.

[0033] Figure 1 This is a schematic diagram of the system module architecture. Detailed Implementation

[0034] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0035] According to one aspect of the invention, such as Figure 1 As shown, an AR-based teaching and training system for the construction, debugging, and maintenance of unmanned factories is provided. It includes a training operation module, which constructs a virtual training environment for unmanned factories based on AR technology, supporting immersive hands-on simulation of unmanned factory construction, equipment debugging, and fault repair. This module includes:

[0036] Precise positioning module: It adopts infrared markers + spatial anchor points + SLAM real-time modeling + dynamic compensation algorithm to achieve precise spatial positioning between virtual devices and physical training platforms;

[0037] Specifically, highly reflective infrared markers are evenly distributed at key locations such as equipment installation interfaces, operation nodes, and fault detection points on the physical training platform. Each marker is assigned a unique ID code to ensure the identifiability and uniqueness of each positioning node. An infrared camera is installed inside the AR glasses to capture images of the markers at a sampling frequency of 30 frames per second. The three-dimensional coordinate information of the markers is quickly analyzed through image recognition algorithms to provide basic data for initial positioning.

[0038] In addition, multiple (e.g., four or more) spatial anchor base stations are deployed in the training site to form a full-coverage positioning network. The anchor base stations send location signals in real time through UWB (Ultra-Wideband) technology. After the AR glasses receive the signal, they are cross-validated with the infrared marker positioning data to initially correct the positioning deviation.

[0039] Finally, the AR glasses are equipped with LiDAR and vision sensors. The SLAM algorithm is used to perform real-time 3D modeling of the training site and physical training platform, generating a dynamically updated environmental map. To address positioning fluctuations caused by occlusion and changes in light that may occur in the industrial environment, a dynamic compensation algorithm is introduced. Based on historical positioning data and environmental map features, the positioning deviation is accurately predicted and corrected in real time, ultimately achieving alignment between the virtual device and the physical training platform.

[0040] Force feedback and visualization module: The physical training platform is equipped with pressure sensors and torque sensors to simulate operating resistance, and AR glasses display the internal structure and data of the equipment in real time;

[0041] Pressure and torque sensors are installed in the core mechanical operating components of the physical training platform (such as virtual robotic arm control handles, valve knobs, etc.). When students operate the physical components, the sensors collect the operating force and torque data in real time and transmit the data at high speed through the 5G edge computing gateway.

[0042] Meanwhile, the system is equipped with a virtual device operating resistance model. After receiving sensor data, it quickly matches the operating resistance parameters of the corresponding device and drives the force feedback device to simulate the operating resistance consistent with the real device, allowing trainees to have a real tactile operating experience.

[0043] Finally, the AR glasses overlay a real-time view of the virtual device's internal structure, key data streams (such as operating parameters like voltage, current, and speed), and standardized operating procedure instructions, enabling trainees to intuitively perceive the device's operating status and completely solving the technical bottleneck of traditional AR training where "you can see it but can't touch it."

[0044] Virtual production line debugging module: Supports the construction of virtual production lines in unmanned factories, equipment wiring simulation, and PLC programming linkage debugging operations;

[0045] Based on the actual production line layout of enterprises, it provides a standardized virtual equipment component library (including core equipment such as motors, sensors, robots, and conveyor belts). According to the training task requirements, trainees can complete equipment selection, layout planning and virtual installation in the virtual environment. The installation process strictly follows the industrial construction standards and verifies in real time whether key parameters such as equipment spacing and installation angle meet the requirements.

[0046] Meanwhile, trainees can perform circuit connection operations between devices in the AR scene. The system verifies the correctness of the wiring in real time. If a wiring error occurs, it will immediately issue a warning and mark the wrong node, and display the correct wiring diagram. In the PLC programming and debugging process, trainees can input PLC control programs. The system simulates the program running effect to realize the linkage control of the virtual production line and intuitively present the impact of program logic on the production line operation.

[0047] In addition, the training process simultaneously simulates problems that may occur in real industrial scenarios, such as equipment interference, signal failure, and program logic errors. Trainees need to solve these problems by checking equipment layout, verifying wiring, and optimizing programs, thereby strengthening their practical adaptability.

[0048] Finally, all setup and debugging operations follow industry standards (such as ISO 10218 robot safety standard) to ensure that the training operations are consistent with the actual production process of enterprises and improve the industry adaptability of the training results.

[0049] The fault generation and assessment module builds upon common equipment and production line fault types in unmanned factories to provide basic data support, generates fault scenarios, and establishes corresponding training assessment standards and evaluation rules. This module includes:

[0050] Fault Knowledge Base Module: Collects related data on common fault types in unmanned factories, their occurrence probability, inducing factors, chain reactions, standard maintenance time, and tool requirements, forming a structured fault knowledge base that includes 3D fault models, AR disassembly animations, troubleshooting flowcharts, standard maintenance procedures, and safety precautions.

[0051] Specifically, in collaboration with leading industrial equipment companies such as Siemens and FANUC, we systematically collected over 1200 common fault types in unmanned factories, covering core fault scenarios such as mechanical transmission jamming, sensor signal loss, PLC program errors, and pneumatic system leaks.

[0052] Then, by deeply mining the associated data of each fault, including the probability of fault occurrence, multi-dimensional inducing factors (such as ambient temperature and humidity, equipment running time, deviation of operating procedures, etc.), fault chain reaction path, industry-standard maintenance time, and a list of special maintenance tools and consumables, a comprehensive fault association data system is constructed.

[0053] Finally, multi-dimensional visualization and practical data are matched for each fault entry, including 3D fault models, step-by-step AR disassembly animations, logically clear fault troubleshooting flowcharts, standardized maintenance procedure texts that conform to industry standards, and targeted safety precautions (such as power-off operation specifications, high-voltage protection requirements, etc.). This results in a structured, reusable, and easily accessible fault knowledge base, providing solid data support for fault scenario generation and practical training.

[0054] Fault generation module: Based on AI learning and analysis of learners' skill levels, it generates differentiated fault scenarios at the beginner, intermediate, and expert levels;

[0055] AI learning analysis assesses students’ skill levels by collecting their initial training test scores (covering dimensions such as equipment structure cognition, basic operating procedures, and simple fault identification) and historical training operation data (such as past fault handling accuracy, training task completion time, and standardization of operating steps), and accurately classifies them into three levels: beginner, intermediate, and expert.

[0056] Meanwhile, differentiated fault scenarios are generated for different skill levels: for example, beginner students focus on single simple faults (basic operational faults such as loose sensor wiring and valves not closing tightly), emphasizing basic troubleshooting and maintenance skills training; advanced students are designed with complex fault scenarios (PLC program errors causing robot positioning deviation and conveyor belt jamming linkage faults), strengthening the ability to analyze and collaboratively handle multiple faults; and expert students are configured with complex cascading faults (power failures inducing multiple actuator failures and superimposed faults of data transmission interruption), training emergency response and system-level fault diagnosis capabilities.

[0057] In addition, the system will adjust the fault status based on the trainee's real-time troubleshooting operations: if the trainee fails to find the core fault point within the specified time, the system will trigger a fault chain reaction. For example, if the sensor signal is lost and not processed, it will cause the robot to misoperate, which will lead to the conveyor belt overload and shutdown, simulating the fault propagation law in a real factory. If the trainee's troubleshooting direction is correct, the system will gradually unlock the fault clues, such as displaying abnormal temperature data and current fluctuation curves of local equipment in real time, guiding the trainee to accurately locate the root cause of the fault.

[0058] Finally, the fault scenario generation process strictly matches the actual fault occurrence logic of industrial equipment. All fault phenomena, correlations, and evolution paths are derived from real production cases of enterprises, ensuring the industry adaptability of the training scenarios.

[0059] Practical training assessment and evaluation module: Real-time recording of trainees' fault identification time and troubleshooting steps, and generating skills assessment reports and improvement suggestions based on the difficulty of the fault;

[0060] Specifically, during the practical training process, the trainees' entire operation data is recorded in real time, including core indicators such as fault identification response time, completeness and standardization of troubleshooting steps, rationality of tool selection, accuracy of repair operation, total fault repair time, and safety and compliance of the repair process.

[0061] Then, a fault difficulty coefficient quantification system is introduced. Based on the fault type (single / compound / chain), troubleshooting complexity, maintenance technical requirements, and industrial scenario adaptability, different difficulty coefficients are assigned to each fault scenario. For example, the difficulty coefficient for entry-level faults is set to 0.1-0.3, for intermediate-level faults to 0.4-0.7, and for expert-level faults to 0.8-1.0.

[0062] Simultaneously, the operational data and the fault difficulty coefficient are weighted and calculated, using the following formula:

[0063]

[0064] In the formula, Assess trainees' overall skills. The difficulty level of the fault. Let i be the weight of the i-th evaluation indicator. For the score of the i-th evaluation indicator, a comprehensive skills score is generated. At the same time, combined with industry standard maintenance requirements and trainee operation performance, a deep analysis of skills shortcomings is conducted, such as insufficient PLC program debugging ability and lack of experience in troubleshooting mechanical transmission faults. Targeted improvement suggestions are made, such as recommending special AR training courses and strengthening practice of certain types of fault cases. The generated skills assessment report also includes operation data visualization charts, links to fault handling process replays, and comparative analysis with the average level of trainees of the same level, so that trainees can clearly understand their own strengths and weaknesses. At the same time, it provides teachers with precise teaching improvement basis and realizes closed-loop management of "training-assessment-reinforcement".

[0065] The collaborative interaction module supports multi-terminal access adaptation, enabling real-time transmission and interaction of training data between different users, and also features the ability to assign operation permissions to different roles. This module includes:

[0066] Terminal adaptation module: Adapts to AR smart glasses, tablets, and PC terminals;

[0067] Firstly, the AR smart glasses are equipped with binocular high-definition cameras, infrared sensors, and force feedback handles, enabling them to accurately respond to students' body movements and gesture commands. They support immersive equipment disassembly, virtual component assembly, and visual annotation of fault points, among other single-person practical training scenarios, meeting students' needs for close-range, highly immersive practical training.

[0068] Then, the tablet computer was adapted for multi-person collaboration functions, and a dedicated collaborative interactive interface was developed to support students to complete functions such as marking suspicious fault points, uploading troubleshooting solutions, real-time synchronization of operation progress, and group task allocation through touch screen operation, adapting to the collaboration needs of students in different positions such as mechanical, electrical, and programming in cross-position collaborative training.

[0069] Finally, the PC terminal is configured with core functional modules for scene editing, data management, and access control, supporting teachers to customize training scene parameters (such as modifying production line layout, adding new fault types, and setting training duration and assessment standards). It also integrates a student training progress monitoring panel, a skills assessment report export function, and a remote assistance instruction sending interface to meet teachers' teaching management and guidance needs.

[0070] Data transmission module: adopts a 5G+edge computing architecture and a UDP+TCP hybrid transmission protocol;

[0071] First, core tasks such as virtual scene rendering, real-time processing of training data, and multi-terminal data synchronization are deployed on edge computing nodes near the training site to shorten the data transmission path and avoid delays caused by transmitting core data through the remote cloud. At the same time, the edge computing nodes have local data caching and fast scheduling capabilities to ensure that training data is not lost and operations are not interrupted when there are sudden network fluctuations.

[0072] The use of a UDP+TCP hybrid transmission protocol for data classification and transmission involves using TCP for critical operational data such as fault diagnosis instructions, operation confirmation signals, and permission change requests. The TCP protocol ensures the reliability and integrity of data transmission through a three-way handshake mechanism, preventing the loss or error of critical operational instructions. For real-time transmission of large amounts of data, such as AR scene video streams, real-time device operation images, and multi-terminal collaborative annotation images, UDP is used, sacrificing the integrity of some non-core data to ensure real-time transmission. At the same time, a lightweight data compression algorithm is used to perform lossless compression processing on virtual scene models and animation frame data, ensuring that the training operation is smooth and lag-free.

[0073] Permission allocation module: Based on the training stage and the student's role, dynamic permissions such as scene editing, equipment operation, data viewing, and remote assistance are allocated;

[0074] Specifically, the entire training process is divided into four stages: training preparation, training operation, group collaboration, and assessment. The system allocates permissions based on two core roles: teachers and students (including students within and between groups).

[0075] For example, in the training preparation stage: teachers have full access to scene editing, training parameter settings, and permission configuration, while students only have access to view training materials and participate in initial training tests, but cannot operate equipment or modify scenes. In the training operation stage: students gain access to equipment operation, troubleshooting, and data recording, while teachers retain access to real-time monitoring, viewing operation data, and remote assistance (e.g., sending operation prompts and marking error nodes), ensuring that teachers can intervene and provide timely guidance while students practice independently. In the group collaboration stage: students within a group share access to viewing and editing operation records, fault labeling data, and troubleshooting solution documents, while data between groups is isolated, allowing students to view only their own group's training progress and task assignments, but not accessing other groups' core operation data. In the assessment stage: student permissions are locked, retaining only access to troubleshooting, repair operations, and result submission, but not access to reference answers, modification of operation records, or external assistance, while teachers have access to real-time viewing of assessment data and generation of assessment reports, ensuring fairness in the assessment.

[0076] The personalized teaching module collects user operational data and skill performance during practical training, and based on this data, provides customized teaching content and training scenarios, and pushes suitable training courses. This module includes:

[0077] Skills assessment module: Analyze trainees' basic skills deficiencies in equipment knowledge and operation through practical training data;

[0078] Specifically, when trainees log into the system for the first time, they are required to complete an initial training test. The test content covers three core dimensions: equipment structure cognition (names of core components, installation locations, and functions), basic operating procedures (equipment start-up and shutdown procedures, and safe operating standards), and simple fault identification (association of common fault phenomena with corresponding fault points). Each dimension has multiple graded questions (e.g., 20-30), and is accompanied by practical simulation tasks to comprehensively collect trainees' theoretical and practical basic data.

[0079] Then the system captures the student's entire training process operation data in real time, including key indicators such as course completion progress, training task time, operation step accuracy, fault identification response speed, tool selection rationality, and number of repeated errors, to ensure that the data covers the entire learning and practice scenario;

[0080] In addition, AI algorithms are used to analyze initial training test data and real-time training data to establish a skills assessment model. The model provides quantitative scores from dimensions such as equipment knowledge, operational proficiency, troubleshooting ability, and program debugging level, accurately identifying students' basic skill deficiencies, such as "insufficient knowledge of robot maintenance equipment" and "weak PLC program debugging logic," thus forming a visualized skills assessment.

[0081] Scenario-based teaching module: Based on real enterprise projects, it designs full-process scenario-based training tasks that include task decomposition and industry standard requirements;

[0082] For example, we can cooperate deeply with companies in fields such as intelligent manufacturing and auto parts production to obtain real project prototypes such as "automated assembly line construction", "intelligent warehousing system debugging" and "new energy vehicle parts production line fault repair", extract the core tasks, technical requirements and process specifications of the projects, and ensure that the training tasks are consistent with the actual needs of enterprises.

[0083] In addition, real-world job role simulations are introduced, allowing trainees to choose roles such as "Project Manager," "Mechanical Engineer," and "Electrical Debugger." Different roles assume corresponding job responsibilities (Mechanical Engineers are responsible for equipment layout and installation, while Electrical Debuggers are responsible for wiring and signal verification). In the process of collaboratively completing the project, the system simulates cross-job communication and collaboration scenarios in real-world work. At the same time, the system simulates unexpected problems during project implementation in real time (such as equipment model mismatch and conflicting debugging parameters), requiring trainees to work together to solve these problems based on their job functions, thus strengthening the integration of theory and practice.

[0084] Course delivery module: Based on the skills assessment results, push differentiated AR training courses, including beginner-level basic operation, intermediate-level complex debugging, and expert-level complex fault diagnosis.

[0085] First, based on the shortcomings identified in the skills assessment module, a precise matching library of "shortcomings-courses" is established. Each skill shortcoming corresponds to multiple specialized AR courses, with course content focusing on improving the shortcomings (for example, "insufficient knowledge of robot operation and maintenance equipment" corresponds to courses such as "3D cognition of core components of industrial robots" and "practical operation of robot installation and layout specifications").

[0086] Then, the courses are divided into different levels according to the students' skill level (taking 100 points as an example): The introductory course focuses on basic knowledge and operation simulation, with supporting animations explaining knowledge points and step-by-step operation guidance (such as the 3-step standard process for starting and stopping equipment, and the wiring demonstration of simple sensors), which is suitable for students with a skill score of less than 60 points. The intermediate course focuses on complex tasks and collaborative debugging, including multi-device linkage debugging cases and complex fault troubleshooting practice (such as the collaborative optimization of PLC programs and robot motion trajectories), which is suitable for students with a skill score of 60-90 points. The expert course focuses on complex system fault handling and technological innovation, covering cascading fault diagnosis and production line optimization solution design (such as debugging solutions for improving the efficiency of intelligent production lines), which is suitable for students with a skill score of 90 points or above.

[0087] Finally, courses are pushed out, and the system monitors students' course learning data in real time (video viewing completion rate, practical task accuracy rate, course test score). If a student's practical accuracy rate in a certain course is low, multiple similar reinforcement training tasks will be pushed out. If a student completes the advanced course ahead of time and scores ≥90 points on the test, they can unlock expert-level courses or choose in-depth courses in professional directions such as robot operation and maintenance, industrial control, and intelligent manufacturing, so as to achieve dynamic and personalized teaching with "one plan for one thousand people".

[0088] The school-enterprise collaborative management module facilitates the transformation and integration of actual production resources from enterprises with teaching and training resources, establishes a channel for joint school-enterprise training projects, records training results, and provides employment resource matching services. This module includes:

[0089] Resource transformation module: Transforms enterprise production line layout and technical standard document resources into standardized training scenarios;

[0090] Specifically, enterprises can upload the latest 3D layout diagrams of production lines, equipment technical parameter manuals, process standard documents, and real fault case videos through the upload port of the university-enterprise collaborative cloud platform. The uploaded resources support multiple mainstream formats such as STEP, PDF, and MP4 to meet the adaptation needs of different types of resources.

[0091] Meanwhile, through the AR conversion engine, the uploaded resources are standardized: the production line layout diagram is transformed into a 1:1 virtual training scene, the technical standard documents are broken down into the operation specifications guide in the training tasks, and the fault case videos are transformed into interactive AR fault demonstration scenes, ensuring that the resources are quickly implemented as training content.

[0092] Project training integration module: Establishes a virtual training task channel based on real enterprise orders, allowing enterprise technical personnel to provide real-time guidance to trainees through a cloud platform;

[0093] Enterprises publish virtual training tasks corresponding to real orders on the cloud platform, specifying task objectives (e.g., debugging accuracy requirements of a certain automotive parts production line), technical standards (e.g., relevant ISO production specifications), deliverables (e.g., debugging data reports, fault handling solutions), and setting parameters such as task start and end times and limits on the number of participants.

[0094] Then, based on the company's task requirements, the school organizes students to participate in groups. Students simulate the entire process of production line setup, equipment debugging, and troubleshooting in an AR scenario. Real-time data during the operation (such as operation steps, debugging parameters, and fault handling records) is uploaded to the cloud platform. Company technicians can view the students' operation progress and standardization through the real-time monitoring panel of the cloud platform. They can send remote guidance instructions through text, voice, and annotated screenshots. They can also score and comment on the students' deliverables after the training, forming a closed-loop training model of "company setting questions - students practicing - expert guidance - result evaluation".

[0095] Blockchain-based evidence storage module: Encrypts and stores student training reports, project results, and skill assessment levels;

[0096] Specifically, after trainees complete their practical training, the system collects core data such as practical training reports (including design schemes, operation records, and debugging data), project results (including virtual production line construction documents and fault repair plans), and skill assessment levels (including comprehensive scores and weakness analysis). At the same time, it links auxiliary verification information such as operation timestamps, equipment operation logs, teacher comments, and enterprise assessment scores to form a complete evidence data package.

[0097] Then, a consortium blockchain architecture is used for encrypted storage. Data packets are processed by a hash algorithm to generate a unique and irreversible encrypted string. Each stored data corresponds to a unique blockchain address and storage certificate, ensuring that the data is tamper-proof and traceable. When recruiting, companies can enter the student's storage certificate number through the cloud platform to verify the authenticity of the data on the blockchain node. Without relying on third-party endorsement, they can directly obtain authoritative proof of the student's skill level.

[0098] Employment Matching Module: Excellent training results are synchronized to the school-enterprise collaborative cloud platform to provide enterprises with talent selection references and realize employment recommendation and matching;

[0099] For example, the system selects student achievements with a comprehensive training score of 90 or above and excellent enterprise evaluations, and generates an "Excellent Training Achievement Database". The database is categorized and labeled according to professional direction (robot operation and maintenance, industrial control), skill specialties (PLC programming, fault repair), and project experience (automated assembly line construction, intelligent warehousing system debugging) to facilitate quick retrieval by enterprises.

[0100] Meanwhile, companies can post recruitment needs on the cloud platform, specifying job skill requirements (such as familiarity with Siemens PLC programming and experience in production line debugging). The system uses AI algorithms to accurately match company needs with trainees' certified achievements and skill assessment levels, and pushes a suitable list of trainee candidates and links to their achievements.

[0101] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention are also within the protection scope of the present invention.

Claims

1. A teaching and training system for building, debugging, and maintaining an unmanned factory based on AR, characterized in that, include: The system comprises four modules: a practical training module, a fault generation and assessment module, a collaborative interaction module, a personalized teaching module, and a school-enterprise collaborative management module. The practical training module utilizes AR technology to construct a virtual training environment for unmanned factories, supporting immersive practical simulations of unmanned factory setup, equipment debugging, and fault repair. The fault generation and assessment module builds basic data support based on common equipment and production line fault types in unmanned factories, generates fault scenarios, and establishes corresponding training assessment standards and evaluation rules. The collaborative interaction module supports multi-terminal access adaptation, enabling real-time transmission and interaction of training data between different users, and also features operation permission divisions for different roles. The personalized teaching module collects user operation data and skill performance during training, providing customized teaching content and training scenarios based on this data, and pushing suitable training courses. The school-enterprise collaborative management module facilitates the conversion and connection between actual enterprise production resources and teaching and training resources, establishes a channel for joint school-enterprise training projects, records training results, and provides employment resource matching services.

2. The AR-based teaching and training system for building, debugging, and maintaining an unmanned factory, as described in claim 1, is characterized in that: The training operation modules include: a precise positioning module, a force feedback and visualization module, and a virtual production line debugging module; Precise positioning module: It adopts infrared markers + spatial anchor points + SLAM real-time modeling + dynamic compensation algorithm to achieve precise spatial positioning between virtual devices and physical training platforms; Force feedback and visualization module: The physical training platform is equipped with pressure sensors and torque sensors to simulate operating resistance, and AR glasses display the internal structure and data of the equipment in real time; Virtual production line debugging module: Supports the construction of virtual production lines in unmanned factories, equipment wiring simulation, and PLC programming linkage debugging operations.

3. The AR-based teaching and training system for building, debugging, and maintaining an unmanned factory, as described in claim 1, is characterized in that: The fault generation and assessment module includes: a fault knowledge base module, a fault generation module, and a practical training assessment and evaluation module. Fault Knowledge Base Module: Collects related data on common fault types in unmanned factories, their occurrence probability, inducing factors, chain reactions, standard maintenance time, and tool requirements, forming a structured fault knowledge base that includes 3D fault models, AR disassembly animations, troubleshooting flowcharts, standard maintenance procedures, and safety precautions. Fault generation module: Based on AI learning and analysis of learners' skill levels, it generates differentiated fault scenarios at the beginner, intermediate, and expert levels; Practical training assessment and evaluation module: Real-time recording of students' fault identification time and troubleshooting steps, and generating skills assessment reports and improvement suggestions based on the difficulty of the fault.

4. The AR-based teaching and training system for building, debugging, and maintaining an unmanned factory, as described in claim 1, is characterized in that: The collaborative interaction module includes: a terminal adaptation module, a data transmission module, and a permission allocation module; Terminal adaptation module: Adapts to AR smart glasses, tablets, and PC terminals; Data transmission module: adopts a 5G+edge computing architecture and a UDP+TCP hybrid transmission protocol; Permission allocation module: Based on the training stage and the student's role, dynamic permissions such as scene editing, equipment operation, data viewing, and remote assistance are assigned.

5. The AR-based teaching and training system for building, debugging, and maintaining an unmanned factory, as described in claim 1, is characterized in that: The personalized teaching module includes: a skills assessment module, a scenario-based teaching module, and a course delivery module; Skills assessment module: Analyze trainees' basic skills deficiencies in equipment knowledge and operation through practical training data; Scenario-based teaching module: Based on real enterprise projects, it designs full-process scenario-based training tasks that include task decomposition and industry standard requirements; Course delivery module: Based on the skills assessment results, push differentiated AR training courses, including beginner-level basic operation, intermediate-level complex debugging, and expert-level complex fault diagnosis.

6. The AR-based teaching and training system for building, debugging, and maintaining an unmanned factory, as described in claim 1, is characterized in that: The school-enterprise collaborative management module includes: a resource transformation module, a project training and docking module, a blockchain evidence storage module, and an employment docking module; Resource transformation module: Transforms enterprise production line layout and technical standard document resources into standardized training scenarios; Project training integration module: Establishes a virtual training task channel based on real enterprise orders, allowing enterprise technical personnel to provide real-time guidance to trainees through a cloud platform; Blockchain-based evidence storage module: Encrypts and stores student training reports, project results, and skill assessment levels; Employment Matching Module: Excellent training results are synchronized to the school-enterprise collaborative cloud platform to provide enterprises with talent selection references and realize employment recommendation and matching.

7. The AR-based teaching and training system for building, debugging, and maintaining an unmanned factory, as described in claim 2, is characterized in that: The infrared markers of the precise positioning module are set at the installation interface, operation node, and fault detection point of the physical training platform equipment, and each marker is assigned a unique ID code.

8. The AR-based teaching and training system for building, debugging, and maintaining an unmanned factory, as described in claim 6, is characterized in that: The blockchain evidence storage module uses the SHA-256 hash algorithm to encrypt the evidence storage data.

Images