Information import method, device and system based on MR technology

Abstract

The invention provides an information import method, equipment and system based on an MR technology, and the technical scheme of the invention simulates the information input operation process of an internal computer network of an enterprise through building a set of MR augmented reality and AI action recognition interaction system, guides the operation of enterprise employees in real time, corrects the error operation in real time, and improves the operation efficiency. According to the method, correct, safe and timely import of information is guaranteed, the working time of operators is saved, the working process is optimized, and the method has very positive influence on shortening of the ship construction period.

Description

Technical Field

[0001] The present application belongs to the field of shipbuilding technology, and in particular, relates to an information import method, device and system based on MR technology. Background Art

[0002] The computer information network of large heavy industrial enterprises such as shipbuilding usually has extremely strict management systems and preventive measures, which inevitably bring many problems such as multiple steps for information system introduction, multiple devices involved, multiple software used, and multiple abnormal failures. Information management will be the responsibility of full-time personnel with strong management and information statistics capabilities to avoid the negative impact caused by other personnel's misoperation. However, when there is a need to import information for urgent and major business, full-time personnel are often unable to cope with it and find it difficult to properly handle emergencies in a timely manner. They cannot follow up in time, which delays emergency work, affects the construction period setting, and causes significant losses. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide an information import method, device and system based on MR technology to better solve the above-mentioned technical problems encountered in the existing ship information management.

[0004] In a first aspect, the present application provides an information import method based on MR technology, the method comprising:

[0005] Step S1, establishing an MR augmented reality management system;

[0006] Step S2, connecting the MR augmented reality management system with the MR glasses device;

[0007] Step S3, establishing multiple AI action recognition models for different body parts of the demonstrator;

[0008] Step S4, scanning the environment, calibrating the environment boundary, and performing point cloud modeling on the environment, and then calibrating the environment boundary in the generated environment model;

[0009] Step S5, the demonstrator wears the MR glasses device and performs the information import process;

[0010] Step S6, in the environment model generated in step S4, calibrate the hardware equipment used for information import and the software used for process import;

[0011] Step S7, performing AI analysis through the MR augmented reality management system according to the environment model generated in step S4 and the calibration result in step S6, to obtain a process model for importing information;

[0012] Step S8, downloading the process model to the MR glasses device for loading and running by the terminal application;

[0013] In step S9, the operator wears the MR glasses device, opens the terminal application, and performs the operation steps in sequence according to the augmented reality guidance of the MR glasses device to complete the information import work.

[0014] Furthermore, in step S3, creating multiple AI action recognition models includes:

[0015] The upper limbs of the demonstrator are photographed through various cameras of the MR glasses device, and the coordinates of key points are collected from the images;

[0016] Construct and draw convex hull points based on the collected key point coordinates, and find the number of external points;

[0017] Classify and store the actions represented by various point combinations;

[0018] The joints of fingers and wrists are used as key point detection objects, some key points are set as convex hull points, and the positions of the remaining key points are calculated to determine the posture of the person and create a hand recognition model.

[0019] Furthermore, in step S7, the AI ​​analysis process includes:

[0020] Based on MediaPipe’s gesture recognition algorithm, gesture actions are recorded and stored in time series to generate a gesture model.

[0021] The behavior prediction model is used to analyze the results of the gesture time series to obtain the probabilities of the two gestures before and after, and the gesture with the maximum probability value is used as the prediction result of the next behavior;

[0022] The connection probability between gestures and objects is analyzed through the process model, the results are stored, and the object with the maximum probability value is used as the prediction result of the next process.

[0023] Furthermore, the behavior prediction model is constructed as follows:

[0024] Sort out the items involved in each step of information import work;

[0025] Collect and annotate photographic samples of items;

[0026] Train an algorithm on the training platform that can recognize multiple objects;

[0027] A table defining "item combination-personnel behavior";

[0028] Arrange the business sequence of the object recognition algorithm and the query behavior table, and execute the business sequence in a loop to obtain the "behavior prediction model".

[0029] Furthermore, before step S8, a key node correction step is also included:

[0030] The demonstrator reviews and corrects the prediction results obtained by the AI ​​analysis in step S7, eliminates meaningless actions, and feeds them back to the MR augmented reality management system for relearning and choreographing the process, repeating steps S6 and S7 until the results meet the preset requirements.

[0031] Furthermore, before step S9, an operator identity authentication step is also included:

[0032] Enter staff information into the MR augmented reality management system;

[0033] When the MR augmented reality management system sends the model, it also sends the staff information to the MR glasses device; when the operator uses the MR glasses device, he first opens the terminal program, uses the MR glasses device to perform face recognition, authenticates and records the identity, and then starts the subsequent system process.

[0034] Furthermore, it also includes:

[0035] Step S10: The demonstrator downloads the video in the MR glasses device and uploads it to the MR augmented reality management system for AI analysis of invalid operations. By comparing the demonstrator's standard operations, the operator's invalid actions are found and marked as deviations from the threshold for the demonstrator to review.

[0036] Step S11, the demonstrator checks the result of step S9, manually improves the operation process, and iterates repeatedly from step S5 to step S10.

[0037] In a second aspect, the present application provides an information import device based on MR technology, which is used to execute the information import method provided by any of the above technical solutions, including:

[0038] Server, used to manage the scheduling of AI training tasks, store various models generated by AI, and compile terminal programs that can be run by MR glasses;

[0039] AI computing unit, including multiple servers that can provide AI computing capabilities, used to complete AI training tasks and accept server scheduling and management;

[0040] The client is used to access the information import system and issue manual instructions;

[0041] A display, used to display the management page of the information import system;

[0042] MR glasses are used to record operating procedures and provide augmented reality guidance during actual operations.

[0043] In a third aspect, the present application provides an information import system based on MR technology, the system comprising:

[0044] Connect the MR glasses module to import the MR glasses device access information into the system, register the serial number and hardware information, adapt the viewing angle and pupil distance adjustment range of the MR glasses, adapt the camera parameters of the MR glasses, and install the terminal application and process AI recognition model;

[0045] The scanning 3D modeling module is used to import the videos and photos taken by the MR glasses into the information system for 3D space modeling, matching the real office environment with the operation process;

[0046] Manual calibration software and hardware module, used to manually calibrate the hardware equipment involved in the operation process, and manually calibrate the operation steps of the software involved in the operation process;

[0047] The AI ​​analysis operation video module is used to record the operation process, perform AI recognition analysis, refine the human behavior and mouse behavior in each operation link, identify the key action nodes, and match the key action nodes with the operation process;

[0048] The manual correction AI analysis module is used to manually review and annotate the results of AI analysis and correct the key action nodes that AI identified incorrectly for AI re-analysis;

[0049] The personnel and face management module is used to maintain the information of the staff, package and compile the information into basic data, and download it to the MR glasses by the terminal program compilation module;

[0050] AI training hardware scheduling module, used to schedule AI training hardware in the information import system, queue and schedule AI analysis tasks, and maximize the use of hardware;

[0051] The glasses terminal program compilation module is used to compile terminal programs that can be run by MR glasses, as well as to package process AI recognition models and staff information. MR glasses run terminal programs by loading models and staff information to realize corresponding functions.

[0052] Compared with the prior art, the technical solution provided by this application has the following beneficial effects:

[0053] The technical solution provided in this application establishes a set of MR augmented reality and AI motion recognition interactive systems to simulate the information input operation process of the enterprise's internal computer network, provide real-time guidance for the operations of enterprise employees, and correct erroneous operations in real time, thereby ensuring the correct, safe and timely import of information, saving the working time of operators, optimizing the work process, and having a very positive impact on shortening the ship construction period. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of an information import method based on MR technology provided in Example 1 of the present application;

[0055] Figure 2 This is a structural diagram of the information import module based on MR technology provided in Example 2 of the present application. DETAILED DESCRIPTION

[0056] The feasible solutions to the thorny problems encountered in the background technology include the following: 1) training as many operators as possible to learn how to import information; 2) the system administrator provides telephone guidance on the operation.

[0057] However, the above methods still have obvious disadvantages: for example, 1) although operators have received several trainings, they do not operate the system on a daily basis, and full-time personnel are responsible for the operation; occasionally the operation is very unfamiliar, and full-time personnel are still needed to provide telephone guidance; 2) The system administrator's telephone guidance is not intuitive, and untrained personnel cannot execute remote control commands, or often make mistakes, resulting in more serious consequences; using video guidance instead will lead to information leakage.

[0058] In order to solve the problems of limited working time of full-time personnel, unfamiliarity of operators, inefficient telephone guidance, and operator errors in the above method, the present invention provides an information import method, device and system based on MR technology. MR usually refers to "Mixed Reality", which is a technology that combines the real world with the virtual world. The following describes the implementation of the present application through specific examples. Those skilled in the art can easily understand other advantages and principles of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods. The details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application.

[0059] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the restrictive conditions for the implementation of this patent, so they have no substantial technical significance. Any structural modification, change in proportional relationship or adjustment of size, without affecting the effects and purposes that can be achieved by this patent, should still fall within the scope of the technical content disclosed by this patent. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and so on quoted in this specification are only for the convenience of description, and are not used to limit the scope of the implementation of this patent. The change or adjustment of their relative relationship should also be regarded as the scope of the implementation of this patent without substantial change in the technical content.

[0060] Embodiment 1:

[0061] This embodiment provides an information import method based on MR technology, including the following steps:

[0062] Step S1, first establish an MR augmented reality management system. Specifically, the system includes several basic functional modules for later call, such as: a functional module for docking MR glasses, a functional module for three-dimensional modeling, a functional module for AI analysis of sample operation process videos, a functional module for manually correcting AI analysis results, a functional module for manually calibrating the operated device, a functional module for compiling the operation process, a functional module for personnel and face management, a functional module for AI training hardware scheduling, etc., to realize the various functions of the system.

[0063] Step S2, connect the MR augmented reality management system to the MR glasses device. Specifically, to connect the MR glasses device to the MR augmented reality management system, it is necessary to register the device serial number and hardware information, adapt the viewing angle and pupil distance adjustment range of the MR glasses device, adapt the various camera parameters of the MR glasses device, install the terminal application to the MR glasses and run the DEMO process. Furthermore, the MR glasses device includes at least a main camera, an infrared camera and a depth camera, that is, it is necessary to adapt the main camera, infrared camera and depth camera parameters of the MR glasses device respectively to provide a more accurate guidance function.

[0064] Step S3, establishing multiple AI action recognition models for different body parts of the demonstrator.

[0065] Specifically, creating a variety of AI action recognition models includes:

[0066] The upper limbs of the demonstrator are photographed through various cameras of the MR glasses device, and the coordinates of key points are collected from the images;

[0067] Construct and draw convex hull points based on the collected key point coordinates, and find the number of external points;

[0068] Classify and store the actions represented by various combinations of points, such as grasping the mouse, clicking the mouse, typing on the keyboard, holding a USB flash drive, taking a CD, and writing with a pen;

[0069] The joints of fingers and wrists are used as key point detection objects, some key points are set as convex hull points, and the positions of the remaining key points are calculated to determine the posture of the person and create a hand recognition model.

[0070] Step S4, scan the environment, calibrate the environment boundary, and perform point cloud modeling on the environment, and then calibrate the environment boundary in the generated environment model. Specifically, the demonstrator wears MR glasses and turns on the video recording, walks in the office where the information is imported, and shoots every corner of the place as detailed as possible, and gradually expands the scope of activities until the boundary area, imports the video file into the MR augmented reality management system, and performs point cloud modeling on the environment; in the generated model, manually calibrate the boundaries of the office. Furthermore, different color film layers are attached to key equipment to enhance the real-time positioning effect in the later stage.

[0071] Step S5: The demonstrator wears the MR glasses and performs the information import process.

[0072] Step S6, in the environment model generated in step S4, calibrate the hardware equipment used for information import and the software used for process import. Specifically, this step of calibration work adopts manual calibration.

[0073] Step S7, based on the environmental model generated in step S4 and the calibration result in step S6, AI analysis is performed through the MR augmented reality management system to extract the action points of the demonstrator, the mouse cursor and the software in each node to obtain a process model for importing information.

[0074] Specifically, the AI ​​analysis process includes: 1) Based on the gesture recognition algorithm of MediaPipe, the gesture actions are recorded and stored in time series to generate a gesture model; 2) The behavior prediction model is used to analyze the results of the gesture time series to obtain the probabilities of the two previous and subsequent gesture actions, and the gesture action with the maximum probability value is used as the prediction result of the next behavior; 3) The connection probability between gesture actions and objects is analyzed through the process model, the results are stored, and the object with the maximum probability value is used as the prediction result of the next process.

[0075] Furthermore, the behavior prediction model can be superimposed with the environmental model to analyze the connection probability between gestures and objects, thereby improving the judgment accuracy.

[0076] Furthermore, the behavior prediction model is constructed as follows:

[0077] 1) Sort out the items involved in each link of the information import work, such as monitors, keyboards, mice, palms, USB flash drives, CDs, computer cases, USB interfaces, registers, doors, etc.

[0078] 2) Collect and label a sample of photos of the objects.

[0079] 3) Train an algorithm on the training platform that can identify multiple objects;

[0080] 4) Define the table of "object combination - person behavior". For example, palm + register = register and import information; finger + USB = take USB; finger + CD = take CD; CD + door = take CD and leave the room, etc.

[0081] 5) Arrange the business sequence of the object recognition algorithm and the query behavior table, and execute the business sequence in a loop to obtain the "behavior prediction model".

[0082] Subsequently, the key node correction step is performed. Specifically, the demonstrator reviews and corrects the prediction results obtained by the AI ​​analysis in step S7, eliminates meaningless actions, and feeds back to the MR augmented reality management system for relearning and choreographing the process, repeating steps S6 and S7 until the results meet the preset requirements.

[0083] Step S8, downloading the process model to the MR glasses device for loading and running by the terminal application;

[0084] Subsequently, the operator identity authentication step is performed. Specifically, the staff information is entered and maintained in the MR augmented reality management system. When the MR augmented reality management system sends the model, it also sends the staff information to the MR glasses device; when the operator uses the MR glasses device, he first opens the terminal program and uses the camera of the MR glasses device to perform face recognition, and after identifying and recording the identity, the subsequent system process is started.

[0085] In step S9, the operator wears the MR glasses device, opens the terminal application, and performs the operation steps in sequence according to the augmented reality guidance of the MR glasses device to complete the information import work. Specifically, after opening the terminal application, the terminal application calls the AI ​​computing power of the mobile CPU to identify and judge the completion of the operation stage and determine the next link; according to the content of the current link, the operator is given augmented reality guidance, and the operator's movements, mouse cursor and software interaction are monitored in real time, and voice warnings and prompts are given for operations that do not follow the guidance, guiding the operator back to the right path. Furthermore, in this process, the MR glasses record the complete operation process for subsequent analysis and improvement.

[0086] In step S10, the demonstrator downloads the video in the MR glasses device and uploads it to the MR augmented reality management system for AI analysis of invalid operations. By comparing the demonstrator's standard operations, the operator's invalid actions are discovered and marked as deviations from the threshold for the demonstrator to review.

[0087] Step S11, the demonstrator checks the result of step S9, manually improves the operation process, and iterates repeatedly from step S5 to step S10.

[0088] By adopting the above information import method, the complicated information import and management work of the enterprise is greatly facilitated, and at least the following beneficial effects are brought about:

[0089] 1) The information import business has been changed from being operated by full-time personnel to being operated by everyone, and timely import has been achieved 24 hours a day, ensuring the timely development of corporate business and avoiding economic losses for the company.

[0090] 2) Improve complex, forgettable and error-prone businesses with real-time “personal” guidance from AI, replacing remote control by telephone or video by full-time personnel, thus improving the accuracy of operations.

[0091] 3) It avoids the situation where full-time staff have to rush to the company late at night and standby for remote support, thus saving manpower and improving the satisfaction of full-time staff.

[0092] 4) It eliminates the risk of information leakage in video guidance and telephone guidance, and protects corporate information security.

[0093] 5) When fine-tuning the business process, there is no need to convene all employees for training. Instead, the process model can be modified and generated in a small range. The MR glasses will prompt and guide the operators to improve and implement the fine-tuning, which is convenient and efficient.

[0094] 6) The identification behaviors at each stage of the process are standardized to avoid operators forgetting or being careless and missing out on key links.

[0095] 7) The operator's entire operation process can be recorded to prepare for subsequent evidence collection and facilitate accountability.

[0096] 8) This method can be further expanded to other business scenarios of internal computer networks and has a wide range of usage scenarios.

[0097] Embodiment 2:

[0098] This embodiment provides an information import system based on MR technology, that is, an MR augmented reality management system, which is isolated from the Internet. At present, the main devices equipped with MR technology in the market are MR glasses, head-mounted displays, smart phones, etc. This patent mainly uses built-in applications and preset operation processes in MR glasses to call the AI ​​computing power of mobile CPU chips to identify the operator's hand movements, identify the operated devices, identify software buttons and mouse arrows, so as to guide correct behaviors, predict and promptly correct wrong behaviors, and record wrong behaviors. In a form similar to a close action coach, guide the operator to complete a complete set of complex information import processes.

[0099] The system includes a docking MR glasses module, a scanning 3D modeling module, a manual calibration software and hardware module, an AI analysis and operation video module, a manual correction AI analysis module, a personnel and face management module, an AI training hardware scheduling module, and a glasses terminal program compilation module. The above functional modules are described in detail below.

[0100] Docking the MR glasses module is used to import the MR glasses device access information into the system, register the serial number and hardware information, adapt the viewing angle and pupil distance adjustment range of the MR glasses, adapt the camera parameters of the MR glasses, such as the parameters of the main camera, infrared camera, depth camera and other components of the MR glasses, and install terminal applications and process AI recognition models.

[0101] The scanning 3D modeling module is used to import the videos and photos taken by the MR glasses into the information system for 3D space modeling, matching the real office environment with the operating procedures.

[0102] The manual calibration software and hardware module is used to manually calibrate the hardware devices involved in the operation process, such as computers, mice, keyboards, monitors, USB drives and CDs, and to manually calibrate the operating steps of the software involved in the operation process.

[0103] The AI ​​analysis operation video module is used to record the operation process, perform AI recognition analysis, refine the human behavior and mouse behavior in each operation link, identify key action nodes, and match and bind the key action nodes with the operation process.

[0104] The manual correction AI analysis module is used to manually review and annotate the results of AI analysis and correct key action nodes that AI has identified incorrectly for AI re-analysis.

[0105] The personnel and face management module is used to maintain staff information, such as work number, photo, rank information, etc., and package the information into basic data, which is downloaded to the MR glasses by the terminal program compilation module; face information is used for operator authentication and identity recording.

[0106] The AI ​​training hardware scheduling module is used to schedule the AI ​​training hardware in the information import system, and queue and schedule AI analysis tasks such as environmental point cloud modeling, process video analysis, and manual annotation result analysis to maximize the use of hardware. Furthermore, this module can be deployed in a distributed manner on one or more high-performance graphics servers, which can achieve high-performance analysis of new tasks without occupying high-end computing resources for a long time.

[0107] The glasses terminal program compilation module is used to compile terminal programs that can be run by MR glasses, as well as to package process AI recognition models and staff information. MR glasses run terminal programs by loading models and staff information to realize corresponding functions.

[0108] This embodiment further provides an information import device based on MR technology, which is used to execute the information import method provided in the first embodiment. The device includes:

[0109] Server, used to manage the scheduling of AI training tasks, store various models generated by AI, and compile terminal programs that can be run by MR glasses;

[0110] AI computing unit, including multiple servers capable of providing AI computing capabilities, to provide high-density, high-performance AI computing capabilities for completing AI training tasks and accepting server scheduling and management;

[0111] The client is used to access the information import system and issue manual instructions;

[0112] A display, used to display the management page of the information import system;

[0113] MR glasses are used to record operating procedures and provide augmented reality guidance during actual operations.

[0114] In summary, the technical solution provided by this application has high industrial utilization value because it effectively overcomes various shortcomings in the prior art.

[0115] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. An information import method based on MR technology, characterized in that: The method comprises: Step S1, establishing an MR augmented reality management system; Step S2, connecting the MR augmented reality management system with the MR glasses device; Step S3, establishing multiple AI action recognition models for different body parts of the demonstrator; Step S4, scanning the environment, calibrating the environment boundary, and performing point cloud modeling on the environment, and then calibrating the environment boundary in the generated environment model; Step S5, the demonstrator wears the MR glasses device and performs the information import process; Step S6, in the environment model generated in step S4, calibrate the hardware equipment used for information import and the software used for process import; Step S7, performing AI analysis through the MR augmented reality management system according to the environment model generated in step S4 and the calibration result in step S6, to obtain a process model for importing information; Step S8, downloading the process model to the MR glasses device for loading and running by the terminal application; In step S9, the operator wears the MR glasses device, opens the terminal application, and performs the operation steps in sequence according to the augmented reality guidance of the MR glasses device to complete the information import work.

2. The information import method based on MR technology according to claim 1, characterized in that: In step S3, multiple AI action recognition models are created including: The upper limbs of the demonstrator are photographed through various cameras of the MR glasses device, and the coordinates of key points are collected from the images; Construct and draw convex hull points based on the collected key point coordinates, and find the number of external points; Classify and store the actions represented by various point combinations; The joints of fingers and wrists are used as key point detection objects, some key points are set as convex hull points, and the positions of the remaining key points are calculated to determine the posture of the person and create a hand recognition model.

3. The information import method based on MR technology according to claim 1, characterized in that: In step S7, the AI ​​analysis process includes: Based on MediaPipe’s gesture recognition algorithm, gesture actions are recorded and stored in time series to generate a gesture model. The behavior prediction model is used to analyze the results of the gesture time series to obtain the probabilities of the two gestures before and after, and the gesture with the maximum probability value is used as the prediction result of the next behavior; The connection probability between gestures and objects is analyzed through the process model, the results are stored, and the object with the maximum probability value is used as the prediction result of the next process.

4. The information import method based on MR technology according to claim 3, characterized in that: The construction method of the behavior prediction model is: Sort out the items involved in each step of information import work; Collect and annotate photographic samples of items; Train an algorithm on the training platform that can recognize multiple objects; A table defining "item combination-personnel behavior"; Arrange the business sequence of the object recognition algorithm and the query behavior table, and execute the business sequence in a loop to obtain the "behavior prediction model".

5. The information import method based on MR technology according to claim 1, characterized in that: Before step S8, a key node correction step is also included: The demonstrator reviews and corrects the prediction results obtained by the AI ​​analysis in step S7, eliminates meaningless actions, and feeds them back to the MR augmented reality management system for relearning and choreographing the process, repeating steps S6 and S7 until the results meet the preset requirements.

6. The information import method based on MR technology according to claim 1, characterized in that: Before step S9, an operator identity authentication step is also included: Enter staff information into the MR augmented reality management system; When the MR augmented reality management system sends the model, it also sends the staff information to the MR glasses device; when the operator uses the MR glasses device, he first opens the terminal program, uses the MR glasses device to perform face recognition, authenticates and records the identity, and then starts the subsequent system process.

7. The information import method based on MR technology according to claim 1, characterized in that: Also includes: Step S10: The demonstrator downloads the video in the MR glasses device and uploads it to the MR augmented reality management system for AI analysis of invalid operations. By comparing the demonstrator's standard operations, the operator's invalid actions are found and marked as deviations from the threshold for the demonstrator to review. Step S11, the demonstrator checks the result of step S9, manually improves the operation process, and iterates repeatedly from step S5 to step S10.

8. An information import device based on MR technology, used to execute the information import method according to any one of claims 1 to 7, characterized in that: include: Server, used to manage the scheduling of AI training tasks, store various models generated by AI, and compile terminal programs that can be run by MR glasses; AI computing unit, including multiple servers that can provide AI computing capabilities, used to complete AI training tasks and accept server scheduling and management; The client is used to access the information import system and issue manual instructions; A display, used to display the management page of the information import system; MR glasses are used to record operating procedures and provide augmented reality guidance during actual operations.

9. An information import system based on MR technology, characterized in that: The system comprises: Connect the MR glasses module to import the MR glasses device access information into the system, register the serial number and hardware information, adapt the viewing angle and pupil distance adjustment range of the MR glasses, adapt the camera parameters of the MR glasses, and install the terminal application and process AI recognition model; The scanning 3D modeling module is used to import the videos and photos taken by the MR glasses into the information system for 3D space modeling, matching the real office environment with the operation process; Manual calibration software and hardware module, used to manually calibrate the hardware equipment involved in the operation process, and manually calibrate the operation steps of the software involved in the operation process; The AI ​​analysis operation video module is used to record the operation process, perform AI recognition analysis, refine the human behavior and mouse behavior in each operation link, identify the key action nodes, and match the key action nodes with the operation process; The manual correction AI analysis module is used to manually review and annotate the results of AI analysis and correct the key action nodes that AI identified incorrectly for AI re-analysis; The personnel and face management module is used to maintain the information of the staff, package and compile the information into basic data, and download it to the MR glasses by the terminal program compilation module; AI training hardware scheduling module, used to schedule AI training hardware in the information import system, queue and schedule AI analysis tasks, and maximize the use of hardware; The glasses terminal program compilation module is used to compile terminal programs that can be run by MR glasses, as well as to package process AI recognition models and staff information. The MR glasses run the terminal program by loading the model and staff information to realize the corresponding functions.

Images