Scene-oriented mixed reality man-machine cooperative computing architecture and system

Abstract

The invention provides a scene-oriented mixed reality man-machine cooperative computing architecture and system, which can provide a more intuitive interaction mode, optimize a working process, realize visual expansion in a limited physical space and visual presentation of a multi-level situation, realize man-machine efficient cooperative scheduling in a limited range and improve the working efficiency. And a man-machine cooperation complex task solving paradigm based on mixed reality intelligent calculation is formed.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction and collaborative computing technology, and in particular to a scene-oriented mixed reality human-computer collaborative computing architecture and system. Background Art

[0002] Mixed reality (MR) is a technology that combines the real world and the virtual world. It expands the real world by adding virtual information to the user's vision, hearing and touch, that is, adding or enhancing virtual information in the user's field of view to provide a richer sensory experience. The core goal of mixed reality technology is to achieve a seamless integration of the real world and digital information to enhance the user's perception and interaction with the surrounding environment. As an important support for mixed reality, human-computer interaction technology is committed to achieving a more natural and efficient way of interaction. Human-computer collaborative computing architecture refers to a computing architecture in which human users and machine systems can collaborate with each other to complete tasks in specific application scenarios. This architecture usually requires processing a large amount of data and requires the system to respond in real time while maintaining a high degree of flexibility and adaptability. The scenario-oriented mixed reality human-computer collaborative computing architecture is designed for specific application scenarios. It combines the characteristics of mixed reality technology and human-computer collaborative computing to provide a more intuitive, efficient and intelligent way of working. However, despite the great progress made in this field, there are still some problems and challenges in the existing mixed reality human-computer interaction technology. Mixed reality applications often require the system to be able to process and feedback large amounts of data in real time. How to efficiently and accurately integrate virtual information with real-world data is a current technical difficulty.

[0003] Huazhong University of Science and Technology proposed a human interaction method based on mixed reality technology in its patent document "A human-computer two-way interaction method and system based on mixed reality technology" (patent application number: CN202410072739.7, publication number: CN117762260A). The method mainly includes the following steps: collecting user gaze points and user field of view images through mixed reality devices; performing target recognition on user field of view images to obtain multiple targets, and selecting user intended targets from these targets based on user gaze points; selecting the action mode of the machine end according to the user intended target, and the machine end executes the action mode; finally, the machine end collects the interaction information between it and the environment and feeds it back to the mixed reality device. In addition, the invention also provides a corresponding human-computer two-way interaction system, electronic device and storage medium. However, its key technology is the machine's understanding of the user's intention. It uses mixed reality devices to collect user gaze points and user field of view images. This patent uses mixed reality devices to integrate virtual and real scenes, and uses mixed reality head-mounted displays and touch operation screens to complete human-computer collaborative tasks in mixed reality scenes.

[0004] The 15th Research Institute of China Electronics Technology Group Corporation proposed a training situation visualization system based on mixed reality technology and its implementation method in its patent application "Training situation visualization system based on hybrid implementation and its implementation method" (patent application number: CN202311405645.9, publication number: CN117130492A). The method improves the visualization performance and command efficiency of training tasks through the collaborative work of lightweight data processing, human-computer interaction and situation visualization modules. The system is mainly composed of a data lightweight driving module, a human-computer interaction module and a situation visualization module, supplemented by a data management module for unified data storage, labeling and management. The system can not only integrate the data of real space and virtual space, provide an interactive experience combining virtual and real, but also perform storage, playback, conversion and export operations on the generated training situation data. In addition, the invention also describes the system's architecture design, specific implementation methods, and the configuration of related electronic devices and computer-readable storage media. However, its focus is on data processing and human-computer interaction, and it does not introduce mixed reality head-mounted displays and touch-operated displays to expand its human-computer interaction capabilities and situation visualization capabilities. Summary of the invention

[0005] The technical problem to be solved by the technical solution of the present invention is: the existing virtual reality method, although it has a strong sense of immersion, limits the visualization and synchronous interaction with the real environment. Although the augmented reality method can directly observe the real world and superimpose virtual information, it loses the sense of immersion. At the same time, the single interaction method is limited in usage scenarios and reduces the flexibility of interaction. In addition, although voice interaction has made certain progress, the recognition accuracy in noisy environments still needs to be improved. Gesture interaction technology still has certain limitations for complex gesture recognition and tracking. Providing users with a touch experience similar to that in the real world is still technically challenging. The response speed and feedback accuracy of touch devices will decrease when the temperature rises. Therefore, it is necessary to better integrate multiple interaction channels such as vision, hearing, and touch to create a consistent user experience.

[0006] In order to solve the above technical problems, the technical solution of the present invention provides a scene-oriented mixed reality human-computer collaborative computing architecture, including a user layer and a hardware platform layer for performing human-computer collaborative hybrid computing with the user layer, a basic environment layer, a technology layer, a function layer and an application layer;

[0007] The user layer includes operators;

[0008] The hardware platform layer includes an edge search end that obtains multi-source perception information, a command center end that performs visual processing on the multi-source perception information and executes and dispatches instructions from the user layer, and a computing and analysis end that optimizes decisions based on instructions from the user layer.

[0009] The command center obtains multi-source perception information from the edge search end and performs visual processing to generate real-time visual images. The operator at the user layer issues instructions based on the real-time visual images by operating the built-in human-computer interaction display console, touch screen and interactive devices of the command center end. The command center dispatches the edge search end and / or the computing and analysis end according to the instructions.

[0010] The basic environment layer includes basic data, operating system environment and hybrid computing engine. The basic data is used to store complex environment data composed of multi-source perception information, search target information corresponding to the multi-source perception information, simulation resource data for operator interaction at the user layer in the command center, and process data generated by computing and analyzing decisions in the computing and analyzing end. The operating system environment includes a hybrid reality operating system, a task operating system and a basic operating system corresponding to the edge search end, the command center end and the computing and analyzing end. The hybrid computing engine includes an operating environment corresponding to the edge search end, the command center end and the computing and analyzing end, which is a combination of a computing engine, an algorithm model library and a rule strategy library, and is used for intelligent scheduling and element classification and hierarchical extraction to obtain computing results.

[0011] The technical layer includes a human-computer interaction collaborative computing framework that combines immersive holographic scene construction technology, virtual-reality fusion high-precision matching technology, and multimodal interaction technology, which is used to provide technical support to the hardware platform layer. The human-computer interaction collaborative computing framework includes a virtual-reality interaction front end that interacts with the user layer to obtain computing requirements, and a hybrid computing back end that obtains the calculation results extracted by the hybrid computing engine according to the computing requirements to perform human-computer hybrid decision iteration;

[0012] Functional layers include mixed reality, human-computer interaction, and collaborative computing;

[0013] Mixed reality is used to process basic data according to the technical layer and the basic environment layer, integrate virtual and real scenes and display holographic situations. Human-computer interaction is used to execute processing according to gesture interaction, voice interaction, touch interaction and keyboard-mouse interaction obtained from the user layer and the hardware platform layer. Collaborative computing is used to perform computational analysis and execution according to the corresponding human-computer collaborative scheduling, multi-level information retrieval, multi-dimensional data intelligent interaction, and iterative optimization of human-computer decision-making at the hardware platform layer.

[0014] The application layer is used for the application of mixed reality, human-computer interaction and collaborative computing in the functional layer, including manned / unmanned search collaboration, decision collaboration, task collaboration, and situation awareness, intent understanding, and contextual cognition in deep situational awareness.

[0015] Preferably, the command center is provided with a mixed reality head-mounted display for presenting multi-source perception information as a three-dimensional scene situation on an extended visual channel.

[0016] Preferably, the computing engine includes graphics rendering, professional simulation and intelligent computing.

[0017] The technical solution of the present invention further provides a scene-oriented mixed reality human-computer collaborative computing system, which adopts the scene-oriented mixed reality human-computer collaborative computing architecture as described above, including:

[0018] The edge search end of the hardware platform layer obtains multi-source perception information and stores it in the basic environment layer. The operating system environment and hybrid computing engine of the basic environment layer provide basic support. The technical layer generates real-time visual images, which are displayed by the command center end. The operator of the user layer generates instructions through human-computer interaction at the command center end based on the real-time visual images.

[0019] The command center dispatches the edge search end to obtain multi-source perception information and / or sends instructions to the computing and analysis end according to the instructions. The computing and analysis end transmits the instructions to the virtual-reality interaction front end. The virtual-reality interaction front end triggers the computing requirements corresponding to the instructions to the hybrid computing engine, performs intelligent scheduling and element classification and hierarchical extraction to generate computing results and transmit them to the hybrid computing back end.

[0020] The functional layer obtains the transmission interaction between the hardware platform layer, the basic environment layer and the technical layer, and provides corresponding functional support. The application layer obtains functional support for application display.

[0021] Preferably, the functions supported include virtual environment and real environment functions, non-blocking real window function, font adaptive scaling function, sandbox overall rotation and movement interactive operation function, process control function, information retrieval function, mode switching function and third-person perspective display function.

[0022] Preferably, the hardware facilities on which the system operates include a high-performance hybrid computing integrated processor, a high-quality Varjo XR-3 head-mounted display device and a touch screen.

[0023] The technical solution of the present invention proposes a scenario-oriented mixed reality human-computer collaborative computing architecture and system, which can provide a more intuitive interaction method, optimize the workflow, realize visual expansion in limited physical space and visualize multi-level situations, realize efficient human-computer collaborative scheduling within a limited range, and form a human-computer collaborative complex task solving paradigm based on mixed reality intelligent computing. The technical solution of the present invention solves core problems such as user experience problems, interactive design defects, and insufficient interface intuitiveness that occur in complex human-computer collaborative workflows. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of a scene-oriented mixed reality human-computer collaborative computing architecture provided by an embodiment of the present invention;

[0025] Figure 2A schematic diagram of a scene-oriented mixed reality human-computer collaborative computing framework provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a system flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0028] like Figure 1 and Figure 2 As shown, the embodiment of the present invention provides a human-computer collaborative computing architecture based on mixed reality, and its specific scheme is as follows:

[0029] The architecture includes hardware platform layer, basic environment layer, technology layer, function layer, application layer and user layer, which can realize deep situational awareness and human-computer collaborative interaction.

[0030] The hardware platform layer includes the multi-level command and control equipment platform in the human-machine collaboration process and the visual computing and human-machine interaction equipment involved, which are divided into three parts: edge search end, command center end, and computing and analysis end. Integrate multi-level command and control equipment, visual computing equipment and human-machine interaction interfaces, such as unmanned search platform, mixed reality head-mounted display, etc. Among them, the mixed reality head-mounted display presents the multi-source perception information processed by the hybrid computing integrated processor as a three-dimensional scene situation on the extended visual channel; the mixed reality head-mounted display displays the auxiliary decision information calculated by the hybrid computing integrated processor; the mixed reality head-mounted display displays the real layout scene, which includes the operator's human-machine interaction display console, touch screen and interactive equipment. The operator can observe the actual position of the touch screen in the mixed reality head-mounted display, and can call a scene on the touch screen into the mixed reality head display through touch and display the corresponding three-dimensional situation; the operator can realize the visualization of the task unit information or status, the task target attributes and the iterative confirmation of the decision information through gesture interaction; the operator can start the call function of the airborne sensor detection information, target search information, and environmental information through voice interaction. The hybrid computing integrated processor collects multi-source perception data, calculates and generates three-dimensional situation information, and renders it in the mixed reality head-mounted display; the hybrid computing integrated processor monitors, receives, and processes multi-channel human-computer interaction instructions, supporting multi-tasking processing, data scheduling, information interaction, and decision optimization in the human-computer hybrid computing process.

[0031] The basic environment layer includes the basic environment required to support mixed reality holographic visualization, human-computer interaction, and human-computer collaborative computing, which is divided into basic data, operating system environment, and hybrid computing engine. Basic data includes complex environment data consisting of search target information and environmental perception information; simulation resource data consisting of virtual environment, role, and physical equipment; process data consisting of human-computer decision-making, behavior records, and system events. The operating system environment includes mixed reality operating system, task operating system, and basic operating system. The hybrid computing engine includes a computing engine consisting of graphics rendering, professional simulation, and intelligent computing, and an operating environment consisting of an algorithm model library and a rule strategy library.

[0032] The technical layer includes the human-computer interaction collaborative computing framework, immersive holographic scene construction technology, virtual-reality fusion high-precision matching technology, and multimodal interaction technology. Among them, the human-computer interaction and collaborative computing framework mainly includes three parts: the virtual-reality interaction front-end, the hybrid computing engine, and the hybrid computing back-end. The virtual-reality interaction front-end is triggered by the user personnel to the hybrid computing engine through human-computer collaborative scheduling; the hybrid computing engine automatically pushes the calculation results of the hybrid computing back-end to the virtual-reality interaction back-end through the intelligent collaborative computing engine with resource on-demand intelligent scheduling and factor classification and hierarchical extraction methods. The virtual-reality interaction back-end completes the virtual-reality interaction through multimodal human-computer interaction and realizes human-computer hybrid decision iteration.

[0033] The functional layer includes various functions of scene-oriented mixed reality human-computer collaborative computing, which are divided into three categories: mixed reality, human-computer interaction and collaborative computing.

[0034] Mixed reality includes the fusion of virtual and real scenes and holographic situation display; human-computer interaction includes gesture interaction, voice interaction, touch interaction, and keyboard and mouse interaction; collaborative computing includes human-computer collaborative scheduling, multi-level information retrieval, multi-dimensional data intelligent interaction, and iterative optimization of human-computer decision-making.

[0035] The application layer includes manned / unmanned search collaboration, decision-making collaboration, task collaboration supported by scene-oriented holographic display and control and human-machine intelligent hybrid computing technology, as well as situation awareness, intention understanding, and contextual cognition in deep situational awareness.

[0036] The user layer includes all personnel involved in human-machine hybrid intelligent computing, including decision makers, operators, information officers, and administrators.

[0037] like Figure 3 As shown, the embodiment of the present invention also provides a human-computer collaborative computing system based on mixed reality, and its specific scheme is as follows:

[0038] The system achieves seamless integration of the virtual and real worlds and is applicable to many fields such as simulation exercises, engineering design, education and training. It provides a realistic multi-dimensional interactive experience through a highly integrated software and hardware environment. The main purpose of the system is to improve the effectiveness of operational training, reduce the cost of simulation experiments, and enhance the convenience of cross-platform multi-modal data processing and interaction.

[0039] The virtual-reality interaction front end of the system is triggered by the user personnel to the hybrid computing engine through human-machine collaborative scheduling; the hybrid computing engine automatically pushes the calculation results of the hybrid computing back end to the virtual-reality interaction back end through the intelligent collaborative computing engine with resource on-demand intelligent scheduling and factor classification and hierarchical extraction methods. The virtual-reality interaction back end completes the virtual-reality interaction through multimodal human-machine interaction and realizes human-machine hybrid decision iteration. In this process, humans and machines work together, using human visual perception and cognitive decision-making to make comprehensive judgments and verifications, and feedback to the machine to improve the machine's intelligence level, thereby realizing the combination of human interpretation knowledge and machine intelligence and improving the efficiency of human-machine collaboration.

[0040] At the front end of virtual-reality interaction, personnel observe the planar situation of the large-screen touch display of the mission cabin display console, and send the interactive instructions of three-dimensional hierarchical perception needs to the human-machine collaborative scheduling module through keyboard and mouse interaction and touch interaction; at the same time, personnel observe the extended window information and three-dimensional holographic situation in the mixed reality head-mounted display, and after comprehensive judgment and verification, send the interactive instructions of multi-dimensional data intelligent display and decision-making iterative optimization needs to the human-machine collaborative scheduling module through multi-modal interaction means such as keyboard and mouse interaction, voice interaction, gesture interaction or handle interaction. The interactive instructions mainly include information protocols for information input, information display, data calculation, iterative optimization, process control, mode switching and other needs in human-machine interaction and collaborative computing, mainly including interaction methods, interaction areas, interaction elements, interaction needs, interaction requirements, etc. The human-machine collaborative scheduling module is responsible for integrating and processing multiple interactive instructions according to the designed information interaction rules to realize multi-channel data.

[0041] In the hybrid computing engine of the hybrid computing integrated processor, the collaborative computing engine is responsible for receiving the multi-channel data transmitted by the human-machine collaborative scheduling module, and decomposing it into multiple computing tasks according to the various interactive requirements of different interactive modes, such as multi-level information retrieval, data calculation, spatial operation, logical rule operation, model reasoning, simulation rendering, etc., scheduling the computing and storage resources of the hybrid computing backend on demand, and intelligently selecting relevant algorithm models to complete intelligent computing; at the same time, the key elements are extracted from the situation data by classification and stratification, and the simulation model is extracted from the simulation resource data, and the relevant data in the human-machine collaborative computing process is stored in the process data for subsequent retrieval. The collaborative computing engine pushes the calculation results and feedback information returned by the hybrid computing backend mixed reality to the graphics rendering engine and the professional simulation engine accordingly. The rendering engine displays the perception, navigation and other data and the digital earth base map data on the touch screen through scene drawing; the professional simulation engine constructs a holographic scene with the situation data and the simulation data, and performs virtual-real fusion matching between the real scene and the virtual scene, and intelligently presents the multi-dimensional data in the mixed reality headset through three-dimensional scene drawing.

[0042] The hybrid computing backend gathers situation data such as target search, environmental detection, and navigation from unmanned search platforms, manned platforms, and command platforms. At the same time, it performs tasks such as resource scheduling, intelligent computing, and element classification and hierarchical extraction of the collaborative computing engine.

[0043] At the hardware platform layer, unmanned or manned search platforms are used for edge search, and the command center includes mixed reality head-mounted displays, interactive devices, etc.; the basic environment layer provides a basic environment for mixed reality holographic visualization and human-computer interaction; the functional layer includes functions such as mixed reality, human-computer interaction and collaborative computing; the application layer provides scene-oriented holographic display and control and collaborative tasks supported by human-computer intelligent hybrid computing technology; the user layer includes decision makers, operators and other participants.

[0044] Among them, the high-performance graphics workstation provides complex computing services as a hybrid computing integrated processor, the positioning base station provides users with real-time positioning services in virtual and real environments, and the large-screen display, head-mounted display and simulated cockpit as well as the traditional keyboard and mouse human-computer interaction provide users with interactive services with mixed reality.

[0045] The main functions of the system:

[0046] 1) Visualization of multi-channel human-computer interaction methods such as gestures, voice, keyboard and mouse, touch, and related functional effects;

[0047] 2) Use mixed reality head mounted displays to optimize the display of multi-dimensional depth situation information such as holographic electronic sandbox and three-dimensional visual window in a mixed reality environment;

[0048] 3) The virtual-reality fusion interface displays both virtual environment and real environment functions;

[0049] 4) The virtual window of the virtual-reality fusion interface does not block the real window function;

[0050] 5) Adaptive font scaling function for virtual window of virtual-reality fusion interface;

[0051] 6) The overall rotation and movement interactive operation function of the sandbox in the mixed reality head-mounted display;

[0052] 7) Multiple interactive modes to realize process control, information retrieval, mode switching and other functions;

[0053] 8) Mixed reality head-mounted display content third-person perspective display function.

[0054] The operation of the system relies on high-performance hybrid computing integrated processors, high-quality Varjo XR-3 head-mounted display devices and touch screens and other hardware facilities, and interconnection between devices is achieved through standardized interfaces. In the development process of the virtual-reality fusion matching and multi-modal interaction technology system, the system development and operation platform is the industry-leading and widely used UnrealEngine 5. This high-performance 3D real-time rendering engine developed by Epic Games provides strong support for the system with its excellent graphics rendering capabilities, efficient development tool chain and rich ecosystem. In addition, C++ programming is used to design voice editing / develop interactive scripts in Unreal.

[0055] The beneficial effects of the embodiments of the present invention are as follows:

[0056] The technical solution provided by the embodiment of the present invention can provide a more intuitive interaction method, optimize the workflow, realize visual expansion in limited physical space and visualization of multi-level situations, realize efficient human-machine collaborative scheduling within a restricted range, and form a human-machine collaborative complex task solving paradigm based on mixed reality intelligent computing.

Claims

1. A scene-oriented mixed reality human-computer collaborative computing architecture, characterized in that: It includes the user layer and the hardware platform layer, basic environment layer, technology layer, function layer and application layer for human-machine collaborative hybrid computing with the user layer; The user layer includes operators; The hardware platform layer includes an edge search end that obtains multi-source perception information, a command center end that performs visual processing on the multi-source perception information and executes and dispatches instructions from the user layer, and a computing and analysis end that optimizes decisions based on instructions from the user layer. The command center obtains multi-source perception information from the edge search end and performs visual processing to generate real-time visual images. The operator at the user layer issues instructions based on the real-time visual images by operating the built-in human-computer interaction display console, touch screen and interactive devices of the command center end. The command center dispatches the edge search end and / or the computing and analysis end according to the instructions. The basic environment layer includes basic data, operating system environment and hybrid computing engine. The basic data is used to store complex environment data composed of multi-source perception information, search target information corresponding to the multi-source perception information, simulation resource data for operator interaction at the user layer in the command center, and process data generated by computing and analyzing decisions in the computing and analyzing end. The operating system environment includes a hybrid reality operating system, a task operating system and a basic operating system corresponding to the edge search end, the command center end and the computing and analyzing end. The hybrid computing engine includes an operating environment corresponding to the edge search end, the command center end and the computing and analyzing end, which is a combination of a computing engine, an algorithm model library and a rule strategy library, and is used for intelligent scheduling and element classification and hierarchical extraction to obtain computing results. The technical layer includes a human-computer interaction collaborative computing framework that combines immersive holographic scene construction technology, virtual-reality fusion high-precision matching technology, and multimodal interaction technology, which is used to provide technical support to the hardware platform layer. The human-computer interaction collaborative computing framework includes a virtual-reality interaction front end that interacts with the user layer to obtain computing requirements, and a hybrid computing back end that obtains the calculation results extracted by the hybrid computing engine according to the computing requirements to perform human-computer hybrid decision iteration; Functional layers include mixed reality, human-computer interaction, and collaborative computing; Mixed reality is used to process basic data according to the technical layer and the basic environment layer, integrate virtual and real scenes and display holographic situations. Human-computer interaction is used to execute processing according to gesture interaction, voice interaction, touch interaction and keyboard-mouse interaction obtained from the user layer and the hardware platform layer. Collaborative computing is used to perform computational analysis and execution according to the corresponding human-computer collaborative scheduling, multi-level information retrieval, multi-dimensional data intelligent interaction, and iterative optimization of human-computer decision-making at the hardware platform layer. The application layer is used for the application of mixed reality, human-computer interaction and collaborative computing in the functional layer, including manned / unmanned search collaboration, decision collaboration, task collaboration, and situation awareness, intent understanding, and contextual cognition in deep situational awareness.

2. A scene-oriented mixed reality human-computer collaborative computing architecture as claimed in claim 1, characterized in that: The command center is provided with a mixed reality head-mounted display for presenting multi-source perception information as a three-dimensional scene situation on an extended visual channel.

3. The scene-oriented mixed reality human-computer collaborative computing architecture according to claim 1, characterized in that: The computing engine includes graphics rendering, professional simulation and intelligent computing.

4. A scene-oriented mixed reality human-computer collaborative computing system, characterized in that: A scene-oriented mixed reality human-computer collaborative computing architecture as claimed in claim 1 is adopted, comprising: The edge search end of the hardware platform layer obtains multi-source perception information and stores it in the basic environment layer. The operating system environment and hybrid computing engine of the basic environment layer provide basic support. The technical layer generates real-time visual images, which are displayed by the command center end. The operator of the user layer generates instructions through human-computer interaction at the command center end based on the real-time visual images. The command center dispatches the edge search end to obtain multi-source perception information and / or sends instructions to the computing and analysis end according to the instructions. The computing and analysis end transmits the instructions to the virtual-reality interaction front end. The virtual-reality interaction front end triggers the computing requirements corresponding to the instructions to the hybrid computing engine, performs intelligent scheduling and element classification and hierarchical extraction to generate computing results and transmit them to the hybrid computing back end. The functional layer obtains the transmission interaction between the hardware platform layer, the basic environment layer and the technical layer, and provides corresponding functional support. The application layer obtains functional support for application display.

5. A scene-oriented mixed reality human-computer collaborative computing system as claimed in claim 4, characterized in that: The functions supported include virtual environment and real environment functions, non-blocking real window function, font adaptive scaling function, sandbox overall rotation and movement interactive operation function, process control function, information retrieval function, mode switching function and third-person perspective display function.

6. A scene-oriented mixed reality human-computer collaborative computing system as claimed in claim 4, characterized in that: The hardware facilities on which the system relies include a high-performance hybrid computing integrated processor, a high-quality Varjo XR-3 head-mounted display device and a touch screen.

Images