MR space interaction system

Abstract

The invention belongs to the technical field of mixed reality, and particularly discloses an MR space interaction system which comprises space computing equipment and a plurality of user terminals, each user terminal comprises MR glasses and a wearing device, and the wearing devices are in communication connection with the MR glasses and the space computing equipment. According to the invention, related information such as accurate position, visual field and interactive action of each user can be accurately measured and calculated, and a spatial position result is calculated by integrating the position information of a virtual object, so that the effect that multiple persons share the same MR spatial scene is achieved; main functional modules can be concentrated on the spatial computing equipment, so that the weight of the MR glasses of the user is reduced, and the experience comfort of the user is improved; the complete body of the user can be fused into the real-time shared MR space in the use process of the user.

Description

Technical Field

[0001] This invention belongs to the field of mixed reality technology, specifically relating to an MR spatial interaction system. Background Technology

[0002] With the development of MR (Mixed Reality) technology, users have placed higher demands on the performance, effects, wearability, and comfort of MR-related devices. Currently, existing MR products on the market still have the following shortcomings and deficiencies:

[0003] 1. In multi-user shared MR scenarios, the same virtual object within the same MR scene and space cannot be accurately anchored to multiple users simultaneously. Furthermore, if a user arbitrarily changes the position, size, or angle of a virtual object, other users cannot update in real time. Solving this problem requires addressing the anchoring method for virtual objects and a spatial calculation system for the MR scene. As user experience demands increase, virtual objects need to be displayed at any location and can be dragged and scaled freely. If the coordinate changes of virtual objects within the entire MR scene space cannot be accurately calculated, it will be difficult to synchronously update the changes to the glasses of other users in the same scene, thus preventing multi-user interaction with the same virtual object within the same scene and space.

[0004] 2. Current MR products concentrate spatial computing, image rendering, and other functional modules into head-mounted MR glasses. This increases the weight of the MR glasses, severely impacting the user's long-term wearing experience and easily causing fatigue and neck pain. Secondly, with all modules concentrated in the MR glasses, space constraints limit the size of related chips and other components, resulting in insufficient overall performance of the MR glasses and failing to meet users' gaming and entertainment needs. Thirdly, the highly concentrated MR glasses make heat dissipation very difficult, causing users to feel hot and stuffy, especially noticeable in summer.

[0005] 3. Current MR glasses cannot capture and scan the user's full-body movements and postures when the user communicates with other users in the MR space, and the interactive experience needs to be improved. Summary of the Invention

[0006] The purpose of this invention is to provide an MR spatial interaction system to solve the above-mentioned problems existing in the prior art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, a real-time (MR) spatial interaction system is provided, comprising a spatial computing device and several user terminals. Each user terminal is worn by a user and includes MR glasses and a wearing device. The wearing device establishes communication connections with both the MR glasses and the spatial computing device. The spatial computing device is used to collect scene information and the location information of each user, and to construct an MR scene with the spatial computing device as the origin based on the scene information and the location information of each user. The spatial computing device is used to obtain virtual object information, first spatial location information, second spatial location information, and user head location information from the corresponding wearing device, and to determine the spatial coordinate information of the virtual object in the MR scene based on the first spatial location information, second spatial location information, and user head location information. It also determines the spatial coordinate information of the virtual object in the MR scene based on the virtual object information and the spatial coordinate information of the virtual object in the MR scene, and adds or updates virtual objects in the MR scene based on the virtual object information and the spatial coordinate information of the virtual object in the MR scene, thereby obtaining a real-time shared MR space. The spatial computing device is used to collect scene information, the location information of each user, and the corresponding first spatial location information and user head location information based on the real-time shared MR space. The real-time MR spatial field-of-view image of each user is determined based on the spatial location information and the corresponding user head position information, and the real-time MR spatial field-of-view image of each user is transmitted to the corresponding wearing device. The wearing device is used to obtain virtual object information and first spatial location information from the corresponding MR glasses, and detect the second spatial location information and angle information of the MR glasses relative to the wearing device. Based on the angle information, the corresponding user head position information is determined, and the virtual object information, first spatial location information, second spatial location information and user head position information are transmitted to the spatial computing device. The device also obtains the real-time MR spatial field-of-view image of the corresponding user from the spatial computing device and transmits the real-time MR spatial field-of-view image to the corresponding MR glasses. The MR glasses are used to determine the virtual object information when the user operates the virtual object, anchor the virtual object relative to the user's first spatial location information, transmit the virtual object information and first spatial location information to the corresponding wearing device, and display the real-time MR spatial field-of-view image transmitted by the wearing device.

[0009] In one possible design, the MR glasses include a left-eye display and a right-eye display, the distance between the left-eye display and the right-eye display of the MR glasses is adjustable, the real-time MR spatial field of view image includes a left-eye real-time MR spatial field of view image and a right-eye real-time MR spatial field of view image, the left-eye display is used to display the left-eye real-time MR spatial field of view image, and the right-eye display is used to display the right-eye real-time MR spatial field of view image.

[0010] In one possible design, the MR glasses, the wearing device, and the spatial computing device are all equipped with a camera and a radar sensor, the camera being used to capture scene information and the radar sensor being used to detect spatial position.

[0011] In one possible design, the camera and radar sensors on the wearable device and spatial computing device are used to detect the user's gestures and obtain gesture tracking information. The spatial computing device is used to determine the user's hand position information and gesture change information based on the gesture tracking information, and to update the real-time shared MR space based on the hand position information and gesture change information of each user.

[0012] In one possible design, the wearable device includes a power supply module and a first information transmission module. The power supply module is used to power the wearable device and the MR glasses. The first information transmission module is used to transmit virtual object information, first spatial location information, second spatial location information and user head location information to a spatial computing device, and to transmit real-time MR spatial field of view images to the corresponding MR glasses.

[0013] In one possible design, the wearable device includes a microphone and a speaker. The microphone is used to collect the first interactive voice of the corresponding user, and the speaker is used to play MR scene sounds or the second interactive voice of other users. The spatial computing device is used to transmit the first interactive voice of the corresponding user to the user terminal of other users, and to transmit the second interactive voice of other users to the user terminal of the corresponding user, or to generate MR scene sounds based on real-time MR spatial field-of-view images and transmit the MR scene sounds to the user terminal of the corresponding user.

[0014] In one possible design, the wearable device further includes a vibration module. The spatial computing device is used to generate vibration control information when the user operates virtual objects or based on real-time MR spatial field-of-view images, and transmit the vibration control information to the wearable device of the corresponding user. The vibration module is used to operate according to the vibration control information and issue vibration prompt information.

[0015] In one possible design, the spatial computing device includes an AI processing module and an audio processing module. The AI ​​processing module is used to perform speech recognition on the first interactive speech of the corresponding user and to perform human-computer interaction with the corresponding user based on the speech recognition result. The audio processing module is used to adjust the sound intensity and sound direction of the MR scene sound or the second interactive speech according to the location information of each user.

[0016] In one possible design, the real-time shared MR space contains avatar models of each user, and the AI ​​processing module is also used to edit and set the avatar models of each user.

[0017] In one possible design, the spatial computing device includes a spatial computing module, a system operation module, a second information transmission module, an image rendering module, a brain-computer interface module, a network expansion module, and a scene storage module. The spatial computing module is used to construct an MR scene based on scene information and the location information of each user; to add or update virtual objects in the MR scene based on virtual object information and the spatial coordinates of the virtual objects within the MR scene, thereby obtaining a real-time shared MR space; and to determine the real-time MR spatial field-of-view image of each user based on the real-time shared MR space, the location information of each user, corresponding second spatial location information, and the corresponding user's head position information. The system operation module is used to control the operation and interaction of each functional module of the spatial computing device. The second information transmission module is used to realize data interaction with each wearable device. The image rendering module is used to render the real-time MR spatial field-of-view image. The brain-computer interface module is used to realize brain-computer interaction between the spatial computing device and an external brain-computer terminal. The network expansion module is used to realize online interaction between the spatial computing device and an external network terminal. The scene storage module is used to archive data in the real-time shared MR space.

[0018] Beneficial effects:

[0019] 1. This invention enables multiple user terminals to communicate with the same spatial computing device in the same scene, accurately calculating the precise location, field of view, interactive actions and other related information of each user, and comprehensively calculating the spatial location result by integrating the location information of virtual objects, so as to achieve the effect of multiple people sharing the same MR spatial scene, and at the same time, the virtual objects in space and space are visible to each other and influence each other.

[0020] 2. This invention significantly reduces the weight of the user-end MR glasses by centrally deploying functional modules such as spatial computing on a spatial computing device, eliminating the need to consider the impact of heat dissipation on user comfort. Simultaneously, by placing the power supply module within the wearable device, the weight of the MR glasses is further reduced, improving user comfort. Since the spatial computing device is not limited by device size and weight, larger and more powerful electronic components and chips can be used, freeing it from the constraints of chip manufacturing processes.

[0021] 3. Because the spatial computing device is set in the MR scene and has radar sensors and cameras, the present invention can integrate the user's entire body into the real-time shared MR space during the user's use. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the interaction in the MR scene in Embodiment 1 of the present invention;

[0024] Figure 2 This is a schematic diagram of the system configuration in Embodiment 1 of the present invention;

[0025] Figure 3 This is a partial schematic diagram of the spatial computing device in Embodiment 2 of the present invention. Detailed Implementation

[0026] It should be noted that the descriptions of these embodiments are intended to aid in understanding the invention and do not constitute a limitation thereof. The specific structural and functional details disclosed herein are merely for describing exemplary embodiments of the invention. However, the invention may be embodied in many alternative forms and should not be construed as being limited to the embodiments described herein.

[0027] It should be understood that, unless otherwise explicitly specified and limited, the term "connection" should be interpreted broadly. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments according to the specific circumstances.

[0028] Specific details are provided in the following description to provide a complete understanding of the exemplary embodiments. However, those skilled in the art will understand that the exemplary embodiments can be implemented without these specific details. For example, the system may be shown in block diagrams to avoid obscuring the example with unnecessary details. In other embodiments, well-known processes, structures, and techniques may be shown without non-essential details to avoid obscuring the embodiments.

[0029] Example 1:

[0030] This embodiment provides an MR spatial interaction system, such as Figures 1 to 2As shown, the system includes a spatial computing device and several user terminals. Each user terminal is worn by a user and includes MR glasses and a wearing device. The wearing device establishes communication connections with both the MR glasses and the spatial computing device. The spatial computing device collects scene information and the location information of each user, and constructs an MR scene with the spatial computing device as the origin based on the scene information and the location information of each user. The spatial computing device obtains virtual object information, first spatial location information, second spatial location information, and user head location information from the corresponding wearing device, and determines the spatial coordinates of the virtual objects in the MR scene based on the first spatial location information, second spatial location information, and user head location information. It adds or updates virtual objects in the MR scene based on the virtual object information and the spatial coordinates of the virtual objects in the MR scene to obtain a real-time shared MR space. The spatial computing device is used to calculate the real-time shared MR space, the location information of each user, and the corresponding second spatial location information. The system determines the real-time MR spatial field-of-view image of each user based on the user's head position information and transmits the real-time MR spatial field-of-view image of each user to the corresponding wearing device. The wearing device is used to acquire virtual object information and first spatial position information from the corresponding MR glasses, detect the second spatial position information and angle information of the MR glasses relative to the wearing device, determine the corresponding user's head position information based on the angle information, and transmit the virtual object information, first spatial position information, second spatial position information and user's head position information to the spatial computing device. It also acquires the real-time MR spatial field-of-view image of the corresponding user from the spatial computing device and transmits the real-time MR spatial field-of-view image to the corresponding MR glasses. The MR glasses are used to determine the virtual object information when the user operates the virtual object, anchor the virtual object relative to the user's first spatial position information, transmit the virtual object information and first spatial position information to the corresponding wearing device, and display the real-time MR spatial field-of-view image transmitted by the wearing device.

[0031] In specific implementation, such as Figure 1As shown in the diagram, M represents the spatial computing device, T1 and T2 represent User 1 and User 2 respectively, G1 and G2 represent the MR glasses worn by User 1 and User 2 respectively, and D1 and D2 represent the wearing devices for User 1 and User 2 respectively. These wearing devices can be worn on the user's neck and shoulders. The MR glasses, wearing devices, and spatial computing device are all equipped with cameras and radar sensors. The cameras are used to capture scene information, and the radar sensors are used for spatial position detection. The spatial computing device is stationary in the real-world scene, scanning and identifying scene information and user position information in real time to construct an MR (Mixed Reality) scene digital space with the spatial computing device as the origin of the coordinate system. This MR scene includes each user. The wearing device worn on the user's neck and shoulders communicates with the spatial computing device, which determines the user's precise position and body orientation in real time. Simultaneously, the wearing device communicates with the user's MR glasses to determine the real-time position and angle information of the MR glasses relative to the wearing device on the user's neck and shoulders, thereby calculating the user's specific field of vision information in the MR space with the spatial computing device as the origin of the coordinate system. When a user interacts with a virtual object, the virtual object appears from a feature point in the MR space. The MR glasses then anchor the feature point where the virtual object appears, along with its position relative to the user, to obtain information about the virtual object and its first spatial position relative to the user. This information is then combined with the position and angle information of the MR glasses relative to the wearing device (i.e., the second spatial position information of the MR glasses relative to the wearing device) and the user's orientation information to calculate the specific position of the virtual object in the MR space with the spatial computing device as the origin. After obtaining the precise position information of the virtual object in the MR space, virtual objects can be added or updated in the MR scene based on the virtual object information and its spatial coordinates, resulting in a real-time shared MR space. Simultaneously, based on the positional relationships between other users and the spatial computing device, the precise positional relationship between the virtual object and other users can be calculated.

[0032] For example, when user 1 interacts with a virtual object, the virtual object appears at a certain location in the MR space. This location can be anchored using AI inference. User 1's MR glasses collect images and radar information within their field of vision. AI inference can then be used to find suitable feature points to release the virtual object, determining its precise location relative to user 1. Once the feature points are anchored, the virtual object generated based on them is anchored. When no user interacts with it, the virtual object remains stationary based on the MR space coordinate system constructed by the spatial computing device. When user 2 looks at the same virtual object, based on the anchored coordinates of the virtual object, user 2's position, field of vision, head position, and MR scene information, the accurate position and angle of the virtual object in user 2's glasses can be calculated. Similarly, this allows multiple users to see, influence, and interact with each other in a shared MR space, with synchronized real-time updates, all targeting the same anchored virtual object. Once a virtual object is anchored, when a user performs operations such as changing the position or scaling the virtual object, the coordinate changes of the virtual object in the real-time shared MR space are updated accordingly in real time with reference to the center point of the spatial computing device.

[0033] Furthermore, the MR glasses include a left-eye display and a right-eye display. The distance between the left-eye and right-eye displays is adjustable to accommodate different users' interpupillary distances. The real-time MR spatial vision image includes a left-eye real-time MR spatial vision image and a right-eye real-time MR spatial vision image. The left-eye display is used to display the left-eye real-time MR spatial vision image, and the right-eye display is used to display the right-eye real-time MR spatial vision image, thereby achieving binocular stereoscopic vision image display. The camera and radar sensors on the wearing device and spatial computing device are used to detect the user's gestures and obtain gesture tracking information. The spatial computing device determines the user's hand position information and gesture change information based on the gesture tracking information, and updates the real-time shared MR space based on the hand position information and gesture change information of each user. By using dual acquisition and cross-validation to identify the user's hand position information and gesture change information, it is more accurate than traditional methods.

[0034] Furthermore, the wearable device includes a power supply module and a first information transmission module. The power supply module supplies power to the wearable device and the MR glasses. The first information transmission module transmits virtual object information, first spatial location information, second spatial location information, and user head position information to a spatial computing device, and transmits real-time MR spatial field-of-view images to the corresponding MR glasses. The wearable device also includes a microphone and a speaker. The microphone collects the first interactive voice of the corresponding user, and the speaker plays MR scene sounds or the second interactive voice of other users. The spatial computing device transmits the first interactive voice of the corresponding user to the user terminal of other users, and transmits the second interactive voice of other users to the user terminal of the corresponding user, or generates MR scene sounds based on the real-time MR spatial field-of-view images and transmits the MR scene sounds to the user terminal of the corresponding user. The wearable device also includes a vibration module. The spatial computing device generates vibration control information when the user operates virtual objects or based on the real-time MR spatial field-of-view images, and transmits the vibration control information to the wearable device of the corresponding user. The vibration module operates according to the vibration control information and emits vibration prompts.

[0035] Furthermore, the spatial computing device includes an AI processing module and an audio processing module. The AI ​​processing module can be configured with a corresponding digital AI assistant to perform speech recognition on the first interactive speech of the corresponding user, and to conduct human-computer interaction with the corresponding user based on the speech recognition results. For example, the digital AI assistant understands the user's needs through voice dialogue, opens the corresponding virtual content or application window, and anchors the virtual content elements or application window to appropriate feature points in the virtual scene based on corresponding spatial computing information, spatial perception capabilities, and AI image recognition capabilities. The audio processing module is used to adjust the sound intensity and direction of the MR scene sound or the second interactive speech according to the location information of each user. Because the sound emitted by the same virtual object should have different intensities and directions heard by different users in different locations, the intensity and direction of the audio are spatially calculated based on the precise location information of the user and the precise location information of the virtual object, and then the corresponding audio is output to the speaker of the user's wearing device. The wearing device can be equipped with multiple speakers in multiple directions to present sound from different directions.

[0036] Furthermore, the real-time shared MR space contains image models of each user, and the AI ​​processing module is also used to edit and set the image models of each user. The computing power of the AI ​​processing module deployed in the spatial computing device is unlimited, and real-time AI face restoration can be used. When User 1 communicates with User 2, the image of User 2 captured by the camera on User 1's MR glasses is an image of a person wearing MR glasses. After the image captured by User 1 is transmitted to the spatial computing device, the AI ​​processing module of the spatial computing device recognizes the image. If a face is recognized, the face is restored, the MR glasses are removed, and appropriate user eyes are restored. The user's eye shape can be pre-trained in the AI ​​processing module and stored in the spatial computing device with a specific number. When the user uses the device, they can bind their pre-trained eye model number to their MR glasses, which can avoid restoration errors in multi-user scenarios. Of course, users can also specify other images to replace their heads. The image is calculated based on MR scene information, user position information, user head position information, and user field of vision information to determine the splicing position and angle information of the image and the person's body. Based on real-time scanning of user body movements and facial expressions by spatial computing devices, the AI ​​processing module enables the user's avatar model to perform appropriate actions and expressions. Simultaneously, cameras deployed on the spatial computing devices can capture real-time full-body information of the user, allowing for full-body communication between users.

[0037] Furthermore, the spatial computing device also includes a spatial computing module, a system operation module, a second information transmission module, an image rendering module, a brain-computer interface module, a network expansion module, and a scene storage module. The spatial computing module is used to construct an MR scene based on scene information and the location information of each user; to add or update virtual objects in the MR scene based on virtual object information and the spatial coordinates of virtual objects within the MR scene, thereby obtaining a real-time shared MR space; and to determine the real-time MR spatial field-of-view image of each user based on the real-time shared MR space, the location information of each user, the corresponding second spatial location information, and the corresponding user's head position information. The system operation module is used to control the operation and interaction of each functional module of the spatial computing device, as well as the presentation of the user interface (UI). The second information transmission module is used to realize data interaction with each wearable device. The image rendering module is used to render the real-time MR spatial field-of-view image. The brain-computer interface module is used to realize brain-computer interaction between the spatial computing device and an external brain-computer terminal, enabling people with disabilities to interact with the system's MR space and virtual objects through brain-computer interface signals. The network expansion module is used to realize online interaction between the spatial computing device and an external network terminal, meeting the data exchange needs between the spatial computing device and external network devices. The scene storage module is used to archive data in the real-time shared MR space so that it can be directly accessed later.

[0038] Example 2:

[0039] This embodiment provides a spatial computing device, such as... Figure 3 As shown, at the hardware level, it includes:

[0040] The data interface is used to establish data communication between the processor and the corresponding external data terminal;

[0041] Memory, used to store instructions;

[0042] The processor is used to read the instructions stored in the memory and implement the operation process of the space computing device in Embodiment 1 according to the instructions.

[0043] Optionally, the system also includes an internal bus. The processor, memory, and data interface can be interconnected via the internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0044] The memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or first-in-last-out (FILO) memory. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0045] Example 3:

[0046] This embodiment provides a computer-readable storage medium storing instructions. When these instructions are executed on a computer, the computer causes the computer to perform the operation of the spatial computing device in Embodiment 1. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems.

[0047] This embodiment also provides a computer program product containing instructions, which, when executed on a computer, cause the computer to perform the operation process of the space computing device in Embodiment 1 according to the instructions. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system.

[0048] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An MR spatial interaction system, characterized in that, The system includes a spatial computing device and several user terminals. Each user terminal is worn by a user and includes MR glasses and a wearing device. The wearing device establishes communication connections with both the MR glasses and the spatial computing device. The spatial computing device collects scene information and the location information of each user, and constructs an MR scene with the spatial computing device as the origin based on the scene information and the location information of each user. The spatial computing device obtains virtual object information, first spatial location information, second spatial location information, and user head location information from the corresponding wearing device, and determines the spatial coordinate information of the virtual object in the MR scene based on the first spatial location information, second spatial location information, and user head location information. It then adds or updates virtual objects in the MR scene based on the virtual object information and the spatial coordinate information of the virtual objects in the MR scene. The system provides a real-time shared MR space. The spatial computing device determines the real-time MR spatial field-of-view image for each user based on the real-time shared MR space, the user's location information, corresponding second spatial location information, and the user's head location information. It then transmits the real-time MR spatial field-of-view image for each user to the corresponding wearing device. The wearing device acquires virtual object information and first spatial location information from the corresponding MR glasses, detects the second spatial location information and angle information of the MR glasses relative to the wearing device, determines the corresponding user's head location information based on the angle information, and transmits the virtual object information, first spatial location information, second spatial location information, and user's head location information to the spatial computing device. It also acquires the real-time MR spatial field-of-view image for the corresponding user from the spatial computing device and transmits the real-time MR spatial field-of-view image to the corresponding MR glasses. The MR glasses are used to determine the information of the virtual object when the user operates the virtual object, anchor the first spatial position information of the virtual object relative to the user, transmit the virtual object information and the first spatial position information to the corresponding wearing device, and display the real-time MR spatial field image transmitted by the wearing device.

2. The MR spatial interaction system according to claim 1, characterized in that, The MR glasses include a left-eye display and a right-eye display. The distance between the left-eye display and the right-eye display is adjustable. The real-time MR spatial vision image includes a left-eye real-time MR spatial vision image and a right-eye real-time MR spatial vision image. The left-eye display is used to display the left-eye real-time MR spatial vision image, and the right-eye display is used to display the right-eye real-time MR spatial vision image.

3. The MR spatial interaction system according to claim 2, characterized in that, The MR glasses, wearing device, and spatial computing device are all equipped with cameras and radar sensors. The cameras are used to capture scene information, and the radar sensors are used to detect spatial positions.

4. The MR spatial interaction system according to claim 1, characterized in that, The camera and radar sensors on the wearable device and spatial computing device are used to detect the user's gestures and obtain gesture tracking information. The spatial computing device is used to determine the user's hand position information and gesture change information based on the gesture tracking information, and to update the real-time shared MR space based on the hand position information and gesture change information of each user.

5. The MR spatial interaction system according to claim 4, characterized in that, The wearable device includes a power supply module and a first information transmission module. The power supply module is used to power the wearable device and the MR glasses. The first information transmission module is used to transmit virtual object information, first spatial location information, second spatial location information and user head location information to the spatial computing device, and to transmit real-time MR spatial field of view images to the corresponding MR glasses.

6. The MR spatial interaction system according to claim 1, characterized in that, The wearable device includes a microphone and a speaker. The microphone is used to collect the first interactive voice of the corresponding user, and the speaker is used to play MR scene sounds or the second interactive voice of other users. The spatial computing device is used to transmit the first interactive voice of the corresponding user to the user terminal of other users, and to transmit the second interactive voice of other users to the user terminal of the corresponding user, or to generate MR scene sounds based on real-time MR spatial field of view images and transmit the MR scene sounds to the user terminal of the corresponding user.

7. The MR spatial interaction system according to claim 6, characterized in that, The wearable device also includes a vibration module. The spatial computing device is used to generate vibration control information when the user operates virtual objects or based on real-time MR spatial field-of-view images, and transmit the vibration control information to the wearable device of the corresponding user. The vibration module is used to work according to the vibration control information and issue vibration prompt information.

8. The MR spatial interaction system according to claim 6, characterized in that, The spatial computing device includes an AI processing module and an audio processing module. The AI ​​processing module is used to perform speech recognition on the first interactive speech of the corresponding user and to perform human-computer interaction with the corresponding user based on the speech recognition result. The audio processing module is used to adjust the sound intensity and sound direction of the MR scene sound or the second interactive speech according to the location information of each user.

9. The MR spatial interaction system according to claim 8, characterized in that, The real-time shared MR space contains avatar models of each user, and the AI ​​processing module is also used to edit and set the avatar models of each user.

10. The MR spatial interaction system according to claim 1, characterized in that, The spatial computing device includes a spatial computing module, a system operation module, a second information transmission module, an image rendering module, a brain-computer interface module, a network expansion module, and a scene storage module. The spatial computing module is used to construct an MR scene based on scene information and the location information of each user, add or update virtual objects in the MR scene based on virtual object information and the spatial coordinate information of virtual objects in the MR scene to obtain a real-time shared MR space, and determine the real-time MR spatial field of view image of each user based on the real-time shared MR space, the location information of each user, the corresponding second spatial location information, and the corresponding user head position information. The system operation module is used to control the operation and interaction of various functional modules of the space computing device; The second information transmission module is used to realize data interaction with each wearable device; the image rendering module is used to render images of real-time MR spatial field of view; the brain-computer interface module is used to realize brain-computer interaction between the spatial computing device and the external brain-computer terminal; the network expansion module is used to realize online interaction between the spatial computing device and the external network terminal; and the scene storage module is used to archive data of the real-time shared MR space.

Images