An AR-based remote virtual interaction method and system

Abstract

Embodiments of the present disclosure provide an AR-based remote virtual interaction method, comprising: obtaining an interaction position and interaction content between a first user and virtual content presented at a first device, the virtual content comprising a first rendering of a three-dimensional (3D) model defined within a virtual coordinate system, the interaction position being defined within the virtual coordinate system, the interaction content being obtained through a camera on an AR; generating AR content, the AR content comprising: a second rendering of the model based on a position of a second user within the virtual coordinate system, the position of the second user being based on information obtained by a second device, and a virtual representation of the interaction based on the interaction position; and outputting and presenting the AR content at the second device. Corresponding systems, electronic devices and computer-readable storage media are also provided. The AR-based remote virtual interaction method and system can break through the barrier of "seeing different things" between the two parties of remote interaction, and expand the application range of AR interaction.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision and augmented reality (AR) technology, and more particularly to an AR-based remote virtual interaction method and system. Background Technology

[0002] AR involves using computer-generated or virtual content to enhance real-world experiences. In some cases, AR involves overlaying virtual content onto real-world physical content. AR differs from Virtual Reality (VR). VR involves the creation of entirely virtual experiences, while AR retains at least some aspects of the real-world experience but alters the perception of that experience using virtual content.

[0003] AR-based remote interaction can significantly shorten the distance in time and space and improve interaction efficiency, and therefore has been widely used in the industrial field. A typical scenario involves on-site personnel wearing AR glasses and viewing the same target simultaneously with remote personnel. Both on-site and remote personnel can mark the screen they see with their hands or a mouse, and the mark is visible to both parties in real time, thereby realizing the intentional interaction between the on-site and remote parties.

[0004] However, the aforementioned interaction methods suffer from a serious drawback: in many interactive scenarios, the objects seen by the interacting parties are not the same. For example, two mechanical designers working from home need to discuss design solutions. Each is facing their own model and cannot simultaneously see the other's model. Even if they could, they cannot arbitrarily mark on the model and make it visible to the other in real time. Therefore, existing multi-party AR interaction methods for the same scenario cannot meet this type of "different views" interaction requirement, resulting in the inability of the two users to effectively exchange intentions and severely impacting collaborative efficiency. Summary of the Invention

[0005] This disclosure addresses the above-mentioned problems by providing an AR-based remote virtual interaction method and system, which can overcome the obstacle of "different views" between the two parties in remote interaction and expand the application scope of AR interaction.

[0006] According to a first aspect of this disclosure, an AR-based remote virtual interaction method is provided, comprising:

[0007] S1, obtaining the interaction position and interaction content between the first user and the virtual content presented on the first device. The virtual content includes a 3D AR model placed in a first virtual coordinate system defined on the first device. The interaction position is the relative position of the first user with respect to the 3D AR model in the first virtual coordinate system. The interaction content is the actions of the first user on the AR model and the posture adjustment actions of the AR model, obtained by the image acquisition device installed on the first device. In this embodiment, the image acquisition device is a camera. Of course, those skilled in the art can also choose other devices that can acquire images, such as 3D gloves, as needed, all of which are within the protection scope of this invention.

[0008] S2, based on the actions of the first user on the AR model, operate the 3D AR model and generate the operated 3D AR model;

[0009] S3, the manipulated 3D AR model is presented to the second user based on the interaction position between the second user and the manipulated 3D AR model and the posture adjustment action of the AR model.

[0010] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein presenting the manipulated 3D AR model to the second user based on the interaction position between the second user and the manipulated 3D AR model and the posture adjustment action of the AR model includes: outputting the manipulated 3D AR model to a second device and forming AR content from the manipulated 3D AR model based on the interaction position of the second user and the posture adjustment action of the AR model, and presenting it to the second user.

[0011] The interaction position of the second user is the relative position of the second user within the second virtual coordinate system relative to the 3D AR model after the operation.

[0012] In addition to the aspects described above and any possible implementation, a further implementation is provided in which the interaction position is an interaction position in a preset state, and the 3D AR model is displayed to the first user and the second user according to the interaction position in the preset state.

[0013] In addition to the aspects described above and any possible implementation, a further implementation is provided in which the user is two or more users, wherein the first user and the second user are defined relative to each other, any user can be used to receive and display the manipulated 3D AR model, and the first user and the second user can be a group of users or a collection of multiple users.

[0014] In addition to the aspects described above and any possible implementation, a further implementation is provided in which obtaining the interaction position between the first user and the virtual content presented at the first device includes: obtaining the first position of the first user in a first virtual coordinate system, wherein the interaction position is determined based on the first position of the first user in the first virtual coordinate system and the position of the three-dimensional AR model in the first virtual coordinate system.

[0015] In addition to the aspects described above and any possible implementation, a further implementation is provided in which the interaction position of the second user with the manipulated 3D AR model includes: obtaining a second position of the second user in a second virtual coordinate system, wherein the interaction position is determined based on the second position of the second user in the second virtual coordinate system and the position of the manipulated 3D AR model in the second virtual coordinate system.

[0016] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the 3D AR model is hand-drawn, a pre-defined special shape, or a cursor controlled by a first user using a first device. The 3D AR model appears in the field of vision of both the first user and the second user through the AR device, and also appears in the field of vision of the image acquisition device. The posture of the 3D AR model changes with the position movement and viewing angle changes of the first user and the second user.

[0017] In addition to the aspects described above and any possible implementations, a further implementation is provided in which the interaction position includes a relative position and an interaction posture.

[0018] In addition to the aspects described above and any possible implementations, a further implementation is provided, wherein the method further includes: presenting the manipulated 3D AR model to the first user based on the interaction position between the first user and the manipulated 3D AR model and the posture adjustment action of the AR model.

[0019] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein the virtual content presented at the first device is first AR content; and the virtual content includes second AR content; wherein the first AR content includes a three-dimensional AR model defined in a first virtual coordinate system or a first rendering of a three-dimensional AR model, and the first AR content is output for presentation at the first device;

[0020] The virtual content presented at the second device is the second AR content, which is generated based on the three-dimensional AR model or the second rendering of the three-dimensional AR model after being operated by the first user. It includes the three-dimensional AR model or the second rendering of the three-dimensional AR model defined in the second virtual coordinate system. The second AR content is output for presentation at the second device.

[0021] According to a second aspect of this disclosure, an AR-based remote virtual interaction device is provided for implementing a remote virtual interaction method, comprising:

[0022] The interactive information acquisition module (101) is used to obtain the interactive position, interactive posture and interactive content between the first user and the virtual content presented at the first device. The virtual content includes a three-dimensional AR model placed in a first virtual coordinate system defined at the first device. The interactive position and the interactive posture are the position and posture of the first user in the first virtual coordinate system. The interactive content is the actions of the first user to operate the AR model and the actions to adjust the posture of the AR model, which are obtained by the image camera installed at the first device.

[0023] AR content generation module (102) is used to operate the three-dimensional AR model according to the actions of the first user on the AR model and generate the operated three-dimensional AR model.

[0024] The virtual interaction module (103) is used to present the operated 3D AR model to the second user based on the interaction position between the second user and the operated 3D AR model and the posture adjustment action of the AR model.

[0025] According to a third aspect of this disclosure, an AR-based remote virtual interaction system is provided, comprising a client and a server, wherein the client includes an image acquisition device and a display device, and the client communicates wirelessly with the server.

[0026] The server includes:

[0027] A remote virtual interaction trigger instruction receiving module is used to receive service trigger instructions sent by the client; and

[0028] The model display module is used to display 3D AR models and the manipulated 3D AR models.

[0029] According to a fourth aspect of this disclosure, an electronic device is provided, including a processor and a memory, the memory storing a plurality of instructions, the processor being configured to read the instructions and execute the method as described in the first aspect.

[0030] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided that stores a plurality of instructions which can be read by a processor and executed as described in the first aspect.

[0031] The beneficial effects of this invention are:

[0032] Based on the AR-based remote virtual interaction method, system, electronic device, and computer-readable storage medium of the present invention, the following beneficial effects have been achieved:

[0033] (1) Generation of AR content for shared AR experiences for multiple users: Multiple users can be separated from each other, so that users cannot physically see each other or interact with each other, but the AR content used for shared AR experiences can be mapped to each user's environment. For example, shared AR experiences allow multiple users to view the same virtual content at the same time, such as the same mechanical model or presentation or battlefield situation.

[0034] (2) The AR content generated for users in a shared AR experience includes a virtual representation of another user. Each user may be able to see how (one or more) other users are positioned within the shared AR experience and / or how (one or more) other users view virtual content within the shared AR experience. This can provide richer content to improve communication between different users, thereby helping to promote collaboration and cooperation in shared AR experiences.

[0035] (3) AR content generated for users in a shared AR experience includes the process of virtual content generation, which can provide a more intuitive and immersive shared AR experience; simultaneously, AR content generated for users in a shared AR experience includes virtual representations of user interactions within the AR experience. These virtual representations of user interactions can convey a user's interaction to one or more other users in a shared AR experience. Examples of such interactions include users providing gestures or gestures with enhanced markers to virtual items in the AR experience, or even touching virtual items. Displaying virtual interactions in an AR experience can help enable non-verbal communication between different users.

[0036] (4) User modifications to shared AR experiences can help improve collaboration and cooperation among different users. Modifying a shared AR experience can include changing the viewpoint of AR content for at least one user. In one example, the viewpoint of one user in the AR experience is modified to match the viewpoint of another user in the AR experience, without either user having to physically move within their environment. In another example, virtual items in the shared AR experience are modified simultaneously for all users.

[0037] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0038] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0039] Figure 1 A flowchart of an AR-based remote virtual interaction method according to an embodiment of the present disclosure is shown;

[0040] Figure 2 An architecture diagram of an AR-based remote virtual interaction device according to an embodiment of the present disclosure is shown.

[0041] Figure 3 A schematic diagram of an electronic device structure according to an embodiment of the present disclosure is shown. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0043] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0044] Example 1

[0045] like Figure 1 As shown, an AR-based remote virtual interaction method includes:

[0046] S1, obtain the interaction position, interaction posture and interaction content between the first user and the virtual content presented on the first device. The virtual content includes a three-dimensional AR model placed in a first virtual coordinate system defined on the first device. The interaction position and the interaction posture are the position and posture of the first user in the first virtual coordinate system. The interaction content is the actions of the first user to operate the AR model and the actions to adjust the posture of the AR model, which are obtained by the image camera installed on the first device.

[0047] S2, based on the actions of the first user on the AR model, the 3D AR model is manipulated and integrated to generate AR content, the AR content including:

[0048] The 3D AR model is an manipulated 3D AR model formed in a second virtual coordinate system based on the virtual content, interaction position, and interaction content described in S1, and the position of the second user. The position of the second user is based on information obtained from the second device and defined by the second virtual coordinate system at the second device.

[0049] S3, the manipulated 3D AR model is presented to the second user based on the interaction position between the second user and the manipulated 3D AR model and the posture adjustment action of the AR model.

[0050] In addition to the aspects described above and any possible implementations, a further implementation is provided where there are two or more users. For simplicity, this embodiment only describes the scenario with two interactors. The method can be extended to multiple interactors without substantial difficulty. Assume the two interactors are located in two different geographical locations and are both wearing AR glasses equipped with cameras. The camera's field of view is calibrated to match the field of view seen by the AR glasses wearer. The AR glasses are interconnected via a network. Each interactor has an object of interest within their field of view. The two interactors need to discuss the specifics of their respective objects.

[0051] In addition to the aspects described above and any possible implementation, a further implementation is provided to obtain the position of a first user in a virtual coordinate system, wherein the position of the interaction is determined based on the position of the first user in the virtual coordinate system; or to obtain the position of a second user in a virtual coordinate system, wherein the position of the interaction is determined based on the position of the second user in the virtual coordinate system.

[0052] In addition to the aspects described above and any possible implementations, a further implementation is provided in which the interactive content also includes features of the first user, typically used when the first user needs to emphasize a part of an object of interest, or in any general operation as a way to enhance the viewpoint. The features of the first user are an AR model.

[0053] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the AR model is a hand-drawn image, a pre-defined special shape, or a cursor controlled by a first user using a first device. The AR model appears in the field of vision of both the first user and the second user through the AR device, and also appears in the field of vision of the camera. The posture of the AR model changes with the position movement and viewing angle changes of the first user and the second user.

[0054] In addition to the aspects described above and any possible implementations, a further implementation is provided in which the AR content presented at the second device includes the AR model.

[0055] In this embodiment, when one of the users needs to emphasize a part of an object of interest, they simply extend a finger. The finger and the object appear in the user's field of vision, as well as in the camera's field of view. The image captured by the camera is input to an image processing device in real time. This device analyzes the image, identifies the finger (if necessary, a marker can be attached to the finger to aid in identification), and tracks the finger's movement. The user changes their working state, and the finger "marks" the object of interest. This marker is actually an AR model "placed" by the user at the location of the object of interest in the visible screen. It can be a hand-drawn circle, a star, or a more complex AR model. The marker appears both in the user's field of vision through the AR glasses and in the camera's field of view. It's important to note that this marker is an AR model; therefore, as the user moves and their perspective changes, the marker's posture in the field of vision (including the camera's field of view) will change accordingly, rather than remaining fixed in one place. Images, including AR models, captured by the camera on the AR glasses of this party are transmitted to the other end via the network. The other party receives the images and displays them in an appropriate manner, allowing them to see the scene within their field of vision and the markings made on it in real time.

[0056] Similarly, the other party can also "mark" objects of interest in their own scene, and the same party can see the objects and their marks in the other party's scene in real time. This enables mutual viewing of AR marks in situations where "what is seen is not the same," thereby improving interaction efficiency.

[0057] When the interactive content also includes the characteristics of the first user, as described above and any possible implementation, a further implementation is provided, the method further including obtaining the orientation of the interaction in the virtual coordinate system including obtaining the orientation of the characteristics of the first user in the virtual coordinate system; and the position of the interaction in the virtual coordinate system is based on the orientation of the characteristics of the first user in the virtual coordinate system.

[0058] In addition to the aspects described above and any possible implementation, an implementation is further provided in which the AR content is first AR content; and the virtual content includes second AR content; wherein the first AR content includes a first rendering of a model defined in a virtual coordinate system, the first AR content being output for rendering at a first device; the second AR content is generated based on a rendering of a modified model defined in a virtual coordinate system and implemented by a second device that renders the AR model, the second AR content being output and rendered at the first device.

[0059] Example 2

[0060] like Figure 2 As shown, an AR-based remote virtual interaction device is provided, comprising:

[0061] The interactive information acquisition module 101 is used to obtain the interactive position, interactive posture and interactive content between the first user and the virtual content presented on the first device. The virtual content includes a three-dimensional AR model placed in a first virtual coordinate system defined on the first device. The interactive position and the interactive posture are the position and posture of the first user in the first virtual coordinate system. The interactive content is the actions of the first user on the AR model and the posture adjustment actions of the AR model obtained by the image camera installed on the first device.

[0062] AR content generation module 102 is used to manipulate the 3D AR model and generate AR content based on the actions of the first user on the AR model, wherein the AR content includes:

[0063] The 3D AR model is an manipulated 3D AR model formed in a second virtual coordinate system based on the virtual content, interaction position, and interaction content described in S1, as well as the position of the second user. The position of the second user is based on information obtained from the second device and defined by the second virtual coordinate system at the second device.

[0064] The virtual interaction module 103 is used to present the manipulated 3D AR model to the second user based on the posture adjustment action of the AR model according to the interaction position between the second user and the manipulated 3D AR model.

[0065] Example 3

[0066] A remote virtual interaction system based on AR, characterized in that it includes a client and a server, wherein the client includes an image acquisition device and a display device, and the client communicates wirelessly with the server;

[0067] The server includes:

[0068] A remote virtual interaction trigger instruction receiving module is used to receive service trigger instructions sent by the client; and

[0069] The model display module is used to display 3D AR models and the manipulated 3D AR models.

[0070] Two typical embodiments of this invention are described. First, in a home-based work environment, software and hardware developers in different locations can collaborate on design using this interactive method, experiencing a similar feeling to working together in person. Second, in a battlefield environment, commanders in different geographical locations can use this interactive method to convey their situation to others, marking key areas requiring attention and displaying relevant information through various markers. This achieves immersive collaborative command.

[0071] For hardware development scenarios under home office conditions, the following are examples of actual interaction scenarios.

[0072] Users A and B are located in different places and are wearing AR glasses. When using the method of this invention, A and B simultaneously face the same virtual machine model and practice machine assembly and disassembly operations.

[0073] Initially, A faces the front of the virtual machine model, while B faces the back. A then begins operating from the front of the virtual machine model, manually adjusting virtual buttons to unlock and remove a virtual part. Simultaneously, B operates from the back of the machine model, manually installing a virtual part and then manually adjusting virtual buttons to lock.

[0074] Then, A rotated the virtual model 180 degrees by hand, and saw the back of the virtual model, including the virtual parts that B had just installed. At the same time, B saw the virtual model rotating and finally saw the front of the virtual model. A also saw what happened after the parts were removed.

[0075] Then, A can offer their opinions on B's part installation. Where A believes the installation is inadequate, A can mark it by hand; this mark will be permanently stored on the virtual model. B can also mark any empty spaces left by A after removing parts to remind A.

[0076] Then, B manually rotated the virtual model another 180 degrees. A could then see again the position where he had removed the virtual part, as well as the markings left by B, and thus understood the points to note when reinstalling the part. At the same time, B also saw the markings left by A at the part's installation position, clarifying the problems with his installation.

[0077] like Figure 3As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301. The memory 302 stores a plurality of instructions, which can be loaded and executed by the processor to enable the processor to perform the method as described in Embodiment 1.

[0078] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0079] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0080] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0081] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0082] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0083] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0084] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0085] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A remote virtual interaction method based on AR, characterized in that, include: S1, obtain the interaction position and interaction content between the first user and the virtual content presented at the first device, wherein the virtual content includes a three-dimensional AR model placed in a first virtual coordinate system defined at the first device, the interaction position is the relative position of the first user relative to the three-dimensional AR model in the first virtual coordinate system, and the interaction content is the actions of the first user on the AR model and the posture adjustment actions of the AR model obtained by the first device. S2, based on the actions of the first user on the AR model, operate the 3D AR model and generate the operated 3D AR model; S3, The manipulated 3D AR model is presented to the second user based on the interaction position between the second user and the manipulated 3D AR model and the posture adjustment action of the AR model; Presenting the manipulated 3D AR model to the second user based on the interaction position between the second user and the manipulated 3D AR model, as well as the posture adjustment action of the AR model, includes: The manipulated 3D AR model is output to the second device, and the manipulated 3D AR model is used to form AR content based on the second user's interaction position and the posture adjustment actions of the AR model, which is then presented to the second user. The interaction position of the second user is the relative position of the second user within the second virtual coordinate system relative to the 3D AR model after the operation; The method further includes: presenting the manipulated 3D AR model to the first user based on the interaction position between the first user and the manipulated 3D AR model and the posture adjustment action of the AR model; The step of obtaining the interaction position between the first user and the virtual content presented at the first device includes: obtaining the first position of the first user in the first virtual coordinate system, wherein the interaction position is determined based on the first position of the first user in the first virtual coordinate system and the position of the three-dimensional AR model in the first virtual coordinate system; The interaction position between the second user and the 3D AR model after the operation includes: obtaining the second position of the second user in the second virtual coordinate system, wherein the interaction position is determined based on the second position of the second user in the second virtual coordinate system and the position of the 3D AR model after the operation in the second virtual coordinate system.

2. The AR-based remote virtual interaction method according to claim 1, characterized in that, The interaction position is a preset interaction position, and the 3D AR model is displayed to the first user and the second user respectively according to the preset interaction position.

3. The AR-based remote virtual interaction method according to claim 1, characterized in that, The 3D AR model is a hand-drawn, pre-defined special shape, or a cursor controlled by the first user using the first device. The 3D AR model will appear in the field of vision of the first user and the second user respectively through the AR device. The posture of the 3D AR model changes with the position movement and viewing angle change of the first user and the second user.

4. The AR-based remote virtual interaction method according to claim 1, characterized in that, The interaction position includes relative position and interaction posture.

5. The AR-based remote virtual interaction method according to claim 1, characterized in that, The virtual content presented at the first device is the first AR content; the first AR content includes a three-dimensional AR model defined in the first virtual coordinate system or a first rendering of the three-dimensional AR model, and the first AR content is output for presentation at the first device. The virtual content presented at the second device is the second AR content, which is generated based on the three-dimensional AR model or the second rendering of the three-dimensional AR model after being operated by the first user. It includes the three-dimensional AR model or the second rendering of the three-dimensional AR model defined in the second virtual coordinate system. The second AR content is output for presentation at the second device.

6. An AR-based remote virtual interaction device, used to implement the remote virtual interaction method according to any one of claims 1-5, characterized in that, include: The interactive information acquisition module (101) is used to obtain the interaction position and interaction content between the first user and the virtual content presented at the first device. The virtual content includes a three-dimensional AR model placed in a first virtual coordinate system defined at the first device. The interaction position is the relative position of the first user relative to the three-dimensional AR model in the first virtual coordinate system. The interaction content is the actions of the first user on the AR model and the posture adjustment actions of the AR model obtained by the first device. AR content generation module (102) operates the three-dimensional AR model according to the actions of the first user on the AR model and generates the operated three-dimensional AR model; The virtual interaction module (103) is used to present the operated 3D AR model to the second user based on the interaction position between the second user and the operated 3D AR model and the posture adjustment action of the AR model.

7. An AR-based remote virtual interaction system for implementing the remote virtual interaction method according to any one of claims 1-5, characterized in that, It includes a client and a server. The client includes an image acquisition device and a display device, and the client communicates wirelessly with the server. The server includes: A remote virtual interaction trigger instruction receiving module is used to receive service trigger instructions sent by the client; and The model display module is used to display 3D AR models and the manipulated 3D AR models.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing multiple instructions, and the processor for reading the instructions and executing the method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions, which can be read by a processor and executed as described in any one of claims 1-5.

Images