Method, system and storage medium for realizing multi-person interaction in virtual scene based on WebRTC

By receiving user access requests in a virtual scene, pushing RTC video streams and generating role models, providing interactive button UI, recording and synchronizing interactive operations, the problem of single interaction mode in the existing technology is solved, and the convenience of multi-person interaction is achieved Sex and efficiency.

CN118741257BActive Publication Date: 2024-11-08VSWORK NETWORK TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411229071.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-08
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

The existing virtual scenes combine WebRTC technology with single interaction methods, which makes the user experience inconvenient and cannot achieve diversified interactive operations.

Method used

By receiving user access requests, pushing RTC video streams and generating role models, providing interactive button UI, recording user interaction operations and syncing to the scene server, executing interactive data with administrator permissions, realizing multi-person interaction.

Benefits of technology

The diversity and efficiency of interaction are increased, and users can dynamically manipulate through the interactive button UI, improving the interactive experience in virtual scenes and avoiding the problem of interaction difficulties in traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118741257B_ABST
    Figure CN118741257B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, system and storage medium for realizing multi-person interaction in a virtual scene based on WebRTC, which belongs to the field of virtual and reality technology. The method includes: receiving a user's access request, pushing an RTC video stream to the user; detecting that the user accesses the RTC video stream, and generating a corresponding role model in a virtual scene; obtaining the user's display interface authority, generating an interactive button UI on the display interface, obtaining the size of the current user's display interface to map out a template interface; recording the user's interactive operation, restoring the interactive operation on the template interface to generate interactive data, and synchronizing the interactive data to a scene server, which is used for the scene server to control an administrator to execute the interactive data. The present invention generates a role model in a virtual scene for a user who accesses the RTC video stream, and synchronizes the interactive data to the scene server by obtaining the interactive data, and executes it using the administrator's authority to realize a multi-person interactive mode.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of virtual and reality technologies, and in particular to a method, system and storage medium for realizing multi-person interaction in a virtual scene based on WebRTC. Background Art

[0002] WebRTC is an open source, cross-platform, real-time communication technology that allows web applications or sites to establish peer-to-peer connections between browsers without the aid of an intermediary, and to transmit video streams, audio streams, or any other data. With the popularization of WebRTC technology, its application areas are also increasing. In virtual reality technology, the transmission of WebRTC video streams can replace the traditional client-side way of running virtual scenes. You only need to select a user as an administrator to load the virtual scene using the client or cloud rendering. Other users can use WebRTC to transmit the administrator's field of view, and they can fully view the content of the virtual scene without loading the virtual scene. Compared with the traditional way of running virtual scenes, this way of running virtual scenes is more efficient and saves resources.

[0003] Although the current virtual scenes are combined with WebRTC technology, from the perspective of user experience, it can only be used to "view" the scene content. If the user wants to experience some functions or interact with other people, he can only enter through voice or load the virtual scene himself. As a result, the virtual scenes using WebRTC technology in the existing technology have the defects of single interaction methods and functions and inconvenience in use. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the above-mentioned WebRTC combined with virtual and reality technologies has a single interaction mode and is not convenient to use, and proposes a method, system and storage medium for realizing multi-person interaction in virtual scenes based on WebRTC, which has the advantages of increasing the diversity of interaction, improving the efficiency of multi-person interaction and facilitating user use.

[0005] In a first aspect, the present invention achieves the above-mentioned purpose through the following technical solutions, a method for realizing multi-person interaction in a virtual scene based on WebRTC, the method comprising the following steps:

[0006] Receive an access request from a user, and push an RTC video stream to the user, where the RTC video stream is a visual image of the administrator in the virtual scene;

[0007] Detecting that the user accesses the RTC video stream, and sending an access message to the scene server, where the access message is used by the scene server to generate a corresponding role model in the virtual scene;

[0008] Obtain the display interface permission of the user, generate an interactive button UI on the display interface, the interactive button UI is used to provide the user with interactive operations, obtain the size of the current display interface of the user and map it out a template interface, the size of the template interface is consistent with the size of the administrator's display interface;

[0009] The interactive operation of the user is recorded, the interactive operation is restored on the template interface to generate interactive data, and the interactive data is synchronized to the scene server so that the scene server controls the administrator to execute the interactive data.

[0010] Preferably, the method for pushing the RTC video stream to the user is: generating an RTC invitation link, sending the RTC invitation link to all users through a communication program, and the RTC invitation link is used for the user to jump to the RTC playback page after clicking the link.

[0011] Preferably, the method also includes setting a custom role model, providing a role selection interface before the user accesses the RTC video stream, and after the user selects the role model, synchronizing the encoding of the selected role model to the scene server, and the scene server generates the corresponding role model according to the encoding.

[0012] Preferably, the interactive button UI generated on the display interface includes a field of view control button, a character movement button and a scene function button.

[0013] Preferably, the size of the display interface includes the aspect ratio of the display interface and the length and width of the display screen.

[0014] Preferably, the method for restoring the interactive operation to generate interactive data on the template interface is:

[0015] The center point of the display interface of the current user is set as the origin to establish a rectangular coordinate system Y1, where the X axis of the rectangular coordinate system Y1 is the length of the display interface and the Y axis is the width of the display interface;

[0016] Obtain the moving track coordinates P1 of the pointer when the user performs interactive operations on the display interface;

[0017] Obtain a rectangular coordinate system Y2 established with the center point of the template interface as the origin, wherein the X axis of the rectangular coordinate system Y2 is the length of the template interface, and the Y axis is the width of the template interface;

[0018] The moving track coordinates P1 are converted into the rectangular coordinate system Y2 to obtain new moving track coordinates P2 as interaction data.

[0019] In a second aspect, the present invention achieves the above-mentioned purpose through the following technical solutions, a system for realizing multi-person interaction in a virtual scene based on WebRTC, the system comprising:

[0020] An RTC push unit is used to receive an access request from a user and push an RTC video stream to the user, wherein the RTC video stream is a visual image of the administrator in the virtual scene;

[0021] A role generation unit, configured to detect that the user accesses the RTC video stream, and send an access message to a scene server, wherein the access message is used by the scene server to generate a corresponding role model in a virtual scene;

[0022] An interface setting unit, used to obtain the display interface authority of the user, generate an interactive button UI on the display interface, the interactive button UI is used to provide the user with interactive operations, obtain the size of the current display interface of the user and map it out a template interface, the size of the template interface is consistent with the size of the display interface of the administrator;

[0023] An interactive synchronization unit is used to record the interactive operation of the user, restore the interactive operation on the template interface to generate interactive data, and synchronize the interactive data to the scene server so that the scene server controls the administrator to execute the interactive data.

[0024] Preferably, the system also includes a role setting unit for setting a custom role model, providing a role selection interface before the user accesses the RTC video stream, and after the user selects the role model, synchronizing the encoding of the selected role model to the scene server, and the scene server generates the corresponding role model according to the encoding.

[0025] Preferably, in the interactive synchronization unit, the method for restoring the interactive operation to generate interactive data on the template interface is:

[0026] The center point of the display interface of the current user is set as the origin to establish a rectangular coordinate system Y1, where the X axis of the rectangular coordinate system Y1 is the length of the display interface and the Y axis is the width of the display interface;

[0027] Obtain the moving track coordinates P1 of the pointer when the user performs interactive operations on the display interface;

[0028] Obtain a rectangular coordinate system Y2 established with the center point of the template interface as the origin, wherein the X axis of the rectangular coordinate system Y2 is the length of the template interface, and the Y axis is the width of the template interface;

[0029] The moving track coordinates P1 are converted into the rectangular coordinate system Y2 to obtain new moving track coordinates P2 as interaction data.

[0030] In a third aspect, the present invention achieves the above-mentioned purpose through the following technical scheme: a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for realizing multi-person interaction in a virtual scene based on WebRTC as described in the first aspect.

[0031] Compared with the prior art, the beneficial effect of the present invention is as follows: the present invention uses a scene server to generate a role model corresponding to the user while the user accesses the RTC video stream. In this way, in the RTC video stream, the user can view his or her own role model with the help of the administrator's field of view, and an interaction button UI is also generated on the user's display interface. The interaction data is recorded through the interaction button UI and synchronized to the scene server, and the administrator's authority is used to execute these interaction data, thereby achieving a method of realizing multi-person interaction using only the RTC video stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of a method for realizing multi-person interaction in a virtual scene based on WebRTC of the present invention.

[0033] Figure 2 This is a schematic diagram of a system for realizing multi-person interaction in a virtual scene based on WebRTC according to the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] like Figure 1 As shown, a method for realizing multi-person interaction in a virtual scene based on WebRTC, the method comprising the following steps:

[0037] Step S1, receiving the user's access request, and pushing an RTC video stream to the user. The RTC video stream is the administrator's field of view in the virtual scene. The administrator's real-time field of view in the virtual scene is generated into an RTC video stream, which can be pushed to all accessing users through WebRTC real-time communication technology. The administrator here can load the virtual scene through the client or cloud rendering, and use technical means to obtain the camera image to obtain the administrator's field of view in real time. The access request here can be in the form of clicking a link. An access link is generated while using WebRTC technology, and whether to send the RTC video stream to the user is determined by detecting whether the user clicks the access link.

[0038] Step S2, detect that the user has accessed the RTC video stream, and send an access message to the scene server, wherein the access message is used by the scene server to generate a corresponding role model in the virtual scene. When the user accesses the RTC video stream, the video content will be played through the web page, so that all users can view the content of the virtual scene with the help of the administrator's field of view. In order to increase the user's sense of participation, when it is determined that the user is playing the RTC video stream, the scene server is notified to load the role model, so that the user can not only see the scene picture during playback, but also see the role model corresponding to himself enter the scene. By using the display of the role model, more functional interactions can be achieved, such as sending emoticons, text conversations, etc., which can be displayed by adding UI to the role model, which is more participatory than the traditional form that cannot be seen between users.

[0039] Step S3, obtaining the display interface authority of the user, generating an interactive button UI on the display interface, the interactive button UI is used to provide the user with interactive operations, obtaining the size of the current display interface of the user to map out a template interface, the size of the template interface is consistent with the size of the display interface of the administrator, the role model described in the above step S2 can only perform some static display effects, although this display form has a good effect, but the function is relatively simple, so by generating an interactive button UI on the user's display interface, the user can perform some dynamic manipulations through the interactive button UI, such as rotating the viewing angle, moving the screen, and other functions of the interactive button UI, because the user's display interface is only used as the playback of the RTC video stream, it is impossible to directly control the role model in the virtual scene for dynamic manipulation, so it is also necessary to use the administrator's role for dynamic manipulation, and how to use the administrator's role for dynamic manipulation will be described in detail in step S4, and no further details will be given here, because the user's display interface and the management interface are not necessarily consistent, and the need to use the administrator's role for dynamic manipulation must be to obtain the user's interactive data, and two interfaces with inconsistent sizes will cause errors in the acquisition of interactive data, so here a template interface is added as a conversion, so that the interactive data obtained on the display interface can be used on the administrator's interface after conversion.

[0040] Step S4, recording the user's interactive operation, restoring the interactive operation on the template interface to generate interactive data, and synchronizing the interactive data to the scene server, so that the scene server controls the administrator to execute the interactive data. Some interactive operations of the user, such as clicking the position of the UI, or moving the perspective, and the distance of sliding the display interface when walking, are recorded as interactive operations. By synchronizing the recorded data to the scene server for operation, the scene server only needs to restore the touch position or touch distance to control the administrator's role to restore the action the user wants to operate, thereby achieving step S3, with the help of the administrator's role to restore the user's desired operation. In a specific implementation, for example, the administrator leads all users to tour the virtual scene, but one of the users wants to trace back the area just passed. At this time, the description is not easy to understand only through voice communication, resulting in low interaction efficiency. At this time, the administrator's role is directly controlled to walk to the passed position. In this way, it is more convenient to use the method of self-operation, and effectively solves the problem of interaction difficulties when using WebRTC technology in virtual scenes. It is worth noting that since each user needs to restore the interactive operation with the help of the administrator's role, the management needs to control the user's interaction authority to ensure that only a single user can use each interaction to avoid conflicts.

[0041] In order to ensure that users can smoothly access the RTC video stream, the method for pushing the RTC video stream to the users is: generating an RTC invitation link, sending the RTC invitation link to all users through a communication program, the RTC invitation link is used for users to jump to the RTC playback page after clicking the link, and pushing the RTC invitation link through a communication program makes the transmission more efficient, and ensures that users can enter the virtual scene anytime and anywhere with the help of the administrator's field of view, without occupying too many resources of their own mobile phones or computer devices.

[0042] Since the user's own role model is mainly used to display some static interactions, the method also includes setting a custom role model, providing a role selection interface before the user accesses the RTC video stream, and after the user selects the role model, the encoding of the selected role model is synchronized to the scene server, and the scene server generates the corresponding role model according to the encoding. The custom model can increase the user's interest in use.

[0043] The interactive button UI generated on the display interface includes a field of view control button, a character movement button and a scene function button. The field of view control button can be set with a directional joystick, and the field of view can be rotated up, down, left and right by the directional joystick, while the character movement button can be a walking button, and the virtual character can be controlled to move forward and backward by the four directional keys of front, back, left and right. Other scene function buttons include expression sending, voice input, brush and other functions. This interactive button UI is widely used in existing products, so each function will not be described one by one.

[0044] The size of the display interface includes the aspect ratio of the display interface and the length and width of the display screen. The specific size of the display interface can be obtained through the length and width data and the aspect ratio, which also provides basic data for the subsequent conversion of the template interface. The method of restoring the interactive operation to generate interactive data on the template interface is:

[0045] The center point of the display interface of the current user is set as the origin to establish a rectangular coordinate system Y1, where the X axis of the rectangular coordinate system Y1 is the length of the display interface and the Y axis is the width of the display interface;

[0046] Obtain the moving track coordinates P1 of the pointer when the user performs interactive operations on the display interface;

[0047] Obtain a rectangular coordinate system Y2 established with the center point of the template interface as the origin, wherein the X axis of the rectangular coordinate system Y2 is the length of the template interface, and the Y axis is the width of the template interface;

[0048] The moving track coordinates P1 are converted into the rectangular coordinate system Y2 to obtain new moving track coordinates P2 as interaction data.

[0049] Since the template interface is the same size as the administrator's display interface, the template interface needs to be re-determined for different administrators. This is to avoid errors when the interactive data is synchronously restored. The coordinate system is created using the center point. No matter how the size of the display interface changes, it can be restored by simply scaling the coordinate data proportionally. In actual operation, if the interaction is just a simple click, only the single coordinate of the point needs to be restored. If there is sliding, the coordinate trajectories of different points need to be calculated.

[0050] Example 2

[0051] like Figure 2 As shown, a system for realizing multi-person interaction in a virtual scene based on WebRTC, the system comprising:

[0052] An RTC push unit is used to receive an access request from a user and push an RTC video stream to the user, wherein the RTC video stream is a visual image of the administrator in the virtual scene;

[0053] A role generation unit, configured to detect that the user accesses the RTC video stream, and send an access message to a scene server, wherein the access message is used by the scene server to generate a corresponding role model in a virtual scene;

[0054] An interface setting unit, used to obtain the display interface authority of the user, generate an interactive button UI on the display interface, the interactive button UI is used to provide the user with interactive operations, obtain the size of the current display interface of the user and map it out a template interface, the size of the template interface is consistent with the size of the display interface of the administrator;

[0055] An interactive synchronization unit is used to record the interactive operation of the user, restore the interactive operation on the template interface to generate interactive data, and synchronize the interactive data to the scene server so that the scene server controls the administrator to execute the interactive data.

[0056] The system also includes a role setting unit for setting a custom role model and providing a role selection interface before the user accesses the RTC video stream. After the user selects the role model, the encoding of the selected role model is synchronized to the scene server, and the scene server generates the corresponding role model according to the encoding.

[0057] The method for pushing the RTC video stream to the user is: generating an RTC invitation link, sending the RTC invitation link to all users through a communication program, and the RTC invitation link is used for the user to jump to the RTC playback page after clicking the link.

[0058] The interactive button UI generated on the display interface includes a field of view control button, a character movement button, and a scene function button. In the interactive synchronization unit, the method for restoring the interactive operation to generate interactive data on the template interface is:

[0059] The center point of the display interface of the current user is set as the origin to establish a rectangular coordinate system Y1, where the X axis of the rectangular coordinate system Y1 is the length of the display interface and the Y axis is the width of the display interface;

[0060] Obtain the moving track coordinates P1 of the pointer when the user performs interactive operations on the display interface;

[0061] Obtain a rectangular coordinate system Y2 established with the center point of the template interface as the origin, wherein the X axis of the rectangular coordinate system Y2 is the length of the template interface, and the Y axis is the width of the template interface;

[0062] The moving track coordinates P1 are converted into the rectangular coordinate system Y2 to obtain new moving track coordinates P2 as interaction data.

[0063] Embodiment 2 is essentially the same as Embodiment 1, so the specific working principles of each unit will not be described in detail.

[0064] Example 3

[0065] This embodiment proposes a storage medium, which includes a program storage area and a data storage area, wherein the program storage area can store an operating system and programs required for running instant messaging functions, etc.; the data storage area can store various instant messaging information and operation instruction sets, etc. The computer program is stored in the program storage area, and when the computer program is executed by the processor, the method for realizing multi-person interaction in a virtual scene based on WebRTC as described in Example 1 is implemented. The processor may include one or more central processing units, or a digital processing unit, etc.

[0066] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

[0067] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for realizing multi-person interaction in a virtual scene based on WebRTC, characterized in that: The method comprises the following steps: Receive an access request from a user, and push an RTC video stream to the user, where the RTC video stream is a visual image of the administrator in the virtual scene; Detecting that the user accesses the RTC video stream, and sending an access message to the scene server, where the access message is used by the scene server to generate a corresponding role model in the virtual scene; Obtain the display interface permission of the user, generate an interactive button UI on the display interface, the interactive button UI is used to provide the user with interactive operations, obtain the size of the current display interface of the user and map it out a template interface, the size of the template interface is consistent with the size of the administrator's display interface; The interactive operation of the user is recorded, the interactive operation is restored on the template interface to generate interactive data, and the interactive data is synchronized to the scene server so that the scene server controls the administrator to execute the interactive data.

2. The method for realizing multi-person interaction in a virtual scene based on WebRTC according to claim 1, characterized in that: The method for pushing the RTC video stream to the user is: generating an RTC invitation link, sending the RTC invitation link to all users through a communication program, and the RTC invitation link is used for the user to jump to the RTC playback page after clicking the link.

3. The method for realizing multi-person interaction in a virtual scene based on WebRTC according to claim 1, characterized in that: The method also includes setting a custom role model, providing a role selection interface before the user accesses the RTC video stream, and after the user selects the role model, synchronizing the encoding of the selected role model to the scene server, which generates the corresponding role model according to the encoding.

4. The method for realizing multi-person interaction in a virtual scene based on WebRTC according to claim 1, characterized in that: The interactive button UI generated on the display interface includes a field of view control button, a character movement button and a scene function button.

5. The method for realizing multi-person interaction in a virtual scene based on WebRTC according to claim 1, characterized in that: The size of the display interface includes the aspect ratio of the display interface and the length and width of the display screen.

6. The method for realizing multi-person interaction in a virtual scene based on WebRTC according to claim 1, characterized in that: The method for restoring the interactive operation to generate interactive data on the template interface is: The center point of the display interface of the current user is set as the origin to establish a rectangular coordinate system Y1, where the X axis of the rectangular coordinate system Y1 is the length of the display interface and the Y axis is the width of the display interface; Obtain the moving track coordinates P1 of the pointer when the user performs interactive operations on the display interface; Obtain a rectangular coordinate system Y2 established with the center point of the template interface as the origin, wherein the X axis of the rectangular coordinate system Y2 is the length of the template interface, and the Y axis is the width of the template interface; The moving track coordinates P1 are converted into the rectangular coordinate system Y2 to obtain new moving track coordinates P2 as interaction data.

7. A system for realizing multi-person interaction in a virtual scene based on WebRTC, characterized in that: The system includes: An RTC push unit is used to receive an access request from a user and push an RTC video stream to the user, wherein the RTC video stream is a visual image of the administrator in the virtual scene; A role generation unit, configured to detect that the user accesses the RTC video stream, and send an access message to a scene server, wherein the access message is used by the scene server to generate a corresponding role model in a virtual scene; An interface setting unit, used to obtain the display interface authority of the user, generate an interactive button UI on the display interface, the interactive button UI is used to provide the user with interactive operations, obtain the size of the current display interface of the user and map it out a template interface, the size of the template interface is consistent with the size of the display interface of the administrator; An interactive synchronization unit is used to record the interactive operation of the user, restore the interactive operation on the template interface to generate interactive data, and synchronize the interactive data to the scene server so that the scene server controls the administrator to execute the interactive data.

8. The system for realizing multi-person interaction in virtual scenes based on WebRTC according to claim 7, characterized in that: The system also includes a role setting unit for setting a custom role model and providing a role selection interface before the user accesses the RTC video stream. After the user selects the role model, the encoding of the selected role model is synchronized to the scene server, and the scene server generates the corresponding role model according to the encoding.

9. The system for realizing multi-person interaction in virtual scenes based on WebRTC according to claim 7, characterized in that: In the interactive synchronization unit, the method for restoring the interactive operation to generate interactive data on the template interface is: The center point of the display interface of the current user is set as the origin to establish a rectangular coordinate system Y1, where the X axis of the rectangular coordinate system Y1 is the length of the display interface and the Y axis is the width of the display interface; Obtain the moving track coordinates P1 of the pointer when the user performs interactive operations on the display interface; Obtain a rectangular coordinate system Y2 established with the center point of the template interface as the origin, wherein the X axis of the rectangular coordinate system Y2 is the length of the template interface, and the Y axis is the width of the template interface; The moving track coordinates P1 are converted into the rectangular coordinate system Y2 to obtain new moving track coordinates P2 as interaction data.

10. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, a method for realizing multi-person interaction in a virtual scene based on WebRTC as described in any one of claims 1 to 6 is realized.