Method and system for providing navigation assistance in 3D (three-dimensional) virtual environments

Abstract

To provide a method and system for providing navigation assistance in 3D (three-dimensional) virtual environments.SOLUTION: The method includes multiple virtual characters corresponding to multiple users, where the multiple virtual characters communicate with each other in the 3D (three-dimensional) virtual environment, and multiple virtual cameras corresponding to the multiple virtual characters. Further, a given virtual camera is movable according to a movement of the corresponding virtual character. Further, at least one predefined navigation path is provided in the 3D (three-dimensional) virtual environment, the navigation path being for directing the multiple virtual characters to move towards at least one virtual area in the 3D (three-dimensional) virtual environment. Further, an indication of scene details in the at least one virtual area is provided. Further, multiple video streams are captured using the multiple virtual cameras, and the multiple video streams are sent to the multiple user devices and displayed.SELECTED DRAWING: Figure 1A

Description

Technical Field

[0001]

[0001] This disclosure generally relates to the field of virtual environments, and more particularly, to methods and systems for providing navigation assistance in a 3D (three-dimensional) virtual environment. Background

[0002]

[0002] Due to the rapid progress of innovative technologies, all aspects of life have been fundamentally changed. In recent years, with the development of 3D (three-dimensional) technology, for example, 3D (three-dimensional) virtual environments have become popular.

[0003]

[0003] 3D (three-dimensional) virtual environments provide a new way to communicate with many users at once in addition to virtual connections. Furthermore, 3D (three-dimensional) virtual environments can be used not only for educational purposes, organizational meetings, video conferencing, but also for traditional learning and distance learning. 3D (three-dimensional) virtual environments are similar to physical places such as conference rooms, classrooms, and museums. Also, 3D (three-dimensional) virtual environments are used for video game play or virtual events, for example, video conferencing that users use to communicate with each other regardless of the location of each user participating in the virtual event. Since conventional 3D (three-dimensional) virtual environments used for video games or virtual events are implemented on a large scale, users may easily overlook specific details that may be required for the provision of an overall experience within the 3D (three-dimensional) virtual environment. In some scenarios, although there may be cutscenes that can provide users with experiences defined by programmers, these cutscenes usually become obstacles or restrictions to the interactions that users may have. Cutscenes (or event scenes) in the case of video games can be defined as non-interactive sequences that interrupt the game play. Usually, cutscenes are used for displaying conversations between characters, setting the atmosphere, introducing new models and game play elements, generating emotional connections, etc. Therefore, there are technical problems regarding methods for improving the user experience in conventional 3D (three-dimensional) virtual environments.

[0004]

[0004] Therefore, in light of the above discussion, it is necessary to overcome the aforementioned shortcomings associated with conventional 3D (three-dimensional) virtual environments. Overview

[0005]

[0005] This disclosure provides a method and system for providing navigation assistance in a 3D virtual environment. This disclosure provides solutions to existing problems in conventional methods for improving the user experience in a 3D virtual environment. The object of this disclosure is to provide solutions that at least partially overcome the problems that arise in the prior art, and to provide an improved method and system for providing navigation assistance in a 3D virtual environment.

[0006]

[0006] One or more objects of this disclosure are achieved by solutions provided in the appended independent claims. Dependent claims further specify advantageous embodiments of this disclosure.

[0007]

[0007] In one embodiment, the present disclosure provides a method for providing navigation assistance in a 3D virtual environment. The method includes the steps of accessing a 3D virtual environment using a plurality of user devices associated with a plurality of users, wherein the 3D virtual environment comprises a plurality of virtual characters corresponding to the plurality of users, the plurality of virtual characters are positioned in different virtual locations, the different virtual locations change dynamically as the plurality of virtual characters navigate the 3D virtual environment, and the plurality of virtual characters communicate with each other in the 3D virtual environment. The 3D virtual environment further comprises a plurality of virtual cameras corresponding to the plurality of virtual characters, wherein a given virtual camera is positioned at a given distance from a corresponding virtual character and views the 3D virtual environment from a given orientation, and the given virtual camera is movable in accordance with the movement of the corresponding virtual character. The method further includes the step of providing a 3D virtual environment with at least one predefined navigation path for instructing multiple virtual characters to move toward at least one virtual area in the 3D virtual environment, wherein the at least one predefined navigation path is divided into multiple subpaths in the at least one virtual area. The method further includes the step of providing a display of scene details in at least one virtual area when at least one of the multiple virtual characters is located in or near the at least one virtual area. The method further includes the step of capturing multiple video streams using multiple virtual cameras, wherein a given video stream is captured from the viewpoint of a given virtual camera associated with a given virtual character, and the given video stream represents at least a region of the 3D virtual environment in which a given virtual character resides; and the step of sending the multiple video streams to multiple user devices for display.

[0008]

[0008] The method of disclosure enhances the user experience in a 3D virtual environment. The method provides improved navigation assistance for multiple users to access the 3D virtual environment so that each user does not miss specific details provided in the 3D virtual environment. The predefined navigation paths disclosed in this method point to at least one virtual area of ​​the 3D virtual environment. Furthermore, the predefined navigation paths are divided into multiple subpaths to assist multiple users exploring at least one virtual area from different directions. In addition, the configuration of different geometries with different virtual positions of multiple virtual characters provides interactive, cinematic experiences and cutscenes that are not typically available on video game or 3D video conferencing platforms.

[0009]

[0009] It is understood that it is also possible to combine all of the above embodiments.

[0010]

[0010] It should be noted that all devices, elements, circuits, units, and means described herein may also be implemented as software or hardware elements, or any combination thereof. In addition to all steps performed by various entities described herein, functions described as being performed by various entities are intended to mean that each entity is adapted or configured to perform each step and function. In the following description of specific embodiments, even if a particular function or step performed by an external entity is not reflected in the description of a specific detailed element of the entity performing that particular step or function, it should be obvious to those skilled in the art that these methods and functions can be implemented as software or hardware elements, or any combination thereof. It will be apparent that the features of this disclosure are easily combined in various combinations without departing from the scope of this disclosure as defined by the appended claims.

[0011]

[0011] Additional aspects, advantages, features, and purposes of the present disclosure will become apparent from the drawings and the detailed description of exemplary embodiments, which will be interpreted in conjunction with the appended claims below.

[0012]

[0012] In addition to the above summary, the following detailed description of exemplary embodiments will be better understood when read in conjunction with the accompanying drawings. For the purpose of illustrating this disclosure, the drawings show exemplary configurations of this disclosure. However, this disclosure is not limited to the specific methods and means disclosed herein. Furthermore, those skilled in the art will understand that the drawings are not to scale. Wherever possible, the same elements are shown with the same numbers. [Brief explanation of the drawing]

[0013]

[0013] Embodiments of the present disclosure will be described with reference to the following drawings, but these are only a few examples. [Figure 1A] Figures 1A to 1E are flowcharts illustrating, as a whole, a method for providing navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. [Figure 1B] Figures 1A to 1E are flowcharts illustrating, as a whole, a method for providing navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. [Figure 1C] Figures 1A to 1E are flowcharts illustrating, as a whole, a method for providing navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. [Figure 1D] Figures 1A to 1E are flowcharts illustrating, as a whole, a method for providing navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. [Figure 1E] Figures 1A to 1E are flowcharts illustrating, as a whole, a method for providing navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. [Figure 2]Figure 2 shows a system that provides navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. [Figure 3A] Figure 3A is a diagram showing an implementation scenario of a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. [Figure 3B] Figure 3B is a diagram illustrating an implementation scenario of a predetermined navigation path in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. Detailed description of the embodiment

[0014]

[0014] In the attached drawings, underlined numbers are used to represent the item in which the underlined number is located or the items adjacent to the underlined number. Ununderlined numbers are associated with the items identified by the line connecting the ununderlined number to the item. When a number is ununderlined and accompanied by an associated arrow, the ununderlined number is used to identify the general item pointed to by the arrow.

[0015]

[0015] The following detailed description illustrates embodiments of the present disclosure and how each can be implemented. While several forms of implementing the present disclosure are disclosed, those skilled in the art will recognize that other embodiments for implementing or realizing the present disclosure are also possible.

[0016]

[0016] Figures 1A to 1E, as a whole, are flowcharts of a method for providing navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. Referring to Figures 1A to 1E, these figures show flowcharts of method 100, which includes steps 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134A, 134B, 134C, 136, 138A, 138B, 138C, 140, and 142. Step 102 includes substeps 102A and 102B. Steps 102, 104, and 106, including 102A and 102B, are shown in Figure 1A; steps 108, 110, 112, 114, and 116 are shown in Figure 1B; steps 118, 120, 122, 124, and 126 are shown in Figure 1C; steps 128, 130, 132, 134A, 134B, and 134C are shown in Figure 1D; and steps 136, 138A, 138B, 138C, and 140 are shown in Figure 1E.

[0017]

[0017] A method 100 for navigation assistance in a 3D virtual environment is provided. Method 100 provides an overall view of the 3D virtual environment by navigation that indicates each point of interest to help the user explore each virtual area of ​​the 3D virtual environment. In Method 100, the 3D virtual environment is explored through a predetermined navigation path by multiple virtual characters controlled by multiple corresponding users. The predetermined navigation path directs multiple users to move multiple virtual characters to at least one virtual area.

[0018]

[0018] Referring to Figure 1A, in step 102, method 100 includes accessing a 3D virtual environment using multiple user devices associated with multiple users. Each of the multiple users is configured to access the 3D virtual environment using one of the multiple user devices. Examples of multiple user devices include, but are not limited to, smartphones, laptops, desktops, and tablets.

[0019]

[0019] In substep 102A of step 102, the 3D virtual environment includes multiple virtual characters corresponding to multiple users. The multiple virtual characters are placed in different virtual locations, which change dynamically as the multiple virtual characters navigate the 3D virtual environment, and the multiple virtual characters communicate with each other in the 3D virtual environment. Generally, the 3D virtual environment is defined as a computer-simulated electronic 3D virtual environment in which multiple users can communicate and interact with each other through multiple virtual characters (e.g., avatars). Typically, a virtual character can be defined as an electronic image representing a user or an electronic image that can be manipulated by a user. Alternatively, multiple virtual characters can be defined as graphic representations of multiple users. Furthermore, multiple users can explore the 3D virtual environment by moving the multiple virtual characters within the 3D virtual environment. In a 3D virtual environment, multiple users can freely move multiple virtual characters, and the position of each virtual character changes dynamically. In one embodiment, each user's virtual character may be controlled by a head tracking device. In another embodiment, each user's virtual character may be controlled by mouse movement. In yet another embodiment, each user's virtual character may be controlled by one or more keyboard keys. Furthermore, multiple users communicate with each other in the 3D virtual environment by clicking on a specific virtual character with which they wish to communicate with other virtual characters. In one embodiment, multiple virtual characters communicate by sharing audio. In another embodiment, multiple virtual characters communicate by sharing video. Accessing the 3D virtual environment is beneficial because multiple users can communicate with each other on a single platform and dynamically explore the entire 3D virtual environment.

[0020]

[0020] In substep 102B of step 102, the 3D virtual environment is a plurality of virtual cameras corresponding to a plurality of virtual characters, further comprising a plurality of virtual cameras that are positioned at a given distance from the corresponding virtual character and that view the 3D virtual environment from a given orientation. Furthermore, the given virtual cameras are movable in accordance with the movement of the corresponding virtual characters. Furthermore, the plurality of virtual cameras corresponding to the plurality of virtual characters capture different views of the 3D virtual environment in the direction in which the corresponding virtual characters are moving. In one embodiment, the given virtual camera is positioned at a given distance from the corresponding virtual character. The given distance may be fixed or set by a user associated with the corresponding virtual character. Furthermore, the given virtual camera captures different views of the 3D virtual environment from a given orientation that may be fixed or changed by a user associated with the corresponding virtual character. For example, after a first user controls a first virtual character and a first virtual camera corresponding to the first virtual character, the first virtual camera captures a specific virtual area that the first virtual character views. Furthermore, the first user views the view captured by the first virtual camera.

[0021]

[0021] In step 104, at any given time, a geometry is constructed in which multiple virtual characters are in different virtual positions, and this geometry is one of a circle geometry, ellipse geometry, polygon geometry, straight line geometry, bow geometry, curve geometry, or freeform geometry. The geometry formed by the multiple virtual characters may be arranged around a virtual structure of the virtual environment, such as a table, one or more sofas, a bonfire, seats in an auditorium or classroom, etc. In one embodiment, the different virtual positions of the multiple virtual characters provide a realistic view of the 3D (three-dimensional) virtual environment in which multiple users can each see each other in a single field of view. In another embodiment, the different virtual positions of the multiple virtual characters provide a hassle-free way for users to communicate with each other, as multiple virtual characters corresponding to multiple users exist in one location that constitutes one of a circle geometry, ellipse geometry, polygon geometry, straight line geometry, bow geometry, curve geometry, freeform geometry, etc., so that each user does not need to search for the other user.

[0022]

[0022] In step 106, method 100 includes a given virtual camera positioned above the height of the corresponding virtual character. In one embodiment, the given virtual camera is positioned above the height of the corresponding virtual character and behind the corresponding virtual character so that one of several users can view the scene in the 3D (three-dimensional) virtual environment as well as the head and torso of the corresponding virtual character. Furthermore, the view of the head and torso of the corresponding virtual character allows the user to analyze the movement of the corresponding virtual character. In another embodiment, the given virtual camera is positioned above the height of the corresponding virtual character and in front of the corresponding virtual character so that a user associated with the corresponding virtual character receives a line-of-sight view of the corresponding virtual character.

[0023]

[0023] Referring now to FIG. 1B, at step 108, method 100 includes controlling a given virtual camera using at least one of a plurality of user devices. For example, when a first user controls a first virtual character using a first user device, at least one virtual area of the 3D (three-dimensional) virtual environment targeted by the first virtual character is visually recognized by the first user. The reason is that, for at least one virtual area visually recognized by the first virtual character, a first virtual camera corresponding to the first virtual character captures a view.

[0024]

[0024] At step 110, method 100 includes providing at least one pre-defined navigation path in the 3D (three-dimensional) virtual environment for instructing a plurality of virtual characters to move towards at least one virtual area in the 3D (three-dimensional) virtual environment. Further, the at least one pre-defined navigation path includes a common starting point. Further, according to the at least one pre-defined navigation path, a plurality of users can move their respective corresponding virtual characters to one or more virtual areas in the 3D (three-dimensional) virtual environment. Further, the at least one pre-defined navigation path guides a plurality of users in exploring different virtual areas of the 3D (three-dimensional) virtual environment so that the plurality of users do not miss specific details provided in the 3D (three-dimensional) virtual environment.

[0025]

[0025] In step 112, method 100 includes providing a visual representation of the position of at least one virtual camera along at least one predefined navigation path, through which a corresponding virtual character moves. For example, when a first user and a second user among multiple users explore a 3D (three-dimensional) virtual environment through a first virtual character and a second virtual character, the first user can anticipate the position (e.g., height) of a virtual camera placed on the second virtual character. The virtual camera placed on the second virtual character is visually recognized by the first user among multiple users. Examples of the visual representation include, but are not limited to, dots, orbs, virtual objects, halos, virtual fog, etc.

[0026]

[0026] In step 114, at least one predefined navigation path is divided into multiple sub-paths in at least one virtual area, and the at least one predefined navigation path is a 3D (three-dimensional) spline including a series of segments, where a given segment is either a linear segment or a curved segment. The at least one predefined navigation path varies according to the 3D (three-dimensional) virtual environment, such as being divided into multiple sub-paths that enable multiple users to experience an overall view of one or more virtual areas from different directions. Further, the multiple sub-paths may have different patterns. In one example, depending on the implementation scenario of the 3D (three-dimensional) virtual environment, the number of predefined navigation paths may be one or more. In one embodiment, the at least one predefined navigation path may be a 3D (three-dimensional) spline, which includes a series of segments, and a given segment is either a linear segment or a curved segment. Examples of the curved segment include, but are not limited to, Bezier segments, Hermite segments, etc.

[0027]

[0027] In step 116, the multiple subpaths are at least one form of a spiral pattern, a linear pattern, or a free-form pattern. In one embodiment, a spiral pattern is obtained by at least one subpath of a predefined navigation path that gives multiple users a 360° view. In another embodiment, a linear pattern is obtained by at least one subpath of a predefined navigation path that gives multiple users a linear view. In yet another embodiment, a free-form pattern is obtained by at least one subpath of a predefined navigation path that allows multiple users to move freely through a specific virtual area.

[0028]

[0028] Referring here to Figure 1C, in step 118, method 100 includes providing a display of scene details in at least one virtual area when at least one of a plurality of virtual characters is located in or near at least one virtual area. For example, in one case, three of the plurality of virtual characters are moving along at least one predetermined navigation path toward at least one virtual area in a 3D (three-dimensional) virtual environment. The visual experience of the virtual characters moving toward at least one virtual area is improved by providing a display of scene details as soon as any one of the three virtual characters approaches or reaches at least one virtual area. For this reason, the display of scene details may be provided to the virtual character that approaches or reaches at least one virtual area, or it may be provided to multiple virtual characters. In one embodiment, the display of scene details in at least one virtual area may be provided by flashing a particular virtual area among the at least one virtual area. In another embodiment, the display of scene details in at least one virtual area may be provided by highlighting a particular virtual area among the at least one virtual area. In yet another embodiment, the display of scene details in at least one virtual area may be provided by playing audio or video in a specific virtual area of ​​at least one virtual area when any of several virtual characters approaches at least one virtual area. The display of scene details in at least one virtual area helps to draw the attention of each of several users. In one embodiment, several users interact within at least one virtual area by clicking in a specific area of ​​at least one virtual area. In another embodiment, several users interact within at least one virtual area by entering a specific area of ​​at least one virtual area.In one example, multiple users interact within at least one virtual area by sending voice commands within a specific area of ​​at least one virtual area. In another example, multiple users interact within at least one virtual area by giving gesture commands within a specific area of ​​at least one virtual area.

[0029]

[0029] In step 120, the display of scene details is in the form of at least one of visual, auditory, or tactile displays. In one example, when a first user among multiple users approaches at least one virtual area, the first user's attention is directed to the at least one virtual area by providing a visual display to the first user. Examples of visual displays include, but are not limited to, playing a video, playing a GIF image, or playing an animated video. In another example, when a first user among multiple users approaches at least one virtual area, the first user's attention is directed to the at least one virtual area by playing an auditory display. Examples of auditory displays include, but are not limited to, playing a ringtone or playing an audio record. In yet another example, when a first user among multiple users approaches at least one virtual area, the first user is made to interact with the at least one virtual area by providing a tactile display to the first user. Examples of haptic displays include, but are not limited to, clicking or moving within at least one virtual area.

[0030]

[0030] In step 122, method 100 includes publishing data associated with a given character among a plurality of virtual characters to a given area of ​​a 3D virtual environment and broadcasting the data to at least one user device that has subscribed to the given area of ​​the 3D environment. The publishing and broadcasting of data is used as a communication model between multiple users in the 3D virtual environment. Data associated with a given character among a plurality of virtual characters (e.g., location, audio, video, etc.) is published to a given area (or a specific area) of the 3D virtual environment. Users who wish to view the published data then subscribe to the given area of ​​the 3D virtual environment. This broadcasts the data to user devices that have subscribed to the given area of ​​the 3D virtual environment. For example, a first user publishes data (i.e., location, audio, video, etc.) in a first virtual area of ​​a 3D virtual environment using a corresponding first user device. Furthermore, the published data is restricted to the first virtual area only. In one embodiment, if a second user requests access to data published by the first user in the first virtual area, they request to subscribe to the first virtual area of ​​the 3D virtual environment. After subscribing to the first virtual area, the published data is broadcast to a second user device operated by the second user. In another embodiment, if a second user requests access to data published by the first user in the first virtual area, they request to move a corresponding second virtual character to the first virtual area to access the first user's published data. Subscribing to a virtual area where data is published is beneficial for multiple users to receive each other's published data.

[0031]

[0031] In one embodiment, data associated with a given virtual character is published by a publish / subscribe module, which is then used by users who have subscribed to a given area of ​​the 3D virtual environment. The processor uses the publish / subscribe module to restrict access to the data to other user devices among multiple user devices that have not subscribed to a given area of ​​the 3D virtual environment. In one example, the publish / subscribe module is used as a communication model. In this case, user data (e.g., location, audio, and video) of a first user is published to a specific virtual area of ​​the 3D virtual environment, and another user who wishes to view it subscribes to that specific virtual area of ​​the 3D virtual environment. The publish / subscribe module corresponds to a communication model based on the creation and consumption of content that can anonymize communication through the publish / subscribe model, meaning that each of the server and / or multiple user devices does not need to know about subscriptions to obtain resources of the virtual environment. Typically, publish / subscribe modules enable information exchange where each participant can publish and receive information, providing greater flexibility than a typical client / server model. The publish / subscribe module of this disclosure enables the dynamic concentration of more resources to each of the multiple user devices that require the most resources, such as high-resource-demand cells that user devices can subscribe to. Furthermore, as the number of published resources increases, the volume of the 3D virtual environment and the level of interaction visible to each user within that 3D virtual environment also increases.

[0032]

[0032] In step 124, the data includes at least one of position, orientation, audio, and video associated with a given character. The data associated with a given character includes one of position, orientation, audio, and video, which is published to a plurality of virtual characters and broadcast to users subscribed to a given area of ​​the 3D (three-dimensional) virtual environment.

[0033]

[0033] In step 126, method 100 includes triggering at least one action when any of the multiple virtual characters enter at least one virtual area via at least one predefined navigation path, the at least one predefined navigation path includes at least one trigger point which is triggered when any of the multiple virtual characters are present. In some cases, when one of the multiple virtual characters enters at least one virtual area via at least one predefined navigation path, at least one action is triggered. This is because the at least one predefined navigation path includes at least one trigger point which is triggered when one of the multiple virtual characters is present at at least one trigger point.

[0034]

[0034] Referring here to Figure 1D, in step 128, at least one action includes at least one of transmitting video, playing audio, and activating an object located in at least one predefined navigation path. At least one action triggered when one of a plurality of virtual characters enters at least one virtual area via at least one predefined navigation path includes at least one of transmitting video, playing audio, and activating an object located in at least one predefined navigation path.

[0035]

[0035] In step 130, method 100 involves capturing multiple video streams using multiple virtual cameras, wherein a given video stream is captured from the viewpoint of a given virtual camera associated with a given virtual character, and the given video stream represents at least a region of a 3D virtual environment in which the given virtual character resides. For example, in some cases, a first user controls a first virtual character, a first virtual camera is associated with the first virtual character, and the video stream captured by the first virtual camera represents at least a region of a 3D virtual environment in which the first virtual character resides. Similarly, multiple virtual cameras capture one or more regions of a 3D virtual environment in which the corresponding multiple virtual characters reside.

[0036]

[0036] In step 132, the given video stream also represents a part of the given virtual character. For example, if a first user is controlling a first virtual character in a 3D virtual environment, the first user can see a part of the first virtual character (e.g., the head and torso) while exploring the 3D virtual environment. Representing a part of the given virtual character is beneficial because it allows tracking of the movement and gestures of the given virtual character. For example, if a given virtual camera is positioned above the head of the virtual character, the given video stream represents the top and back of the head and the back of the torso of the virtual character. In such a case, the feet of the virtual character may not be in the viewpoint of the given virtual camera.

[0037]

[0037] In step 134A, method 100 includes a user device receiving at least two video streams from at least two other user devices among a plurality of user devices. For example, if three users, such as a first user, a second user, and a third user, are using a 3D (three-dimensional) virtual environment, and the first and second users are exploring at least one virtual area, then the video stream of the at least one virtual area being explored by the first and second users is broadcast to the third user.

[0038]

[0038] In step 134B, method 100 includes the step of rearranging at least two video streams as a primary stream and at least one auxiliary stream based on the virtual position of a virtual character corresponding to a user of a user device and the relative virtual positions of a plurality of virtual characters corresponding to at least two video streams with respect to the viewing direction of a virtual camera corresponding to the virtual character, wherein the plurality of virtual characters correspond to at least two users of at least two other user devices. In the aforementioned case where three users, such as a first user, a second user, and a third user, use the 3D virtual environment, the first user explores at least one virtual area of ​​the 3D virtual environment after approaching a specific area, and the second user explores at least one virtual area of ​​the 3D virtual environment without approaching a specific area. Furthermore, the video stream captured by the first virtual camera of the first virtual character corresponding to the first user is considered the primary stream. Similarly, the video stream captured by the second virtual camera of the second virtual character corresponding to the second user is considered the auxiliary stream. In this case, the third user receives the video stream from the first user as the primary stream and the video stream from the second user as the secondary stream. Therefore, the captured video streams are reorganized into a primary stream and at least one secondary stream based on the position and orientation of the corresponding virtual characters (e.g., the first and second virtual characters) (e.g., the orientation of the first and second virtual cameras). Reorganizing the two received video streams into a primary and secondary stream is beneficial because it improves the viewing experience for users among multiple users who receive the two video streams (e.g., the third user).

[0039]

[0039] In step 134C, method 100 provides and displays a primary stream and at least one auxiliary stream at varying video quality on the user device, wherein the video quality of the primary stream is higher than that of the at least one auxiliary stream. For example, if a first virtual character is located near at least one virtual area, the video stream captured by the first virtual camera corresponding to the first virtual character (i.e., the primary stream) is of high quality and high pixel rate. Similarly, if a second virtual character is not located outside near at least one virtual area, the video stream captured by the second virtual camera corresponding to the second virtual character (i.e., the auxiliary stream) is of low quality and low pixel rate.

[0040]

[0040] Referring here to Figure 1E, in step 136, method 100 includes the step of rearranging at least two video streams, which includes determining a video stream as an auxiliary stream if the virtual position of the virtual character corresponding to the video stream is at a distance greater than a predetermined threshold distance from the virtual position of the virtual character corresponding to the user on the user device, and the position of the virtual character corresponding to the video stream is at an angle greater than a predetermined threshold angle from the viewing direction of the virtual camera corresponding to the virtual character. In one embodiment, the predetermined threshold distance from the virtual position of the virtual character corresponding to the user that determines the auxiliary stream is greater than 0.5m. In another embodiment, the predetermined threshold angle from the viewing direction of the virtual camera corresponding to the virtual character that determines the auxiliary stream is greater than 30°. Whether a video stream is determined as a primary stream or an auxiliary stream is determined by the position of the virtual character among a plurality of virtual characters, as well as the viewing direction and orientation of the virtual camera of the virtual character. For example, if the first virtual character is located within a predetermined threshold distance of at least one virtual area, the video stream captured by the first virtual camera corresponding to the first virtual character is the primary stream. In another situation, if the first virtual character is located at a distance greater than the predetermined threshold distance of at least one virtual area, the video stream captured by the first virtual camera corresponding to the first virtual character is the auxiliary stream. Furthermore, if the position of the first virtual character is at an angle greater than a predetermined threshold angle from the viewing direction of the first virtual camera corresponding to the first virtual character, the video stream captured by the first virtual camera in such a case is considered to be the auxiliary stream.

[0041]

[0041] In step 138A, method 100 includes capturing multiple video feeds representing multiple users using the cameras of multiple user devices. For example, if a first user, a second user, and a third user among the multiple users are using a 3D (three-dimensional) virtual environment through their respective corresponding user devices, the cameras of the corresponding user devices capture the video of each user. Alternatively, the camera of the first user device captures the video of the first user, the camera of the second user device captures the video of the second user, and the camera of the third user device captures the video of the third user.

[0042]

[0042] In step 138B, method 100 includes removing the background of each user from the corresponding video feed for multiple users. Furthermore, the background of the captured video is removed by processing the video captured by the user device's camera. In one embodiment, the background of the captured video may be cropped by the processor. In another embodiment, the background of the captured video may be blurred by the processor. In yet another embodiment, the background of the captured video may be modified by the processor (e.g., color change, brightness change, etc.). Removing the background of the captured video is beneficial because it allows focus on the corresponding user's face.

[0043]

[0043] In step 138C, method 100 includes merging video cutouts of multiple users without backgrounds with corresponding virtual characters in a 3D (three-dimensional) virtual environment. After removing the backgrounds from the captured videos, only the users' faces and / or entire torsos remain in the video stream and are inserted into the virtual environment. Furthermore, in one embodiment, the video stream may be pasted onto the corresponding user's virtual character such that the corresponding virtual character's face is represented as the corresponding user's face. Thus, the appearance of the virtual character becomes the appearance of the user into which the video cutout was inserted into the virtual environment, giving the impression that the user actually exists in the virtual environment and contributes to its virtual presence.

[0044]

[0044] In step 140, method 100 includes broadcasting metadata indicating the position coordinates of each virtual camera to which the corresponding virtual character moves along at least one predetermined navigation path from the user device associated with the user of the virtual character to at least one other user device among a plurality of user devices. For example, metadata for a first virtual character corresponding to a first user is broadcast to a second user device corresponding to a second user (e.g., via a WebSocket). Furthermore, the broadcast of metadata allows the second user to search for and reach the first virtual character of the first user in a 3D (three-dimensional) virtual environment. By receiving the metadata, the second user tracks the coordinates of the first virtual character and moves the second virtual character to reach the first virtual character.

[0045]

[0045] In step 142, method 100 includes sending and displaying multiple video streams to multiple user devices. In one embodiment, the multiple video streams captured by multiple virtual cameras may be sent to a selected user among multiple users or to each of the multiple users.

[0046]

[0046] Method 100 enhances the user experience in a 3D virtual environment. Method 100 provides improved navigation assistance for multiple users to access the 3D virtual environment so that each user does not miss specific details provided in the 3D virtual environment. The predefined navigation paths disclosed in Method 100 point to at least one virtual area of ​​the 3D virtual environment. Furthermore, the predefined navigation paths are divided into multiple subpaths to assist multiple users exploring at least one virtual area from different directions. In addition, the configuration of different shapes of geometry with different virtual positions of multiple virtual characters provides interactive, cinematic experiences and cutscenes that are not typically available on video game or 3D video conferencing platforms.

[0047]

[0047] Steps 102 to 142 are illustrative, and other options are available without departing from the claims of this specification, including the addition of one or more steps, the removal of one or more steps, or the provision of one or more steps in a different order.

[0048]

[0048] Figure 2 shows a system that provides navigation assistance in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. Figure 2 is described in conjunction with the elements of Figure 1. Referring to Figure 2, this figure shows a system 200 that provides navigation assistance in a 3D (three-dimensional) virtual environment 202. Furthermore, the system 200 comprises a server 204 that accesses the 3D (three-dimensional) virtual environment 202. The server 204 comprises a processor 206 and memory 208. Furthermore, a plurality of user devices such as a first user device 210A and a second user device 210B are shown, which are operated by a plurality of users such as a first user 212A and a second user 212B, respectively. Furthermore, there are multiple virtual characters, such as a first virtual character 214A and a second virtual character 214B, each of which is associated with one of several users; for example, the first virtual character 214A is associated with the first user 212A, and the second virtual character 214B is associated with the second user 212B. The multiple virtual characters, such as the first virtual character 214A and the second virtual character 214B, have multiple virtual cameras, such as a first virtual camera 216A and a second virtual camera 216B. The system 200 is configured to perform method 100 (shown in Figures 1A to 1E).

[0049]

[0049] The 3D virtual environment 202 represents a virtual configuration (e.g., a virtual model) designed by any suitable 3D modeling technique and computer-aided design (CAD) method, enabling exploration and communication between users through corresponding virtual characters. For this reason, the 3D virtual environment 202 may be a virtual meeting platform or video conferencing platform with a virtual setting, where users can walk around, see each other's virtual characters, and communicate with each other. Examples of the 3D virtual environment 202 include, but are not limited to, a 3D roller coaster, a 3D haunted house in a theme park, a theme park, a video game, a 3D museum, a 3D city, a school, a factory, or any venue.

[0050]

[0050] Server 204 may include preferred logic, circuitry, interfaces, and / or code that are communicatively coupled to a plurality of user devices, such as a first user device 210A and a second user device 210B. Alternatively, Server 204 may be configured to provide access to the 3D virtual environment 202 to a plurality of users, such as a first user 212A and a second user 212B, using a plurality of user devices, such as a first user device 210A and a second user device 210B. Server 204 may be further configured to provide a live feed of the actions performed by the plurality of users, such as a first user 212A and a second user 212B, in the 3D virtual environment 202. Examples of implementations of Server 204 include, but are not limited to, a storage server, a cloud-based server, a web server, an application server, or a combination thereof.

[0051]

[0051] The processor 206 may include suitable logic, circuitry, interfaces, or code configured to control a plurality of virtual characters, such as a first virtual character 214A and a second virtual character 214B, in accordance with commands provided by a plurality of users, such as a first user 212A and a second user 212B, in the 3D (three-dimensional) virtual environment 202 by executing instructions stored in the memory 208 of the server 204. Examples of the processor 206 include, but are not limited to, a processor, a digital signal processor (DSP), a microprocessor, a microcontroller, a complex instruction set computer (CISC) processor, an application-specific integrated circuit (ASIC) processor, a reduced instruction set (RISC) processor, a very long instruction word (VLIW) processor, a state machine, a data processing unit, a graphics processing unit (GPU), and other processors or control circuits.

[0052]

[0052] The memory 208 may include suitable logic, circuitry, and / or interfaces configured to store data associated with the 3D virtual environment 202 (including, but not limited to, virtual areas of the 3D virtual environment 202, positions of multiple virtual characters, audio, video, etc.). In one embodiment, the memory 208 may be configured to store data and instructions executable by multiple users, such as a first user 212A and a second user 212B. Examples of implementations of the memory 208 include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) card, solid-state drive (SSD), and / or CPU cache memory.

[0053]

[0053] Each of the user devices, such as the first user device 210A and the second user device 210B, may be communicatively coupled to the server 204 and may include suitable logic, circuitry, interfaces, and / or code configured to enable each of the user devices, such as the first user 212A and the second user 212B, to access the 3D (three-dimensional) virtual environment 202. Examples of each of the user devices include, but are not limited to, Internet of Things (IoT) devices, smartphones, machine-type communication (MTC) devices, computer equipment, IoT controllers, drones, hardware customized for wireless telecommunications, transmitters, or any other portable or non-portable electronic devices.

[0054]

[0054] Each of the multiple virtual characters explores the 3D (three-dimensional) virtual environment 202 under the control of each of the multiple users. For example, the first virtual character 214A is controlled by the first user 212A, and similarly, the second virtual character 214B is controlled by the second user 212B.

[0055]

[0055] Multiple cameras, such as the first virtual camera 216A and the second virtual camera 216B, are each used to capture different views in the 3D virtual environment 202 when multiple virtual characters, such as the first virtual character 214A and the second virtual character 214B, move around the 3D virtual environment 202. The captured views are then displayed on each of the multiple user devices, such as the first user device 210A and the second user device 210B.

[0056]

[0056] A system 200 is provided for navigation assistance in a 3D virtual environment 202. Navigation assistance in a 3D virtual environment 202 means that one or more specific directions are provided to multiple users, such as a first user 212A and a second user 212B, when moving toward a point of interest in the 3D virtual environment 202. In one embodiment, the system 200 may have N users, and therefore N user devices, N virtual characters, and N virtual cameras, but for simplicity, Figure 2 shows only two users, and therefore two user devices, two virtual characters, and two virtual cameras.

[0057]

[0057] The system 200 comprises a plurality of user devices associated with a plurality of users, and at least one server, such as a server 204, which is communicatively coupled to the plurality of user devices. In one embodiment, a plurality of user devices, such as a first user device 210A, are associated with a first user 212A. Similarly, a second user device 210B is associated with a second user 212B. The system 200 further comprises a server 204 which is communicatively coupled to the first user device 210A and the second user device 210B, respectively, thereby enabling the first user 212A and the second user 212B to access the 3D (three-dimensional) virtual environment 202.

[0058]

[0058] At least one server, such as server 204, is configured to allow multiple user devices to access the 3D virtual environment. Each of the multiple user devices accesses the 3D virtual environment 202 through at least one server, such as server 204. For example, a first user 212A accesses the 3D virtual environment 202 by connecting a first user device 210A to at least one server, such as server 204. Similarly, a second user 212B accesses the 3D virtual environment 202 by connecting a second user device 210B to at least one server, such as server 204.

[0059]

[0059] The 3D virtual environment 202 includes multiple virtual characters corresponding to multiple users, the multiple virtual characters are placed in different virtual positions, the different virtual positions change dynamically when the multiple virtual characters navigate the 3D virtual environment 202, and the multiple virtual characters communicate with each other in the 3D virtual environment 202. In one embodiment, in the 3D virtual environment 202, the first user 212A moves the corresponding first virtual character 214A, and in the 3D virtual environment 202, the second user 212B moves the corresponding second virtual character 214B. The virtual positions of the first virtual character 214A and the second virtual character 214B change dynamically to provide multiple users with different views of all virtual areas of the 3D virtual environment 202. The communication between each of the multiple virtual characters is described in detail, for example, in Figures 1A to 1E.

[0060]

[0060] The 3D (three-dimensional) virtual environment 202 further comprises multiple virtual cameras corresponding to multiple virtual characters, where a given virtual camera is positioned at a given distance from the corresponding virtual character and views the 3D (three-dimensional) virtual environment 202 from a given orientation, and the given virtual camera is movable in accordance with the movement of the corresponding virtual character. In one example, when a first user 212A moves a first virtual character 214A from one position to another, the corresponding first virtual camera 216A moves in accordance with the movement of the first virtual character 214A. In another example, when a second user 212B moves a second virtual character 214B from one position to another, the corresponding second virtual camera 216B moves in accordance with the movement of the second virtual character 214B. The movement of multiple virtual cameras in accordance with the movement of multiple virtual characters is beneficial because it provides an overall view of the virtual area of ​​the 3D (three-dimensional) virtual environment 202 in which the corresponding virtual characters reside.

[0061]

[0061] At least one server, such as server 204, is further configured to provide the 3D virtual environment 202 with at least one predefined navigation path for instructing multiple virtual characters to move toward at least one virtual area in the 3D virtual environment 202. In one example, a first user 212A operates a first user device 210A to move a first virtual character 214A toward at least one virtual area in the 3D virtual environment 202 via at least one predefined navigation path. In another example, a second user 212B operates a second user device 210B to move a second virtual character 214B toward at least one virtual area in the 3D virtual environment 202 via at least one predefined navigation path. Furthermore, the at least one predefined navigation path is divided into multiple subpaths so that each of the multiple users can explore and experience the at least one virtual area from multiple directions. In one embodiment, at least one predefined navigation path is divided into multiple sub-paths within at least one virtual area. Furthermore, the at least one predefined navigation path is divided into multiple sub-paths that allow multiple users to experience an overall view of one or more virtual areas from different directions. Moreover, the multiple sub-paths may have different patterns.

[0062]

[0062] At least one server, such as server 204, is further configured to provide a display of scene details in at least one virtual area when at least one of the multiple virtual characters is located in or near at least one virtual area. In one embodiment, when a first user 212A moves the corresponding first virtual character 214A closer to at least one virtual area, the display of scene details is provided to the first virtual character 214A by highlighting a particular virtual area or playing audio or video in a particular virtual area. The display of scene details in at least one virtual area is provided to the first virtual character 214A so that the first user 212A can observe the details of a particular scene and not miss them.

[0063]

[0063] At least one server, such as server 204, is configured to capture multiple video streams using multiple virtual cameras, such that a given video stream is captured from the viewpoint of a given virtual camera associated with a given virtual character, and the given video stream represents at least a portion of a given virtual character and an area of ​​the 3D virtual environment 202 in which the given virtual character resides. For example, a first virtual camera 216A associated with a first virtual character 214A captures a video stream. The captured video stream includes different views of a particular virtual area in which the first virtual character 214A resides. Furthermore, the captured video stream includes a portion of the corresponding first virtual character 214A. Furthermore, the video stream captured by the first virtual camera 216A provides a first user 212A with different views of the 3D virtual environment 202 by display on a first user device 210A.

[0064]

[0064] At least one server, such as server 204, is further configured to send and display multiple video streams to multiple user devices. In one example, a video stream captured by a first virtual camera 216A is displayed on a first user device 210A. In another example, a video stream captured by a second virtual camera 216B is displayed on a second user device 210B. By displaying the captured video streams on multiple user devices, a third-eye view of at least one virtual region is provided to multiple users.

[0065]

[0065] According to one embodiment, at least one server, such as server 204, comprises at least one processor, such as processor 206, and at least one memory, such as memory 208, which are coupled to communicate with each other. At least one processor, such as processor 206, is configured to implement a virtual environment platform that hosts at least one 3D virtual environment 202, and at least one memory, such as memory 208, is configured to store information and / or data related to at least one 3D virtual environment 202. In one embodiment, the processor 206 of server 204 performs tasks in the 3D virtual environment 202. Furthermore, at least one memory, such as memory 208 of server 204, provides storage for data related to the 3D virtual environment 202.

[0066]

[0066] One embodiment is a computer program product that provides navigation assistance in a 3D virtual environment 202, and comprises a non-transitory machine-readable data storage medium on which program instructions are stored when accessed by a processing device. In one embodiment, the computer program product includes a set of instructions and commands that need to be executed in response to inputs given by multiple users. For example, if a first user 212A instructs the first virtual character 214A to move, the computer program product executes the command after analyzing the instructions for the command to move the first virtual character 214A. The computer program product is useful for providing hassle-free functionality in the 3D virtual environment 202. In one example, instructions are implemented on a computer-readable medium, including but not limited to electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drives (HDDs), flash memory, secure digital (SD) cards, solid-state drives (SSDs), computer-readable storage media, and / or CPU cache memory. The non-temporary computer-readable medium causes the computer to perform actions including the movement of multiple virtual characters in response to receiving commands from multiple user devices, and the display of objects when one of the multiple virtual characters approaches at least one virtual area. Furthermore, the non-temporary computer-readable medium is configured to store data updated by any of the multiple users and to broadcast the data to specific users who have subscribed to at least one virtual area on which the data was uploaded. Furthermore, the non-temporary computer-readable medium is configured to display at least one virtual area, visible to multiple virtual characters, on corresponding multiple virtual devices.

[0067]

[0067] Figure 3A shows an implementation scenario of a 3D virtual environment according to one embodiment of the present disclosure. Figure 3A is described in conjunction with the elements of Figures 1 and 2. Referring to Figure 3A, this figure shows an implementation scenario 300A of a 3D virtual environment 302. It further shows a server 304 that provides access to the 3D virtual environment 302 to multiple users, such as a first user 306A, a second user 306B, a third user 306C, and a fourth user 306D. Furthermore, the 3D virtual environment 302 includes at least one virtual area 314, a predefined navigation route 316, and a route camera 318.

[0068]

[0068] The 3D virtual environment 302 shown in Figure 3A corresponds to the 3D virtual environment 202 in Figure 2. The server 304 shown in Figure 3A corresponds to the server 204 in Figure 2. Similarly, multiple users correspond to multiple users in Figure 2, and multiple user devices correspond to multiple user devices in Figure 2.

[0069]

[0069] In implementation scenario 300A, at least one virtual area 314 of the 3D (three-dimensional) virtual environment 302 is considered to be a virtual conference room for video conferencing among multiple users such as a first user 306A, a second user 306B, a third user 306C, and a fourth user 306D. Each of the multiple users is configured to access at least one virtual area 314 (i.e., the virtual conference room) using one of the multiple user devices such as a first user device 308A, a second user device 308B, a third user device 308C, and a fourth user device 308D. Subsequently, each of the multiple virtual characters is controlled by one of the multiple users through one of the multiple user devices. For example, the first virtual character 310A needs to reach or approach at least one virtual area 314 (i.e., a virtual conference room) by moving along a predetermined navigation path 316, which is determined by the first user 306A using the first user device 308A. Similarly, the second virtual character 310B, the third virtual character 310C, and the fourth virtual character 310D each need to reach or approach at least one virtual area 314 (i.e., a virtual conference room) by moving along a predetermined navigation path 316, which is determined by the second user 306B, the third user 306C, and the fourth user 306D, respectively, using the second user device 308B, the third user device 308C, and the fourth user device 308D. If one of the multiple virtual characters is in or near at least one virtual area 314 (i.e., a virtual conference room), a point of interest is provided to the virtual character approaching at least one virtual area 314 (i.e., a virtual conference room). For example, when the first virtual character 310A approaches at least one virtual area 314 (i.e., a virtual conference room), the flashing or highlighting of at least one virtual area 314 (i.e., a virtual conference room) begins to attract the attention of the first user 306A corresponding to the first virtual character 310A. Furthermore, each of the multiple virtual cameras is configured to capture the video stream of each of the multiple virtual characters.For example, a first virtual camera 312A associated with a first virtual character 310A is configured to capture a video stream of the movement of the first virtual character 310A toward at least one virtual area 314 (i.e., a virtual conference room) in the 3D virtual environment 302. Similarly, second virtual cameras 312B, third virtual camera 312C, and fourth virtual camera 312D associated with a second virtual character 310B, a third virtual character 310C, and a fourth virtual character 310D, respectively, are configured to capture a video stream of the movement of the second virtual character 310B, third virtual character 310C, and fourth virtual character 310D toward at least one virtual area 314 (i.e., a virtual conference room) in the 3D virtual environment 302. The captured video streams are displayed on the corresponding user device. For example, the video stream of the first virtual character 310A captured by the first virtual camera 312A is displayed on the first user device 308A.

[0070]

[0070] Furthermore, a path camera 318 is shown, which is provided in conjunction with a predetermined navigation path 316. The predetermined navigation path 316 shown in Figure 3A is circular in shape, but any other suitable shape (detailed in detail in Figure 3B, for example) may be used so that the user's virtual character moves from one place to another within at least one virtual area 314 of the 3D (three-dimensional) virtual environment 302 by the predetermined navigation path 316. For example, the predetermined navigation path 316 can be a straight line that crosses a virtual room from end to end, or a square, oval, or rectangular path that surrounds the room, providing the user with a predetermined experience of the virtual environment that still gives the user the freedom to interact with elements of the camera path, such as looking around (for example by rotating the path camera 318), zooming in or out, tilting, and interacting with other users (for example by conversing with other users, sending and receiving texts, or sharing files). The path camera 318 is configured to capture a video stream in a 3D virtual environment 302. In some embodiments, the path camera 318 is configured to move along a predetermined navigation path 316 while maintaining a constant distance and a constant viewing angle in at least one virtual area 314 (i.e., a virtual conference room). The path camera 318 is accessible to any user through a corresponding virtual character using a user device, and the view from the path camera 318 is provided to the user. When the path camera 318 moves, or when the viewpoint is adjusted due to a change in the viewing angle such as tilting or rotating the path camera 318, the viewpoint is updated in the 3D virtual environment 302. Alternatively, the path camera 318 provides a view of a third person. For example, if a first user 306A needs to see a second virtual character 310B corresponding to a second user 306B, but a third virtual character 310C is blocking the first user 306A's line of sight, the path camera 318 provides a view of the second virtual character 310B by changing its position and orientation, so that the first user 306A does not need to change the position of the corresponding first virtual character 310A.The path camera 318 is beneficial because it provides many viewpoints without changing the positions of multiple virtual characters. Each of the multiple virtual cameras provides a line-of-sight view of the corresponding virtual character, while the path camera 318 provides a third-eye view to multiple users; therefore, each of the multiple virtual cameras is different from the path camera 318. For example, if the first virtual character 310A is positioned in front of the second virtual character 310B, the first user 306 can only see the second virtual character 310B. This is because the second virtual character 310B is in the line of sight of the first virtual camera 312A. Furthermore, the first user 306A can use the path camera 318 to view other directions without moving the first virtual character 310A.

[0071]

[0071] Figure 3B shows an implementation scenario of a predetermined navigation path in a 3D (three-dimensional) virtual environment according to one embodiment of the present disclosure. Figure 3B is described in conjunction with the elements of Figures 1A to 1E, Figure 2, and Figure 3A. Referring to Figure 3B, this figure shows an implementation scenario 300B of a predetermined navigation path in a 3D (three-dimensional) virtual environment 302.

[0072]

[0072] In one embodiment, a predetermined navigation path 316 branches into a plurality of sub-paths starting from a common path, and these sub-paths branch in a spiral manner to the seats of multiple users in the 3D virtual environment 302. For example, the first predetermined navigation path 320 branches into a plurality of sub-paths such as the first sub-path 320A, the second sub-path 320B, the third sub-path 320C, the fourth sub-path 320D, the fifth sub-path 320E, the sixth sub-path 320F, and the seventh sub-path 320G, and these branch in a spiral manner to the seats of multiple users around the table 326 in the 3D virtual environment 302. Similarly, as shown in Figure 3B, the second predetermined navigation path 322 and the third predetermined navigation path 324 also branch into a plurality of sub-paths around the table 328 and the table 330, respectively. Similarly, without limiting the scope of this disclosure, a number of other possible predefined navigation routes and subroutines can branch off.

[0073]

[0073] The embodiments of this disclosure described above can be improved without departing from the scope of this disclosure as set forth in the attached claims. The expressions “including,” “comprising,” “incorporating,” “have,” and “is” used in the description and claims of this disclosure are intended to be non-exclusive, i.e., they may also exist for items, components, or elements not explicitly mentioned. Singular references are to be interpreted as relating to plurals as well. The word “exemplary” is used herein to mean “acting as an example, case, or illustration.” Any embodiment described as “exemplary” is not necessarily interpreted as being preferable or advantageous to other embodiments, and / or as excluding the incorporation of features from other embodiments. The word “optionally” is used herein to mean “provided in some embodiments and not in other embodiments.” Certain features of this disclosure are described in the context of separate embodiments for clarity, but it is understood that they may also be provided in combination as a single embodiment. Conversely, various features of this disclosure are described in the context of a single embodiment for simplification, but they may also be provided separately, in any preferred combination, or in any other preferred embodiment described herein. [Explanation of Symbols]

[0074] 100 ways Steps 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134A, 134B, 134C, 136, 138A, 138B, 138C, 140, 142 102A, 102B substep 200 Systems 202 3D (3-dimensional) virtual environment 204 Server 206 processors 208 memory 210A First User Device 210B Second User Device 212A First User 212B Second User 214A First Virtual Character 214B Second Virtual Character 216A First virtual camera 216B Second virtual camera 300A, 300B Implementation Scenarios 302 3D (3-dimensional) virtual environment 304 Server 306A First User 306B Second User 306C Third User 306D The fourth user 308A First User Device 308B Second User Device 308C Third User Device 308D Fourth User Device 310A First virtual character 310B Second Virtual Character 310C Third Virtual Character 310D The fourth virtual character 312A First virtual camera 312B Second virtual camera 312C Third virtual camera 312D The fourth virtual camera 314 Virtual Area 316 Predefined navigation routes 318 Path Cameras 320 First predefined navigation route 320A First secondary route 320B Second alternate route 320C Third secondary route 320D Fourth secondary route 320E Fifth secondary route 320F 6th secondary route 320G Seventh secondary route 322 Second predefined navigation route 324 Third predefined navigation route Tables 326, 328, and 330

Claims

1. A method for providing navigation assistance in a three-dimensional virtual environment, The step of accessing the three-dimensional virtual environment using multiple user devices associated with multiple users, wherein the three-dimensional virtual environment is The system comprises multiple virtual characters corresponding to multiple users, the multiple virtual characters being positioned in different virtual locations, the different virtual locations changing dynamically as the multiple virtual characters navigate the three-dimensional virtual environment, and the multiple virtual characters communicating with each other in the three-dimensional virtual environment. The aforementioned three-dimensional virtual environment further, The system comprises multiple virtual cameras corresponding to the multiple virtual characters, each virtual camera positioned at a given distance from the corresponding virtual character and viewing the three-dimensional virtual environment from a given orientation, and the given virtual cameras are movable in accordance with the movement of the corresponding virtual characters, and the system comprises the steps of: The steps include providing the three-dimensional virtual environment with at least one predefined navigation path for instructing the plurality of virtual characters to move toward at least one virtual area in the three-dimensional virtual environment, The steps include providing a display of scene details in the at least one virtual area when at least one of the plurality of virtual characters is located in or near the at least one virtual area, The step of capturing multiple video streams using the multiple virtual cameras, wherein a given video stream is captured from the viewpoint of a given virtual camera associated with a given virtual character, and the given video stream represents at least a region of the three-dimensional virtual environment in which the given virtual character resides. The steps of sending and displaying the plurality of video streams to the plurality of user devices, Methods that include...

2. The method according to claim 1, wherein the at least one predefined navigation path is divided into a plurality of subpaths in the at least one virtual area, and the at least one predefined navigation path is a three-dimensional spline comprising a series of segments, wherein a given segment is either a linear segment or a curved segment.

3. The method according to claim 1, wherein the plurality of subpaths are at least one form of a spiral pattern, a linear pattern, and a free-form pattern.

4. The method according to claim 1, wherein the display of the scene details is in at least one form from among visual, auditory, and tactile displays.

5. The method according to claim 1, wherein the given video stream also represents a portion of the given virtual character.

6. The method according to claim 1, wherein the given virtual camera is positioned above the height of the corresponding virtual character.

7. The method according to claim 1, wherein, at any given time, the different virtual positions of the plurality of virtual characters constitute a geometry, and the geometry is one of a circular geometry, an elliptical geometry, a polygonal geometry, a linear geometry, a bow geometry, a curved geometry, or a freeform geometry.

8. The steps include publishing data associated with a given character among the plurality of virtual characters to a given area of ​​the three-dimensional virtual environment, The steps include broadcasting the data to at least one user device that the user has subscribed to in the given area of ​​the three-dimensional environment, The method according to claim 1, further comprising:

9. The method according to claim 8, wherein the data includes at least one of position, orientation, audio, and video associated with the given character.

10. The method according to claim 1, further comprising the step of controlling the given virtual camera using at least one user device among the plurality of user devices.

11. The method according to claim 1, further comprising the step of providing a visual representation of the position of at least one virtual camera in which a corresponding virtual character moves along the at least one predetermined navigation path.

12. The steps include: receiving at least two video streams from at least two other user devices among the plurality of user devices on a user device; The steps include rearranging the at least two video streams into a primary stream and at least one auxiliary stream based on the virtual position of a virtual character corresponding to a user of the user device and the relative virtual positions of the plurality of virtual characters corresponding to the at least two video streams with respect to the viewing direction of a virtual camera corresponding to the virtual character, wherein the plurality of virtual characters correspond to at least two users of the at least two other user devices, and The steps include providing and displaying the main stream and the at least one auxiliary stream in a user device with varying video quality, wherein the video quality of the main stream is higher than the video quality of the at least one auxiliary stream; The method according to claim 1, further comprising:

13. The step of rearranging the at least two video streams is, If the virtual position of the virtual character corresponding to the video stream is located at a distance greater than a predetermined threshold distance from the virtual position of the virtual character corresponding to the user on the user device, and / or If the position of the virtual character corresponding to the video stream is at an angle greater than a predetermined threshold angle from the viewing direction of the virtual camera corresponding to the virtual character, The method according to claim 12, further comprising determining the video stream as an auxiliary stream.

14. The method according to claim 1, further comprising the step of broadcasting metadata indicating the position coordinates of each virtual camera to which the corresponding virtual character moves along the at least one predetermined navigation path, from a user device associated with the user of the virtual character to at least one other user device among the plurality of user devices.

15. The process further includes triggering at least one action when any of the plurality of virtual characters enters the at least one virtual area via the at least one predetermined navigation path, The method according to claim 1, wherein the at least one predefined navigation path includes at least one trigger point in which the at least one action is triggered if any of the plurality of virtual characters are present.

16. The method according to claim 15, wherein the at least one operation includes at least one of transmitting video, playing audio, and activating an object located on the at least one predetermined navigation path.

17. The steps include capturing multiple video feeds representing the multiple users using the cameras of the multiple user devices, The steps include removing the background of each of the multiple users from the corresponding video feed, The steps include merging video clips of each of the aforementioned users, which lack backgrounds, with the corresponding virtual characters in the three-dimensional virtual environment, The method according to claim 1, further comprising:

18. A system that provides navigation assistance in a 3D virtual environment, Multiple user devices associated with multiple users, At least one server that is communicatively connected to the aforementioned plurality of user devices, It is equipped with, The aforementioned at least one server is The objective is to provide access to the three-dimensional virtual environment to the multiple user devices, and the three-dimensional virtual environment is The system comprises multiple virtual characters corresponding to multiple users, the multiple virtual characters are positioned in different virtual locations, the different virtual locations change dynamically when the multiple virtual characters navigate the three-dimensional virtual environment, and the multiple virtual characters communicate with each other in the three-dimensional virtual environment. The aforementioned three-dimensional virtual environment further, The system comprises multiple virtual cameras corresponding to the multiple virtual characters, where a given virtual camera is positioned at a given distance from the corresponding virtual character and views the three-dimensional virtual environment from a given orientation, and the given virtual camera is movable in accordance with the movement of the corresponding virtual character. Providing the three-dimensional virtual environment with at least one predetermined navigation path for instructing the plurality of virtual characters to move toward at least one virtual area in the three-dimensional virtual environment, When at least one of the plurality of virtual characters is located in or near the at least one virtual area, the display of scene details in the at least one virtual area is provided. The method involves capturing multiple video streams using the aforementioned multiple virtual cameras, wherein a given video stream is captured from the viewpoint of a given virtual camera associated with a given virtual character, and the given video stream represents at least a portion of the given virtual character and the region of the three-dimensional virtual environment in which the given virtual character exists. Sending and displaying the aforementioned multiple video streams to the aforementioned multiple user devices, A system configured to perform the following actions.

19. The system according to claim 18, wherein the at least one server comprises at least one processor and at least one memory, which are coupled together in a manner that allows communication between them, the at least one processor being configured to implement a virtual environment platform that hosts at least one three-dimensional virtual environment, and the at least one memory being configured to store information and / or data relating to the at least one three-dimensional virtual environment.

20. A computer program product that provides navigation assistance in a three-dimensional virtual environment, comprising a non-temporary machine-readable data storage medium that stores program instructions causing a processing device to execute the steps of the method described in claim 1 when accessed by the processing device.

Images