Resolution of Visibility Discrepancies of Virtual Objects in an Extended Reality Device

Abstract

An embodiment is provided for resolving a visibility mismatch of virtual objects in an extended reality device. The embodiment can include receiving, during an extended reality session, one or more virtual objects and settings for each virtual object from one or more users. The embodiment can also include identifying an owner of each virtual object. The embodiment may further include identifying a virtual object of interest. The embodiment can also include displaying a warning notification to the owner before sharing, in response to determining that the virtual object of interest is visible only to the owner. The embodiment may further include querying the owner as to whether to share the virtual object of interest with one or more users other than the owner. The embodiment can also include sharing the virtual object of interest, in response to determining that the owner desires to share the virtual object of interest.

Description

Technical Field

[0001] The present invention generally relates to the field of computing, and more particularly to a system for resolving visibility inconsistencies of virtual objects in extended reality devices.

Summary of the Invention

Problems to be Solved by the Invention

[0002] Extended reality (XR) devices and software have recently become popular both for consumers and for businesses. Examples of such devices include XR headsets or XR glasses, and examples of such software include XR games and socializing applications. Such XR devices and software can enable a user to simply view the surrounding real-world environment through an XR headset or glasses, or to interact with multiple other users and display a complete virtual (e.g., fictional) or augmented version of that surrounding real-world environment. As technology continues to improve and begins to impact everyday interactions between individuals, the demand for XR devices and software is expected to increase in the coming years.

Means for Solving the Problems

[0003] According to one embodiment, a method, a computer system, and a computer program product for resolving a visibility mismatch of virtual objects in an extended reality device are provided. An embodiment can include receiving, during an extended reality conference, one or more virtual objects and settings for each virtual object from one or more users. An embodiment can also include identifying an owner of each virtual object based on the settings of each virtual object. An embodiment may further include identifying a target virtual object based on one or more captured actions of one or more users. An embodiment can also include displaying a warning notification to the owner before sharing the target virtual object in response to determining that the target virtual object is visible only to the owner. An embodiment may further include querying the owner as to whether to share the target virtual object with one or more users other than the owner. An embodiment can also include sharing the target virtual object with one or more users other than the owner in response to determining that the owner desires to share the target virtual object.

[0004] These and other objects, features, and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments read in conjunction with the accompanying drawings. The drawings are provided to clarify the invention to facilitate understanding by those skilled in the art of the manner in which the invention may be practiced, and the various features of the drawings are not drawn to scale.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

MODE FOR CARRYING OUT THE INVENTION

[0006] Although detailed embodiments of the claimed structures and methods are disclosed herein, it can be understood that the disclosed embodiments merely exemplify the claimed structures and methods that can be embodied in various forms. However, the present invention can be implemented in many different forms and should not be construed as limited to the exemplary embodiments described herein. Well-known features and technical details may be omitted herein to avoid unnecessarily obscuring the presented embodiments.

[0007] Unless the context clearly dictates otherwise, it should be understood that the singular forms “a,” “an,” and “the” include plural referents. Thus, for example, a reference to “a component surface” includes a reference to one or more such surfaces unless the context clearly dictates otherwise.

[0008] Embodiments of the present invention relate to the field of computing, and more particularly, to a system for resolving visibility discrepancies of virtual objects in an Extended Reality (XR) device. The exemplary embodiments described below provide a system, method, and program product that, in particular, identify a reference virtual object based on one or more captured actions of one or more users and, in response, determine whether an owner of the reference virtual object is sharing the reference virtual object with one or more users who cannot see the reference virtual object. Thus, the present embodiments have the ability to improve XR technology by dynamically detecting and resolving visibility discrepancies among participants during an XR conference.

[0009] As described above, XR devices and software have recently become popular among both consumers and businesses. Examples of such devices include XR headsets or XR glasses, and examples of such software include XR games and socializing applications. Such XR devices and software can enable a user to simply view the surrounding real-world environment through an XR headset or glasses, or to interact with multiple other users and display a complete virtual (e.g., fictional) or augmented version of that surrounding real-world environment. As technology continues to improve and starts to impact the daily interactions that individuals have with each other, the demand for XR devices and software is expected to increase in the coming years. During an XR collaboration session, virtual objects may not be visible to all participants. For example, one user may be able to see a virtual car, while another user may not be able to see that virtual car. This issue is typically addressed by making virtual objects visible to all participants. However, making these virtual objects visible to all participants reveals important information about the business or one or more participants or both.

[0010] Therefore, it may be essential to provide a system for detecting and resolving visibility discrepancies among participants during an XR session. Accordingly, embodiments of the present invention can provide advantages including, but not limited to, dynamically detecting and resolving visibility discrepancies among participants during an XR session, notifying the owner of a virtual object of interest of the status of the virtual object of interest before sharing the virtual object of interest, and enabling the owner of the virtual object of interest to select which participants can view the virtual object of interest. The present invention does not require that all advantages must be incorporated into all embodiments of the present invention.

[0011] According to at least one embodiment, during an XR conference, one or more virtual objects and settings for each virtual object can be received from one or more users in order to identify the owner of each virtual object based on the settings of each virtual object. Once the owner is identified, the virtual object of interest can be identified based on one or more captured actions of the one or more users. According to at least one embodiment, in response to determining that the virtual object of interest is visible only to the owner, a warning notification may be displayed to the owner before sharing the virtual object of interest with one or more users other than the owner, so that the owner can be queried as to whether to share the virtual object of interest. In response to determining that the owner desires to share the virtual object of interest, the virtual object of interest can be shared with one or more users other than the owner.

[0012] According to at least one other embodiment, in response to determining that the virtual object of interest is visible to users other than the owner, a warning notification may be displayed to the owner before sharing the virtual object of interest with one or more unauthorized users, so that the owner can be queried as to whether to share the virtual object of interest with one or more unauthorized users. In response to determining that the owner desires to share the virtual object of interest with one or more unauthorized users, the virtual object of interest can be shared with one or more unauthorized users.

[0013] The present invention can be a system, method, or computer program product, or a combination thereof, at any possible level of integration of technical details. The computer program product may include a computer-readable storage medium (or multiple computer-readable storage media) having computer-readable program instructions for causing a processor to implement aspects of the present invention.

[0014] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction-executing device. The computer-readable storage medium can be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof, but is not limited thereto. A non-exhaustive listing of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick (R), floppy disk (R), mechanically encoded devices such as punch cards or raised structures engraved in grooves in which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through an electrical wire.

[0015] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within the individual computing / processing device.

[0016] Computer-readable program instructions for carrying out the operations of the present invention may be source code or object code written in any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or object-oriented programming languages such as Smalltalk(R), C++, and procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer as a stand-alone software package, partly on the user's computer, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions to individually configure the electronic circuit by utilizing the state information of the computer-readable program instructions to carry out aspects of the present invention.

[0017] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0018] These computer-readable program instructions, when executed via the processor of a computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of a flowchart, a block diagram, or both, and may thus provide to the processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus to create a machine. These computer-readable program instructions may also be stored in a computer-readable storage medium that includes a manufactured article including instructions for implementing the aspects of the functions / acts specified in one or more blocks of a flowchart, a block diagram, or both, and can direct a computer, programmable data processing apparatus, or other device or combination thereof to function in a particular manner.

[0019] The computer-readable program instructions may also be loaded onto a computer, other programmable apparatus, or other device to create a computer-implemented process such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / operations specified in one or more blocks of a flowchart, a block diagram, or both, and cause a series of operational steps to be executed on the computer, other programmable apparatus, or other device.

[0020] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions represented by the blocks may occur in a different order than shown in the drawings. For example, two blocks shown in succession may actually be executed simultaneously, substantially simultaneously, or may sometimes be executed in the reverse order depending on the functionality involved. It should also be noted that each block of the block diagram or flowchart or both, and combinations of blocks of the block diagram or flowchart or both, can be implemented by a dedicated hardware-based system that performs the specified function or action, or that executes a combination of dedicated hardware and computer instructions.

[0021] The exemplary embodiments described below provide a system, method, and program product that identify a reference virtual object based on one or more captured actions of one or more users and, accordingly, determine whether the owner of the reference virtual object is willing to share the reference virtual object with one or more users who cannot view the reference virtual object.

[0022] Referring to FIG. 1, an exemplary network computer environment 100 according to at least one embodiment is illustrated. The network computer environment 100 can include client computing devices 102, servers 112, and Internet of Things (IoT) devices 118 interconnected via a communication network 114. According to at least one implementation, the network computer environment 100 can include a plurality of client computing devices 102 and servers 112, and only one of each is shown for simplicity of illustration.

[0023] The communication network 114 can include various types of communication networks such as a wide area network (WAN), local area network (LAN), telecommunications network, wireless network, public switched network, or satellite network, or a combination thereof. The communication network 114 may include wired or wireless communication links, or connections such as fiber optic cables. It will be understood that FIG. 1 merely provides an example of one implementation and does not imply any limitation with respect to environments in which different embodiments may be implemented. Many changes can be made to the depicted environment based on design and implementation requirements.

[0024] According to one embodiment of the present invention, the client computing device 102 can include a processor 104 and a data storage device 106, host and execute a software program 108 and an XR collaboration program 110A, and communicate with the server 112 and the IoT device 118 via a communication network 114. The client computing device 102 can be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of executing a program and accessing a network. As will be discussed with reference to FIG. 4, the client computing device 102 can include internal components 402a and external components 404a, respectively.

[0025] According to an embodiment of the present invention, the server computer 112 can be a laptop computer, a netbook computer, a personal computer (PC), a desktop computer, or any programmable electronic device or any network of programmable electronic devices that can host and execute an XR collaboration program 110B and a database 116 and communicate with the client computing device 102 and the IoT device 118 via a communication network 114. As will be discussed with reference to FIG. 4, the server computer 112 can include internal components 402b and external components 404b, respectively. The server 112 can also operate in a cloud computing service model such as SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service). The server 112 may be deployed in a cloud computing deployment model such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0026] The IoT device 118 may be a line-of-sight tracking sensor, a motion sensor, an XR device such as XR smart glasses and / or an XR headset, a microphone embedded in or external to the client computing device 102, or any other IoT device 118 known in the art that is connectable to the communication network 114 for tracking body movements and / or capturing audio outputs and capable of transmitting and receiving data with the client computing device 102 and the server 112, or a combination thereof.

[0027] According to this embodiment, the XR collaboration programs 110A, 110B may be programs capable of performing the following: receiving one or more virtual objects and settings for each virtual object from one or more users during an XR meeting; identifying a virtual object of interest based on one or more captured actions of one or more users; determining whether the owner of the virtual object of interest is willing to share the virtual object of interest with one or more users who cannot see the virtual object of interest; dynamically detecting and resolving visibility inconsistencies among participants (i.e., users) during the XR meeting; notifying the owner of the virtual object of interest of the status of the virtual object of interest before sharing the virtual object of interest; and enabling the owner of the virtual object of interest to select which participants can view the virtual object of interest. The method of XR collaboration will be described in further detail below with reference to FIGS. 2A and 2B.

[0028] Next, referring to FIGS. 2A and 2B, an operational flowchart for resolving visibility inconsistencies of virtual objects in an XR device in a visibility inconsistency resolution process 200 according to at least one embodiment is depicted. At 202, the XR collaboration programs 110A, 110B receive one or more virtual objects and settings for each virtual object from one or more users during an XR meeting. Using the software program 108, one or more users can arbitrarily create one or more virtual objects and place them in the 3D XR space. For example, "User 1" can create a virtual car and place it on their computer keyboard, and "User 2" can create a virtual animal and place it on their TV stand or table. The above examples are not intended to be limiting, and it should be understood that in embodiments of the present invention, virtual objects can represent various real-world objects and can be placed in a variety of different locations.

[0029] Once one or more virtual objects are created, the user who created each virtual object can associate settings with the virtual object. According to at least one embodiment, the settings may be private virtual objects. According to at least one other embodiment, the settings may be exclusively shared virtual objects. According to at least one further embodiment, the settings may be shared virtual objects. The settings for each virtual object are described in more detail below with respect to step 204.

[0030] Next, at 204, the XR collaboration programs 110A, 110B identify the owner of each virtual object. The owner is identified based on the settings of each virtual object as described above with respect to step 202. As used herein, the "owner" of each virtual object is the user who created the virtual object and associated the settings with the virtual object. Continuing with the example of step 202 above, "User 1" can be the owner of the virtual car, and "User 2" can be the owner of the virtual animal. As described above with respect to step 202, the owner can associate settings with the virtual object.

[0031] In embodiments where the virtual object has private settings, the virtual object may initially be visible only to the owner. For example, if "User 1" makes the settings of the virtual car private, "User 1" may be the only user who can see the virtual car in the XR meeting. In embodiments where the virtual object has exclusively shared settings, the virtual object may initially be visible not only to the owner but also to at least one other user designated by the owner. For example, if "User 1" makes the settings of the virtual car exclusively shared, "User 1" can designate "User 3", "User 6", and "User 10" as "authorized users" who can view the virtual object together with the owner (i.e., "User 1"). In embodiments where the virtual object has shared settings, the virtual object may be visible to all users participating in the XR meeting. For example, if "User 2" makes the settings of the virtual animal shared, each of one or more users participating in the XR meeting will be able to view the virtual object.

[0032] Next, at 206, the XR collaboration programs 110A, 110B identify the virtual object of interest. The virtual object of interest is identified based on one or more captured actions of one or more users. The virtual object of interest may be any virtual object that is referred to by one or more users via one or more captured actions during an XR session, as will be described in more detail below.

[0033] According to at least one embodiment, the captured action may be the voice of one or more users. The voice of one or more users can be captured by an XR device such as XR smart glasses or an XR headset, or an IoT device 118 including a microphone or a combination thereof, but is not limited thereto. In particular, the voice of one or more users may refer to a specific virtual object in the surrounding environment of one or more users. The XR collaboration programs 110A, 110B can use a voice recognition module as further illustrated and described in more detail below with respect to FIG. 3 to convert the voice of one or more users into text. Next, a speech analyzer module, also as further illustrated and described in more detail below with respect to FIG. 3, performs natural language processing (NLP) on the text and identifies the object referred to in the text. For example, "User 1" might say, "That car in the middle of the room is an electric vehicle." In this example, the referred virtual object could be the car in the middle of the room. In another example, "User 2" might say, "The animal on my TV stand is a species at risk of extinction." In this example, the referred virtual object could be the animal on the TV stand. In addition, when one or more virtual objects are shared or exclusively shared, users other than the owner may refer to the virtual object. For example, "User 3" might say, "How old is the helicopter on the table in the middle of the room?" In this example, the referred virtual object could be the helicopter.

[0034] According to at least one other embodiment, the voices of one or more users may not be sufficient to identify the virtual object of interest. For example, "User 1" might say, "The car over there is an electric vehicle." However, there may be multiple virtual cars in the surrounding environment of "User 1". In this embodiment, the action to be captured may be the line of sight of one or more users. The line of sight can be captured by an IoT device 118 including, but not limited to, an XR device such as XR smart glasses or an XR headset, or a line-of-sight tracking sensor as further illustrated and described in more detail below with respect to FIG. 3, or a combination thereof. Continuing with the above example, "User 1" may be directly looking at an electric vehicle placed on a table in the center of the room. In this example, the virtual object of interest may be the electric vehicle placed on the table in the center of the room and not the other two electric vehicles on both sides of the room. Additionally, when one or more virtual objects are shared or exclusively shared, users other than the owner may refer to the virtual object. For example, "User 3" might say, "How old is the helicopter on the table in the center of the room?" However, there may be multiple virtual helicopters in the surrounding environment that "User 3" is looking at. In this example, the line of sight of "User 3" can indicate that "User 3" is directly looking at a helicopter on a TV stand, and the virtual object of interest may be the helicopter on the TV stand.

[0035] According to at least one further embodiment, the voices and gazes of one or more users may not be sufficient to identify the virtual object of interest. For example, "User 2" might say, "The animal over there can be seen in many places across the country." However, there may be multiple virtual animals in the surrounding environment of "User 2", and these virtual animals may be within a few inches (1 inch = 2.54 cm) of each other in the surrounding environment. In this embodiment, the actions to be captured may be the movement of the hand or finger or both of one or more users. The movement of the hand or finger or both can be captured by an XR device such as XR smart glasses or an XR headset, or an IoT device 118 including a motion sensor or a combination thereof, but is not limited thereto. Continuing with the above example, "User 2" may be directly looking at three virtual animals on a table in the center of the room. When "User 2", or one or more virtual objects are shared or exclusively shared, another user physically adjacent to "User 2", or both, may be directly pointing at the virtual animal on the far right. In this example, the virtual object of interest may be the animal on the far right and may not be the other two animals on the left of the animal being referred to. The above example is not intended to be limiting, and it should be understood that in the embodiments of the present invention, the virtual object of interest can represent various real-world objects and can be placed in a variety of different locations.

[0036] Next, at 208, the XR collaboration programs 110A, 110B determine whether the virtual object of interest is visible only to the owner. The determination is made based on one or more captured actions and the settings associated with the virtual object of interest.

[0037] As described above with respect to step 204, the setting of each virtual object may be either private, exclusive sharing, or sharing. This setting is applied to the reference virtual object selected from one or more virtual objects by one or more users. If the reference virtual object is a private virtual object, the reference virtual object may initially be visible only to the owner. For example, if "User 1" creates a virtual car that is the reference object and sets the reference virtual car, "User 1" may be the only user who can see the reference virtual car in the XR meeting. If the reference virtual object is a virtual object that is exclusively shared, the reference virtual object may initially be visible not only to the owner but also to at least one other user specified by the owner. For example, if "User 1" creates a virtual car that is the reference object and sets the reference virtual car, "User 1" can specify "User 3", "User 6", and "User 10" as "authorized users" who can view the reference virtual object together with the owner (i.e., "User 1"). If the reference virtual object is a virtual object that is shared, the reference virtual object may be visible to all users participating in the XR meeting. For example, if "User 2" creates a virtual animal that is the reference object and sets the reference virtual animal, each of one or more users participating in the XR meeting will be able to see the reference virtual animal.

[0038] One or more captured actions can reinforce the determination as to whether the reference virtual object is visible only to the owner. For example, due to a technical glitch, a private virtual object may be displayed to users other than the owner. This other user can see the virtual object and refer to the virtual object by voice, line of sight, or movement of the hand or finger, or a combination thereof.

[0039] In response to determining that the virtual object of interest is visible only to the owner (step 208, "Yes" branch), the visibility conflict resolution process 200 proceeds to step 210 and displays a warning notification to the owner before sharing the virtual object of interest. In response to determining that the virtual object of interest is visible to others besides the owner (step 208, "No" branch), the visibility conflict resolution process 200 proceeds to step 218 and displays a warning notification to the owner before sharing the virtual object of interest with one or more unauthorized users.

[0040] Next, at 210, the XR collaboration programs 110A, 110B display a warning notification to the owner before sharing the virtual object of interest. According to at least one embodiment, the warning notification may be a visual cue displayed to the owner. Examples of visual cues include changes in the appearance of the virtual object of interest (e.g., changes in color, shape, transparency, lighting, or contour enhancement, or a combination thereof), or a pop-up message (e.g., a pop-up with the text "User 3 cannot view the virtual object of interest"), or both, but are not limited thereto. In at least one other embodiment, the warning notification may be an audible cue spoken to the owner. For example, the audio output may be "User 3 cannot view the virtual object of interest."

[0041] According to at least one other embodiment, one or more users other than the owner may be informed of the existence and location of the virtual object of interest without actually seeing the virtual object of interest by displaying a placeholder of the virtual object of interest to such one or more other users. Examples of placeholders include, but are not limited to, a cloud bubble at the location of the virtual object of interest, or a translucent object with a blurred outline, or both. In this embodiment, the placeholder can be displayed to one or more users other than the owner before sharing the virtual object of interest.

[0042] Next, at 212, the XR collaboration programs 110A, 110B query the owner as to whether to share the virtual object of interest with one or more users other than the owner. When displaying a warning notice, the XR collaboration programs 110A, 110B can prompt the owner using an audio message, or a visual message, or both, asking the owner whether to share the virtual object of interest with one or more users other than the owner. For example, in a pop-up message, the owner can be asked "Do you want to share the virtual object of interest with fellow attendees?"

[0043] Next, at 214, the XR collaboration programs 110A, 110B determine whether the owner desires to share the virtual object of interest with one or more users other than the owner. The determination can be made based on the owner's response to the query presented in step 212. Continuing with the example described above with respect to step 212, the owner's response to the pop-up message can be "Yes" or "No". According to at least one embodiment, if the owner desires to share the virtual object of interest, the owner may set a time limit for sharing the virtual object of interest. For example, the owner can decide to share the virtual object of interest for 5 minutes.

[0044] In response to determining that the owner desires to share the virtual object of interest (step 214, "Yes" branch), the visibility conflict resolution process 200 proceeds to step 216 to share the virtual object of interest with one or more users other than the owner. In response to determining that the owner does not desire to share the virtual object of interest (step 214, "No" branch), the visibility conflict resolution process 200 ends and the virtual object of interest is not shared.

[0045] Next, at 216, the XR collaboration programs 110A, 110B share the virtual object of interest with one or more users other than the owner. The virtual object of interest can be shared by creating a digital copy of the private virtual object of interest and sharing the digital copy via the XR devices of one or more other users. For example, if the private virtual object of interest is a virtual car, a digital copy of the virtual car of interest can be displayed to one or more other users via the XR devices. According to at least one embodiment, if the owner decides to share the virtual object of interest for five minutes, the virtual object of interest can be displayed to one or more other users for only five minutes.

[0046] Next, at 218, the XR collaboration programs 110A, 110B display a warning notification to the owner before sharing the reference virtual object with one or more unauthorized users. As used herein, an "unauthorized user" may be a user who does not currently have the right to view the reference virtual object. For example, if "User 1" makes the settings of a virtual car exclusive, "User 1" can specify "User 3", "User 6", and "User 10" as "authorized users", and the users who are not specified become unauthorized users. According to at least one embodiment, as described above with respect to step 210, the warning notification may be a visual cue displayed to the owner. Examples of visual cues include changes in the appearance of the reference virtual object (e.g., changes in color, shape, transparency, lighting, or contour enhancement, or a combination thereof), or a pop-up message (e.g., a pop-up with the text "User 4 cannot view the reference virtual object"), or both, but are not limited thereto. According to at least one other embodiment, the warning notification may be an audible cue addressed to the owner. For example, the audio output may be "User 4 cannot view the reference virtual object."

[0047] According to at least one other embodiment, one or more unauthorized users may be informed of the existence and location of the reference virtual object without actually seeing the reference virtual object by displaying a placeholder of the reference virtual object to such one or more unauthorized users. Examples of placeholders include a cloud bubble at the location of the reference virtual object, or a semi-transparent object with a blurred outline, or both, but are not limited thereto. In this embodiment, the placeholder can be displayed to one or more unauthorized users before sharing the reference virtual object with one or more unauthorized users.

[0048] Next, at 220, the XR collaboration programs 110A, 110B inquire of the owner whether to share the virtual object of interest with one or more unauthorized users. When displaying a warning notice, the XR collaboration programs 110A, 110B can prompt the owner using an audio message, a visual message, or both, asking the owner whether to share the virtual object of interest with one or more unauthorized users. For example, in a pop-up message, the owner can be asked "Do you want to share the virtual object of interest with users who have not been previously specified?"

[0049] Next, at 222, the XR collaboration programs 110A, 110B determine whether the owner desires to share the virtual object of interest with one or more unauthorized users. The determination can be made based on the owner's response to the inquiry presented at step 220. Continuing with the example described above with respect to step 220, the owner's response to the pop-up message can be "Yes" or "No". According to at least one embodiment, if the owner desires to share the virtual object of interest, the owner may set a time limit for sharing the virtual object of interest. For example, the owner can decide to share the virtual object of interest for 5 minutes.

[0050] In response to determining that the owner desires to share the virtual object of interest with one or more unauthorized users (step 222, "Yes" branch), the visibility conflict resolution process 200 proceeds to step 224 to share the virtual object of interest with one or more unauthorized users. In response to determining that the owner does not desire to share the virtual object of interest with one or more unauthorized users (step 222, "No" branch), the visibility conflict resolution process 200 ends and the virtual object of interest is not shared with one or more unauthorized users.

[0051] Next, at 224, the XR collaboration programs 110A, 110B share the virtual object of interest with one or more unauthorized users. By changing the setting of the virtual object of interest from exclusive sharing to sharing and associating the virtual object of interest with the XR devices of one or more unauthorized users, the virtual object of interest can be shared. In an embodiment of the present invention, when the setting of the virtual object of interest is changed to sharing, it will be understood that one or more unauthorized users become one or more authorized users for that particular virtual object of interest. For example, if the virtual object of interest being shared exclusively is a virtual car, the virtual car being referred to can be associated with the XR devices of users who were previously unauthorized users but are now authorized users. According to at least one embodiment, if the owner decides to share the virtual object of interest for five minutes, the virtual object of interest can be displayed to the current authorized users for only five minutes.

[0052] Next, referring to FIG. 3, FIG. 300 depicting an operational example of the interaction between solution components of the processes of FIGS. 2A and 2B according to at least one embodiment is shown. In FIG. 300, an Extended Reality Collaboration Service 302 may provide a spatial model 304A of a 3D XR space. The spatial model 304A can include a shared virtual model 304B or a private virtual model 304C or both. In the shared virtual model 304B, the settings of one or more virtual objects 305 may be either shared or shared exclusively. In the private virtual model 304C, the settings of one or more virtual objects 305 are private and may be visible only to the current owner 306. According to at least one embodiment, the owner 306 can speak towards a microphone 308, and an audio recognition module 310 can convert the utterance into text. The text can then be fed to a speech analyzer module 312 that can perform NLP on the text to detect one or more words referring to an object. For example, the speech analyzer module 312 can analyze text such as "The car in the middle of the room is an electric vehicle" and determine that the car is the virtual object being referred to. According to at least one other embodiment, if the virtual object being referred to cannot be accurately identified solely by the utterance, a gaze tracking sensor 314 can track the line of sight of the owner 306 to assist in identifying the virtual object being referred to. In this embodiment, once the virtual object being referred to is identified, a visibility checker module 316 can be utilized to determine whether the virtual object being referred to is visible only to the owner 306 or is visible to the owner 306 and at least one other user (i.e., an authorized user). In response to the owner 306 being determined to be attempting to share a virtual object being referred to that is visible only to the current owner 306, a model updater module 318 can update the status of the virtual object being referred to from private to shared.In response to determining that owner 306 desires to share a reference virtual object that is currently visible not only to the owner but also to at least one other user (i.e., an authorized user), the model updater module 318 can update the status of the reference virtual object from exclusive sharing to sharing.

[0053] FIGS. 2A and 2B and FIG. 3 merely provide examples of one implementation form and should be understood not to imply any limitation regarding how various implementation forms can be implemented. Many changes can be made to the depicted environment based on design and implementation requirements.

[0054] FIG. 4 is a block diagram 400 of the internal and external components of the client computing device 102 and the server 112 depicted in FIG. 1, according to an embodiment of the present invention. FIG. 4 merely provides an example of one implementation form and it should be understood that it does not imply any limitation regarding the environment in which various implementation forms can be implemented. Many changes can be made to the depicted environment based on design and implementation requirements.

[0055] Data processing systems 402, 404 represent any electronic device capable of executing machine-readable program instructions. The data processing systems 402, 404 can represent smartphones, computer systems, PDAs, or other electronic devices. Examples of computing systems, environments, or configurations or combinations thereof that can be represented by the data processing systems 402, 404 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputer systems, and distributed cloud computing environments including any of the above systems or devices.

[0056] Client computing device 102 and server 112 may each include a respective set of internal components 402a, b and external components 404a, b shown in FIG. 4. Each set of internal components 402 includes, on one or more buses 426, one or more processors 420, one or more computer-readable RAMs 422, and one or more computer-readable ROMs 424, as well as one or more operating systems 428 and one or more computer-readable tangible storage devices 430. The one or more operating systems 428, software program 108 and XR collaboration program 110A within client computing device 102, and XR collaboration program 110B within server 112 are stored in one or more of the respective computer-readable tangible storage devices 430 for execution by one or more of the respective processors 420 via one or more of the respective RAMs 422 (typically including cache memory). In the embodiment shown in FIG. 4, each of the computer-readable tangible storage devices 430 is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices 430 is a semiconductor storage device such as ROM 424, EPROM, flash memory, or any other computer-readable tangible storage device capable of storing computer programs and digital information.

[0057] Each set of internal components 402a, b also includes an R / W drive or interface 432 for reading from and writing to one or more portable computer-readable tangible storage devices 438 such as CD-ROMs, DVDs, memory sticks (R), magnetic tapes, magnetic disks, optical disks, or semiconductor storage devices. Software programs such as XR collaboration programs 110A, 110B can be stored in one or more of the respective portable computer-readable tangible storage devices 438, read via the respective R / W drives or interfaces 432, and loaded onto the respective hard drives 430.

[0058] Each set of internal components 402a, b also includes a network adapter or interface 436 such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G or 4G wireless interface card, or other wired or wireless communication links. Software programs 108 and XR collaboration programs 110A within the client computing device 102, and XR collaboration program 110B within the server 112 can be downloaded from an external computer to the client computing device 102 and the server 112 via a network (e.g., the Internet, a local area network, or other wide area network) and the respective network adapters or interfaces 436. From the network adapter or interface 436, the software programs 108 and XR collaboration programs 110A within the client computing device 102, and the XR collaboration program 110B within the server 112 are loaded onto the respective hard drives 430. The network can include copper wires, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers or edge servers or combinations thereof.

[0059] Each set of external components 404a, b can include a computer display monitor 444, a keyboard 442, and a computer mouse 434. The external components 404a, b can also include a touch screen, a virtual keyboard, a touch pad, a pointing device, and other human interface devices. Each set of internal components 402a, b also includes a device driver 440 for interfacing with the computer display monitor 444, the keyboard 442, and the computer mouse 434. The device driver 440, the R / W drive or interface 432, and the network adapter or interface 436 include hardware and software (stored in the storage device 430 or the ROM 424 or both).

[0060] Although the present disclosure includes a detailed description of cloud computing, it should be understood in advance that the implementation of the teachings presented herein is not limited to a cloud computing environment. Rather, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0061] Cloud computing is a service delivery model that enables convenient on-demand network access to a shared pool of configurable computing resources such as networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services, and that can rapidly provision and release these resources with minimal management effort or interaction with a service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0062] The characteristics are as follows. On-demand self-service: Cloud users can automatically and unilaterally provision computing capabilities such as server time and network storage as needed, without the need for human interaction with the service provider. Broad network access: The capabilities are available over the network and can be accessed using standard mechanisms, facilitating use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). Resource pooling: The provider's computing resources are pooled and provided to multiple users using a multi-tenant model, and various physical and virtual resources are dynamically assigned and re-assigned according to demand. Although users generally have the sense of being location-independent in that they are usually unaware of and do not manage the exact location of the resources provided, at a higher level of abstraction, the location (e.g., country, state, or data center) can be specified. Rapid elasticity: The capabilities can be provisioned quickly and flexibly, sometimes automatically, scale out rapidly, and be released quickly and scale in rapidly. The capabilities available for provisioning often appear to users to be available in any quantity at any time without limit. Measured service: Cloud systems automatically control and optimize resource use at an appropriate level of abstraction for the type of service (e.g., storage, processing, bandwidth, and active user accounts) by leveraging measurement capabilities. The amount of resource use can be monitored, controlled, and reported, providing transparency to both the provider and the user of the service being utilized.

[0063] The service model is as follows. SaaS (Software as a Service): The capabilities provided to users are the use of the provider's applications running on cloud infrastructure. Those applications can be accessed from various client devices via a thin-client interface such as a web browser (e.g., web-based email). Users do not manage or control the underlying cloud infrastructure, which includes the network, servers, operating systems, storage, or individual application features, except for limited user-specific application configuration settings. PaaS (Platform as a Service): The capabilities provided to users are to deploy the applications created or acquired by the users, which are created using programming languages and tools supported by the provider, onto the cloud infrastructure. Users do not manage or control the underlying cloud infrastructure, which includes the network, servers, operating systems, or storage, but can control the deployed applications and, in some cases, the configuration of the application hosting environment. IaaS (Infrastructure as a Service): The capabilities provided to users are the provisioning of processing, storage, network, and other basic computing resources, and users can deploy and run any software that can include operating systems and applications. Users do not manage or control the underlying cloud infrastructure, but can control the operating systems, storage, deployed applications, and, in some cases, can have limited control over selected network components (e.g., host firewalls).

[0064] The deployment models are as follows. Private Cloud: The cloud infrastructure is operated only for an organization. It can be managed by that organization or a third party and can exist on-premises or off-premises. Community Cloud: The cloud infrastructure is shared by multiple organizations and supports a specific community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-premises or off-premises. Public Cloud: The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization that sells cloud services. Hybrid Cloud: The cloud infrastructure remains a unique entity but is a composite of two or more clouds (private, community, or public) joined by standardized or proprietary technologies (e.g., cloud bursting for load distribution between clouds) that enable data and application portability.

[0065] The cloud computing environment is a service-oriented environment that emphasizes statelessness, loose coupling, modularity, and semantic interoperability. At the center of cloud computing is an infrastructure with a network of interconnected nodes.

[0066] Referring now to FIG. 5, an exemplary cloud computing environment 50 is shown. As illustrated, cloud computing environment 50 includes one or more cloud computing nodes 100 that can communicate with a local computing device (e.g., a personal digital assistant (PDA) or cellular phone 54A, a desktop computer 54B, a laptop computer 54C, or an automotive computer system 54N, or a combination thereof) used by a cloud user. The nodes 100 can communicate with each other. These can be physically or virtually grouped (not shown) within one or more networks such as the private cloud, community cloud, public cloud, or hybrid cloud described above, or a combination thereof. Thereby, cloud computing environment 50 can provide an infrastructure, platform, or SaaS, or a combination thereof, that a cloud user need not maintain resources on a local computing device. The types of computing devices 54A - 54N shown in FIG. 5 are for illustration only, and it is understood that cloud computing nodes 100 and cloud computing environment 50 can communicate with any type of computer controlled device via any type of network or network addressable connection (e.g., a connection using a web browser) or both.

[0067] Next, referring to FIG. 6, a set of functional abstraction layers 600 provided by cloud computing environment 50 is shown. It should be understood in advance that the components, layers, and functions shown in FIG. 6 are for illustration only and that embodiments of the present invention are not limited thereto. As illustrated, the following layers and corresponding functions are provided.

[0068] Hardware and software layer 60 includes hardware components and software components. Examples of hardware components include mainframe 61, RISC (Reduced Instruction Set Computer) architecture-based server 62, server 63, blade server 64, storage device 65, and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0069] Virtualization layer 70 provides an abstraction layer that can provide the following examples of virtual entities: virtual server 71, virtual storage 72, virtual network 73 including virtual private network, virtual applications and operating systems 74, and virtual client 75.

[0070] In one example, the management layer 80 can provide the functions described below. Resource provisioning 81 performs dynamic procurement of computing resources and other resources used to execute tasks within a cloud computing environment. Measurement and pricing 82 performs cost tracking when resources are utilized within a cloud computing environment, and billing or invoicing for the utilization of those resources. In one example, these resources can include application software licenses. Security performs identity verification of cloud users and tasks, and protects data and other resources. The user portal 83 provides access to the cloud computing environment to users and system administrators. Service level management 84 performs allocation and management of cloud computing resources to meet the required service levels. Service level agreement (SLA) planning and execution 85 performs advance preparation and procurement of cloud computing resources for which future demands are anticipated, in accordance with the SLA.

[0071] The workload layer 90 provides examples of functionality that can be utilized in a cloud computing environment. Examples of workloads and functions that can result from this layer include: mapping and navigation 91, software development and life cycle management 92, virtual classroom education delivery 93, data analysis processing 94, transaction processing 95, and resolution of visibility discrepancies of virtual objects in XR devices 96. Resolution of visibility discrepancies of virtual objects in XR devices 96 may be related to identifying a target virtual object based on one or more captured actions of one or more users to determine whether an owner of the target virtual object is sharing the target virtual object with one or more users who cannot see the target virtual object.

[0072] The description of various embodiments of the present invention is presented for illustrative purposes and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements found in the marketplace, or to enable other skilled artisans to understand the embodiments disclosed herein.

Claims

1. A computer-based method for resolving a visibility discrepancy of virtual objects in an extended reality device, comprising: receiving, during an extended reality conference, one or more virtual objects and settings of each virtual object from one or more users; identifying an owner of each virtual object based on the settings of each virtual object; identifying a target virtual object based on one or more captured actions of the one or more users; determining whether the target virtual object is visible only to the owner based on the one or more captured actions and the settings associated with the target virtual object; in response to determining that the target virtual object is visible only to the owner, displaying a warning notification to the owner before sharing the target virtual object; and querying the owner as to whether to share the target virtual object with the one or more users other than the owner. A computer-based method as described above.

2. determining whether the owner intends to share the target virtual object with the one or more users other than the owner; in response to determining that the owner intends to share the target virtual object, sharing the target virtual object with the one or more users other than the owner. The computer-based method according to Claim 1, further comprising the above steps.

3. In response to determining that the target virtual object is visible to users other than the owner, displaying the warning notification to the owner before sharing the target virtual object with one or more unauthorized users; and querying the owner as to whether to share the target virtual object with the one or more unauthorized users. The computer-based method according to Claim 1, further comprising the above steps.

4. determining whether the owner intends to share the target virtual object with the one or more unauthorized users. In response to determining that the owner is about to share the virtual object of interest with the one or more unauthorized users, sharing the virtual object of interest with the one or more unauthorized users The computer-based method according to claim 3, further comprising:

5. The computer-based method according to claim 1, wherein the setting is selected from the group consisting of a private virtual object, an exclusively shared virtual object, and a shared virtual object.

6. The computer-based method according to claim 1, wherein the warning notification is a change in the appearance of the virtual object of interest.

7. The computer-based method according to claim 1, wherein the captured action is selected from the group consisting of hand movements of the one or more users, eye lines of the one or more users, and voices of the one or more users.

8. A computer system, comprising: One or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored in at least one of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, the computer system comprising: Receiving one or more virtual objects and settings of each virtual object from one or more users during an extended reality conference; Identifying an owner of each virtual object based on the setting of each virtual object; Identifying a virtual object of interest based on one or more captured actions of the one or more users; Determining whether the virtual object of interest is visible only to the owner based on the one or more captured actions and the setting associated with the virtual object of interest; In response to determining that the virtual object of interest is visible only to the owner, Displaying a warning notification to the owner before sharing the virtual object of interest inquiring of the owner whether to share the virtual object of interest with the one or more users other than the owner A computer system capable of executing a method including this.

9. determining whether the owner desires to share the virtual object of interest with the one or more users other than the owner; in response to determining that the owner desires to share the virtual object of interest, sharing the virtual object of interest with the one or more users other than the owner The computer system according to claim 8, further including this.

10. in response to determining that the virtual object of interest is visible to others than the owner, displaying the warning notification to the owner before sharing the virtual object of interest with one or more unauthorized users; inquiring of the owner whether to share the virtual object of interest with the one or more unauthorized users The computer system according to claim 8, further including this.

11. determining whether the owner desires to share the virtual object of interest with the one or more unauthorized users; in response to determining that the owner desires to share the virtual object of interest, sharing the virtual object of interest with the one or more unauthorized users The computer system according to claim 10, further including this.

12. The computer system according to claim 8, wherein the setting is selected from the group consisting of a private virtual object, an exclusively shared virtual object, and a shared virtual object.

13. The computer system according to claim 8, wherein the warning notification is a change in the appearance of the virtual object of interest.

14. The computer system according to claim 8, wherein the captured action is selected from the group consisting of the hand movement of the one or more users, the eye line of the one or more users, and the voice of the one or more users.

15. A computer program product One or more computer-readable tangible storage media, and program instructions stored in at least one of the one or more computer-readable tangible storage media, the program instructions being executable by a processor capable of executing a method, the method comprising receiving, during an extended reality session, one or more virtual objects and settings for each virtual object from one or more users identifying an owner of each virtual object based on the settings of each virtual object identifying a reference virtual object based on one or more captured actions of the one or more users determining whether the reference virtual object is visible only to the owner based on the one or more captured actions and the settings associated with the reference virtual object in response to determining that the reference virtual object is visible only to the owner displaying a warning notification to the owner before sharing the reference virtual object querying the owner as to whether to share the reference virtual object with the one or more users other than the owner A computer program product comprising determining whether the owner desires to share the reference virtual object with the one or more users other than the owner in response to determining that the owner desires to share the reference virtual object, sharing the reference virtual object with the one or more users other than the owner The computer program product according to claim 15, further comprising in response to determining that the reference virtual object is visible to others than the owner displaying the warning notification to the owner before sharing the reference virtual object with one or more unauthorized users querying the owner as to whether to share the reference virtual object with the one or more unauthorized users The computer program product according to claim 15, further comprising ​ ​ ​ Determining whether the owner intends to share the virtual object of interest with the one or more unauthorized users; In response to determining that the owner intends to share the virtual object of interest with the one or more unauthorized users, sharing the virtual object of interest with the one or more unauthorized users The computer program product according to claim 17, further comprising. **Claim 19** The computer program product according to claim 15, wherein the setting is selected from the group consisting of a private virtual object, an exclusively shared virtual object, and a shared virtual object. **Claim 20** The computer program product according to claim 15, wherein the warning notification is a change in the appearance of the virtual object of interest.

Images