Resolving visibility differences for virtual objects in augmented reality devices

Abstract

Embodiments are provided for resolving visibility differences of virtual objects in an augmented reality device. The embodiment may include receiving one or more virtual objects and settings for each virtual object from one or more users during an augmented reality conference. The embodiment may also include identifying an owner of each virtual object. The embodiment may also include identifying the referenced virtual object. The embodiment may also include, in response to determining that the referenced virtual object is only visible to the owner, displaying a warning notification to the owner prior to sharing. The embodiment may also include querying the owner whether to share the referenced virtual object with one or more users other than the owner. The embodiment may also include sharing the referenced virtual object in response to determining that the owner wants to share the referenced virtual object.

Description

Background Art

[0001] The present invention relates generally to the field of computing, and more particularly to a system for resolving visibility discrepancies of virtual objects in an extended reality device.

[0002] Extended reality (XR) devices and software have become popular for both consumers and businesses in recent memory. These devices and software may include XR head-mounted devices (headsets) or XR glasses, and XR games and social applications, respectively. Such XR devices and software may allow a user to simply gaze at a real-world surrounding environment through an XR head-mounted device or glasses, or interact with several other users, and display a completely virtual (e.g., fictional) or enhanced version of the real-world surrounding environment. As the technology continues to improve and impact the daily interactions that individuals have with each other, the demand for XR devices and software is expected to increase in the coming years. Summary of the invention

[0003] According to one embodiment, a method, computer system, and computer program product for resolving visibility differences of virtual objects in an extended reality device are provided. The embodiment may include receiving one or more virtual objects and settings for each virtual object from one or more users during an extended reality meeting. The embodiment may also include identifying the owner of each virtual object based on the settings for each virtual object. The embodiment may also include identifying the referenced virtual object based on one or more captured actions of one or more users. The embodiment may also include displaying a warning notification to the owner before sharing the referenced virtual object in response to determining that the referenced virtual object is visible only to the owner. The embodiment may also include asking the owner whether to share the referenced virtual object with one or more users other than the owner. The embodiment may also include sharing the referenced virtual object with one or more users other than the owner in response to determining that the owner wants to share the referenced virtual object. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof read in conjunction with the accompanying drawings. The various features of the drawings are not to scale, as for the sake of clarity, the illustrations are intended to facilitate understanding of the present invention by those skilled in the art in conjunction with the specific embodiments. In the drawings:

[0005] Figure 1 An exemplary networked computer environment is shown in accordance with at least one embodiment.

[0006] Figure 2A and Figure 2BAn operational flow diagram for resolving visibility differences of virtual objects in an extended reality (XR) device during a visibility difference resolution process is shown in accordance with at least one embodiment.

[0007] Figure 3 is a diagram depicting interactions between solution components of the process in FIG. 2 according to at least one embodiment.

[0008] Figure 4 According to at least one embodiment Figure 1 A functional block diagram of the internal and external components of a computer and server is depicted in FIG.

[0009] Figure 5 A cloud computing environment according to an embodiment of the present invention is depicted.

[0010] Figure 6 Abstract model layers according to an embodiment of the present invention are depicted. DETAILED DESCRIPTION

[0011] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it is understood that the disclosed embodiments are merely illustrative of the claimed structures and methods that may be implemented in various forms. However, the present invention may be implemented in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0012] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces unless the context clearly dictates otherwise.

[0013] Embodiments of the present invention relate to the field of computing, and more specifically to systems for resolving visibility differences of virtual objects in extended reality (XR) devices. The exemplary embodiments described below provide a system, method, and program product for identifying a referenced virtual object based on one or more captured actions of one or more users, and accordingly determining whether the owner of the referenced virtual object wants to share the referenced virtual object with one or more users who cannot see the referenced virtual object. Therefore, the present embodiments have the ability to improve XR technology by dynamically detecting and resolving visibility differences between participants during an XR conference.

[0014] As previously mentioned, XR devices and software have become popular for both consumers and businesses in recent memory. These devices and software may include XR head-mounted devices or XR glasses, as well as XR games and social applications, respectively. Such XR devices and software may allow users to simply gaze at the real-world surroundings through XR head-mounted devices or glasses, or interact with several other users, and display a completely virtual (e.g., fictional) or enhanced version of the real-world surroundings. As technology continues to improve and affect the daily interactions that individuals have with each other, the demand for XR devices and software is expected to increase in the next few years. During an XR collaborative meeting, virtual objects may not be seen by all participants. For example, one user may see a virtual car, but another user may not see the virtual car. This problem is usually solved by making virtual objects visible to all participants. However, making these virtual objects visible to all participants exposes sensitive information about the enterprise and / or one or more participants.

[0015] Therefore, it may be necessary to have a system in place to detect and resolve visibility differences between participants during an XR conference. Therefore, embodiments of the present invention may provide advantages including, but not limited to, dynamically detecting and resolving visibility differences between participants during an XR conference, notifying the owner of a referenced virtual object of the state of the referenced virtual object before sharing the referenced virtual object, and allowing the owner of the referenced virtual object to select which participants can view the referenced virtual object. The present invention does not require that all advantages need to be incorporated into every embodiment of the present invention.

[0016] According to at least one embodiment, during an XR conference, one or more virtual objects and settings for each virtual object may be received from one or more users so as to identify the owner of each virtual object based on the settings for each virtual object. After identifying the owner, the referenced virtual object may be identified based on one or more captured actions of the one or more users. According to at least one embodiment, in response to determining that the referenced virtual object is visible only to the owner, a warning notification may be displayed to the owner before sharing the referenced virtual object, so that the owner may be asked whether to share the referenced virtual object with one or more users other than the owner. In response to determining that the owner wants to share the referenced virtual object, the referenced virtual object may be shared with one or more users other than the owner.

[0017] According to at least one other embodiment, in response to determining that the referenced virtual object is not visible only to the owner, a warning notification can be displayed to the owner before sharing the referenced virtual object with one or more unauthorized users, so that the owner can be asked whether to share the referenced virtual object with the one or more unauthorized users. In response to determining that the owner wants to share the referenced virtual object with the one or more unauthorized users, the referenced virtual object can be shared with the one or more unauthorized users.

[0018] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, and the computer-readable program instructions are used to cause a processor to perform various aspects of the present invention.

[0019] Computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage medium includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device (such as a punch card or a convex structure in a groove) with instructions recorded thereon, and any suitable combination of the foregoing. Computer readable storage medium as used herein should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse transmitted by an optical fiber cable), or an electrical signal transmitted by a wire.

[0020] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0021] The computer-readable program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform various aspects of the present invention, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute the computer-readable program instructions by personalizing the electronic circuit using the state information of the computer-readable program instructions.

[0022] Various aspects of the present invention are described herein with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to embodiments of the present invention. It will be understood that each frame of the flow chart and / or block diagram and the combination of frames in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0023] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0024] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0025] The flow charts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations of possible implementations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each frame in the flow chart or block diagram may represent a module, segment or portion of an instruction, which includes one or more executable instructions for implementing a specified (one or more) logical function. In some alternative implementations, the functions noted in the frame may not occur in the order noted in the figure. For example, the two frames shown in succession may actually be performed simultaneously or substantially simultaneously, or these frames may sometimes be performed in reverse order, depending on the functions involved. It will also be noted that each frame of the block diagram and / or flow chart and the combination of frames in the block diagram and / or flow chart may be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0026] The exemplary embodiments described below provide a system, method, and program product for identifying a referenced virtual object based on one or more captured actions of one or more users, and accordingly determining whether the owner of the referenced virtual object wants to share the referenced virtual object with one or more users who cannot see the referenced virtual object.

[0027] refer to Figure 1 , depicts an exemplary networked computer environment 100 according to at least one embodiment. The networked computer environment 100 may include a client computing device 102, a server 112, and an Internet of Things (IoT) device 118 interconnected via a communication network 114. According to at least one implementation, the networked computer environment 100 may include multiple client computing devices 102 and servers 112, only one of each of which is shown for simplicity of illustration.

[0028] The communication network 114 may include various types of communication networks, such as a wide area network (WAN), a local area network (LAN), a telecommunications network, a wireless network, a public switching network, and / or a satellite network. The communication network 114 may include connections such as wired, wireless communication links, or fiber optic cables. It is understood that Figure 1 This merely provides an illustration of one implementation and does not imply any limitation on the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0029] According to one embodiment of the present invention, the client computing device 102 may include a processor 104 and a data storage device 106, which is capable of hosting and running a software program 108 and an XR collaboration program 110A, and communicating with a server 112 and an IoT device 118 via a communication network 114. The client computing device 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop, a tablet computer, a desktop computer, or any type of computing device capable of running a program and accessing a network. Figure 4 As discussed, the client computing device 102 can include internal components 402a and external components 404a, respectively.

[0030] According to an embodiment of the present invention, the server computer 112 may be a laptop computer, a netbook computer, a personal computer (PC), a desktop computer, or any programmable electronic device or any network of programmable electronic devices that can host and run the XR collaboration program 110B and the database 116, and communicate with the client computing device 102 and the IoT device 118 via the communication network 114. Figure 4 As discussed, the server computer 112 may include internal components 402b and external components 404b, respectively. The server 112 may also operate in a cloud computing service model, such as software as a service (SaaS), platform as a service (PaaS), or infrastructure as a service (IaaS). The server 112 may also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0031] The IoT device 118 may be an eye tracking sensor, a motion sensor, an XR device such as XR smart glasses and / or an XR head mounted device, a microphone embedded in or external to the client computing device 102, and / or any other IoT device 118 known in the art for tracking body movements and / or capturing audio output that is capable of connecting to the communication network 114 and sending and receiving data with the client computing device 102 and the server 112.

[0032] According to the present embodiment, the XR collaboration program 110A, 110B may be a program that is capable of receiving one or more virtual objects and settings for each virtual object from one or more users during an XR conference, identifying a referenced virtual object based on one or more captured actions of the one or more users, determining whether the owner of the referenced virtual object wants to share the referenced virtual object with one or more users who cannot see the referenced virtual object, dynamically detecting and resolving visibility differences between participants (i.e., users) during the XR conference, notifying the owner of the referenced virtual object of the state of the referenced virtual object before sharing the referenced virtual object, and allowing the owner of the referenced virtual object to select which participants can view the referenced virtual object. The XR collaboration method will be described below with reference to Figure 2A and Figure 2B Further detailed explanation.

[0033] Reference now Figure 2A and Figure 2B , depicts an operational flow chart for resolving visibility differences of virtual objects in XR devices in a visibility difference resolution process 200 according to at least one embodiment. At 202, the XR collaboration programs 110A, 110B receive one or more virtual objects and settings for each virtual object from one or more users during an XR conference. Using the software program 108, one or more users can arbitrarily create one or more virtual objects and place them in a 3D XR space. For example, "User 1" can create a virtual car and place it on top of their computer keyboard, and "User 2" can create a virtual animal and place it on top of their TV stand or table. It will be understood that the above examples are not intended to be limiting, and in various embodiments of the present invention, virtual objects can represent different real-world objects and can be placed in a variety of different locations.

[0034] Once one or more virtual objects are created, the user who created each virtual object can associate settings with the virtual object. According to at least one embodiment, the settings can be private virtual objects. According to at least one other embodiment, the settings can be exclusive shared virtual objects. According to at least one other embodiment, the settings can be shared virtual objects. The settings for each virtual object are described in further detail below with respect to step 204.

[0035] Then, at 204, the XR collaboration programs 110A, 110B identify the owner of each virtual object. The owner is identified based on the settings for each virtual object described above with respect to step 202. As used herein, the "owner" of each virtual object is the user who created the virtual object and associated settings with the virtual object. Continuing with the example in step 202 above, "User 1" may be the owner of a virtual car, and "User 2" may be the owner of a virtual animal. As described above with respect to step 202, the owner may associate settings with the virtual object.

[0036] In an embodiment of a virtual object where the setting is private, the virtual object may initially be visible only to the owner. For example, where "User 1" makes the setting for the virtual car private, "User 1" may be the only user in the XR conference who can see the virtual car. In an embodiment of a virtual object where the setting is exclusively shared, the virtual object may initially be visible to the owner and at least one other user specified by the owner. For example, where "User 1" makes the setting for the virtual car exclusively shared, "User 1" may specify "User 3," "User 6," and "User 10" as "authorized users" who can see the virtual object with the owner (i.e., "User 1"). In an embodiment of a virtual object where the setting is shared, the virtual object may be visible to all users participating in the XR conference. For example, where "User 2" makes the setting for the virtual animal shared, each of the one or more users participating in the XR conference will be able to see the virtual object.

[0037] Next, at 206, the XR collaboration program 110A, 110B identifies a referenced virtual object. The referenced virtual object is identified based on one or more captured actions of one or more users. The referenced virtual object can be any virtual object referenced by one or more users during the XR conference via one or more captured actions, which will be described in more detail below.

[0038] According to at least one embodiment, the captured action may be the voice of one or more users. The voice of one or more users may be captured by an IoT device 118, which includes but is not limited to an XR device (such as XR smart glasses or an XR head-mounted device) and / or a microphone. Specifically, the voice of one or more users may reference a specific virtual object in the surrounding environment of one or more users. The XR collaboration program 110A, 110B may convert the voice of one or more users into text using a voice recognition module, which is described below with reference to Figure 3 The speech analyzer module then performs natural language processing (NLP) on the text to identify objects referenced in the text, and the speech analyzer module is also referenced below. Figure 3 Further details are shown and described. For example, "User 1" may say, "The car in the middle of the room is an electric car". In this example, the referenced virtual object may be the car in the middle of the room. In another example, "User 2" may say, "The animal on my TV stand is part of an endangered species". In this example, the referenced virtual object may be the animal on the TV stand. In addition, when one or more virtual objects are shared or exclusively shared, users other than the owner may reference the virtual objects. For example, "User 3" may say, "The helicopter on the table in the middle of the room was produced in what year". In this example, the referenced virtual object may be the helicopter.

[0039] According to at least one other embodiment, the speech of one or more users alone may not be sufficient to identify the referenced virtual object. For example, "User 1" may say, "That car over there is an electric car." However, there may be multiple virtual cars in the surrounding environment of "User 1." In this embodiment, the captured action may be the line of sight of one or more users. The line of sight may be captured by IoT devices 118, which include but are not limited to XR devices (such as XR smart glasses or XR head-mounted devices) and / or eye tracking sensors, which are described below with reference to Figure 3 Shown and described in further detail. Continuing with the example above, "User 1" may be looking directly at an electric car on a table in the middle of the room. In this example, the referenced virtual object may be the electric car on the table in the middle of the room, rather than the two other electric cars on either side of the room. In addition, when one or more virtual objects are shared or exclusively shared, users other than the owner may reference the virtual object. For example, "User 3" may say, "In which year was the helicopter on the table in the middle of the room produced." However, there may be multiple virtual helicopters in the surrounding environment that "User 3" is looking at. In this example, "User 3's" line of sight may indicate that "User 3" is looking directly at the helicopter on the TV stand, and the referenced virtual object may be the helicopter on the TV stand.

[0040] According to at least one additional embodiment, the voice and sight of one or more users may not be sufficient to identify the referenced virtual object. For example, "User 2" may say, "The animal over there can be seen in many places across the country." However, there may be multiple virtual animals in the surrounding environment of "User 2," and these virtual animals may be within a few inches of each other in the surrounding environment. In this embodiment, the captured action may be the hand and / or finger movement of one or more users. The hand and / or finger movement may be captured by the IoT device 118, which includes, but is not limited to, an XR device (such as an XR smart glasses or an XR head-mounted device) and / or a motion sensor. Continuing with the above example, "User 2" may be looking directly at three virtual animals on a table in the middle of the room. When one or more virtual objects are shared or exclusively shared, "User 2" and / or another user physically located near "User 2" may point directly to the virtual animal all the way to the right. In this example, the referenced virtual object may be the animal all the way to the right, rather than the two other animals to the left of the referenced animal. It will be appreciated that the above examples are not intended to be limiting, and in various embodiments of the present invention, the referenced virtual objects may represent different real-world objects and may be placed in a variety of different locations.

[0041] Then, at 208, the XR collaboration program 110A, 110B determines whether the referenced virtual object is visible only to the owner. The determination is made based on the one or more captured actions and settings associated with the referenced virtual object.

[0042] As described above with respect to step 204, the settings for each virtual object may be private, exclusively shared, or shared. The settings are applied to the referenced virtual object selected by one or more users in one or more virtual objects. In the case where the referenced virtual object is a private virtual object, the referenced virtual object may initially be visible only to the owner. For example, in the case where "user 1" creates the referenced virtual car and makes settings for the referenced virtual car, "user 1" may be the only user in the XR conference who can see the referenced virtual car. In the case where the referenced virtual object is an exclusively shared virtual object, the referenced virtual object may initially be visible to the owner and at least one other user specified by the owner. For example, in the case where "user 1" creates the referenced virtual car and makes settings for the referenced virtual car, "user 1" may specify "user 3", "user 6", and "user 10" as "authorized users" who can see the referenced virtual object with the owner (ie, "user 1"). In the case where the referenced virtual object is a shared virtual object, the referenced virtual object may be visible to all users participating in the XR conference. For example, in a case where “User 2 ” creates the referenced virtual animal and makes settings for the referenced virtual animal, each of the one or more users participating in the XR conference will be able to see the referenced virtual animal.

[0043] One or more captured actions may support a determination as to whether a referenced virtual object is visible only to the owner. For example, due to a technical malfunction, a private virtual object may be displayed to a user other than the owner. The other user may see the virtual object and reference the virtual object through voice, sight, and / or hand or finger movement.

[0044] In response to determining that the referenced virtual object is visible only to the owner (step 208, "yes" branch), the visibility difference resolution process 200 proceeds to step 210 to display a warning notification to the owner before sharing the referenced virtual object. In response to determining that the referenced virtual object is not visible only to the owner (step 208, "no" branch), the visibility difference resolution process 200 proceeds to step 218 to display a warning notification to the owner before sharing the referenced virtual object with one or more unauthorized users.

[0045] Next, at 210, the XR collaboration programs 110A, 110B display a warning notification to the owner before sharing the referenced virtual object. According to at least one embodiment, the warning notification may be a visual cue displayed to the owner. Examples of visual cues include, but are not limited to, changes in the appearance of the referenced virtual object (e.g., changes in color, shape, transparency, lighting, and / or outline enhancement) and / or a pop-up message (e.g., a pop-up with text stating “User 3 cannot see the referenced virtual object”). According to at least one other embodiment, the warning notification may be an audio cue spoken to the owner. For example, the audio output may be “User 3 cannot see the referenced virtual object.”

[0046] According to at least one other embodiment, the presence and location of the referenced virtual object can be notified to one or more users other than the owner by displaying a placeholder for the referenced virtual object to these one or more other users without actually seeing the referenced virtual object. Examples of placeholders include, but are not limited to, cloud bubbles and / or semi-transparent objects with blurred outlines at the location of the referenced virtual object. In this embodiment, the placeholder can be displayed to one or more users other than the owner before sharing the referenced virtual object.

[0047] Then, at 212, the XR collaboration program 110A, 110B asks the owner whether to share the referenced virtual object with one or more users other than the owner. When displaying the warning notification, the XR collaboration program 110A, 110B may prompt the owner with an audio and / or visual message, asking the owner whether to share the referenced virtual object with one or more users other than the owner. For example, the pop-up message may ask the owner: "Do you want to share the referenced virtual object with your fellow participants?"

[0048] Next, at 214, the XR collaboration program 110A, 110B determines whether the owner wants to share the referenced virtual object with one or more users other than the owner. The determination may be made based on the owner's response to the inquiry presented in step 212. Continuing with the example described above with respect to step 212, the owner's response to the pop-up message may be "yes" or "no". According to at least one embodiment, when the owner wants to share the referenced virtual object, the owner may set a time limit on sharing the referenced virtual object. For example, the owner may decide to share the referenced virtual object for five minutes.

[0049] In response to determining that the owner wants to share the referenced virtual object (step 214, "yes" branch), visibility difference resolution process 200 proceeds to step 216 to share the referenced virtual object with one or more users other than the owner. In response to determining that the owner does not want to share the referenced virtual object (step 214, "no" branch), visibility difference resolution process 200 ends and the referenced virtual object is not shared.

[0050] Then, at 216, the XR collaboration program 110A, 110B shares the referenced virtual object with one or more users other than the owner. The referenced virtual object may be shared by creating a private digital copy of the referenced virtual object and sharing the digital copy via the XR devices of one or more other users. For example, where the private referenced virtual object is a virtual car, the digital copy of the referenced virtual car may be displayed to one or more other users via the XR devices. According to at least one embodiment, where the owner decides to share the referenced virtual object for five minutes, the referenced virtual object may be displayed to the one or more other users for only five minutes.

[0051] Next, at 218, the XR collaboration program 110A, 110B displays a warning notification to the owner before sharing the referenced virtual object with one or more unauthorized users. As used herein, an "unauthorized user" may be any user who does not currently have permission to view the referenced virtual object. For example, in the case where "User 1" makes the setting for the virtual car exclusive sharing, "User 1" may designate "User 3", "User 6", and "User 10" as "authorized users", and any user not designated will be an unauthorized user. According to at least one embodiment, as described above with respect to step 210, the warning notification may be a visual prompt displayed to the owner. Examples of visual prompts include, but are not limited to, changes in the appearance of the referenced virtual object (e.g., changes in color, shape, transparency, lighting, and / or contour enhancement) and / or a pop-up message (e.g., a pop-up with text stating "User 4 cannot see the referenced virtual object"). According to at least one other embodiment, the warning notification may be an audio prompt spoken to the owner. For example, the audio output may be "User 4 cannot see the referenced virtual object".

[0052] According to at least one other embodiment, the one or more unauthorized users may be notified of the presence and location of the referenced virtual object by displaying a placeholder for the referenced virtual object to the one or more unauthorized users without actually seeing the referenced virtual object. Examples of placeholders include, but are not limited to, cloud bubbles and / or translucent objects with blurred outlines at the location of the referenced virtual object. In this embodiment, the placeholder may be displayed to the one or more unauthorized users before the referenced virtual object is shared with the one or more unauthorized users.

[0053] Then, at 220, the XR collaboration program 110A, 110B asks the owner whether to share the referenced virtual object with one or more unauthorized users. When displaying the warning notification, the XR collaboration program 110A, 110B may prompt the owner with an audio and / or visual message, asking the owner whether to share the referenced virtual object with one or more unauthorized users. For example, the pop-up message may ask the owner: "Do you want to share the referenced virtual object with any users that you have not previously specified?"

[0054] Next, at 222, the XR collaboration program 110A, 110B determines whether the owner wants to share the referenced virtual object with one or more unauthorized users. The determination may be made based on the owner's response to the inquiry presented in step 220. Continuing with the example described above with respect to step 220, the owner's response to the pop-up message may be "yes" or "no". According to at least one embodiment, when the owner wants to share the referenced virtual object, the owner may set a time limit on sharing the referenced virtual object. For example, the owner may decide to share the referenced virtual object for five minutes.

[0055] In response to determining that the owner wants to share the referenced virtual object with one or more unauthorized users (step 222, "yes" branch), visibility difference resolution process 200 proceeds to step 224 to share the referenced virtual object with the one or more unauthorized users. In response to determining that the owner does not want to share the referenced virtual object with the one or more unauthorized users (step 222, "no" branch), visibility difference resolution process 200 ends and does not share the referenced virtual object with the one or more unauthorized users.

[0056] Then, at 224, the XR collaboration program 110A, 110B shares the referenced virtual object with one or more unauthorized users. The referenced virtual object can be shared by changing the setting of the referenced virtual object from exclusive sharing to sharing and mapping the referenced virtual object to the XR devices of one or more unauthorized users. It will be understood that in various embodiments of the present invention, when the setting for the referenced virtual object is changed to sharing, one or more unauthorized users become one or more authorized users for the particular referenced virtual object. For example, in the case where the referenced virtual object shared exclusively is a virtual car, the referenced virtual car can be mapped to the XR device of a previously unauthorized user, who is now an authorized user. According to at least one embodiment, in the case where the owner decides to share the referenced virtual object for five minutes, the referenced virtual object can be displayed to the now authorized user for only five minutes.

[0057] Reference now Figure 3, a diagram 300 of an operation example of the interaction between the solution components of the process in FIG. 2 is shown according to at least one embodiment. In diagram 300, an extended reality collaboration service 302 may provide a spatial model 304A of a 3D XR space. The spatial model 304A may include a shared virtual model 304B and / or a private virtual model 304C. In the shared virtual model 304B, the settings of one or more virtual objects 305 may be shared or exclusively shared. In the private virtual model 304C, the settings of one or more virtual objects 305 may be private and currently visible only to the owner 306. According to at least one embodiment, the owner 306 may speak into a microphone 308, and a speech recognition module 310 may convert the speech into text. The text may then be fed to a speech analyzer module 312, which may perform NLP on the text to detect one or more words that reference an object. For example, the speech analyzer module 312 may analyze the text "The car in the middle of the room is an electric car" and determine that the car is the referenced virtual object. According to at least one other embodiment, when the referenced virtual object cannot be accurately identified only by speech, the eye tracking sensor 314 can track the gaze of the owner 306 to help identify the referenced virtual object. In this embodiment, once the referenced virtual object is identified, the visibility checker module 316 can be used to determine whether the referenced virtual object is visible only to the owner 306 or to the owner 306 and at least one other user (i.e., authorized user). In response to determining that the owner 306 wants to share the referenced virtual object that is currently visible only to the owner 306, the model updater module 318 can update the state of the referenced virtual object from private to shared. In response to determining that the owner 306 wants to share the referenced virtual object that is currently visible to the owner and at least one other user (i.e., authorized user), the model updater module 318 can update the state of the referenced virtual object from exclusive sharing to shared.

[0058] Understandably, Figure 2A , Figure 2B and Figure 3 This merely provides an illustration of one implementation and does not imply any limitation on how different embodiments may be implemented.Many modifications may be made to the described environment based on design and implementation requirements.

[0059] Figure 4 According to an embodiment of the present invention Figure 1 4 is a block diagram 400 of the internal and external components of the client computing device 102 and the server 112 depicted in FIG. It should be understood that Figure 4This merely provides an illustration of one implementation and does not imply any limitation on the environments in which different embodiments may be implemented. Many modifications to the described environment may be made based on design and implementation requirements.

[0060] Data processing systems 402, 404 represent any electronic device capable of executing machine-readable program instructions. Data processing systems 402, 404 may represent smart phones, computer systems, PDAs, or other electronic devices. Examples of computing systems, environments, and / or configurations that may be represented by data processing systems 402, 404 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0061] The client computing device 102 and the server 112 may include Figure 4 , and the corresponding sets of internal components 402a, 402b and external components 404a, 404b shown in . Each set of internal components 402 includes one or more processors 420 on one or more buses 426, one or more computer-readable RAMs 422, and one or more computer-readable ROMs 424, as well as one or more operating systems 428 and one or more computer-readable tangible storage devices 430. One or more operating systems 428, software programs 108, and XR collaboration programs 110A in the client computing device 102, and the XR collaboration program 110B in the server 112 are stored on one or more corresponding computer-readable tangible storage devices 430 for execution by one or more corresponding processors 420 via one or more corresponding RAMs 422 (typically including cache memory). In Figure 4 In the embodiment shown in , each of the computer-readable tangible storage devices 430 is a disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices 430 is a semiconductor memory device such as ROM 424, EPROM, flash memory or any other computer-readable tangible storage device that can store computer programs and digital information.

[0062] Each set of internal components 402a, 402b also includes a R / W drive or interface 432 to read from and write to one or more portable computer-readable tangible storage devices 438, such as a CD-ROM, DVD, memory stick, tape, disk, optical disk, or semiconductor storage device. Software programs such as the XR collaboration program 110A, 110B can be stored on one or more corresponding portable computer-readable tangible storage devices 438, read via the corresponding R / W drive or interface 432, and loaded into the corresponding hard drive 430.

[0063] Each set of internal components 402a, 402b also includes a network adapter or interface 436, such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G or 4G wireless interface card, or other wired or wireless communication link. The software program 108 and the XR collaboration program 110A in the client computing device 102 and the XR collaboration program 110B in the server 112 can be downloaded to the client computing device 102 and the server 112 from an external computer via a network (e.g., the Internet, a local area network or other wide area network) and a corresponding network adapter or interface 436. From the network adapter or interface 436, the software program 108 and the XR collaboration program 110A in the client computing device 102 and the XR collaboration program 110B in the server 112 are loaded into the corresponding hard disk drive 430. The network may include copper wires, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers.

[0064] Each set of external components 404a, 404b may include a computer display monitor 444, a keyboard 442, and a computer mouse 434. External components 404a, 404b may also include touch screens, virtual keyboards, touch pads, pointing devices, and other human-computer interface devices. Each set of internal components 402a, 402b also includes a device driver 440 to interface to a computer display monitor 444, a keyboard 442, and a computer mouse 434. Device driver 440, R / W driver or interface 432, and network adapter or interface 436 include hardware and software (stored in storage device 430 and / or ROM 424).

[0065] It is understood in advance that although the present disclosure includes detailed descriptions about cloud computing, the implementation of the teachings recorded herein is not limited to a cloud computing environment. Instead, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0066] Cloud computing is a service delivery model for enabling convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the provider of the service. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0067] Features are as follows:

[0068] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capacity (such as server time and network storage) as needed without requiring manual interaction with the service provider.

[0069] Broad Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0070] Resource pooling: The computing resources of a provider are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is location independence in the sense that consumers generally do not control or know the exact location of the provided resources, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0071] Rapid elasticity: Capacity can be provisioned quickly and elastically, and in some cases automatically, to scale out quickly and released quickly to scale in quickly. To the consumer, the capacity available for provisioning often appears to be unlimited and can be purchased in any quantity at any time.

[0072] Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0073] The service model is as follows:

[0074] Software as a Service (SaaS): The capability provided to consumers is to use the provider's applications running on the cloud infrastructure. The applications are accessible from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0075] Platform as a Service (PaaS): The capability provided to consumers is to deploy consumer-created or acquired applications onto the cloud infrastructure, where the applications are created using programming languages ​​and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possible configuration of the application hosting environment.

[0076] Infrastructure as a Service (IaaS): The capability provided to consumers is the provisioning of processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0077] The deployment model is as follows:

[0078] Private Cloud: Cloud infrastructure is operated only for the organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0079] Community cloud: The cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0080] Public cloud: Cloud infrastructure is available to the general public or large industry groups and is owned by an organization that sells cloud services.

[0081] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0082] The cloud computing environment is service-oriented, with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is the infrastructure consisting of a network of interconnected nodes.

[0083] Reference now Figure 5, depicts an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 100, with which local computing devices used by cloud consumers can communicate, the local computing devices such as, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 100 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure as a service, platform as a service, and / or software as a service, for which cloud consumers do not need to maintain resources on local computing devices. It should be understood that Figure 5 The types of computing devices 54A-54N shown in are intended to be illustrative only, and computing node 100 and cloud computing environment 50 may communicate with any type of computerized device over any type of network and / or network addressable connection (eg, using a web browser).

[0084] Reference now Figure 6 , shows a set of functional abstraction layers 600 provided by the cloud computing environment 50. It should be understood in advance that Figure 6 The components, layers, and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0085] The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include: mainframes 61; servers based on RISC (Reduced Instruction Set Computer) architecture 62; servers 63; blade servers 64; storage devices 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0086] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71 ; virtual storage 72 ; virtual networks 73 , including virtual private networks; virtual applications and operating systems 74 ; and virtual clients 75 .

[0087] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources for performing tasks within a cloud computing environment. Metering and pricing 82 provides cost tracking when resources are utilized within a cloud computing environment, as well as billing or invoicing for consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that the required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-scheduling and procurement of cloud computing resources, where future demand is anticipated based on the SLA.

[0088] The workload layer 90 provides examples of functionality that can take advantage of a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics processing 94; transaction processing 95; and resolving visibility differences of virtual objects in XR devices 96. Resolving visibility differences of virtual objects in XR devices 96 can involve identifying a referenced virtual object based on one or more captured actions of one or more users to determine whether the owner of the referenced virtual object wants to share the referenced virtual object with one or more users who cannot see the referenced virtual object.

[0089] The description of various embodiments of the present invention has been given for the purpose of illustration, but it is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found on the market, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A computer-based method for resolving visibility differences of virtual objects in an extended reality device, the method comprising: receiving one or more virtual objects and settings for each virtual object from one or more users during an extended reality conference; identifying an owner of each virtual object based on settings for each virtual object; identifying the referenced virtual object based on the one or more captured actions of the one or more users; determining whether the referenced virtual object is visible only to the owner based on the one or more captured actions and settings associated with the referenced virtual object; as well as In response to determining that the referenced virtual object is visible only to the owner: displaying a warning notification to the owner before sharing the referenced virtual object; as well as The owner is asked whether to share the referenced virtual object with the one or more users other than the owner.

2. The computer-based method of claim 1 , further comprising: determining whether the owner wants to share the referenced virtual object with the one or more users other than the owner; as well as In response to determining that the owner wants to share the referenced virtual object, the referenced virtual object is shared with the one or more users other than the owner.

3. The computer-based method of claim 1 , further comprising: In response to determining that the referenced virtual object is not visible only to the owner: displaying the warning notification to the owner before sharing the referenced virtual object with one or more unauthorized users; as well as The owner is asked whether to share the referenced virtual object with the one or more unauthorized users.

4. The computer-based method of claim 3, further comprising: determining whether the owner wants to share the referenced virtual object with the one or more unauthorized users; as well as In response to determining that the owner wants to share the referenced virtual object with the one or more unauthorized users, the referenced virtual object is shared with the one or more unauthorized users.

5. The computer-based method of claim 1, wherein: The setting is selected from the group consisting of a private virtual object, an exclusively shared virtual object, and a shared virtual object.

6. The computer-based method of claim 1, wherein: The warning notification is a change in the appearance of the referenced virtual object.

7. The computer-based method of claim 1, wherein: The captured motion is selected from the group consisting of hand movement of the one or more users, sight of the one or more users, and voice of the one or more users.

8. A computer system, comprising: One or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored on at least one of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing a method comprising: receiving one or more virtual objects and settings for each virtual object from one or more users during an extended reality conference; identifying an owner of each virtual object based on settings for each virtual object; identifying the referenced virtual object based on the one or more captured actions of the one or more users; determining whether the referenced virtual object is visible only to the owner based on the one or more captured actions and settings associated with the referenced virtual object; and In response to determining that the referenced virtual object is visible only to the owner: displaying a warning notification to the owner before sharing the referenced virtual object; and The owner is asked whether to share the referenced virtual object with the one or more users other than the owner.

9. The computer system of claim 8, further comprising: determining whether the owner wants to share the referenced virtual object with the one or more users other than the owner; as well as In response to determining that the owner wants to share the referenced virtual object, the referenced virtual object is shared with the one or more users other than the owner.

10. The computer system according to claim 8, further comprising: In response to determining that the referenced virtual object is not visible only to the owner: displaying the warning notification to the owner before sharing the referenced virtual object with one or more unauthorized users; as well as The owner is asked whether to share the referenced virtual object with the one or more unauthorized users.

11. The computer system of claim 10, further comprising: determining whether the owner wants to share the referenced virtual object with the one or more unauthorized users; as well as In response to determining that the owner wants to share the referenced virtual object with the one or more unauthorized users, the referenced virtual object is shared with the one or more unauthorized users.

12. The computer system according to claim 8, wherein: The setting is selected from the group consisting of a private virtual object, an exclusively shared virtual object, and a shared virtual object.

13. The computer system according to claim 8, wherein: The warning notification is a change in the appearance of the referenced virtual object.

14. The computer system according to claim 8, wherein: The captured motion is selected from the group consisting of hand movement of the one or more users, sight of the one or more users, and voice of the one or more users.

15. A computer program product, the computer program product comprising: One or more computer-readable tangible storage media and program instructions stored on at least one of the one or more computer-readable tangible storage media, the program instructions being executable by a processor capable of performing a method comprising: receiving one or more virtual objects and settings for each virtual object from one or more users during an extended reality conference; identifying an owner of each virtual object based on settings for each virtual object; identifying the referenced virtual object based on the one or more captured actions of the one or more users; determining whether the referenced virtual object is visible only to the owner based on the one or more captured actions and settings associated with the referenced virtual object; and In response to determining that the referenced virtual object is visible only to the owner: displaying a warning notification to the owner before sharing the referenced virtual object; and The owner is asked whether to share the referenced virtual object with the one or more users other than the owner.

16. The computer program product of claim 15, further comprising: determining whether the owner wants to share the referenced virtual object with the one or more users other than the owner; as well as In response to determining that the owner wants to share the referenced virtual object, the referenced virtual object is shared with the one or more users other than the owner.

17. The computer program product of claim 15, further comprising: In response to determining that the referenced virtual object is not visible only to the owner: displaying the warning notification to the owner before sharing the referenced virtual object with one or more unauthorized users; as well as The owner is asked whether to share the referenced virtual object with the one or more unauthorized users.

18. The computer program product of claim 17, further comprising: determining whether the owner wants to share the referenced virtual object with the one or more unauthorized users; as well as In response to determining that the owner wants to share the referenced virtual object with the one or more unauthorized users, the referenced virtual object is shared with the one or more unauthorized users.

19. The computer program product of claim 15, wherein: The setting is selected from the group consisting of a private virtual object, an exclusively shared virtual object, and a shared virtual object.

20. The computer program product of claim 15, wherein: The warning notification is a change in the appearance of the referenced virtual object.

Images