Spatially interactive zones for 3D interactive environments

Abstract

A method according to some embodiments includes obtaining scene description data for a 3D scene. The scene description data includes primitive information describing at least one primitive object in the scene. The primitive information includes a first boundary parameter, and the primitive information is associated with at least one action triggered by an interaction with the primitive object. An interaction region is determined, the interaction region being a volume between a first surface and a second surface. The first surface is a surface of the primitive object, the second surface is a surface of a scaled version of the primitive object, where the scaling is performed by a factor indicated by the first boundary parameter. In response to an interaction within the determined interaction region, at least one action is performed in the scene.

Description

Cross-references to related applications

[0001] This application claims priority to European Patent Application No. 23306753.7, filed on October 10, 2023, entitled “Primitive Boundaries for Interactive Environments,” which is incorporated herein by reference in its entirety. Background Technology

[0002] This disclosure relates to 3D scene and object interaction in immersive environments.

[0003] Various technologies can be used to generate, process, and render virtual 3D scenes. The information characterizing a 3D scene (called a scene description) can be time-dependent, allowing the 3D scene to change in a manner similar to video playback. This behavior can be achieved by relying on the framework defined in the MPEG media scene description document, namely Information Technology—Encoded Representation of Immersive Media—Part 14: MPEG Media Scene Description, ISO / IEC DIS 23090-14:2021 (E). A scene update mechanism based on the JSON Patch protocol defined in IETF RFC 6902 can be used to synchronize virtual content to the MPEG media stream. Summary of the Invention

[0004] A method according to some embodiments includes acquiring scene description data for a 3D scene. The scene description data includes primitive information describing at least one primitive object in the scene. The primitive information includes a first boundary parameter and is associated with at least one action triggered by interaction with the primitive object. An interaction region is determined, which is a volume between a first surface and a second surface. The first surface is a surface of the primitive object, and the second surface is a scaled version of the surface of the primitive object, wherein the scaling is performed according to a factor indicated by the first boundary parameter. In response to an interaction within the determined interaction region, at least one action is performed in the scene.

[0005] In some embodiments, the primitive information is further associated with a flag, which may be referred to as a phantom parameter, indicating whether the primitive object is considered a collisionable object. A collisionable object can be created based on the primitive information in response to determining that the flag indicates the primitive object is considered a collisionable object.

[0006] In some embodiments, the primitive object is one of the following: a cuboid, a planar region, a cylindrical region, a capsule-shaped region, or a sphere.

[0007] In some embodiments, the first surface and the second surface have a common centroid.

[0008] In some embodiments, interactions within the defined interaction area include scene elements having positions within the predetermined volume.

[0009] In some embodiments, the primitive information describes at least a second primitive object in the scene, the primitive information including a second boundary parameter associated with the second primitive object. In some such embodiments, the method further includes deactivating the second primitive object in response to determining that the second boundary parameter is equal to one.

[0010] In some embodiments, scaling includes scaling each linear dimension of the primitive object by an amount proportional to the first boundary parameter. For example, in embodiments where the primitive object has width, height, and depth, scaling may include scaling each of the width, height, and depth by an amount proportional to the first boundary parameter. Where the primitive object has at least a radius, scaling may include scaling the radius by an amount proportional to the first boundary parameter.

[0011] An apparatus according to some embodiments includes one or more processors configured to perform any of the methods described herein.

[0012] A computer-readable medium according to some embodiments includes instructions for causing one or more processors to perform any of the methods described herein. The computer-readable medium may be a non-transitory storage medium.

[0013] A computer program product according to some embodiments includes instructions that, when executed by the program by one or more processors, cause the one or more processors to perform any of the methods described herein.

[0014] According to some embodiments, the signal includes scene description data for a 3D scene, wherein the scene description data includes: primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, the primitive information further being associated with at least one action triggered by interaction with the primitive object; and information associating the primitive object with the at least one action.

[0015] A computer-readable medium according to some embodiments includes scene description data for a 3D scene, wherein the scene description data includes: primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, the primitive information further being associated with at least one action triggered by interaction with the primitive object; and information associating the primitive object with the at least one action.

[0016] The method according to some embodiments includes encoding scene description data for a 3D scene, wherein the scene description data includes: primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, the primitive information further being associated with at least one action triggered by an interaction with the primitive object; and information associating the primitive object with the at least one action. Attached Figure Description

[0017] Figures 1A to 1D The diagram schematically illustrates primitive objects and associated boundary surfaces based on different values ​​of boundary parameters.

[0018] Figure 2 This is a flowchart illustrating a method performed in some embodiments.

[0019] Figure 3 An example of an MPEG-I node hierarchy that supports scene interactivity elements is shown.

[0020] Figure 4 This is a functional block diagram of an apparatus for implementing some embodiments. Detailed Implementation

[0021] In extended reality (XR) applications, scene descriptions are used to combine an explicit and easily parsed description of the scene structure with a binary representation of some media content. This disclosure describes a mechanism for correctly handling and defining interactive regions of basic primitives in a 3D scene.

[0022] In time-based media streaming, the scene description itself can evolve over time to provide relevant virtual content for each sequence of the media stream. For example, for advertising purposes, a virtual bottle could be displayed during a video sequence of people drinking a beverage.

[0023] This behavior can be achieved by relying on the framework defined in the MPEG media scene description document, namely Information Technology—Encoding Representation of Immersive Media—Part 14: MPEG Media Scene Description, ISO / IEC DIS 23090-14:2021 (E). A scene update mechanism based on the JSON Patch protocol defined in IETF RFC 6902 can be used to synchronize virtual content to the MPEG media stream.

[0024] In addition to using timed media, the MPEG-I scene description framework can be used to provide a description of how users interact with scene objects at runtime to obtain an immersive XR experience. Figure 3 An example of an MPEG-I node hierarchy supporting scene interactivity elements is shown. In an exemplary embodiment, as... Figure 3The scene interactivity extensions shown can have the following semantics as shown in Table 1.

[0025]

[0026] To provide interactivity within a scene description, triggers can be defined within the scene description, and actions can be associated with those triggers. Types of actions that can be performed in response to triggers include, for example, activating or disabling nodes, transforming nodes, performing animations, playing or stopping media content, placing nodes in a location, enabling user manipulation of nodes, changing the material of nodes, or changing the lighting properties in the scene.

[0027] In some cases, triggers can be activated by proximity or collision conditions. Proximity conditions are satisfied when a scene element falls within a predetermined volume. This volume can be defined at least partially by a primitive, which can be a 3D geometry such as a cuboid, planar region, cylindrical region, capsule-shaped region, or sphere. In some embodiments, collision conditions are satisfied when a scene element collides with this primitive.

[0028] This disclosure relates to a processing model that can be used to support primitive boundaries in interactive 3D environments (e.g., in MPEG-I scene description (SD)) with and without a physics engine.

[0029] Current interactive frameworks allow basic primitives to be attached to trigger functions to initiate predefined events in immersive experiences. Primitives contain boundary volumes that represent the area of ​​interaction with other objects. This area is responsible for activating triggers and initiating events. However, known implementations do not provide clarity regarding the behavior of different triggers, such as proximity and collision. Collision triggers utilize physics engine properties, and known frameworks do not explicitly state what behavior primitives and boundaries should exhibit in this case.

[0030] In this disclosure, exemplary embodiments provide a mechanism to clearly define and implement the behavior of primitive boundaries in the presence of proximity triggers and collision triggers.

[0031] Examples of interactive and known primitive object definitions in glTF are shown in Table 2.

[0032]

[0033] Boundaries define interactive regions. A zero value for the boundary parameter indicates that the entire volume of the primitives is interactive, although the metric for this number was not previously explicitly defined if the value is greater than zero. Furthermore, the definition of the upper bound of the boundary was previously unclear and undefined.

[0034] Boundaries can behave differently in terms of interacting with objects, depending on the triggers to which they are attached. For example, a collision trigger that utilizes physical properties can make a primitive boundary a rigid body and allow it to collide with other objects in the scene. This behavior differs from a proximity trigger, where the primitive boundary is not collision-prone and acts as a region for initiating events. These behaviors were not previously well-defined and could lead to inconsistencies in interpreting primitive functionality in immersive systems.

[0035] The exemplary embodiments described herein provide systems and methods for defining and manipulating boundaries in 3D scenes. In the exemplary embodiments, the boundaries of primitives are defined with respect to interactivity triggers, as shown in Table 3.

[0036]

[0037] In this example, the "phantom" property, when set to "False," transforms the primitive object into a rigid body, allowing the physics engine to treat it as such. If the "phantom" property is set to "True" (which can be the default value), it allows objects to pass through without considering any rigid body collisions. An example use of the phantom flag is to trigger an event when a collision mesh penetrates a phantom mesh. Defining the volume as the "zone" with which it collides may be more practical than defining a single or multiple values. Game engines with physics engines can use this system to transmit events to the application.

[0038] The "boundary" property can be used to provide a ratio or scaling value for the original primitive size. If the "boundary" property has a value of zero, the entire volume of the primitive is valid for trigger activation. If the value is one, the boundary is scaled to the size of the primitive volume, and as a result, no boundary is displayed and there is no region for trigger activation, so the primitive does not activate the trigger. If the value is greater than one, the boundary grows outward along the normal direction (like linear scaling of any 3D object), and therefore, the region for trigger activation maintains the volume between the boundary and the primitive's limits.

[0039] The following Python code is an illustrative example of generating "BV_CUBOID" primitives and boundaries. The method indicated by this code can be implemented using any other programming language.

[0040]

[0041]

[0042]

[0043]

[0044] As described above, a proximity-based trigger can be satisfied when a scene element falls within a predetermined volume (which may be referred to as the interaction region). In an exemplary embodiment, the predetermined volume is defined in a scene description file using information defining primitives as shown in Table 2 above. This primitive is associated with a first surface, which may be referred to as a primitive boundary, representing the outer surface of the shape defined by the primitive (e.g., the outer surface of a cuboid when the primitive is a cuboid). The primitive is further associated with a second surface, which may be referred to as a boundary. In an exemplary embodiment, the boundary is a scaled version of the primitive boundary. Scaling can be performed along each axis by a factor indicated by the parameter "boundary" associated with the primitive in the scene description file. In an exemplary embodiment, the boundary and the primitive share a common centroid. The interaction region can be defined by the volume between the boundary surface and the primitive surface.

[0045] As an example, Figures 1A to 1D The cuboid primitive 100 is shown with centroid 102 and a “boundary” parameter with four different values. Figure 1A The case where the "boundary" parameter is equal to zero is shown. In this case, the boundary surface does not actually exist, and the entire volume within the cuboid primitive 100 is considered the interaction region.

[0046] exist Figure 1B In the example, the "boundary" parameter is 0.5. As a result, the boundary surface 104 is scaled down by a factor of 0.5 relative to the cuboid primitive 100, but both share the same centroid 102. In this case, the interaction region is the volume within the cuboid primitive 100 but outside the boundary 104. In a 3D scene, an action can be triggered by the presence of an appropriate scene element within this interaction region, but not necessarily by the presence of that scene element outside the cuboid primitive 100 or within the boundary 104.

[0047] exist Figure 1C In the example, the "boundary" parameter is 1.0. As a result, the boundary surface has the same size and shape as the cuboid primitive 100, with no space between them. This can be considered as a situation where there is no interaction area, and the primitive can be deactivated so that it no longer acts as a trigger in the scene.

[0048] exist Figure 1DIn the example, the "boundary" parameter is 2.0. As a result, the boundary surface 106 is scaled up relative to the cuboid primitive 100 by a factor of 2.0, but both share the same centroid 102. In this case, the interaction region is the volume outside the cuboid primitive 100 but within the boundary 106. In a 3D scene, an action can be triggered by the presence of an appropriate scene element within this interaction region, but not necessarily by the presence of that scene element within the cuboid primitive 100 or outside the boundary 106.

[0049] Although Figures 1A to 1D The example is presented using cuboid primitives, but it should be understood that similar scaling can be applied to other primitive types.

[0050] In some embodiments, perform as follows Figure 2 The method is as follows: Obtain a scene description file, which includes information defining interactive triggers and primitives associated with those triggers. Primitive processing begins as shown at 202. Based on the boundary parameters associated with the primitive, a boundary is created at 204. If the boundary parameters are zero as determined at 206, processing continues at 208, where the entire volume enclosed by the primitive is used as the interactive region. If the boundary parameters are one as determined at 210, the primitive can be deactivated at 212 because no interactive region exists. Otherwise, the primitive can be scaled according to the boundary parameters, and a new object can be created in the scene, the type of which (collisive or non-collisive) is determined based on whether the "phantom" parameter is true at 214. If the "phantom" parameter is set to "false", the boundary of the new object can be scaled at 216, and the object can be implemented as a collideable object at 218. Alternatively, if “phantom” is set to “true”, the boundaries of the new object can be scaled at 220, and the object can be implemented as a non-collisionable object at 222.

[0051] Extended reality display devices, along with their control electronics, can be used for example Figure 4 It is implemented through systems like the system itself. Figure 4This is a block diagram illustrating examples of systems implementing various aspects and embodiments. System 1000 may be embodied as a device including various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, networked home appliances, and servers. Elements of system 1000 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete devices. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete devices. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described herein.

[0052] System 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing various aspects, such as those described herein. Processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0053] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide an encoded or decoded scene, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents multiple modules that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both encoding and decoding modules. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or it may be incorporated within processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0054] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described herein may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of a variety of items during the execution of the processes described herein. Such stored items may include, but are not limited to, input scenes, decoded scenes or portions thereof, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0055] In some embodiments, the memory within processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing during encoding or decoding. However, in other embodiments, external memory (e.g., processor 1010 or encoder / decoder module 1030) is used for one or more of these functions. External memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for encoding and decoding operations, such as those for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Various Video Coding, a new standard being developed by the Joint Video Experts Group JVET).

[0056] Inputs to the components of System 1000 can be provided by a variety of input devices as indicated in box 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) component (COMP) input terminals (or a set of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals.

[0057] In various embodiments, the input device of block 1130 has corresponding input processing elements as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal); (ii) down-converting the selected signal; (iii) band limiting the signal again to a narrower band to select, for example, a signal band that may be referred to as a channel in some embodiments; (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements (e.g., frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers) to perform these functions. The RF section may include a tuner that performs a variety of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via wired (e.g., cable) media and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In many embodiments, the RF section includes an antenna.

[0058] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, on demand within a separate input processing IC or within the processor 1010. Similarly, aspects of USB or HDMI interface processing may be implemented on demand within a separate interface IC or within the processor 1010. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements (e.g., including the processor 1010, and an encoder / decoder 1030 operating in conjunction with memory and storage elements) to process the data stream on demand for presentation on the output device.

[0059] Various components of system 1000 can be provided within an integrated housing. Within this integrated housing, various components can be interconnected and transmit data therebetween using suitable connection means 1140, such as internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards.

[0060] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, within wired and / or wireless media.

[0061] In various embodiments, data is provided to system 1000 via streaming or otherwise using a wireless network such as Wi-Fi (e.g., IEEE 802.11, where IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, Wi-Fi signals are received via a communication channel 1060 and a communication interface 1050 adapted for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top services to communicate. Other embodiments use a set-top box that delivers data via an HDMI connection of input box 1130 to provide streaming data to system 1000. Still other embodiments use an RF connection of input box 1130 to provide streaming data to system 1000. As indicated above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0062] System 1000 can provide output signals to a variety of output devices, including display 1100, speaker 1110, and other peripheral devices 1120. Display 1100 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. Display 1100 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. Display 1100 can also be integrated with other components (e.g., as in smartphones) or stand alone (e.g., as an external monitor for a laptop computer). In examples of various embodiments, other peripheral devices 1120 include one or more of a standalone digital video disc (or digital versatile disc) (DVR, for both terms), an optical disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, an optical disc player performs the function of playing the output of system 1000.

[0063] In various embodiments, signaling, such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control (with or without user intervention), is used to communicate control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120. Output devices can be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to system 1000 via communication interface 1050 using communication channel 1060. Display 1100 and speaker 1110 can be integrated into a single unit with other components of system 1000 in an electronic device such as a television. In various embodiments, display interface 1070 includes a display driver, such as a timing controller (TCon) chip.

[0064] Display 1100 and speaker 1110 may alternatively be separate from one or more other components, for example, if the RF section of input 1130 is part of a separate set-top box. In various embodiments where display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0065] System 1000 may include one or more sensor devices 1095. Examples of sensor devices that may be used include one or more GPS sensors, gyroscope sensors, accelerometers, light sensors, cameras, depth cameras, microphones, and / or magnetometers. Such sensors can be used to determine information such as the user's position and orientation. Where system 1000 is used as a control module (such as control modules 124, 1254) for an extended reality display, the user's position and orientation can be used to determine how image data is rendered so that the user perceives the correct portion of a virtual object or scene from the correct perspective. In the case of a head-mounted display device, the position and orientation of the device itself can be used to determine the user's position and orientation for the purpose of rendering virtual content. In the case of other display devices (such as telephones, tablets, computer monitors, or televisions), other inputs can be used to determine the user's position and orientation for the purpose of rendering content. For example, the user can use a touchscreen, keypad or keyboard, trackball, joystick, or other inputs to select or adjust the desired viewpoint and / or viewing direction. When the display device has sensors such as accelerometers and / or gyroscopes, the viewpoint and orientation used for rendering content can be selected or adjusted based on the movement of the display device.

[0066] This embodiment can be executed by computer software implemented by processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, this embodiment can be implemented by one or more integrated circuits. Memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. Processor 1010 can be of any type suitable for the technical environment and can encompass one or more of the following: microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as non-limiting examples.

[0067] A method according to some embodiments includes acquiring scene description data for a 3D scene. The scene description data includes primitive information describing at least one primitive object in the scene. The primitive information includes a first boundary parameter and is associated with at least one action triggered by interaction with the primitive object. An interaction region is determined, which is a volume between a first surface and a second surface. The first surface is a surface of the primitive object, and the second surface is a scaled version of the surface of the primitive object, wherein the scaling is performed according to a factor indicated by the first boundary parameter. In response to an interaction within the determined interaction region, at least one action is performed in the scene.

[0068] In some embodiments, the primitive information is further associated with a flag, which may be referred to as a phantom parameter, wherein the flag indicates whether the primitive object is considered a collideable object. A collideable object can be created based on the primitive information in response to determining that the flag indicates the primitive object is considered a collideable object.

[0069] In some embodiments, the primitive object is one of the following: a cuboid, a planar region, a cylindrical region, a capsule-shaped region, or a sphere.

[0070] In some embodiments, the first surface and the second surface have a common centroid.

[0071] In some embodiments, interactions within the defined interaction area include scene elements having positions within the predetermined volume.

[0072] In some embodiments, the primitive information describes at least a second primitive object in the scene, the primitive information including a second boundary parameter associated with the second primitive object. In some such embodiments, the method further includes deactivating the second primitive object in response to determining that the second boundary parameter is equal to one.

[0073] In some embodiments, scaling includes scaling each linear dimension of the primitive object by an amount proportional to the first boundary parameter. For example, in embodiments where the primitive object has width, height, and depth, scaling may include scaling each of the width, height, and depth by an amount proportional to the first boundary parameter. Where the primitive object has at least a radius, scaling may include scaling the radius by an amount proportional to the first boundary parameter.

[0074] An apparatus according to some embodiments includes one or more processors configured to perform any of the methods described herein.

[0075] A computer-readable medium according to some embodiments includes instructions for causing one or more processors to perform any of the methods described herein. The computer-readable medium may be a non-transitory storage medium.

[0076] A computer program product according to some embodiments includes instructions that, when executed by the program by one or more processors, cause the one or more processors to perform any of the methods described herein.

[0077] According to some embodiments, the signal includes scene description data for a 3D scene, wherein the scene description data includes: primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, the primitive information further being associated with at least one action triggered by interaction with the primitive object; and information associating the primitive object with the at least one action.

[0078] A computer-readable medium according to some embodiments includes scene description data for a 3D scene, wherein the scene description data includes: primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, the primitive information further being associated with at least one action triggered by interaction with the primitive object; and information associating the primitive object with the at least one action.

[0079] The method according to some embodiments includes encoding scene description data for a 3D scene, wherein the scene description data includes: primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, the primitive information further being associated with at least one action triggered by an interaction with the primitive object; and information associating the primitive object with the at least one action.

[0080] A method according to some embodiments includes: determining an interactive region in a 3D scene, the interactive region being a volume between a first surface and a second surface, the first surface being a surface of a primitive object in the 3D scene, the second surface being a scaled version of the surface of the primitive object, the scaling being performed according to a factor indicated by a first boundary parameter; and encoding scene description data for the 3D scene, wherein the scene description data includes primitive information describing the primitive object in the scene, the primitive information including the first boundary parameter, wherein the first boundary parameter indicates a scaling factor for the scaled version of the primitive object, and wherein the primitive information is associated with at least one action triggered by interaction with the primitive object.

[0081] This disclosure describes various aspects, including tools, features, embodiments, models, approaches, etc. Many of these aspects are specifically described, and are generally described in a manner that may sound restrictive, at least for the purpose of showing the various features. However, this is for clarity of description and does not limit the scope of this disclosure or those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, the aspects described can also be combined and interchanged with aspects described in previous applications.

[0082] The aspects described and contemplated in this disclosure can be implemented in many different forms. While some embodiments are specifically shown, other embodiments are contemplated, and the discussion of particular embodiments does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the methods, and / or computer-readable storage media storing bitstreams generated according to any of the methods.

[0083] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first," "second," etc., can be used in various embodiments to modify elements, components, steps, operations, etc., such as "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding need not be performed before the second decoding, and can occur, for example, before, during, or in a time period overlapping with the second decoding.

[0084] For example, various numerical values ​​may be used in this disclosure. Specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0085] The embodiments described herein can be implemented by computer software, either by a processor or other hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. The processor can be of any type suitable for the technical environment and can encompass one or more of the following: microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as a non-limiting example.

[0086] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding apparatus. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0087] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the features under discussion can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, in appropriate hardware, software, and firmware. Methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0088] References to “an embodiment” or “an embodiment” or “an implementation” or “an implementation” and other variations thereof mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases “in an embodiment” or “in an embodiment” or “in an implementation” or “in an implementation” and any other variations thereof appearing throughout this disclosure do not necessarily all refer to the same embodiment.

[0089] Furthermore, this disclosure may involve "determining" various types of information. Determining information may include one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory.

[0090] Furthermore, this disclosure may relate to "accessing" various types of information. Accessing information may include one or more of the following: receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0091] Additionally, this disclosure may relate to "receiving" various types of information. "Receiving," like "accessing," is intended to be a broad term. Receiving information may include one or more of the following: accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" generally refers to, in some way during operation, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0092] It should be recognized that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the words “ / ,” “and / or,” and “at least one of…” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). This can be extended to any number of items listed.

[0093] Furthermore, as used herein, the term "signal" specifically refers to instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals a specific parameter among several parameters used for region-based filter parameter selection for artifact removal filtering. Thus, in embodiments, the same parameter is used at both the encoder and decoder sides. Therefore, for example, the encoder can transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use that same specific parameter. Conversely, if the decoder already has that specific parameter as well as other parameters, it can simply allow the decoder to know and select that specific parameter without transmission (implicit signaling). Bit savings are achieved in many embodiments by avoiding the transmission of any actual functionality. It should be recognized that signaling can be implemented in various ways. For example, in many embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. While the foregoing refers to the verb form of the term "signal," the term "signal" can also be used herein as a noun.

[0094] Implementations can generate various signals that are formatted to carry information that can be stored or transmitted, for example. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding the data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal can be transmitted via a variety of different wired or wireless links as known. The signal may be stored on a processor-readable medium.

[0095] We have described several embodiments. Features of these embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, across various claim classes and types: • A bitstream or signal that includes one or more of the described syntax elements, or variations thereof.

[0096] • A bitstream or signal that includes a syntax for conveying information generated according to any of the described embodiments.

[0097] • Create and / or transmit and / or receive and / or decode bitstreams or signals, including one or more of the described syntax elements, or variations thereof.

[0098] • Creation and / or transmission and / or reception and / or decoding according to any of the described embodiments.

[0099] • A method, process, apparatus, medium for storing instructions, medium for storing data, or signal according to any of the embodiments described.

[0100] Note that the various hardware elements of one or more embodiments described herein are referred to as “modules” that perform (i.e., carry out, implement, etc.) the various functions described herein in connection with the respective modules. As used herein, a module includes hardware considered suitable for a given implementation (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more memory devices). Each of the modules may also include executable instructions for performing one or more functions as described in the description performed by the respective module, and it should be noted that those instructions may employ or include hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and / or the like, and may be stored in any suitable non-transitory computer-readable medium or media, such as commonly referred to as RAM, ROM, etc.

[0101] Although the features and elements have been described above in specific combinations, each feature or element may be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented as computer programs, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROMs and digital versatile optical discs (DVDs)). The processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A method comprising: Acquire scene description data for a 3D scene, wherein the scene description data includes primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, and the primitive information further being associated with at least one action triggered by interaction with the primitive object; Determine an interaction region, which is the volume between a first surface and a second surface, the first surface being the surface of the primitive object, and the second surface being a scaled version of the surface of the primitive object, the scaling being performed according to a factor indicated by the first boundary parameter; as well as The action is performed in response to an interaction within the defined interaction area.

2. An apparatus comprising one or more processors configured to perform at least the following operations: Acquire scene description data for a 3D scene, wherein the scene description data includes primitive information describing at least one primitive object in the scene, the primitive information including a first boundary parameter, and the primitive information further being associated with at least one action triggered by interaction with the primitive object; Define an interaction region, which is the volume between a first surface and a second surface, the first surface being the surface of the primitive object, and the second surface being a scaled version of the surface of the primitive object, the scaling being performed according to a factor indicated by the first boundary parameter; and The action is performed in response to an interaction within the defined interaction area.

3. The method of claim 1, or the apparatus of claim 2, wherein the primitive information is further associated with a phantom parameter, the phantom parameter indicating whether the primitive object is considered a collideable object, further comprising: In response to determining that the phantom parameter indicates that the primitive object is considered a collisionable object, a collisionable object is created based on the primitive information.

4. The method as described in claim 1 or as described in claim 3 when it is dependent on claim 1, or the apparatus as described in claim 2 or as described in claim 3 when it is dependent on claim 2, wherein the primitive object is one of the following: a cuboid, a planar region, a cylindrical region, a capsule-shaped region, and a sphere.

5. The method as described in claim 1 or as described in claims 3-4 when dependent on claim 1, or the apparatus as described in claim 2 or as described in claims 3-4 when dependent on claim 2, wherein the first surface and the second surface have a common centroid.

6. The method as described in claim 1 or as described in claims 3-5 when dependent on claim 1, or the apparatus as described in claim 2 or as described in claims 3-5 when dependent on claim 2, wherein the interaction within the determined interaction area includes moving scene elements to a position within the predetermined volume.

7. The method as described in claim 1 or as described in claims 3-6 when dependent on claim 1, or the apparatus as described in claim 2 or as described in claims 3-6 when dependent on claim 2, wherein the primitive information describes at least a second primitive object in the scene, the primitive information including a second boundary parameter associated with the second primitive object, further comprising: In response to determining that the second boundary parameter is equal to one, the second primitive object is deactivated.

8. The method as described in claim 1 or as described in claims 3-7, which are dependent on claim 1, or the apparatus as described in claim 2 or as described in claims 3-7, which are dependent on claim 2, wherein the scaling comprises scaling each linear dimension of the primitive object by an amount proportional to the first boundary parameter.

9. The method as claimed in claim 1 or as claimed in claims 3-8, which are dependent on claim 1, or the apparatus as claimed in claim 2 or as claimed in claims 3-8, which are dependent on claim 2, wherein the primitive object has a width, a height, and a depth, and wherein the scaling includes scaling each of the width, height, and depth by an amount proportional to the first boundary parameter.

10. The method as claimed in claim 1 or as claimed in claims 3-9, which are dependent on claim 1, or the apparatus as claimed in claim 2 or as claimed in claims 3-9, which are dependent on claim 2, wherein the primitive object has at least a radius, and wherein the scaling includes scaling the radius by an amount proportional to the first boundary parameter.

11. A method comprising: Determine the interactive area in the 3D scene, the interactive area being the volume between a first surface and a second surface, the first surface being the surface of a primitive object in the 3D scene, and the second surface being a scaled version of the surface of the primitive object, the scaling being performed according to a factor indicated by the first boundary parameter; as well as Scene description data for the 3D scene is encoded, wherein the scene description data includes primitive information describing the primitive objects in the scene, the primitive information including a first boundary parameter, wherein the first boundary parameter indicates a scaling factor for a scaled version of the primitive object, and wherein the primitive information is associated with at least one action triggered by interaction with the primitive object.

12. An apparatus comprising one or more processors configured to perform at least the following operations: Determine an interactive region in a 3D scene, the interactive region being the volume between a first surface and a second surface, the first surface being the surface of a primitive object in the 3D scene, and the second surface being a scaled version of the surface of the primitive object, the scaling being performed according to a factor indicated by a first boundary parameter; and Scene description data for the 3D scene is encoded, wherein the scene description data includes primitive information describing the primitive objects in the scene, the primitive information including a first boundary parameter, wherein the first boundary parameter indicates a scaling factor for a scaled version of the primitive object, and wherein the primitive information is associated with at least one action triggered by interaction with the primitive object.

13. The method of claim 11, or the apparatus of claim 12, further comprising encoding a phantom parameter indicating whether the primitive object is considered a collideable object.

14. The method as described in claim 11 or as described in claim 13 when it is dependent on claim 11, or the apparatus as described in claim 12 or as described in claim 13 when it is dependent on claim 12, wherein the primitive object is one of the following: a cuboid, a planar region, a cylindrical region, a capsule-shaped region, and a sphere.

15. The method as claimed in claim 11 or as claimed in claims 13-14, which are dependent on claim 11, or the apparatus as claimed in claim 12 or as claimed in claims 13-14, which are dependent on claim 12, wherein the first surface and the second surface have a common centroid.

Images