Automatic placement and arrangement of content items in a 3D environment

Abstract

A computing device for automatically placing and arranging objects in a computer-based 3D environment is disclosed herein. In one embodiment, the computing device is configured to provide a user interface on a display, the user interface comprising a workspace with a template of a 3D environment and a gallery of models of two-dimensional (2D) or 3D content items. The computing device is then capable of detecting user input via the user interface, the user input selecting one of the models from the gallery to be inserted as an object into the template of the 3D environment. In response to detecting the user input, the computing device is capable of drawing and displaying a graphical representation corresponding to the 2D or 3D content item of the selected model on the display at a position spaced a preset radial distance from a default viewer position of a viewer of the 3D environment along an arc.

Description

Background Art

[0001] In computing, a three-dimensional (3D) environment generally refers to a computer-based simulated 3D platform in which 3D representations of images, videos, sounds, or other digital content can be presented to a viewer. A 3D environment can provide a more immersive viewing experience than a two-dimensional (2D) environment. However, authoring content in a 3D environment remains challenging due to the complexity of accurately representing objects in a 3D environment and interactive control. Due to this complexity, only a small fraction of creators today have the necessary technical skills to create suitable content in a 3D environment. Summary of the invention

[0002] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0003] It is challenging to create content in a 3D environment due to the lack of a full set of creative activities in the creation in a 2D environment. For example, creative activities such as 3D content placement and arrangement, 3D interaction with the placed content items, and motion specifications of 3D content items are lacking in 2D creation. In 2D creation, when an object is inserted into a 2D environment, the inserted object can be extended horizontally, vertically, or both from any existing object. However, such one-dimensional or two-dimensional extensions can cause occlusions in the 3D environment because existing objects can be partially or completely located in front of or behind the inserted object along the depth dimension. Occlusion can cause the created content to be blurred or confusing to the viewer. Even without occlusion, due to the difference in distance from the viewer, the one-dimensional or two-dimensional extension of the inserted object in the 3D environment can make the object look smaller or larger than the existing object. In this way, the uniformity of the appearance of the object may be lacking in the 3D environment.

[0004] To address the above placement challenges, creators of 3D content often need to experiment with a large number of positions through trial and error to determine the best placement for inserted objects in a 3D environment. Such experimentation can be labor-intensive, inconsistent, and may not even produce the best placement and / or arrangement of objects in the 3D environment. When viewing content in a 3D environment, such inaccurate placement and / or arrangement can distract the user's experience or even cause headaches, dizziness, or other negative physiological reactions in the viewer.

[0005] Several embodiments of the disclosed technology can address at least some aspects of the above challenges by automatically placing and arranging supported 2D or 3D content items as objects in a 3D environment, taking into account a combination of the following: (i) the depth perception of the viewer; (ii) the field of view of the viewer; (iii) the relative position of an object with respect to adjacent objects in the 3D environment; and (iv) the field of view of the AR / VR / MR head mounted device. An environment data file can then be generated based on the automatically placed objects to create a file containing 3D content that can be distributed to other computing devices for use in reproducing the 3D environment.

[0006] In an exemplary implementation, the authoring application can be configured to provide a template of a 3D environment having, for example, a background (e.g., a blue sky), a scene (e.g., a mountain), a sound (e.g., the sound of wind blowing), and one or more background objects (e.g., trees on a mountain). The template of the 3D environment can also include one or more anchor points at which content items (e.g., 2D or 3D representations of cars, buses, airplanes, etc.) can be positioned within the 3D environment. The authoring application can be configured to provide a catalog of available 2D or 3D content items as a gallery, list, or other suitable interface format.

[0007] Upon receiving user input selecting a content item from a catalog for insertion into the 3D environment, the authoring application can be configured to resize the selected content item (e.g., height, width, etc.) as needed based on a preset container size, and determine the location of the content item from the viewer in the viewer's field of view based on a preset distance. For example, the preset container size can be a cube with a volume of 1, 1.5, or 2 cubic meters. The preset distance from the viewer's position along the viewer's line of sight can be 2.5, 3.0, 3.5 meters. The authoring application can then place the resized content item as an object in the 3D environment at the preset distance from the viewer in the viewer's field of view.

[0008] The authoring application can also be configured to determine the relative position of the inserted objects relative to other objects in the 3D environment. According to an embodiment of the disclosed technology, it has been recognized that placing objects in a Cartesian coordinate system may not provide a suitable view to the viewer. For example, if objects are arranged in one or two dimensions along a longitudinal or transverse direction, some objects will appear larger than other objects even if the objects are all the same size. In order to address the above challenges, several embodiments of the disclosed technology utilize a cylindrical coordinate system to place objects along an arc around the viewer's position. In this way, each object can be placed at a roughly equal distance from the viewer's position and have corresponding different angle values ​​in the cylindrical coordinate system. Since all objects in a 3D environment generally appear to be the same size, such a placement arrangement can provide a suitable viewing experience for the viewer.

[0009] The authoring application can also be configured to modify the relative position of the inserted object relative to other adjacent objects in the 3D environment in response to the position, position change, or size of the object in the 3D environment. For example, when viewing all objects, the viewer can select one of the objects for an enlarged view. In response to the enlarged size of the selected object, the authoring application can be configured to recalculate the relative position of the adjacent objects by determining the angle occupied by the enlarged object, and move the adjacent objects along an arc while maintaining the original angular interval relative to the enlarged object. For example, the authoring application can determine that the angle occupied by the enlarged object has changed from (-5°-5°) to (-10°-10°). In this way, the authoring application can move the adjacent objects away from the enlarged object by -5° on each side. By moving the adjacent objects in this way, the risk of overlapping the adjacent objects with the enlarged object can be reduced or even eliminated.

[0010] In this way, several embodiments of the disclosed technology can provide a user-friendly authoring environment that allows users to intelligently place supported 2D or 3D content items into a template of a 3D environment. By automatically positioning and / or repositioning the objects in the 3D environment according to a cylindrical coordinate system, several embodiments of the disclosed technology can eliminate repeated trial and error experiments to determine the optimal placement and / or arrangement of the inserted objects, and thereby improve the productivity and user-friendliness of creating 3D content. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a schematic diagram illustrating a computing architecture that enables automatic placement and arrangement of content items in a 3D environment according to an embodiment of the disclosed technology.

[0012] Figures 2A-2D is a diagram illustrating an embodiment according to the disclosed technology Figure 1A partial schematic diagram of specific hardware / software components of a computing architecture.

[0013] Figure 3A and Figure 3B is a schematic diagram illustrating a specific position adjustment in a 3D environment during a specific stage of operation according to an embodiment of the disclosed technology.

[0014] Figure 4-5B is a flow chart illustrating a specific process for automatic placement and arrangement of objects in a 3D environment according to an embodiment of the disclosed technology.

[0015] Figure 6 Is suitable for Figure 1 A computing device that is a specific component of a computing architecture. DETAILED DESCRIPTION

[0016] The following describes specific embodiments of systems, devices, components, modules, routines, data structures, and processes for automatic placement and arrangement of content items in a 3D environment. In the following description, specific details of the components are included to provide a thorough understanding of specific embodiments of the disclosed technology. Those skilled in the relevant art will also understand that the technology can have additional embodiments. It is also possible to not refer to the following embodiments. Figure 1-6 The techniques are practiced without the need for several details of the described embodiments.

[0017] As used herein, a "three-dimensional environment" or "3D environment" generally refers to a computer-based simulated 3D platform in which 2D or 3D representations of images, videos, sounds, or other digital content items can be presented to a viewer. A 3D environment can be a virtual space, such as a virtual reality (VR) world, or can be a real-world space in which content can be displayed or layered on top of the real world, such as via augmented reality (AR), mixed reality (MR), or other VR technologies.

[0018] The term "model" also used in this article generally refers to data representing a two-dimensional (2D) or 3D content item that can be drawn in a 3D environment. Exemplary models can include data representing a virtual room, a virtual scene, or any other sub-part of a virtual world. Exemplary content items can include images, videos, sounds, documents, or other appropriate types of content. The terms "object" or "virtual object" also used in this article generally refer to a visual representation of a 2D or 3D content item drawn in a 3D environment. Exemplary objects can include 3D images, video recordings, etc.

[0019] Aspects of the present disclosure relate to the creation and generation of 3D environments using authoring applications. The 3D environment can contain one or more models that individually include a virtual room, a virtual scene, or any other sub-portion of a virtual world. A user of the authoring application can graphically select supported two-dimensional (2D) or 3D representations of models of content items and insert the selected representations into the 3D environment. In response, the authoring application can automatically determine the placement and / or arrangement of the inserted content items to obtain a suitable view of the corresponding objects in the 3D environment. The 3D environment can then be stored as an environment data file that contains information related to one or more models and / or content items in the 3D environment.

[0020] In certain embodiments, different types of content items can be embedded or included in the 3D environment. Exemplary content types can include 3D objects (e.g., 3D models, graphics, shapes, etc.) or 2D objects (e.g., files, images, presentations, documents, websites, videos, remote resources, etc.), among others. In other embodiments, the 3D environment can be a virtual space, such as a virtual reality (VR) world, or can be a real-world space in which content can be displayed or layered on top of the real world, such as via augmented reality (AR), mixed reality (MR), or other VR technologies. The 3D environment with the inserted model can then be stored as an environment data file, which is then used to reproduce the 3D environment with the 3D rendering of the inserted model.

[0021] Then, the 3D environment creation according to the various aspects disclosed herein can be consumed using a viewing application on a viewing device such as a desktop computer or a smart phone. In one example, the 3D environment can be experienced across a variety of computing devices from low-end devices (e.g., GOOGLECARDBOARD) to high-end devices (e.g., MICROSOFT HOLOLENS, OCULOUS RIFT, HTC VIVE, etc.). By using a desktop or mobile computing device to generate the 3D environment, additional related overhead (e.g., the transmission of all required textures, light maps, audio files, etc.) can be avoided. In addition, device-specific or platform-specific details can be handled by the viewing application, thereby making such details invisible to both the end user and the creator of the 3D environment.

[0022] The viewing application can include computing resources associated with the model used by the environment data file. In some examples, the environment data file can include computing resources used when drawing a 3D environment, or resources that can be retrieved from a server or other remote location. When drawing a 3D environment, the viewing application can identify one or more anchor points when stitching adjacent or connected models specified by the environment data file together in the 3D environment. As an example, a model can include an entry anchor point and an exit anchor point. The entry anchor point can indicate a doorway or other entrance into a model, and the exit anchor point can indicate a doorway or other exit leaving a model. Therefore, when multiple models (e.g., adjacent or connected models) are stitched together, the exit anchor point of the first model can be used to locate the entry anchor point of the second model (and by extension, the second model), thereby creating a continuous combination of models.

[0023] It is challenging to create content in a 3D environment due to the lack of a full set of creative activities in the 2D environment. For example, the placement and arrangement of objects in a 3D environment will be completely different from that in a 2D environment. In 2D creation, when an object is inserted into a 2D environment, the inserted object can be extended horizontally, vertically, or both from any existing object. However, such a one-dimensional or two-dimensional extension can lead to occlusion in the 3D environment because the existing object can be partially or completely in front of the inserted object along the depth dimension. Even without occlusion, the one-dimensional or two-dimensional extension of the inserted object in the 3D environment can make the object appear smaller or larger than the existing object due to the distance from the viewer along the depth dimension. In this way, the uniformity of the appearance of the object is lacking in the 3D environment.

[0024] Several embodiments of the disclosed technology can address at least some aspects of the above challenges by automatically placing and arranging 2D or 3D content items as objects in a 3D environment taking into account a combination of: (i) the viewer's depth perception; (ii) the viewer's field of view; (iii) the relative position of an object with respect to adjacent objects in the 3D environment; and (iv) the field of view of an AR / VR / MR headset. An environment data file can then be generated based on the automatically placed content items to create a file containing 3D content that can be distributed to other computing devices, as described below with reference to Figure 1-6 Described in more detail.

[0025] Figure 1 is a schematic diagram illustrating a computing architecture 100 for automatically placing and arranging content items in a 3D environment according to an embodiment of the disclosed technology. Figure 1As shown in , computing architecture 100 can include an authoring device 102 corresponding to an author 101 and one or more viewing devices 104 corresponding to viewers 103 (shown as a first viewer 103a and a second viewer 103b). Authoring device 102 and viewing device 104 can individually include mobile computing devices, laptop computers, tablet computers, desktop computers, or other suitable types of computing devices. Although for illustration purposes only, the present invention is not limited to the following. Figure 1 Only one authoring device 101 and two viewing devices 103 are shown, but in other embodiments, the computing architecture 100 can facilitate content creation for additional authors 101 and / or viewers 103 with corresponding authoring and viewing devices (not shown). Figure 6 Exemplary configurations of authoring device 102 and viewing device 104 are described in greater detail.

[0026] As in Figure 1 As shown in , the authoring device 102 can include an authoring application 108, a model repository 112 containing data records of the model 107, and an output repository 114 containing data records of the 3D environment data file 109. The authoring application 108 can be configured to provide a user interface 130 representing the 3D environment to the author 101 (e.g., Figure 2A ), which has a 3D workspace to facilitate authoring content in a 3D environment. In certain embodiments, authoring application 108 can be a web-based application accessible by author 101 via a web browser. In other examples, authoring application 108 can be an executable application that can be retrieved and executed by a processor of authoring device 102.

[0027] In one embodiment, the authoring application 108 can be configured to display 2D or 3D representations of one or more models 107 of available content items in a gallery, list, or other suitable format. The author 101 can then select a content item and insert the content item into the provided 3D environment as an object corresponding to the content item. In other embodiments, the authoring application 108 can provide a variety of themes. Different models 107 can be associated with one or more themes, or can change or adapt based on the selected theme (e.g., color, texture, lighting, etc.). As described below with reference to Figures 2A-2D Described in more detail, the authoring application 108 can contain additional modules and routines configured to automatically position and arrange inserted objects, enabling the author 101 to place content items as objects into a 3D environment without extensive trial and error experimentation.

[0028] The model repository 112 can store one or more models 107 that represent corresponding content items that can be used to author a 3D environment. In one example, the models 107 can be associated with one or more themes. When the creator 101 selects a theme, the authoring application 108 can provide one or more models 107 associated with the selected theme. In some examples, a collection of models 107 can be designed so that stitching a model 107 with another model 107 from the same collection can form a seemingly continuous model 107. In other examples, various aspects of the models 107 stored in the model repository 112 can be generated dynamically or programmatically. In certain embodiments, the creator 101 can use the authoring application 108 to create the models 107. In other embodiments, the models 107 can be retrieved from, for example, a third-party supplier of 2D or 3D content items or from other suitable sources.

[0029] In certain embodiments, a model 107 may indicate that a particular aspect may be replaced depending on another model 107 with which the original model 107 can be stitched. As an example, a first model 107 may indicate that a wall or an archway may be replaced with a door. Thus, the entry point of the second model may be stitched with the first model at the door. In other embodiments, other suitable replacement or model generation techniques may be used to generate various models 107.

[0030] The authoring application 108 can also be configured to output the authored 3D environment as an environment data file 109 containing 3D environment data to, for example, an output repository 114. In one implementation, the environment data file 109 can include data associated with the selected model 107 (e.g., model identifier, model name, model type, etc.), positioning information (e.g., coordinates, anchor point identifiers, etc.), content information (e.g., which content should be displayed for one or more anchor points, content to be displayed, references to content, etc.), custom resources (e.g., custom textures, sounds, etc.), and other information. As shown in Figure 1 As shown in , the output repository 114 can be configured to store one or more environment data files 109. As used herein, an "environment data file" can include a file on a file system, an entry in a database, or can be stored using any of a variety of other data storage techniques.

[0031] As in Figure 1As shown in , the viewing devices 104 can each include a viewing application 110, which is configured to generate, view, explore and / or interact with a 3D environment based on an environment data file 109. In one example, the viewing application 110 can be a web-based application accessible using a web browser. In other examples, the viewing application 110 can be an executable application for the viewing device 104. In operation, the viewing application 110 can be configured to evaluate the environment data file 109 to identify one or more models 107 of the 3D environment. If the environment data file 109 references multiple models 107, the models 107 can be stitched together when drawing the 3D environment. The viewing application 110 can fill the drawn 3D environment with content based on the content specified by the environment data file 109. In one example, the viewing application 110 can use any of a variety of 3D rendering engines and can handle device-specific and / or engine-specific implementation details when drawing the 3D environment.

[0032] In certain embodiments, the viewing application 110 can be configured to retrieve the environment data files 109 from the output repository 114, which can be used to generate the 3D environment along with one or more models 107 from the model repository 112. In other embodiments where the viewing application 110 is a locally executed application, the model repository 112 can be stored locally and / or remotely to the viewing device 104 executing the viewing application 110, and at least a portion of the environment data files 109 can be retrieved from the output repository 114. In further embodiments, the environment data files 109 can be streamed or retrieved in chunks from the output repository 114 to the viewing device 104.

[0033] Figures 2A-2D is a diagram illustrating an embodiment according to the disclosed technology Figure 1 A partial schematic diagram of specific hardware / software components of the computing architecture 100. Figure 2A As shown in , the authoring application 108 can include an interface component 120, a resizing component 122, a position component 124, and an output component 126 that are operably coupled to each other. Figure 2A Specific components are shown in FIG. 1 , but in other embodiments, the authoring application 108 can also include an input component or other suitable types of components.

[0034] exist Figure 2AIn the figures and in other figures herein, individual software components, objects, classes, modules and routines may be computer programs, procedures or processes written as source code in C, C++, C#, Java and / or other appropriate programming languages. Components may include, but are not limited to, one or more modules, objects, classes, routines, attributes, processes, threads, executable files, libraries or other components. Components may be in source form or binary form. Components may include aspects of source code (e.g., classes, attributes, procedures, routines) before compilation, compiled binary units (e.g., libraries, executable files) or products (e.g., objects, processes, threads) that are instantiated and used at runtime.

[0035] Components within the system can take different forms within the system. As an example, a system including a first component, a second component, and a third component can include, but is not limited to, a system in which the first component is a property in source code, the second component is a binary compiled library, and the third component is a thread created at runtime. A computer program, procedure, or process can be compiled into an object, intermediate code, or machine code and presented for execution by one or more processors of a personal computer, a network server, a laptop, a smart phone, and / or other suitable computing device.

[0036] Likewise, a component may include a hardware circuit. One of ordinary skill in the art will recognize that hardware may be viewed as petrochemical software, and software may be viewed as liquefied hardware. As just one example, the software instructions in a component may be burned into a programmable logic array circuit, or may be designed as a hardware circuit with an appropriate integrated circuit. Likewise, hardware may be emulated by software. Various implementations of source code, intermediate code, and / or object code and associated data may be stored in a computer memory, which may include a read-only memory, a random access memory, a disk storage medium, an optical storage medium, a flash memory device, and / or other suitable computer-readable storage media in addition to a propagating signal.

[0037] As in Figure 2A As shown in , the interface component 120 of the authoring application 108 can be configured to provide a user interface 130 to facilitate the author 101 to create and / or modify content in the 3D environment. In the illustrated example, the user interface 130 can include a menu bar 131 that includes one or more menu groups, such as "File", "Edit", and "Help". Each of the aforementioned menu groups can be expanded for additional menu items such as "New", "Open", "Save", etc. In other examples, the menu bar 131 can include other suitable types of menu items.

[0038] As in Figure 2A, the user interface 130 can also include a 3D workspace 133 and a gallery 132, which contains a plurality of objects 135 corresponding to the models 107 in the model repository 112. A template 141 of a 3D environment or a previously saved 3D environment corresponding to the environment data file 109 in the output repository 114 can be first loaded into the 3D workspace. In the illustrated example, a template 141 of a 3D environment with mountains, sky, and open ground in front of the mountains is loaded into the 3D workspace 133. The objects 135 include cars, buses, bicycles, and airplanes. In other examples, the template 141 can include a forest, a building, or other suitable types of 3D environments, and the objects can include tents, clothes, or any other suitable types of objects.

[0039] The interface component 120 can also be configured to provide one or more anchor points 137 in the template 141 for placing 2D or 3D objects from, for example, the gallery 132. Figure 2A , anchor point 137 is represented as a cross. In other embodiments, anchor point 137 can also be represented as an arrow, a star, or other suitable representation. In certain embodiments, creator 101 can specify anchor point 137 by, for example, placing anchor point 137 at a location selected by the creator. In other embodiments, anchor point 137 can be automatically determined by interface component 120 at a location determined by position component 124 as described below, and set in the 3D workspace as one of a plurality of default anchor points 137. In further embodiments, interface component 120 can allow creator 101 to place anchor point 137 at a location within a particular limited area of ​​the 3D environment.

[0040] As in Figure 2A , the interface component 120 can also be configured to detect that the author 101 selects an object (e.g., bicycle 135c) from the gallery 132 (e.g., using a cursor 139), and the user inserts the bicycle 135c into the 3D workspace at the anchor point 137, for example, via drag and drop as indicated by the dashed arrow. The interface component 120 can then pass the detected user input to the resizing component 122, which is used to determine whether the selected object 135 needs to be resized.

[0041] In one embodiment, the sizing component 122 can be configured to determine whether the selected object 135 needs to be resized by fitting the selected object 135 into a container of a preset size. For example, in a specific implementation, the sizing component 122 can be configured to fit the bicycle 135c into a cube having a volume of one cubic meter. In other examples, the sizing component 122 can be configured to fit the selected object 135 into a volume of a sphere, cylinder, or other suitable shape having a suitable size.

[0042] In response to determining that the bicycle 135c exceeds the container in at least one dimension, the resizing component 122 can resize the bicycle 135c so that the bicycle 135c fits exactly inside the container. On the other hand, when the bicycle 135c is too small, for example, does not have at least one dimension within 90%, 95% or other suitable thresholds of the corresponding dimension of the container, the resizing component 122 can also enlarge the bicycle 135c to resize it to fit exactly into the container. Therefore, such resizing can draw all inserted objects 135 to be approximately the same size for optimal viewing in the 3D environment.

[0043] After completing the above-described resizing operation, the resizing component 122 can pass control to the position component 124 to determine the position of the inserted object 135. According to embodiments of the disclosed technology, it has been recognized that placing the objects 135 in a Cartesian coordinate system in a 3D environment may not provide a suitable view to the viewer 103. For example, if the objects 135 are arranged along one or two dimensions along a longitudinal or transverse direction, some objects 135 may appear larger than other objects even if the objects have the same size.

[0044] To address the aforementioned challenges, several embodiments of the disclosed technology utilize a cylindrical coordinate system to place objects 135 along an arc 140 around the position of the viewer 103. In this way, each object 135 can be placed at a substantially equal radial distance from the viewer 103, with a relative position relative to the default viewing position (at Figure 2A Since all objects in the 3D environment appear to be roughly the same size, such a placement arrangement can provide a suitable viewing experience for the viewer 103. Figure 2A , the position component 124 can be configured to determine a line of sight 142 for the viewer 103 from a default position of the viewer 103. Upon obtaining the direction of the line of sight 142, the position component 124 can be configured to place the selected bicycle 135c at a preset distance (e.g., 3 meters) along an arc 140 around the viewer 103. In other embodiments, other suitable preset distances may also be used, such as 2.5 meters, 3.5 meters, etc.

[0045] As in Figure 2BAs shown in , when determining the position for the selected bicycle 135c, the position component 124 can be configured to place the bicycle 135c at the determined position at or near the anchor point 137. Once the bicycle 135c is placed in the 3D environment, the author 101 can choose to insert another object 135 (e.g., a car 135a) into the 3D environment near the bicycle 135c at or near another anchor point 137'. In response to this user input, the size adjustment component 122 can be configured to adjust the size of the car 135a as needed, as described above with reference to Figure 2A The location component 124 can then determine a location at a preset distance (eg, 3 meters) from the viewer 103 along the arc 140 based on the line of sight 142 ′ of the viewer 103 .

[0046] According to an embodiment of the disclosed technology, the location component 124 can also be configured to determine whether the inserted object 135 (e.g., car 135a) has any neighboring objects 135 by, for example, searching for cars 135 around the determined location along all three dimensions. In the illustrated example, the location component 124 will detect that the bicycle 135c is nearby. The location component 124 can then be configured to determine whether the car 135a and the bicycle 135c are separated by at least a threshold angle (e.g., 0°, 1°, 2°, 5°, etc.) in the cylindrical coordinate system. Figure 3A-3B Techniques for such determination are described in more detail.

[0047] In response to determining that the car 135a and the bicycle 135c are separated by a threshold angle, the position component 124 can be configured to place the car 135a at the determined position. In response to determining that the car 135a and the bicycle 135c are not separated by a threshold angle, the position component 124 can be configured to move the position of one or both of the car 135a or the bicycle 135c along the arc 140 so that the threshold angle between the car 135a and the bicycle 135c is reached. For example, as in Figure 2C As shown in FIG. 1 , the bicycle 135c can be moved to the right around the viewer 103 so that the car 135a can be placed at a determined position.

[0048] The aforementioned adjacent object arrangement techniques can at least reduce, if not eliminate, the risk of occlusion of object 135. For example, in the example illustrated above, if car 135a and bicycle 135c are placed too close to each other, car 135a may appear to partially overlap bicycle 135c. Such an overlap may cause confusion or other negative effects to viewer 103. By maintaining a threshold angular separation between adjacent objects 135, both car 135a and bicycle 135c can be properly presented in workspace 133. For example, the above operations can be repeated to add, for example, bus 135b to the 3D environment, such as in Figure 2D As shown in .

[0049] As in Figure 2D , all inserted objects 135 (i.e., bus 135b, car 135a, and bicycle 135c) will appear to the viewer 103 as being approximately the same size and at approximately equal distances. Without being bound by theory, it will be appreciated that such an arrangement in the 3D environment can allow for proper or even optimal viewing of the inserted objects 135 in the 3D environment. Thus, by automatically arranging and placing the objects 135 in such a manner, the author 101 can create a usable 3D environment with suitable content items by simply inserting the objects 135 from the gallery 132. Thus, a 3D environment with rich content can be created without the need for labor-intensive trial and error to find suitable locations for objects in the 3D environment.

[0050] When the author 101 completes inserting the object 135 into the 3D environment, the output component 126 can be configured to generate an environment data file 109 to be stored in the output repository 126. The environment data file 210 can contain data representing the template 141 of the 3D environment and the identity, position, size, relative location, or other suitable information of the objects inserted into the template 141.

[0051] Figure 3A and Figure 3B is a schematic diagram illustrating a specific position adjustment in a 3D environment during a specific stage of operation according to an embodiment of the disclosed technology. Figure 3A As shown in , various inserted objects 135 can be placed along the arc 140 in a cylindrical coordinate system, where each object 135 has an angle and a distance relative to the viewer 103 and lies on the same plane. Based on the size of the object 135 and the distance from the viewer 103, the angle (α) of each object can be derived, for example, by the following formula:

[0052]

[0053] Where width is the viewing width of the individual object and distance is the distance from the viewer 103 .

[0054] exist Figure 3A In the illustrated example, the line of sight 142 can represent 0°. Thus, the bus 135b can have an angle of (-10°, -15°); the car 135a can have an angle of (+5°, -5°); and the bicycle 135c can have an angle of (+10°, +15°). Based on the aforementioned angle values, the angular spacing (σ) between the adjacent objects 135 can be derived. For example, in the above example, the angular spacing between the bus 135b and the car 135a is 5°, while the same spacing exists between the car 135a and the bicycle 135c. As described above with reference to Figures 2A-2D As described, the location component 124 ( Figure 2A ) can be configured to compare the angular interval with a threshold and determine whether the corresponding object 135 is moved.

[0055] As in Figure 3B , the viewer 103 can select a more detailed view of the car 135a by, for example, zooming in (and / or zooming in) the car 135a to the viewer 103. As a result, the angle (β) of the car 135a changes, for example, from (+5°, -5°) to (+8°, -8°). In this way, the angular interval (σ) between the car 135a and the bicycle 135c and between the car 135a and the bus 135b is reduced to 3°. According to an embodiment of the disclosed technology, in response to the reduction in the angular interval, the position component 124 can be configured to move the adjacent objects 135, such as the bicycle 135c and the bus 135b, in corresponding directions along the arc 140 so that the angular interval threshold (e.g., 5°) can be maintained, as indicated by the arrow 145.

[0056] Figure 4-5B is a flowchart illustrating a specific process of automatically placing and arranging objects in a 3D environment according to an embodiment of the disclosed technology. Figure 1-2D The process is described in the computing architecture 100 of FIG. 1 , but in other embodiments, the process may also be implemented in a computing architecture having additional and / or different components.

[0057] like Figure 4 As shown, the process 200 of automatically placing and arranging objects in a 3D environment includes: at stage 202, receiving a selection of a model corresponding to a content item to be placed as an object in the 3D environment. In one embodiment, it is possible to detect the author 101 ( Figure 2A ) Drag and drop a graphical representation of a model into the 3D environment to receive selections, such as in Figure 2AIn other embodiments, the selection can be received by detecting an insert command, a copy / paste command, and / or other suitable commands.

[0058] Upon receiving a selection of a model, process 200 can include, at stage 204, resizing the virtual object described by the model. In certain embodiments, resizing the virtual object can initially include, for example, resizing the virtual object using Figure 2A In response to determining that resizing is required, the virtual object can be resized proportionally along three dimensions, for example, so that the object is not too large or too small compared to the container. In other embodiments, resizing the virtual object can include modifying at least one dimension of the object based on a preset value without determining whether the object needs to be resized. Figure 5A Exemplary operations for resizing an object are described in more detail. In other embodiments, resizing a virtual object can be omitted from process 200.

[0059] Then, process 200 can include, at stage 206, positioning the object in the 3D environment. In certain embodiments, the object can be positioned in the 3D environment based on a cylindrical coordinate system. Each object can be placed according to an azimuth, a radial distance, and an altitude. For example, viewer 103 ( Figure 2C ) can be the origin of the coordinate system, and Figure 2C Car 135a( Figure 2C ) is placed at coordinates (0°, 3 meters, 0 meters), corresponding to a position on the same plane as the viewer 103, along the line of sight 142 of the viewer 103, and spaced 3 meters from the viewer 103. In other embodiments, objects can be placed in the 3D environment based on a Cartesian coordinate system, a spherical coordinate system, or other suitable type of coordinate system.

[0060] In a specific implementation, the position of the object can be along the arc 140 ( Figure 2A ), and is spaced radially apart from the viewer 103. Figures 2A-2D As described, it has been recognized that such placement of objects can allow the viewer 103 to optimally view the objects without making some objects appear smaller or larger than other objects. In other implementations, the position of the objects can be along a spherical surface, centered on the viewer 103, or in other suitable ways. Positioning the objects can also include monitoring neighboring objects and adjusting the spacing between the neighboring objects based on a threshold spacing. Figure 5B Example operations for monitoring separation from neighboring objects are described.

[0061] Although the above describes the operation of positioning an object in a 3D environment in the context of receiving a selection of a model, in other embodiments, such positioning operations can be performed before receiving a selection of a model. For example, in certain implementations, once the author specifies a default viewer position, the aforementioned positioning operations can be performed to generate one or more anchor points 137 ( Figure 2A ), for example at a predetermined radial distance. The creator can then select the image from the gallery 132 ( Figure 2A ) to insert an object into a pre-filled anchor point.

[0062] The process 200 can then include a decision stage 208 to determine whether additional input for selecting a model is received. In response to determining that an additional model is selected, the process 200 returns to receive a selection of a model at stage 202. Otherwise, the process 200 can continue to generate an environment data file at stage 210. The environment data file 210 can contain data representing the template 141 of the 3D environment, as well as the identity, location, size, relative position, or other suitable information of objects inserted into the template 141.

[0063] Figure 5A An exemplary operation for resizing a selected model is illustrated. Figure 5A As shown in , the operation can include determining the object size of the object corresponding to the selected model. For example, bicycle 135c ( Figure 2A ) can be expressed as height, width, and length. The values ​​of the aforementioned parameters can then be combined with the container (e.g. Figure 2A ) for comparison.

[0064] As in Figure 5A As shown in , the operation can then include a decision stage 214 to determine whether resizing is required. In one embodiment, resizing is required when at least one value of the height, width, or length exceeds the corresponding value of the container. In another embodiment, resizing is also required when at least one value of the height, width, or length is less than 90%, 95%, or other suitable percentages of the corresponding value of the container. In other embodiments, resizing can be indicated based on other suitable criteria. In response to determining that resizing is required, the operation can include resizing the object according to, for example, the height, width, or length of the container. Then, the operation can include providing the resized object for position determination at stage 218. In response to determining that resizing is not required, the operation can proceed directly to provide the object for position determination.

[0065] Figure 5B An exemplary operation for monitoring and adjusting the position of adjacent objects is illustrated. Figure 5BAs shown in , the operation can include: at stage 222, monitoring the positions of adjacent objects. In one implementation, the interval between two adjacent objects can be represented as an azimuth increment in a cylindrical coordinate system, as described above with reference to Figure 3A and Figure 3B In other embodiments, the interval can also be expressed as a zenith angle and / or a normal angle in a spherical coordinate system, or as other suitable values.

[0066] The operation can then include a decision stage 224 to determine whether the interval is less than a threshold. In one implementation, the threshold can be an interval angle of 0°, 2°, 5°, 10°, or other suitable value, such as in Figure 3A and Figure 3B In other implementations, the threshold can be a separation distance, a separation zenith angle, a separation normal angle, or other suitable values. In response to determining that the separation is less than the threshold, the operation can include moving one or more adjacent objects at stage 226 so that the separation between them is not less than the threshold. Otherwise, the operation reverts to monitoring adjacent positions at stage 222.

[0067] Figure 6 Is suitable for Figure 1 The computing device 300 may be adapted to include specific components of the computing architecture 100. For example, the computing device 300 may be adapted to include a Figure 1 The authoring device 102 or viewing device 104 of the present invention can be a computer system. In a very basic configuration 302, the computing device 300 can include one or more processors 304 and a system memory 306. A memory bus 308 can be used to communicate between the processor 304 and the system memory 306.

[0068] Depending on the desired configuration, the processor 304 can be of any type, including but not limited to: a microprocessor (μR), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. The processor 304 can include a multi-level cache such as a first-level cache 310 and a second-level cache 312, a processor core 314, and registers 316. The exemplary processor core 314 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An exemplary memory controller 318 can also be used with the processor 304, or in some implementations, the memory controller 318 can be an internal part of the processor 304.

[0069] Depending on the desired configuration, system memory 306 can be of any type, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 306 can include an operating system 320, one or more applications 322, and program data 324. Figure 6 The depicted basic configuration 302 is illustrated in FIG. 3 by those components within the inner dashed line.

[0070] The computing device 300 can have additional features or functions and additional interfaces to facilitate communication between the basic configuration 302 and any other devices and interfaces. For example, a bus / interface controller 330 can be used to facilitate communication between the basic configuration 302 and one or more data storage devices 332 via a storage interface bus 334. The data storage device 332 can be a removable storage device 336, a non-removable storage device 338, or a combination thereof. Examples of removable storage devices and non-removable storage devices include magnetic disk devices (such as floppy disk drives and hard disk drives (HDDs), optical disk drives (such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSDs) and tape drives, etc.). Exemplary computer storage media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer readable instructions, data structures, program modules or other data). The term "computer-readable storage medium" or "computer-readable storage device" does not include propagated signals and communication media.

[0071] System memory 306, removable storage device 336, and non-removable storage device 338 are examples of computer-readable storage media. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disks (DVD) or other optical storage, cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by computing device 300. Any such computer-readable storage media can be part of computing device 300. The term "computer-readable storage media" does not include propagated signals and communication media.

[0072] The computing device 300 may also include an interface bus 340 for facilitating communication from various interface devices (e.g., output devices 342, peripheral interfaces 344, and communication devices 346) to the basic configuration 302 via the bus / interface controller 330. Exemplary output devices 342 include a graphics processing unit 348 and an audio processing unit 350, which can be configured to communicate with various external devices (such as a display or speakers) via one or more A / V ports 352. Exemplary peripheral interfaces 344 include a serial interface controller 354 or a parallel interface controller 356, which can be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I / O ports 358. Exemplary communication devices 346 include a network controller 360, which can be arranged to facilitate communication with one or more other computing devices 362 over a network communication link via one or more communication ports 364.

[0073] A network communication link can be an example of a communication medium. Communication media can generally be embodied by computer-readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and can include any information transfer medium. A "modulated data signal" can be a signal having one or more characteristics set or changed in a manner to encode information in a signal. By way of example and not limitation, communication media can include wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term "computer-readable medium" as used herein can include both storage media and communication media.

[0074] The computing device 300 may be implemented as part of a small form factor portable (or mobile) electronic device, such as a cellular phone, a personal data assistant (PDA), a personal media player device, a wireless network viewing device, a personal headset device, a dedicated device, or a hybrid device including any of the above functions. The computing device 300 can also be implemented as a personal computer including laptop and non-laptop configurations.

[0075] For the purpose of illustration, a specific embodiment of the technology has been described above. However, various modifications can be made without departing from the foregoing disclosure. In addition, many elements of one embodiment can be added to or replaced with elements of other embodiments and combined with other embodiments. Therefore, except for the attached claims, the technology is not limited.

Claims

1. A method for automatically placing and arranging objects in a computer-based three-dimensional (3D) environment when authoring content using a computing device having a display and a processor, the method comprising: utilizing the processor of the computing device, providing a template of a 3D environment with a background, scene or sound on the display of the computing device; receiving user input selecting a model representing a two-dimensional (2D) or 3D content item to be inserted as an object into the template of the 3D environment; as well as In response to receiving the user input selecting the model, Fitting the selected object into a container of a preset size and determining whether the selected object exceeds the container in at least one dimension or is smaller than the container in at least one dimension by a preset threshold; In response to determining that the selected object exceeds the container in at least one dimension or is smaller than the container in at least one dimension by a preset threshold, adjusting the size of the selected object to fit inside the container; automatically determining a location to place the resized object based on a preset radial distance from a default viewer position of a viewer of the 3D environment and along an arc spaced the preset radial distance from the default viewer position; as well as A graphical representation of the 2D or 3D content item is drawn and placed as the resized object at the determined location in the 3D environment, thereby enabling proper presentation of the 2D or 3D content item in the 3D environment.

2. The method according to claim 1, wherein: The object is a first object; The content item is a first content item; as well as The method further comprises: receiving another user input selecting another model representing another two-dimensional (2D) or 3D content item to be inserted into the template of the 3D environment as a second object; and In response to receiving the further user input, automatically determining another location to place the second object based on the preset radial distance from the default viewer position of the viewer of the 3D environment and along the same arc spaced apart from the default viewer position by the preset radial distance; and Another graphical representation of the 2D or 3D content item is drawn and placed as the second object at the determined further location in the 3D environment.

3. The method according to claim 2, further comprising: determining whether a distance between the first object and the second object in the 3D environment is less than a threshold; as well as In response to determining that the interval between the first object and the second object in the 3D environment is less than the threshold, one or both of the first object or the second object in the 3D environment are moved along the arc so that the interval is not less than the threshold.

4. The method according to claim 2, wherein: The determined positions of the first object and the second object are positions expressed in a cylindrical coordinate system; and The method further comprises: determining whether a separation angle between the first object and the second object in the 3D environment is less than a threshold angle; and In response to determining that the interval angle between the first object and the second object in the 3D environment is less than the threshold angle, one or both of the first object or the second object in the 3D environment are moved along the arc so that the interval angle is not less than the threshold angle.

5. The method according to claim 2, wherein: The determined positions of the first object and the second object are positions expressed in a cylindrical coordinate system; and The method further comprises: determining whether a spacing angle between the first object and the second object in the 3D environment is less than a threshold angle of 0°; and In response to determining that the interval angle between the first object and the second object in the 3D environment is less than the threshold angle, one or both of the first object or the second object in the 3D environment are moved along the arc so that the interval is not less than the threshold angle.

6. The method according to claim 2, wherein: The determined positions of the first object and the second object are positions expressed in a cylindrical coordinate system; and The method further comprises: enlarging the view of the first object based on a further user request; In response to the further user request, determining an orientation angle of the enlarged first object in the 3D environment; recalculating whether the interval angle between the first object and the second object in the 3D environment is less than a threshold angle based on the determined azimuth of the enlarged first object; and In response to determining that the separation angle between the first object and the second object in the 3D environment is now less than the threshold angle, the second object in the 3D environment is moved away from the enlarged first object along the arc so that the separation angle is not less than the threshold angle.

7. The method according to claim 2, wherein: The determined positions of the first object and the second object are positions expressed in a cylindrical coordinate system; and The method further comprises: based on another user request, reducing a radial distance of the first object from the predetermined radial distance in the 3D environment; In response to the further user request, determining an azimuth angle of the first object in the 3D environment based on the reduced radial distance; recalculating whether the interval angle between the first object and the second object in the 3D environment is less than a threshold angle based on the determined azimuth angle of the first object; and In response to determining that the separation angle between the first object and the second object in the 3D environment is now less than the threshold angle, the second object in the 3D environment is moved away from the first object along the arc so that the separation angle is not less than the threshold angle.

8. The method according to claim 1, wherein: The preset radial distance is approximately three meters from the default viewer position.

9. The method according to claim 1, wherein: The determined position of the object is expressed as coordinates in a cylindrical coordinate system, the coordinates having an azimuth, a radial distance and an altitude, and wherein the radial distance is set to the preset radial distance and the altitude is set to zero meters.

10. A computing device comprising: processor; monitor; as well as A memory operatively coupled to the processor and the display, the memory containing instructions executable by the processor to cause the computing device to perform the method of one of claims 1-9.

Images