Positioning input 3D object in input 3D scene

Abstract

To relate to a computer-implemented method for automatically positioning an input 3D object representing a real object in an input 3D scene representing a room.SOLUTION: A method comprises obtaining a dataset including information about objects of a plurality of rooms. The method comprises executing computer program instructions that cause attempting to identify first, second and / or third pairs. The method comprises outputting one or more pairs among a set consisting of each identified pair and the counts of the one or more identified pairs. The method comprises, for each output pair, determining positions of input 3D objects in an input 3D scene. The method comprises positioning the input 3D objects according to the positions determined for one of the one or more output pairs. The method improves the positioning of an input 3D object in an input 3D scene.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the field of computer programs and systems, and more particularly to methods, systems, and programs for automatically positioning an input 3D object representing a real object in an input 3D scene representing a room. [Background technology]

[0002] Several systems and programs for designing, engineering, and manufacturing objects are available on the market. CAD is an acronym for Computer-Aided Design, which refers to software solutions for designing objects. CAE is an acronym for Computer-Aided Engineering, which refers to software solutions for simulating the physical behavior of future products. CAM is an acronym for Computer-Aided Manufacturing, which refers to software solutions for defining manufacturing processes and operations. In such computer-aided design systems, the graphical user interface plays an important role in the efficiency of the techniques. These techniques may be embedded within product lifecycle management (PLM) systems. PLM refers to a business strategy that helps companies across the extended enterprise to share product data, apply common processes, and leverage enterprise knowledge for the development of products from the beginning to the end of their lifespan. PLM solutions offered by Dassault Systèmes (under the trademarks CATIA, ENOVIA, 3DVIA and DELMIA) provide an Engineering Hub that organizes product engineering knowledge, a Manufacturing Hub that manages manufacturing engineering knowledge, and an Enterprise Hub that enables enterprise integration and connectivity to both the Engineering and Manufacturing Hubs. Taken together, the systems deliver an open object model that links products, processes and resources to enable dynamic, knowledge-based product creation and decision support that drives optimized product definition, manufacturing preparation, production and service.

[0003] In this context, applications for 3D scene completion have been developed. These applications generally propose to create, manipulate and serve 3D scenes, especially (though not exclusively) for touch-sensitive devices (e.g., smartphones or tablets). One task of these applications is the calculation of a consistent location of a given 3D object to be added to a partially furnished 3D scene.

[0004] When attempting to tackle this task, most existing solutions include a step of predicting the position of new items to be added in a 3D scene. Referring to the list of prior art documents below, among these methods, some use deep learning algorithms, such as Non-Patent Documents 1, 2, and 3, others use data-driven and optimization techniques, such as Non-Patent Documents 4 and 5, and others implement procedural approaches that rely on manually designed rules, such as Non-Patent Document 6.

[0005] The methods described in Non-Patent Documents 1 and 2 use a neural network to predict the location of a previously predicted item to be added to a 3D scene. In Non-Patent Document 1, the neural network receives as input a category of the object to be added, a top-down view image of the room, and some spatial constraints that the added object must adhere to (e.g., adding a TV with the constraint that the TV must be in front of the bed). This method does not work well if the constraints are inconsistent or cannot be satisfied (e.g., if the bed is in front of a window or a wardrobe, the TV cannot be placed in front of the window or the wardrobe). In Non-Patent Document 2, the neural network receives as input a category of the object to be added and a top-down view image of the room. Both methods in Non-Patent Documents 1 and 2 use a second neural network to predict the orientation of the object to be added.

[0006] The method described in Non-Patent Document 3 uses a transformer neural network to complete a 3D scene. The transformer predicts the features of the object to be added to the scene, and four attribute extractors (MLP networks) predict the object's class, location, orientation, and size in that exact order. If the object's class is constrained, the location, orientation, and size of the object are predicted, and then the object of the given class with the closest size is selected from the catalog, and the object can be added to the scene at the predicted location and orientation.

[0007] The methods described in Non-Patent Documents 1, 2, and 3 only consider the object category and not the 3D model itself, which limits their use. For example, depending on how finely the classes are defined, it may not distinguish a table lamp from a floor lamp, a wall lamp, or a ceiling lamp, and therefore may predict inappropriate positions for these objects. Furthermore, all of these methods rely on neural networks, which may limit their integration and deployment in applications. Indeed, deep learning-based algorithms require specific infrastructure to be deployed.

[0008] The method described in Non-Patent Document 4 uses a genetic algorithm to optimize the layout of 3D furniture in a 3D room according to manually defined interior design KPIs (e.g., circulation or alignment). This method uses manually defined rules and KPIs and the computation time can be extremely long. Therefore, this method is unsuitable for end-user applications and for furniture catalogs that may evolve over time as new categories or objects are added to the catalog.

[0009] The method described in Non-Patent Document 5 uses a diagram-based approach to furnish a 3D room with a given set of 3D furniture. It groups objects together and uses templates to calculate the relative positions of the objects within the group. The templates are obtained using a data-driven method based on the relative positions of furniture classes in a 3D scene dataset. The positions of the groups within the 3D room are then calculated using an optimization algorithm. Because the optimization algorithm takes furniture classes into account for template calculation, the optimization algorithm relies on accurate annotations of the furniture.

[0010] Finally, [6] presents a procedural method for handcrafted rule-based 3D scene generation. In particular, each 3D piece of furniture in a dataset is precisely annotated with the 3D furniture class, dimensions, and class-dependent location rules (e.g., "on the floor" for a sofa or "on the object" for a plate). Templates are manually designed for groups of furniture classes that are common in indoor scenes (e.g., four chairs around a table). Then, several groups of objects are sampled and repeatedly placed in a 3D room. This method relies heavily on handcrafted rules and templates defined at the furniture class scale and presents the same drawbacks as the previous method. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] European Patent Application Publication No. 4075316 [Non-patent literature]

[0012] [Non-Patent Document 1] Wang, K., Lin, Y., Weissmann, B., Savva, M., Chang, AX, & Ritchie, D. (2019). PlanIT: planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Trans. Graph., 38, 132:1-132:15. [Non-patent document 2] RITCHIE,Daniel,WANG,Kai,et LIN,Yu-an.Fast and flexible indoor scene synthesis via deep convolutional generative models.In :Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019.p.6182-6190. [Non-patent document 3] Despoina Paschalidou,Amlan Kar,Maria Shugrina,Karsten Kreis,Andreas Geiger,Sanja Fidler.2021.ATISS:Autoregressive Transformers for Indoor Scene Synthesis.NeurIPS 2021. [Non-patent document 4] Peter Kan and Hannes Kaufmann.2017.Automated Interior Design Using a Genetic Algorithm.In Proceedings of VRST2017,Gothenburg,Sweden,November 2017,10 pages. [Non-Patent Document 5] Yang L.,Zhang S.,Zhang S.,Xie W.,Fu H.(2020).Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extraction.arXiv:2002.00328v2 [Non-patent document 6] Matt Deitke,Eli VanderBilt,Alvaro Herrasti,Luca Weihs,Jordi Salvador,Kiana Ehsani,Winson Han,Eric Kolve,Ali Farhadi,Aniruddha Kembhavi†,Roozbeh Mottaghi.2022.ProcTHOR:Large-Scale Embodied AI Using Procedural Generation.NeurIPS 2022. [Non-Patent Document 7] Scikit-learn library implementations of Gaussian Mixture Models and silhouette scores. https: / / scikit-learn.org / stable / modules / generated / sklearn.mixture.GaussianMixture.html https: / / scikit-learn.org / stable / modules / generated / sklearn.metrics.silhouette_score.html Within this context, there remains a need for improved solutions for locating input 3D objects within an input 3D scene. Summary of the Invention

[0013]

[0009] Accordingly, a computer-implemented method is provided for automatically positioning input 3D objects representing real objects in an input 3D scene representing a room. The input 3D scene includes a plurality of positioned 3D objects. The plurality of positioned 3D objects includes positioned furniture 3D objects, each representing a piece of real furniture. The method includes acquiring a dataset including information about the plurality of room objects. The information includes classifications of the plurality of room objects. The information includes counts, each count of the set consisting of a respective pair of occurrences including the same first real object and the same second real object in the room. The respective first real object and second real object of each respective pair of occurrences are positioned according to a mutual positioning relationship from a predetermined set of one or more mutual positioning relationships. The information includes information regarding the relative positioning of the first real object and the second real object in each respective pair of occurrences.

[0014] The method includes executing computer program instructions that cause the computer to attempt to identify, in the dataset, one or more first pairs, each including an input 3D object of the input 3D scene and a respective positioned 3D object. If the first pair is not identified, the computer program instructions further cause the computer to attempt to identify, in the dataset, one or more second pairs, each including only one of the input 3D object of the input 3D scene and each positioned 3D object and an equivalent object of the other. A given real object is an equivalent object of another given real object when the given real object has the same classification as the other given real object, and the given real object is closest in size to the other given real object among a set of real objects of the same classification present in the dataset. If the first pair is not identified and the second pair is not identified, the computer program instructions also cause the computer to attempt to identify, in the dataset, one or more third pairs, each including an equivalent object of the input 3D object of the input 3D scene and an equivalent object of the respective positioned 3D object.

[0015] The method includes outputting one or more pairs in a set consisting of each of the identified pairs and a count of the one or more identified pairs. The outputting is based on a set consisting of each of the counts of each identified pair in the set consisting of each of the identified pairs. The method includes, for each of the output pairs, determining a position of a respective input 3D object in the input 3D scene based on information regarding the relative positioning of the first real object and the second real object at each occurrence of the output pair. The method includes positioning the input 3D object according to the respective position determined for one of the one or more output pairs.

[0016] The method may include one or more of the following:

[0017] the predetermined set of one or more mutual placement relationships includes an adjacent relationship and / or a vertical overlap relationship; - the dataset pairs comprise pairs including a decoration object and a furniture object; - the pairs of data sets comprise pairs including a decoration object and another decoration object; - determining the respective positions for each of the output pairs; Relatively positioning the input 3D object with the other object of the pair in the input 3D scene based on information about the relative positioning of the first real object and the second real object in each of the output occurrences of the pair; Snapping relatively positioned input 3D objects onto elements of the 3D scene Including, - for each output pair, the information about the relative positioning of the first real object and the second real object in each occurrence of the output pair includes a respective relative positioning for each occurrence, and the method further comprises fitting parameters of a Gaussian mixture model to the relative positioning of the objects in the occurrence of the pair prior to the relative positioning for each output pair, wherein the relative positioning includes sampling in a distribution represented by the Gaussian mixture model for the fitted parameters, - each of the output pairs has a count higher than a predetermined number, - each of the rooms in the dataset is of the same type as the room represented by the input 3D scene, - for each of the output pairs, the respective positions of the input 3D objects in the input 3D scene determined for the output pair do not have conflicts with any other 3D objects among the plurality of positioned 3D objects of the input 3D scene; the one or more output pairs include several output pairs, and the method further comprises, prior to the positioning, user selecting one of the one or more output pairs in which the input 3D object is to be positioned; the one or more output pairs include several output pairs, and the method further comprises, prior to positioning, ranking the output pairs according to a count of the output pairs in the dataset, the output pair to which the input 3D object is positioned being the pair with the highest ranking; and / or the method further comprises, after positioning the input 3D object, displaying the positioned input 3D object.

[0018] There is further provided a computer program comprising instructions for carrying out the method.

[0019] There is further provided a computer readable storage medium having a computer program recorded thereon.

[0020] There is also provided a system comprising a processor coupled to a memory, the memory having a computer program recorded thereon. The system may comprise a graphical user interface.

[0021] Further provided is a device comprising a data storage medium having a computer program recorded thereon. The device may form or function as a non-transitory computer-readable medium, such as in a software as a service (SaaS) or other server- or cloud-based platform. Alternatively, the device may comprise a processor coupled to the data storage medium. In this manner, the device may form, in whole or in part, a computer system (e.g., the device is a subsystem of an overall system). The system may further comprise a graphical user interface coupled to the processor. [Brief explanation of the drawings]

[0022] Non-limiting examples will now be described with reference to the accompanying drawings. [Figure 1] 1 is a flowchart of an example of the method. [Figure 2] FIG. 1 shows an example of a diagram. [Figure 3] 1 is a flowchart of an example of the offline and online phases. [Figure 4] FIG. 10 is a diagram showing an example of the appearance of pairs. [Figure 5] FIG. 10 is a diagram illustrating an example of counting. [Figure 6] FIG. 10 is a diagram showing an example of each determined position. [Figure 7] FIG. 10 is a diagram illustrating an example of fitting a Gaussian mixture model. [Figure 8] FIG. 10 is a diagram illustrating an example of fitting a Gaussian mixture model. [Figure 9] FIG. 10 is a diagram illustrating an example of fitting a Gaussian mixture model. [Figure 10] FIG. 10 is a diagram showing an example of ranking of output pairs. [Figure 11] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 12] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 13] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 14] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 15] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 16] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 17] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 18] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 19] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 20] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 21] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 22] FIG. 10 illustrates an example of positioning an input 3D object. [Figure 23] FIG. 2 illustrates an example of a graphical user interface of the system. [Figure 24] FIG. 1 illustrates an example of the present system. DETAILED DESCRIPTION OF THE INVENTION

[0023] Referring to the flowchart of FIG. 1 , a computer-implemented method for automatically positioning input 3D objects representing real objects in an input 3D scene representing a room is proposed. The input 3D scene includes a plurality of positioned 3D objects. The plurality of positioned 3D objects include positioned furniture 3D objects, each representing a piece of real furniture. The method includes obtaining S10 a dataset including information about the plurality of room objects. The information includes classifications of the plurality of room objects. The information includes counts. Each of the counts is a count of a set consisting of each respective pair of occurrences including the same first real object and the same second real object in the room. The respective first real object and second real object of each respective pair of occurrences are positioned according to a mutual positioning relationship from a predetermined set of one or more mutual positioning relationships. The information includes information regarding the relative positioning of the first real object and the second real object in each respective pair of occurrences.

[0024] The method includes executing S20 computer program instructions that cause the system to attempt to identify one or more first pairs in the dataset, each pair including an input 3D object and a respective positioned 3D object. If the first pair is not identified, the system then attempts to identify one or more second pairs in the dataset, each pair including only one of the input 3D object and each positioned 3D object of the input 3D scene and an equivalent object of the other. A given real object is an equivalent object of another given real object when the given real object has the same classification as the other given real object, and the given real object is closest in size to the other given real object among a set of real objects of the same classification present in the dataset. If the first pair is not identified and the second pair is not identified, the system then attempts to identify one or more third pairs in the dataset, each pair including an equivalent object of the input 3D object of the input 3D scene and an equivalent object of the respective positioned 3D object.

[0025] The method includes outputting S30 one or more pairs in a set consisting of each of the identified pairs and a count of the one or more identified pairs. The outputting is based on a set consisting of each of the counts of each identified pair in the set consisting of each of the identified pairs. The method includes determining S40, for each output pair, a position of a respective input 3D object in the input 3D scene based on information about the relative positioning of the first real object and the second real object at each occurrence of the output pair. The method includes positioning S50 the input 3D object according to the respective position determined for one of the one or more output pairs.

[0026] Such a method provides an improved solution for positioning an input 3d object in an input 3d scene.

[0027] In particular, the method allows for the calculation of consistent positions of input 3D objects in an input 3D scene based on the positions of furniture 3D objects in the input 3D scene. In particular, the calculated positions are consistent with respect to physical laws and internal design rules. Indeed, counting pairs in a dataset allows for statistical extraction of these physical laws and internal design rules from the 3D scenes contained in the dataset. The claimed method allows for the positioning of input 3D objects in a manner that respects the physical constraints of the objects statistically determined on the 3D scenes of the dataset. For example, the method allows for the calculation of the positioning of a bedside lamp on top of a bedside table, a cushion on a bed, or a photo on the wall above a headboard. This is particularly useful for (although not exclusively) touchscreen devices, which may have difficulty handling 3D environments.

[0028] Moreover, the claimed method is data-driven and does not rely on neural networks. This is an advantage for integrating neural networks in existing applications using current infrastructure. It is also fast in inference, which allows for immediate use from the user's perspective (location calculations take only about one second). In fact, the method can identify pairs by simply examining them in a dataset and output pairs solely from count statistics. The method is therefore based solely on identifying similar pairs and comparing the number of their respective occurrences (this comparison is quick because it is based on numbers). These statistical considerations are not necessarily more computationally intensive (and therefore faster) than using neural networks, which would necessarily be more computationally intensive to implement. The claimed method is therefore more efficient and less demanding in terms of computer resources. This is an advantage because, for end-user applications, the method needs to be fast so that insertions appear instantaneous.

[0029] Furthermore, the claimed method allows for considering, when possible, the exact 3D model to be added to the 3D scene without abstracting it into a classification of the 3D model. Indeed, the executed computer program instructions allow for pairs of identical input 3D objects to be considered first. Moreover, the claimed method also allows for such abstraction, if necessary. Indeed, the executed computer program instructions allow for considering second and third pairs of equivalent objects when the first pair is not identified. Thus, the claimed method allows for an adaptive search for similar pairs according to classification, or not for the input 3D objects, thereby improving the relevance of the calculated positioning.

[0030] The method is computer-implemented, meaning that the steps (or substantially all steps) of the method are performed by at least one computer, or any similar system. Thus, the method steps are performed by a computer, possibly fully automatically or semi-automatically. In an example, triggering of at least some of the method steps may be performed through user-computer interaction. The level of user-computer interaction required may depend on the level of automatism balanced with the need to foresee and implement the user's wishes. In an example, this level may be user-defined and / or predefined.

[0031] For example, the method may include, prior to performing S20, selecting an input 3D object to be positioned. The selection may be made from a virtual catalog containing a plurality of potential input 3D objects. This virtual catalog may be updated by object manufacturers (e.g., whenever a new product is released). For example, the virtual catalog may be stored in a database (e.g., available online), and each of the manufacturers may upload the objects they manufacture to the catalog. The selection may be performed by a user who is currently designing a room, for example. The selection may be performed in any manner in the virtual catalog. For example, the user may select one of the objects in the catalog using an object reference. Alternatively or additionally, the method may include displaying the objects of the catalog (e.g., with a filter for a classification of the objects), and the user may select one of the displayed objects (e.g., by performing a user interaction on the object, such as a touch with a finger or stylus when the object is displayed on a touchscreen or touchpad, or a mouse click when the user uses a mouse).

[0032] A typical example of a computer implementation of the method is performing the method using a system adapted for this purpose. The system may include a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program including instructions for performing the method. The memory may also store a database. The memory is any hardware adapted for such storage, possibly comprising several physically distinct parts (e.g., one for the program and possibly one for the database). Optionally, the system may include a touchscreen or touchpad. The touchscreen or touchpad may be coupled to the processor and the memory. A user may perform user interactions on the touchscreen or touchpad. The method may display a GUI (e.g., including a 3D scene) on the touchscreen or touchpad.

[0033] Data sets may be stored in a database. By "database" is meant any collection of data (i.e., information) organized for search and retrieval (e.g., a relational database, e.g., based on a predetermined structured language, e.g., SQL). When stored in memory, a database allows for rapid search and retrieval by a computer. Databases are actually structured to facilitate the storage, retrieval, modification, and deletion of data along with various data processing operations. A database may consist of a file or set of files that can be broken down into records, each of which consists of one or more fields. A field is the basic unit of data storage. Users may retrieve data primarily through queries. Using keywords and sort commands, users can quickly search, sort, group, and select fields in many records to retrieve or create reports on specific aggregations of data according to the rules of the database management system being used.

[0034] The method generally manipulates modeled (3D) objects. A modeled object is any object defined by data stored, for example, in a database. By extension, the expression "modeled object" refers to the data itself. Depending on the type of system, the modeled object may be defined by different kinds of data. A system may actually be any combination of a CAD system, a CAE system, a CAM system, a PDM system, and / or a PLM system. In these different systems, the modeled object is defined by corresponding data. One may therefore refer to CAD objects, PLM objects, PDM objects, CAE objects, CAM objects, CAD data, PLM data, PDM data, CAM data, and CAE data. However, these systems are not exclusive to one another, as a modeled object may be defined by data corresponding to any combination of these systems. A system may therefore very well be both a CAD system and a PLM system.

[0035] By CAD system, any system at least adapted for designing a modeled object based on a graphical representation of the modeled object, such as CATIA, is further meant. In this case, data defining the modeled object includes data enabling the representation of the modeled object. A CAD system may provide a representation of a CAD modeled object using, for example, edges or lines, in some cases together with faces or surfaces. Lines, edges, or surfaces may be represented in various ways, for example, with non-uniform rational B-splines (NURBS). In particular, a CAD file contains specifications, and a shape may be generated from the specifications, which in turn enables the representation to be generated. The specifications of a modeled object may be stored in a single CAD file or in multiple ones.

[0036] In the context of CAD, a modeled object may typically be a 3D modeled object that represents a real object (e.g., a furniture object or a decorative object) that is (or is to be) positioned in, for example, a real room. By "3D modeled object" is meant any object modeled by data that allows for a 3D representation of the object. The 3D representation allows for the part to be viewed from all angles. For example, a 3D modeled object, when represented in 3D, can be manipulated and rotated around any of the 3D modeled object's axes or around any axis in the screen on which the representation is displayed. This specifically excludes 2D icons that are not 3D modeled. Any 3D modeled object may also be a CAD object, including, for example, a mesh or B-rep (e.g., resulting from calculations that define a history of geometric features).

[0037] Each of the 3D objects may represent the shape of a real object positioned (or to be positioned) in a real room. The real object may be manufactured in the real world following completion of a virtual design of the real object (e.g., using a CAD software solution or CAD system). The method considers a classification of objects, and each of the objects (real or 3D representations of the objects) may belong to a class of objects in the classification of objects. The classification may include each class of a category of objects generally present in a room. The dataset includes classifications of objects for multiple rooms (e.g., the dataset may include a label for each of the objects present in each of the rooms, the label indicating the class to which the object belongs). The input 3D object may also belong to a class of the same classification. The classification may include a class of furniture objects and a class of decoration objects. Each of the rooms (in the dataset and represented by the input 3D scene) may include one or more furniture objects. Each of the rooms may also include one or more decoration objects. The input 3D object may be a decoration object, and the method may position the decoration object relative to one of the other 3D objects (decorative objects or furniture objects) in the 3D scene.

[0038] In examples, the rooms represented by the input 3D scene and / or the rooms of the dataset may be indoor rooms. For example, the rooms represented by the input 3D scene and / or the rooms of the dataset may include rooms in a residential building (e.g., a house or apartment), such as a kitchen, bathroom, bedroom, living room, garage, laundry room, attic, office (e.g., private or shared), conference room, children's room, nursery, hallway, dining room, and / or library (this list may include other types of rooms). Alternatively or additionally, the rooms represented by the input 3D scene and / or the rooms of the dataset may include other indoor rooms, such as those of a factory, museum, and / or theater. Alternatively or additionally, the rooms represented by the input 3D scene and / or the rooms of the dataset may include outdoor scenes, such as a garden, terrace, or amusement park.

[0039] Any furniture object herein may have a furnishing function in the room it is placed in. For example, the classification may include any combination of one or more chair classes, one or more lamp classes, one or more cabinet classes, one or more shelf classes, one or more sofa classes, one or more table classes, one or more bed classes, one or more sideboard classes, one or more nightstand classes, one or more desk classes, and / or one or more wardrobe classes. For example, the classification may include any combination of each of the classes for each one of the following objects: separator, wall, air conditioner, armchair, wardrobe, bar, base cabinet, mirror, single sink, double sink, sink, storage cabinet, bathtub, bookcase, sideboard, bunk bed, ceiling lamp, children's bed, fireplace tools, clothes organizer, shoe organizer, coffee table, cooktop, desk, chair, table, bed, double bed, dresser, dryer, light switch, TV, electronics and entertainment, home cinema, faucet, fireplace, fitness equipment, floor lamp, freezer, headboard, high cabinet, home automation, intercom, house alarm, hood, kitchen islands, junior chair chairs, lighting, microwave, nursery, changing table, bed, cribs, pergola, gazebo, play area, lead pipes, swimming pool, radiator, range, refrigerator, shelf, shower column, shower door, shower, side table, single bed, sofa, stool, table lamp, toilets, media furniture, wall cabinet, wall lamp, wall shelf, washing machine, bench, sun lounger, oven, dishwasher, nightstand, vehicle, mailbox, tree, front door, security door, patio door, screen door, simple interior doors, double interior doorsdoors, sliding doors, interior doors, interior openings, garage doors, side entrances, simple windows, casement windows, simple sliding windows, double sliding windows, basement windows, fixed windows, skylights, stairs, outdoor activities, round cushions (poufs), wall stickers, dressing tables, wood stoves, oven cabinets, under-sink cabinets, sliding storage cabinets, corner cabinets, safety detectors, openings, fountains, bridges, wells, pillars, water heaters, furnaces, boilers, barbecues, gardening, patio heaters, spiral staircases, straight staircases, quarter-turn staircases, double quarter-turn staircases, kitchen sets, folding doors, push windows windows, bay windows, balustrades, podiums, cupboards, mattresses, wine cellars, hanging lamps, meridians, foot stools, showcases, leisure chairs, office chairs, bed stools, outdoor sofas, outdoor chairs, outdoor benches, outdoor tables, swinging patio doors, sliding patio doors, bathroom mirrors, kids desks, kids bedroom furniture, gates, garden lounges, straight sofas, corner sofas, modular sofas, armless chairs, four poster beds, mezzanine beds, floor standing mirrors, wall mirrors, kitchen accessories, baby bathtubs, bathroom base cabinets, bathroom columns, bathroom vanity units, cradles, garden lighting, balconies, fronts, doors, double doors, double glass doors, drawers, glass doors, cabinet shelves, filler strip angles, wall paneling, free standing filler strips, wall panel edges, roof windows and / or half height cabinets.

[0040] Any decorative object herein may have a decorative function in the room they are placed in. For example, the classifications may include any combination of one or more accessory classes, one or more plant classes, one or more book classes, one or more picture frame classes, one or more kitchen accessory classes, one or more cushion classes, one or more lamp classes, one or more curtain classes, one or more vase classes, one or more rug classes, one or more mirror classes, and one or more classes for electronic objects (e.g., refrigerators, freezers, and / or washing machines).For example, the classification may include any combination of each of the classes for each one of the following objects: accessories, mirrors, blinds, ceiling lamps, fireplace tools, curtains, decorative accessories, plugs and meters, electronics and entertainment, Hifi, fences, floor lamps, smoke detectors, lighting, decorations, people, plants and flowers, rugs, table lamps, toys, wall decorations, wall lamps, small appliances, fish tanks, coat stands, vases, kids activities, beams, towel holders, round cushions, screens, garbage cans, picture frames, candles, candlesticks, paintings, books, kids mobiles, height charts, weather vanes, statues and sculptures, clocks, posters, swings, sandboxes, trampolines, kitchen utensils, crockery, pet supplies stuffs, ornaments, hanging lamps, tapestries, towel dryers, toboggans, hammocks, carpets, hardwood, wallpaper, paints, vinyl, tiling, mosaic, stone, slate, brick, concrete, marble, linoleum, shingles, fiberboard, wall clothes, cobblestones, cushions, bathroom mirrors, wainscoting, decking tiles, decking, DIY items, filler strips, security products, floor standing mirrors, wall mirrors, bathroom accessories, office supplies, kids decor, outdoor decor, garden lighting, beach umbrellas, gravel, plaster, worktops, plinths, boxes, legs, wall tiles, joints, operations, wall edge strips, deco strips, quartz, resources, cornices, worktop edges, drains, wall edge strip edges, cover panels and / or cover handles.

[0041] A room among the multiple rooms may contain objects belonging to several classes of a taxonomy (thus, these classes are represented in the dataset) (i.e., all objects in each of the rooms may be grouped together). For example, a room among the multiple rooms may contain at least one object belonging to each of the classes of a taxonomy (each of the classes of the taxonomy is then represented). The taxonomy may have been determined to include each class of a category of objects in different rooms in the dataset. The input 3D scene may also contain objects belonging to another number of classes in the taxonomy (this number may be smaller than the number of classes represented in the multiple rooms in the dataset, but may be, for example, equivalent on average to the number of classes represented in each of the rooms in the dataset). For example, each of the objects in the input 3D scene may belong to a class represented in the dataset.

[0042] The method may be included in a real-life room design (i.e., effective placement) process that, after performing the method, may include using the input 3D scene to indicate a room to be placed (either from scratch or, for example, by adding an object represented by the input 3D object within an already furnished and / or decorated room). For example, this example may be for a user, such as the owner of a house in which the room is being placed. This example may include determining an image and / or video of the input 3D object positioned within the input 3D scene. The determined image and / or video may be used by the user to determine whether to collect the object and may assist the user in selecting the object by showing the object within the room. During the real-life room design process, the method may be repeated to place various objects within the input 3D scene. This repetition of the method may be used to indicate a complete virtual interior of the room (using several objects positioned using the method).

[0043] Alternatively or additionally (e.g., prior to illustration), the real-life room design process may include populating an input 3D scene representing a room with one or more decorative objects. Populating may include, for each of the decorative objects, repeating the method to position a respective input 3D object representing the decorative object in the input 3D scene. The real-life room design process enables the creation of more luxurious and appealing environments (for animation, advertising, and / or to generate virtual environments, e.g., for simulation). The real-life room design process may be used to generate virtual environments (e.g., the Metaverse). The real-life room design process may be included in a general process, which may include repeating the real-life room design process for several input 3D scenes, thereby populating a set of input 3D scenes with decorative objects.

[0044] Alternatively or additionally (e.g., after populating), the real-life room design process may include evaluating one or more layout metrics for the room represented by the input 3D scene. The one or more layout metrics may include a circulation metric, a lighting metric, an accessibility metric (e.g., representing the ability to use functional furniture), and / or an ergonomics metric. The real-life room design process may include using these evaluated one or more metrics to ensure ease of movement within the room, accessibility of some objects, safety in the room (e.g., a room to be constructed later), and / or a certain level of ergonomics.

[0045] Alternatively or additionally, the real-life room design process may include, after performing the method, physically arranging a room (i.e., a real room) so that the room design matches the input 3D scene. For example, a room (without the objects represented by the input 3D scene) may already exist in the real world, and the real-life room design process may include positioning the real objects represented by the input 3D objects within the already-existing room (i.e., in the real world). The real objects may be positioned according to positions calculated by the method. When the method is repeated to position various 3D objects, the real-life room design process may include positioning various real objects within the room. Alternatively, the room may not yet exist when the method is performed. In that case, the real-life room design process may include constructing the room according to the provided 3D scene (i.e., starting from scratch and placing objects as in the provided 3D scene). Because the method improves the positioning of 3D objects in the 3D scene, the method also improves the construction of a room corresponding to the 3D scene, thus increasing the productivity of the real-life room design process.

[0046] The method may include forming a dataset from multiple rooms (dataset formation step). In that case, acquiring S10 may include the dataset formation step. The dataset formation step may be simple and fast. For example, the dataset formation step may include a single loop for examining the dataset, counting each pair of objects, and recording the relative positioning of the objects. The formation of the dataset may be fully automated (i.e., without manual intervention).

[0047] The dataset formation step may include determining information about the objects in the multiple rooms (i.e., information about classification, count, and / or relative positioning of each occurrence of each pair). For example, the dataset formation step may include counting occurrences of pairs in the multiple rooms. After the dataset formation step, obtaining S10 may include recording the calculated dataset in a database, and step S20 may be performed by accessing the recorded dataset. The multiple rooms may be recorded in the same database, for example, or may be recorded in separate databases. The calculation of the dataset may be performed by examining the recorded multiple rooms.

[0048] Alternatively, the method may include retrieving a dataset in a database. The dataset may, in such a case, optionally have been obtained in accordance with the dataset formation step described above (prior to performing the method). In that case, obtaining S10 may include providing access to the dataset in the database. The method may use this access to identify (in step S20) the first, second and / or third pairs in the dataset.

[0049] The calculation of the dataset (i.e., the dataset formation step) may be performed during an offline phase. In that case, the calculated dataset may, for example, be recorded in a database during this offline phase. The dataset may, for example, be calculated by a server having access to information about a plurality of rooms (e.g., 3D representations of these rooms). Steps S20 to S50 (and step S10 when the dataset has already been calculated) may then be performed during the online phase. For example, these steps may be executed by a client computer having access to the dataset calculated during the offline phase.

[0050] The method may include updating the dataset (e.g., periodically). For example, a manufacturer may add new objects to the catalog by, for example, adding new rooms (containing the new objects) to the database, and the method may update the dataset to include these new objects. In that case, updating may include determining new pairs containing these new objects. The method may then comprise counting the occurrences of these new pairs and including the counts of occurrences of these new pairs (along with their relative positions) in the dataset. This improves efficiency because the dataset is updated only for new objects to be added (without recalculating statistics for pairs of objects that already exist).

[0051] The objects and rooms in the dataset may or may exist in the real world (already at the time of computing the dataset or in the future). For example, the rooms may be actual, real rooms (in terms of layout) in the real world, and the objects may be positioned inside these real rooms as specified in the information in the dataset. The rooms may include rooms designed (e.g., by an interior designer) and then implemented in the real world (i.e., the rooms correspond to virtually designed rooms that have been or may be recreated in people's homes). In an example, each of the rooms in the dataset is of the same type as a room represented by the input 3D scene. For example, all the rooms in the dataset may be a kitchen, bathroom, bedroom, living room, garage, laundry room, attic, office (e.g., private or shared), conference room, children's room, nursery, hallway, dining room, or library (this list may include other types of rooms). The dataset allows for more accurate locations to be provided, increasing the stability of the method.

[0052] The classification of the plurality of room objects may include a class (i.e., a category) to which the object belongs. The category may be one of the categories described above for each of the furniture or decorative objects. The classification may be determined during computation of the dataset. For example, the computation may include determining a classification of the plurality of room objects according to a predetermined list of object classes. Determining the classification may include identifying to which class each of the respective objects in the room belongs.

[0053] The information of the dataset also includes counts of the pairs. Each count may be the number of elements in a set consisting of each occurrence of a respective pair including the same first real object and the same second real object in a room. By the same real object, it is meant an object that has the same properties as the object in each of the other occurrences. For example, an object may have the same shape (e.g., outer envelope, texture, and / or color) in each of the other occurrences of a respective pair and / or may be the same product made by the same manufacturer (e.g., have the same product standards). Each set may include one or more occurrences of a respective pair. Each set may include occurrences of the pair in the same room and / or in multiple different rooms (i.e., one for each room). The information includes a respective count for each of the pairs (in the same room and in different rooms). The count is the number of occurrences of each pair (e.g., a count with a value of n when multiple rooms include n occurrences of each pair, where n is an integer). The dataset pairs may include one or more pairs each including a decorative object and a furniture object, the decorative object and the furniture object being arranged relative to one another in one or more rooms of the plurality. Alternatively or additionally, the dataset pairs may include one or more pairs each including a decorative object and another decorative object, the two decorative objects being arranged relative to one another in one or more rooms of the plurality.

[0054] For each occurrence of a pair, the mutual placement relationship between the first real object and the second real object is one of a predetermined set of one or more mutual placement relationships. Each of the mutual placement relationships specifies a particular type of spatial arrangement between the first real object and the second real object in the room (i.e., with respect to the positions and orientations of the first real object and the second real object). The spatial arrangement between the objects may be categorized (e.g., in front of, to the side of, or one above the other), and the predetermined set may include a mutual placement relationship for each of the categories.

[0055] In an example, the occurrences of pairs for the same room may be represented by a diagram, which may include nodes representing objects in the room and edges linking each pair of two nodes. Each of the edges may represent a respective occurrence of the pair between the two nodes linked by the edge. Each of the edges may include a label indicating the mutual placement relationship between the two objects represented by the nodes linked by the edge. The calculation of the count may be based on these diagrams. For example, the method may count the occurrences of the same pair in multiple rooms by browsing the diagrams of multiple rooms. The dataset may include a respective diagram for each room in the multiple rooms. In that case, the calculation of the count may be based on these diagrams. Alternatively, the diagrams may be recorded in a database, and the method may calculate the count by accessing the database.

[0056] In an example, the predetermined set may include an adjacency relationship. Two objects may be positioned according to an adjacency relationship when they are separated from each other by less than a predetermined distance (e.g., 5 centimeters). For example, the distance may be the distance between bounding boxes representing two objects in a 3D scene representing a room (e.g., a distance defined along one of the axes of one of the two objects). Alternatively or additionally, the predetermined set may include a vertical superposition relationship. Two objects may be positioned according to an adjacency relationship when one of them is positioned above the other (e.g., a lamp on a ceiling located above a table).

[0057] The information of the dataset also includes information about the relative positioning of the real objects in each of the occurrences for each of the pairs. This information may include the respective relative positioning of the real objects for each of the occurrences. The relative positioning may be in the form of a vector with coordinates representing the distance between the first object and the second object in different directions of a coordinate system (e.g., between bounding boxes representing the two objects or between the centers of these bounding boxes). Alternatively, the information may be general information about all occurrences. For example, the information may include the average of the relative positioning of the real objects in all occurrences (e.g., the average of the above vectors).

[0058] The computer program instructions executed in step S20 may be recorded on a memory. The computer program instructions are instructions for performing the identification of pairs in a dataset. The computer program instructions may be blocks of data that represent an algorithm and can generate different sequences of steps depending on the circumstances, in particular whether a "when" condition is verified or not. The exact steps in the method are therefore context-dependent. In one example, the method executes all branches of the algorithm (i.e., all attempts at identification) at least once. Attempting to identify means launching a process that operates and whose final result depends on the circumstances. For example, the final result may be the output of an identified pair, if any, or, if none, doing nothing or outputting a "fail" message.

[0059] Performing S20 may include providing one or more pairs among all pairs of the dataset (the one or more pairs being one or more pairs identified during execution of the computer program instructions). The computer program instructions cause three successive identification attempts, where a first identification attempt searches for pairs with the same object, a second identification attempt searches for pairs with only one of the two objects of the pair (and an equivalent of the second one), and a third identification attempt searches for pairs with equivalent objects. The computer program instructions may include instructions for each of these identification attempts (although not all of these identification attempts are always performed). Performing S20 begins with performing the first identification attempt. When at least one first pair is identified, the second and third identification attempts are not performed, and the identified pair is the identified first pair. Otherwise, a second identification attempt is performed. In that case, when at least one second pair is identified, the third identification attempt is not performed, and the identified pair is the identified second pair. If not, a third identification attempt is performed and the identified pair is the identified third pair (a dataset may always contain at least one third pair).

[0060] Outputting S30 is performed based on the pairs identified during executing S20. The output pairs are among the identified pairs. For example, the output pairs may include all identified pairs. Alternatively, the output pairs may include each of the identified pairs having a count higher than a predetermined number. In that case, each of the output pairs may have a count higher than this predetermined number (e.g., the predetermined number is higher than 5 or 10 and / or lower than 15 or 20). In that case, outputting S30 may include filtering the identified pairs based on the counts of the identified pairs, thereby outputting only the pairs having counts higher than the predetermined number. Alternatively, the number of output pairs may be predetermined, and outputting S30 may include selecting a predetermined number of pairs from the identified pairs (the output pairs are selected pairs). In that case, the selection of the predetermined number of pairs may be based on the counts of the identified pairs. For example, outputting S30 may include ranking the identified pairs according to a count of the identified pairs and selecting a predetermined number of first-ranked pairs (e.g., the k first-ranked pairs, where k is an integer, e.g., higher than 1 and / or lower than 10, e.g., equal to 5) (the outputted pairs are the k first-ranked pairs).

[0061] After outputting S30, the method includes determining S40 a respective position of the input 3D object for each of the output pairs. The respective determined positions may be different for each output pair. For each output pair, the respective determined positions may be such that the object is positioned similarly (e.g., on average) as in each occurrence of the output pair. For example, when the output pair is a first pair, the input 3D object may be the same as one of the 3D objects of the pair, and the 3D scene may include a given object that is the same as the other object in the pair. In that case, the respective determined positions may be such that the input 3D object is positioned relative to the given object as the two objects in the occurrence of the pair. The same may apply when the output pair is a second or third pair with equivalent objects (of the input 3D object, the given object, or both objects). The steps of outputting S30 and determining S40 may be fast. For example, the method may complete the steps of outputting S30 and determining S40 in as little as one second. When positioning S50 is performed automatically, the method may calculate the position of the input 3D object (i.e., including performing the steps of outputting S30 and determining S40) in a total of just one second.

[0062] After determining S40, the method includes positioning S50 the input 3D object. Positioning S50 the input 3D object is performed according to the respective positions determined for one of the one or more output pairs. When the method outputs only one pair, the method may position the input 3D object strictly in accordance with the relative position determined for the only output pair. Alternatively, the method may output several pairs. In that case, the method may include selecting (among the output pairs) the pair in which the object is to be positioned prior to positioning S50. For example, this selection may be random. Alternatively, the selection of the pair may be performed by a user. In that case, selecting may include performing a user interaction to select one of the displayed positions. For example, the method may include displaying a set of different icons or graphisms each associated with a respective location, and a selection may be performed by a user by clicking one of the displayed icons or by performing a user interaction on the set (e.g., a touch with a finger or stylus when the icon or graphism is displayed on a touchscreen or touchpad, or a mouse click when the user is using a mouse). Alternatively or additionally, the selection may be performed according to a location specified by the user (e.g., also based on a touch or click). In that case, the selected location may be the location closest to the location specified by the user.

[0063] Alternatively, the selection of the pair may be automatic. For example, the output pairs may be ranked (e.g., according to the count of the output pairs), and the output pair to which the input 3D object is positioned may be the one with the highest ranking. In that case, the method may further include, prior to positioning S50, ranking the output pairs according to the count of the output pairs in the dataset and automatically selecting the output pair with the higher ranking (i.e., the higher count) (the automatically selected pair is the pair to which the input 3D object is positioned). When at least two pairs have the highest ranking, the method may randomly select one of the at least two pairs.

[0064] After positioning S50, the method may include displaying S60 the positioned input 3D object (i.e., in the input 3D scene and together with each of the other 3D objects in the input 3D scene). In an example, the method may repeat the steps of positioning S50 and displaying S60. For example, the method may repeat positioning S50 each time the user selects a new position for the input 3D object in the output pair. For example, the method may repeat positioning S50 each time the user selects a new icon (when position selection is performed via a displayed set of icons) or moves the cursor to a new position (when selection is performed using a cursor). In that case, the method may repeat displaying S60 the input 3D object at the new selected position.

[0065] In an example, for each output pair, determining S40 may be performed in two steps. In a first step, determining S40 may include relative positioning the input 3D object with the other object of the pair in the input 3D scene based on information about the relative positioning of the first real object and the second real object in each occurrence of the output pair. For example, the method may relatively position the input 3D object as an average across all occurrences of the output pair. In that case, the method may include determining an average relative positioning of the two objects in all occurrences of the output pair (e.g., taking into account average values ​​of each of the axes of the coordinate system), and may relatively position the input 3D object with the other object of the pair in the input 3D scene to comply with this average relative positioning.

[0066] Alternatively, when the information regarding the relative positioning of the first and second real objects in each output pair of occurrences includes the respective relative positioning for each occurrence, the method may further include fitting parameters of a Gaussian mixture model to the relative positioning of the objects in each output pair of occurrences prior to the respective relative positioning of the pair of occurrences. For example, the Gaussian mixture model may include respective parameters for each coordinate of a coordinate system (this coordinate system is defined relative to one of the objects of the pair, e.g., one already present in the scene, for calculating the position of the one to be added, and this object is the same for each occurrence). In that case, the relative positioning may include sampling in the distribution represented by the Gaussian mixture model for the fitted parameters (the relative position of the input 3D object corresponds to the relative position sampled in this distribution). Sampling allows for taking into account the variability of the dataset. One advantage of a Gaussian mixture model compared to simple position averaging is the identification of several Gaussian components (as shown in the examples of FIGS. 7 and 8). The Gaussian mixture model also allows for providing a new position of the input 3D object. For example, the method may repeat the sampling, thereby providing a new relative position of the input 3D object in each repetition (this new relative position is within the distribution).

[0067] In the second step, determining S40 may include snapping the relatively positioned input 3D object onto an element of the 3D scene. Snapping may be performed as known in the art. Snapping may include automatically attaching the input 3D object onto the element of the 3D scene while remaining within the relative positioning range determined in the previous step S41. For example, snapping may include determining an element of the 3D scene to which the input 3D object will be snapped and then snapping the input 3D object onto the determined element. The determined element may depend on the category of the input 3D object and may be closest to the positioned input 3D object. For example, the determined element may be the ceiling when the input 3D object is a ceiling lamp, or the nearest wall when the input 3D object is a wall object (wall lamp, picture frame, wall mirror, etc.). In all other cases, the method may snap the inserted object to the nearest horizontal surface (e.g., floor, top of a shelf, or furniture shelf), taking gravity into account and modifying only the z-coordinate of the input 3D object. Snapping improves the realism of the provided position.

[0068] In an example, for each output pair, the respective positions of the input 3D object in the input 3D scene determined for the output pair may not have conflicts with any other 3D objects among the multiple positioned 3D objects of the input 3D scene. For example, the method may include verifying that the intersection between the input 3D object and all other 3D objects in the input 3D scene (e.g., excluding the object on which the input 3D object is positioned) is zero. Alternatively, each of the 3D objects of the input 3D scene may be represented by a respective bounding box, and the method may include verifying that the intersection between the bounding box representing the input 3D object and the bounding box representing any other 3D object in the input 3D scene is zero. The bounding box of a 3D object may be the box with the smallest measure (area or volume) within which all points of the 3D object lie inside the box. To achieve this, the method may include filtering the identified pairs (in step S20) so that all identified intersections are discarded. The method may perform this filtering prior to ranking the identified pairs and may ignore discarded pairs, so that only pairs that provide positions where the input 3D objects do not intersect may be present in the output pairs.

[0069] With reference to Figures 2 to 24, examples of implementations of the method will now be discussed.

[0070] The method calculates consistent locations of input 3D objects to be added in a 3D scene. The method calculates the positions of so-called decorative objects (different from functional objects), whose positions depend on other objects (i.e., functional objects as well as decorative objects) already present in the room (e.g., a table lamp is located on a table / desk / nightstand).

[0071] The method is based on statistics calculated on a dataset of a 3D (e.g., indoor) scene (representing multiple rooms) and an extracted scene diagram corresponding to the 3D scene, as defined in U.S. Patent Application Publication No. 2010 / 0129994, which is incorporated herein by reference. The method uses frequent locations of decorative objects relative to other objects in the room to calculate consistent locations of new 3D decorative furniture to be added into the 3D room.

[0072] The method may proceed in two stages. The first stage may be an offline stage for calculating statistics used for the rest of the pipeline on a given 3D scene dataset. In this stage, the method may use a scene schema of the 3D scene in the dataset (e.g., the schema discussed in U.S. Patent No. 6,277,393) to calculate the occurrence of decoration-functional object pairs and, optionally, decoration-decoration object pairs, organized by class. The method may also collect the relative positions of each of the pairs as seen in the dataset into a relative position dictionary.

[0073] The second stage may be an online stage for calculating the location of a given 3D decoration object (i.e., input 3D object) to be added to the 3D scene (i.e., input 3D scene). In this step, the method first includes selecting potential "anchors" among other 3D objects already present in the 3D scene. For each of these candidate anchors, the method may include calculating a location based on a relative position dictionary. These positions are then refined and ranked to be physically valid. The object to be added is then inserted into the 3D scene at the first location or by user interaction, where the user selects one location among all output locations.

[0074] The concept of a 3D scene is discussed next. A 3D scene may include a 3D layout (in other words, the shape of the environment, i.e., the set of walls and boundaries, their positions and dimensions), a collection of 3D objects (i.e., a set of CAD models with their size, position and orientation in 3D space) with semantic annotations such as the type of 3D object (e.g., chair, table, sofa...), model ID and functional type ("functional", "decorative" or both), and, optionally, a set of 3D openings (i.e., doors and windows in the environment). These 3D openings can also be specifically treated as 3D objects.

[0075] The method considers the classification of 3D objects between two classes: functional and decorative. Functional objects are objects that are typically placed in a room with respect to the room's architecture (walls and openings) and define the room. Functional objects are typically larger and more important with respect to the room's use (e.g., a bed in a bedroom or a sofa in a living room). Decorative objects do not define the room and are used for decoration, creating an atmosphere without changing the room's function. Decorative objects are typically placed in a room with respect to other objects (e.g., cushions on a sofa or a lamp on a nightstand). Some objects are considered both functional and decorative (e.g., lamps and mirrors). The method may relate to the automatic positioning of decorative objects inside a partially furnished room.

[0076] Each 3D scene can be associated with a scene schema corresponding to the 3D scene. These scene schemas are discussed, for example, in U.S. Patent Application Publication No. 2009 / 0129990. The scene schema represents the spatial relationships between 3D elements in the 3D scene (3D elements include walls, openings, and objects). The scene schema contains the essential spatial relationships between elements in the scene that represent the most meaningful and functional constraints.

[0077] The scene diagram may be structured as follows: The scene diagram may include a list of nodes, each corresponding to a 3D element in the room. Each of the nodes may include the following attributes: node ID, category of the 3D element, 3D bounding box (length, width, and height) of the 3D element, transformation matrix of the 3D element (to obtain the 3D element's absolute position and orientation in the 3D room), 3D model corresponding to each of the 3D elements, macro category of the 3D element (optional), list of ingoing edges, list of outgoing edges, parent and / or superstructure tags, if any (e.g., "is_hub", "is_spoke", "is_chain"). The scene diagram may include a list of edges to represent relationships between the 3D elements in the room. Each edge may include the following attributes: edge ID, start node ID, end node ID, direction (e.g., "front," "back," "left," "right," "support"), optional tag (e.g., "contained"), functionality (optional), distance tag, the exact value of the distance between the bounding box of the edge's start node and the bounding box of the end node, and / or superstructure tag, if any (e.g., "is_spoke," "is_chain"). The distance tag may, for example, be "adjacent" if the minimum distance between the bounding box of the edge's start node and the bounding box of the edge's end node is less than a "adjacent" threshold (e.g., 5 cm). The distance tag may, for example, be "proximal" if the minimum distance between the bounding box of the edge's start node and the bounding box of the edge's end node is greater than a "adjacent" threshold and less than another "proximal" threshold (e.g., 50 cm). A distance tag may be, for example, "distal" if the smallest distance between the bounding box of the edge's start node and the bounding box of the edge's end node is greater than a "proximal" threshold.

[0078] 2 shows an example of a room diagram 100. In this example, the attributes are: For "Node 5", the attributes are: "category": "Double Bed", "macrocategory": "Bed", "transformation_matrix": [0,1,0,0,-1,0,0,0,0,0,1,0,-13302.5,-5203.2,455.0,1], "bounding_box": [2000,1800,1000], #mm, "3D_model": #Ref3DModel, "ingoing_edges": [Edge 1], "outgoing_edges": [Edge 2], "parent": ["Floor", "Default"], "superstructure_tags": []. For “edge 2”, the attributes are “starting_node”: “node 5”, “ending_node”: “node 6”, “direction”: “right”, “optional_tag”: “normal”, “distance_tag”: “proximal”, “distance_value”: 25, #mm, “superstructure_tag”: [].

[0079] FIG. 3 shows a flow chart of an example of the offline stage S110 and the online stage S120.

[0080] The offline stage S110 involves the calculation of the dataset (statistical calculation). The offline stage S110 corresponds to the calculation of various statistical files used for positioning. All these statistics are calculated based on a representative dataset of the 3D scene (each having a scene schema corresponding to the 3D scene). Statistical analysis of the dataset makes it possible to extract the number of occurrences of decoration-functional object pairs, and optionally decoration-decorative object pairs. From the number of occurrences of these pairs, the method may include calculating the frequency of each pair's appearance. The occurrences and frequencies of each pair are organized by object category, so that, if necessary, it is possible to abstract the exact 3D model into its category. Furthermore, the method may further include extracting the relative location of each pair of objects as they appear in the dataset and storing the relative locations in a relative location dictionary organized by object category and model. Optionally, the method includes fitting a Gaussian mixture model to the relative locations of each pair of objects and saving the parameters of the Gaussian mixture model distribution for each pair in a specific file.

[0081] The online stage S120 is concerned with location calculation when adding a new decoration 3D object (input 3D object) in a 3D scene. The online stage S120 receives as input the 3D scene, the decoration 3D object to be added in the 3D scene, and a statistical file (calculated dataset) calculated in the offline stage. Based on the statistical file with the occurrence and frequency of object pairs, the method includes selecting anchor candidates for the object to be added among the 3D objects already present in the 3D scene. For each of these candidate anchors, the method calculates a location based on a relative position dictionary (or, if a relative position dictionary exists, by sampling in the distribution of a corresponding Gaussian mixture model stored in an optional file). The positions of the candidate anchors are then refined and ranked to be physically valid (by snapping the candidate anchor to its anchor object or filtering the candidate anchor if it conflicts with other 3D objects in the 3D scene). The component outputs a ranked list of locations of the 3D decoration objects in the 3D scene. The added decoration 3D object is then inserted into the 3D scene at the first location in the list or by user interaction, selected by the user among all output locations.

[0082] The offline stage S110 is discussed in more detail next. All steps detailed in this subsection may be run at time intervals or with different data sets in order to update the statistical files to the most recent or most representative data at a given moment. For example, the method may regenerate these files periodically (e.g., monthly) to take into account updates in the 3D object database (e.g., after the addition of a new 3D object in the catalog discussed previously).

[0083] The offline stage S110 comprises calculating pair occurrence statistics S111. Based on a representative dataset of 3D indoor scenes with a scene schema corresponding to the 3D indoor scene defined above, the method comprises extracting a number of statistics regarding the occurrence of objects in a project of objects as well as the occurrence of pairs of objects.

[0084] A representative data set can be created to include only rooms of a particular type (eg, only bedrooms or only living rooms) or a combination of several room types.

[0085] For each project, based on the project's scene schema, calculation S111 counts associated function-decoration object and optionally decoration-decoration object pairs, considering "adjacent" and "supporting" edges between function objects and decoration objects, and optionally between two decoration objects. Calculation S111 formalizes pair naming by anchor-decoration pairs, where the anchor can be either a function object or, optionally, a decoration object. Calculation S111 aggregates all pair occurrences in the dataset into a count file organized by decoration category.

[0086] 4 shows an example of the occurrence of pairs. In this example, Object 5 101 and Object 6 103 are functions, and Object 7 104 and Object 8 105 are decoration objects. The pairs including Objects 5 and 7 and Objects 6 and 8 are considered pairs of associated function-decoration objects.

[0087] The structure of the file to be calculated is as follows:

[0088] {<decorative_category> : {<decorative_model_ID_of_that_category> : {<anchor_category_associated_with_it> : {"total": the total number of occurrences of that decoration model ID pair with objects in the anchor's category, <anchor_model_ID_of that category> : the number of occurrences of the pair of the decoration model ID with the anchor's model ID, …}, …}, …}, …} From this first count file, calculation S111 includes deriving a second file of frequencies of occurrence of anchor-decoration object pairs. For each pair of objects in the count file, calculation S111 includes deriving a count of the pair occurrence into a frequency. Specifically, for a "Total" key, calculation S111 includes dividing the value by the sum of all "Total" values ​​for this decoration model ID. For an anchor key corresponding to a model ID, calculation S111 includes dividing the count by the value of the "Total" key for this particular anchor category. Thus, calculation S111 includes obtaining a frequency value between 0 and 1. The sum of all "Total" fields for a given decoration model ID is equal to 1. The sum of all associated model IDs with a model ID from the same anchor category is also equal to 1.

[0089] The frequency file is organized similarly to the count file. Figure 5 shows an example of a frequency file obtained for model ID "8901."

[0090] The idea behind this frequency file is to identify what other objects a decoration object most frequently decorates, both in terms of object category and object model. Referring to the example in Figure 5, the decoration object with ID "8901" decorates shelves in 75% of its occurrences in the dataset, and only models of shelves "1747", "80", and "585".

[0091] Going beyond object category to object model ID allows geometric compatibility to be taken into account. Indeed, for a sofa model, the method is able to identify the most geometrically compatible cushion model based on the frequency of cushion model occurrence in the dataset.

[0092] The offline stage S110 then includes calculating S112 relative position statistics. Based on the same representative data set as for the pair occurrence statistics, calculation S112 includes extracting information about the relative positions of objects in pairs of objects. By relative position, the method refers to the position of a decorative object relative to the anchor object it decorates, i.e., the position of the decorative object in the coordinate system of the anchor object.

[0093] For each occurrence of an object pair counted in the previous step S111, calculation S112 involves saving the relative position of the decoration object with respect to the anchor object as a transformation matrix. To save memory space, this transformation matrix may be reduced to four values: the x, y, and z coordinates in the anchor object's coordinate system, and an orientation value. Calculation S112 thus obtains, for each anchor-decoration pair, a list of the relative positions of the decoration object with respect to its anchor. These positions are all stored in a relative positions file with the following structure:

[0094] {<anchor_category> : {<anchor_model_ID_of_that_category> : {“bbox”: the bbox parameter of the anchor model, <decorative_category_associated_with_it> : {<decorative_model_ID_of that category> : {"bbox": the bbox parameter of the decoration model, "rel_transfos": a list of relative positions found in the dataset, stored as transformation matrices, …}, }, …}, …} An example of the determined positions is shown in Figure 6. In this example, the determined positions of model "4852" are shown.

[0095] Fitting a Gaussian mixture model S113 is discussed next. This step is optional. For example, the offline stage S110 may include this fitting S113 step when there are enough occurrences of anchor-decoration object pairs. Fitting S113 may include fitting a Gaussian mixture model (GMM) to the relative positioning of the objects in these pairs.

[0096] This step S113 allows fitting a mixture of Gaussian distributions to the relative positions of the objects and sampling from Gaussian distributions, instead of just selecting positions found in the relative position file obtained from the representative dataset. Moreover, since the method can only consider the bounding boxes of 3D objects without their exact 3D shapes, fitting a GMM allows for a better understanding of the anchor 3D model.

[0097]

[0033] Figure 7 shows a first example of fitting a Gaussian mixture model. This figure shows a lamp position (point 110) relative to a nightstand model (volume 112). When fitting a GMM to this position, the method obtains two 3D Gaussian components 114, indicating that the nightstand has a hole in the center where a lamp can be placed. Figure 8 shows a second example of fitting a Gaussian mixture model. This figure shows a cushion position (point 120) relative to a sofa model (volume 122). When fitting a GMM to this position, the method obtains two 3D Gaussian components 124, indicating that the sofa has an L-shape.

[0098] In particular, for each anchor-decorative object pair with sufficient occurrences (at least 10 occurrences in our case), the method performs the following steps:

[0099] In a first step, the method involves determining the number of Gaussian components that best fits our data, here the relative position of the decorative object with respect to the anchor. To do so, the method may use the scikit-learn library's implementation of Gaussian mixture model fitting [7], which, given the dataset and the number of Gaussian components, estimates the parameters of each of the components, i.e., their weights, means, and covariances. The method iteratively fits a GMM to the data by iterating over the number of Gaussian components from 1 to 10. For each of these models with different numbers of components, the method calculates a silhouette score, e.g., also based on the scikit-learn library [7], and retains the model with the best silhouette score (i.e., the model with the largest silhouette score).

[0100] In a second step, the method includes fitting a GMM to a model with a previously determined number of components. The method may retain only components with weights >0.05 (5%) to avoid components formed by outliers in the dataset. The method may save the parameters (weights, means, and covariances) of each of the components in a list.

[0101] This process is also performed on an aggregation of all decoration objects of the same category for a given anchor, in order to aggregate information about the anchor even when the anchor is decorated by various decoration objects of the same category (although each of the decoration objects considered separately may represent only a few occurrences). The method may save the GMM parameters in a separate file organized similarly to the previously calculated statistics file.

[0102] Figure 9 shows a sample example of model "4975". The GMM for all curtain positions decorating "4975" has two components, while the GMM for "15915" curtains has three components.

[0103] The online phase S120 is discussed in more detail below.

[0104] The online phase S120 includes determining candidate anchors S121. Given a partially furnished 3D scene with 3D objects, a 3D decoration object to be added to the scene and a frequency file calculated on a representative dataset, the method tries to identify potential anchors for the inserted 3D decoration object (i.e., functional objects to decorate). The model ID of the decoration object to be added is indicated by deco_ID and the category of the decoration object to be added by deco_cat.

[0105] Determining candidate anchors S121 includes executing computer program instructions that cause an attempt to identify pairs. In particular, the computer program instructions may include instructions for performing the following steps to create a list of candidate anchors:

[0106] - initializing an empty list of candidate anchors; - Check if deco_ID is in the primary key of the frequency file. If it is not there, it means that the object did not exist in the dataset used for the object's creation. This can happen when adding objects that were added in the database of 3D objects after the frequency file calculation. If the model is unknown, deco_ID takes the value of the model ID of the object with the closest size from the deco_cat category.

[0107] - For each 3D object in the scene, obj_ID takes into account the model ID of the 3D object and obj_cat its category; If the pair obj_ID-deco_ID exists in the frequency file, then add the 3D object in the list of candidate anchors.

[0108] o Otherwise, if this pair does not appear in the frequency file, but there are pairs with obj_ID and other decoration objects of the deco_cat category, among these pairs, select the pair with the decoration object closest in size from the pair to be added into the scene. The decoration model ID of this pair is indicated by closest_deco_ID. Then add the 3D object into the list of candidate anchors along with the closest pair information.

[0109] Otherwise, if this pair does not appear in the frequency file, determine the object among the obj_cat objects that is closest in size to the 3D object and indicate the model ID of that object by closest_obj_ID.

[0110] If the pair closest_obj_ID-deco_ID exists in the frequency file, then add the 3D object into the list of candidate anchors together with the closest pair information.

[0111] Otherwise, if this pair does not exist but there is a pair with closest_obj_ID and another decoration object of the deco_cat category, select among these pairs the pair with the decoration object closest in size to the one to be added in the scene. The decoration model ID of this pair is indicated by closest_deco_ID. Then add the 3D object to the list of candidate anchors along with the closest pair information.

[0112] Otherwise, consider the considered 3D object not possible to be an anchor and move to the next 3D object in the 3D scene.

[0113] By executing the computer program instructions, a list of candidate anchors for 3D decoration objects to be added in the 3D scene is thus obtained. Each element of the candidate anchor list points to an object already in the 3D scene, and is itself a list containing:

[0114] - idx: index of the 3D object in the 3D scene, freq: the frequency taken from the frequency file for the pair used to add the 3D object in the candidate list (according to the step in which the 3D object was added), - ref_deco_model: closest_deco_ID if used, "none" otherwise, - ref_anchor_model: closest_obj_ID if used, "none" otherwise.

[0115] After identifying the pairs, determining candidate anchors S121 includes outputting pairs among the identified pairs. The outputting may be based on a ranking by count of the identified pairs. For example, the method may include first ranking the obtained anchor candidate list from highest to lowest by their frequency (i.e., the second value of the sublist). The method may then re-rank the anchor candidate list to first include candidate anchors that appeared as pairs in the frequency file with the original decoration model ID to be added to the 3D scene, i.e., candidate anchors whose ref_deco_model is "none." Figure 10 shows an example of a ranked list of candidate anchors obtained by the method using these computer program instructions.

[0116] If no 3D objects in the 3D scene are added to the list of candidate anchors, the decoration object is inserted in a default location in the 3D scene (e.g., the center of the room or the free space in the room visible on the user's screen).

[0117] Once the list of candidate anchors is obtained, the online stage S120 includes calculating S122 the locations of the output pairs (i.e., determining for each of the output pairs the respective position of the input 3D object in the input 3D scene). Calculating S122 the locations includes calculating the 3D position of the decoration object to be inserted for each of the candidate anchors in the obtained list.

[0118] In particular, for each candidate anchor in the previously calculated list, the method may execute computer program instructions to perform the following:

[0119] If i.ref_deco_model is "none" (i.e. the deco_ID of the selected pair is the original model ID of the decoration model to be added into the 3D scene), a.If ref_anchor_model is "none" (i.e. the selected pair is obj_ID-deco_ID), randomly select a relative position from the "rel_transfos" list for the pair obj_ID-deco_ID in the relative positions file. Then apply the relative position to the inserted decoration object with respect to the anchor candidate to get the absolute position of the inserted decoration object (as a transformation matrix).

[0120] b. Otherwise, when ref_anchor_model is not "None" (i.e., the selected pair is ref_anchor_model-deco_ID), randomly select a relative position from the "rel_transfos" list of the pair ref_anchor_model-deco_ID in the relative positions file. Then apply the relative position to the inserted decoration object with respect to the anchor candidate to obtain the absolute position of the inserted decoration object (as a transformation matrix).

[0121] ii. Otherwise, if ref_deco_model is not "none" (i.e. the selected pair does not contain the original model ID deco_ID of the decoration model to be added into the 3D scene, but contains the closest one in size from the same category), a.If ref_anchor_model is "none" (i.e. the selected pair is obj_ID-ref_deco_model), randomly select a relative position from the "rel_transfos" list of the pair obj_ID-ref_deco_model in the relative positions file. Then apply the relative position to the inserted decoration object with respect to the anchor candidate to get the absolute position of the inserted decoration object (as a transformation matrix).

[0122] b. Otherwise, when ref_anchor_model is not "None" (i.e., the selected pair is ref_anchor_model-ref_deco_model), randomly select a relative position from the "rel_transfos" list of the pair ref_anchor_model-ref_deco_model in the relative positions file. Then apply the relative position to the inserted decoration object with respect to the anchor candidate to obtain the absolute position of the inserted decoration object (as a transformation matrix).

[0123] Alternatively, instead of selecting a 3D location directly from the relative position file, the method may sample the 3D location using GMM parameters in a GMM parameter file if they have been calculated in an offline phase. Alternatively, the method may sample the 3D location of each Gaussian component of the GMM. Alternatively, instead of sampling directly from the 3D location, the method may select a position from the relative position file that is closest to the center of the GMM component. These different alternatives result in different behaviors and different levels of 3D positioning versatility.

[0124] Validation of no intersection is discussed next (conflict management). Once the absolute 3D position of the decorative object to be inserted is obtained, the method may include verifying that the absolute 3D position does not collide with other objects in the 3D scene. Since the method may only consider the bounding box of the 3D object, the method may not consider collisions with the anchor of the inserted object. In fact, some collisions may not be an issue in the bounding box sense (e.g., a cushion on a sofa will have its bounding box contained within the bounding box of the sofa, but is a valid location).

[0125] In particular, for each object in the 3D scene, excluding the inserted object's anchor, the method may include calculating the volume of the intersection of that object's bounding box with the inserted object's bounding box. The method may also include calculating their union volume. The method may then include using an Intersection over Union (IoU) ratio to determine whether the collision should be considered a conflict. If the IoU ratio is greater than a threshold (0.05 in our implementation), the method may include considering the collision to be a conflict and may include ignoring the solution.

[0126] Alternatively, in case of a conflict, the method may comprise selecting another position from the "rel_transfos" list of the corresponding pair depending on the case in which the insertion is made (i.a., i.b., ii.a. or ii.b.), or alternatively, if the method is computing the GMM parameters in an offline phase, the method may comprise sampling a new position, for example, from another component.

[0127] Snapping is discussed in more detail next. In cases ib, ii.a, and ii.b, if the method uses other objects that are not objects in the 3D scene or the 3D object to be added, i.e., ref_anchor_model or ref_deco_model, or both, it will lead to calculation of an incomplete 3D position of the object to be inserted relative to the anchor of the object to be inserted. In practice, the inserted object may float above or collide with the anchor of the inserted object. To realign the inserted object, the method may use a "snap" function to improve and correct its 3D position.

[0128] The snap function takes as input the inserted object, an origin on this object's bounding box, and a direction to search for a surface to which the inserted object should be snapped.

[0129] The method may distinguish between three cases for snapping:

[0130] When the inserted object is a ceiling lamp, the method may snap it to the ceiling. The top of the ceiling lamp's bounding box may be placed relative to the surface of the ceiling, and the z-coordinate of the ceiling lamp may be modified accordingly. In this case, the origin is the upper center of the ceiling lamp's bounding box, and the direction is upwards (towards the ceiling).

[0131] When the inserted object is a wall object (e.g., a wall lamp, a picture frame, a wall mirror), the method may snap this object to the nearest wall. The back of the wall object may be placed relative to the wall surface, and the x and y axes of the wall object are modified. In this case, the origin is the center of the back side of the wall object's bounding box, and the direction is the normal to the back side of the bounding box. The first wall surface found in this direction may be the one to which the object will be snapped.

[0132] In all other cases, the method may snap the inserted object to the nearest horizontal surface, taking gravity into account and modifying only the z coordinate of the inserted object. In this case, the origin is the center of the lower side of the bounding box of the inserted object, and the method may consider two directions: one direction downwards (towards the floor) and the other upwards (towards the ceiling). The method may collect the nearest horizontal surface for each direction and finally perform a snap to the nearest surface between these two directions.

[0133] This also allows for 3D position correction of inserted objects in cases where the dataset used for the calculation of the statistical file in the offline phase is not of sufficiently good quality (the user may misplace an object in the object's project).

[0134] The last two steps ensure that the 3D positions calculated for the inserted decoration objects are physically valid (no collisions and no floating objects).

[0135] With reference to Figures 11 to 22, examples of positioning an input 3D object will now be discussed.

[0136] In each of Figures 11 through 18, two different objects belonging to the same category are positioned by the method in two different bedrooms. For example, Figure 11 shows a first bedroom 210 in which a first lamp 211 is positioned by the method, and a second bedroom 220 in which a second lamp 221 is positioned by the method. For each of the bedrooms, the figure shows the different relative positions calculated by the method. For the first lamp 211 in the first bedroom 210, the method provides three respective positions 212, 213, and 214 (corresponding respectively to each output pair). The user can therefore select whether to position the lamp between the three respective positions 212, 213, and 214. For the second lamp 221 in the second bedroom 210, the method provides four respective positions 222, 223, 224, and 225 (the pairs output by the method include an additional pair due to the presence of an additional table in the second bedroom 210). The user can therefore select whether to position the lamp between each of the four determined positions 222, 223, 224 and 225.

[0137] 12 shows examples of locating vertical overlapping relationships. In these examples, the method identifies that the dataset includes a pair of occurrences including lamps 231 and 241 (added to the bedroom) positioned according to a vertical overlapping relationship with other 3D objects in the bedroom (the bed or the bedroom table). The method therefore calculates the positions of lamps 231 and 241 above these objects.

[0138] Similarly, Figures 13 to 18 show the positions calculated by the method for the same bedroom discussed previously when the input 3D objects are a cushion, a plant, a picture frame, a mirror, a rug or a curtain, respectively.

[0139] In Figures 19-21, two different objects are positioned by the method, this time in two different living rooms. For example, Figure 19 shows a first living room 310 to which a first lamp 311 is added, and a second living room 320 to which a second lamp 321 is added. For each of the living rooms, the figure shows different relative positions 312 calculated by the method. The user can therefore select whether to position the lamp between these different relative positions 312. Similarly, Figures 20 and 21 show positions calculated by the method for the same living room when the input 3D object is a cushion or a picture frame, respectively.

[0140] Figure 22 shows the positions calculated by the method when the input 3D object is a book for a living room with different shelves, in which case the method allows calculating positions at different shelf levels.

[0141] FIG. 23 shows an example of the GUI of this system, which is a CAD system.

[0142] The GUI 2100 may be a typical CAD-like interface with standard menu bars 2110, 2120 and bottom and side toolbars 2140, 2150. As known in the art, such menu bars and toolbars include a set of user-selectable icons, each of which is associated with one or more operations or functions. Some of these icons are associated with software tools adapted for editing and / or processing the 3D modeled object 2000 displayed in the GUI 2100. The software tools may be grouped into workbenches. Each of the workbenches includes a subset of the software tools. In particular, one of the workbenches is an editing workbench suitable for editing geometric features of the modeled product 2000. During operation, a designer may, for example, pre-select a portion of the object 2000 and then initiate an operation (e.g., change dimensions, color, etc.) or edit geometric constraints by selecting the appropriate icon. For example, a typical CAD operation is modeling a hole or a fold in the 3D modeled object displayed on the screen. The GUI may, for example, display data 2500 related to the displayed product 2000. In the illustrated example, the data 2500 displayed as a "feature tree" and their 3D representation 2000 relate to a brake assembly including a brake caliper and disc. The GUI may further present various types of graphic tools 2130, 2070, 2080, for example, to facilitate 3D orientation of objects, to trigger simulations of the edited product's motion, or to render various attributes of the displayed product 2000. A cursor 2060 may be controlled by a haptic device to allow a user to interact with the graphic tools.

[0143] FIG. 24 shows an example of this system, which is a client computer system, such as a user's workstation.

[0144] The client computer in this example includes a central processing unit (CPU) 1010 connected to an internal communication bus 1000 and a random access memory (RAM) 1070 also connected to the bus. The client computer is further provided with a graphical processing unit (GPU) 1110 associated with a video random access memory 1100 connected to the bus. The video RAM 1100 is also known in the art as a frame buffer. A mass storage controller 1020 manages access to mass memory devices such as a hard drive 1030. Mass memory devices suitable for tangibly containing computer program instructions and data include all forms of non-volatile memory, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks and removable disks, and magneto-optical disks. Any of the above may be augmented by or incorporated in specially designed ASICs (application-specific integrated circuits). A network adapter 1050 manages access to a network 1060. The client computer may also include a cursor control device, a haptic device 1090 such as a keyboard, etc. A cursor control device is used in the client computer to allow a user to selectively position a cursor at any desired location on the display 1080. In addition, the cursor control device allows the user to select various commands and input control signals. The cursor control device includes several signal generating devices for inputting control signals to the system. Typically, the cursor control device may be a mouse, and the mouse buttons are used to generate the signals. Alternatively or additionally, the client computer system may be equipped with a sensitive pad and / or a sensitive screen.

[0145] A computer program may include computer-executable instructions, which include means for causing the system to perform the method. The program may be recordable on any data storage medium, including the system's memory. The program may be implemented, for example, in digital electronic circuitry, or in computer hardware, firmware, software, or a combination thereof. The program may also be implemented as an apparatus, for example, as an article of manufacture tangibly embodied in a machine-readable storage device for execution by a programmable processor. The method steps may be performed by a programmable processor executing a program of instructions to perform the functions of the method by operating on input data and generating output. The processor, thus programmable, may be coupled to receive and transmit data and instructions from and to a data storage system, at least one input device, and at least one output device. The application program may be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language, if desired. In either case, the language may be a compiled or interpreted language. The program may be a full installation program or an update program. Application of the program on a system, in either case, produces instructions for performing the method. The computer program may alternatively be stored and executed on a server in a cloud computing environment, the server being in communication with one or more clients via a network, in which case the processing unit executes the instructions contained by the program, thereby causing the method to be performed on the cloud computing environment.

Claims

1. 1. A computer-implemented method for automatically positioning an input 3D object representing a real object in an input 3D scene representing a room, the input 3D scene comprising a plurality of positioned 3D objects, the plurality of positioned 3D objects comprising positioned furniture 3D objects each representing a piece of real furniture, the method comprising: Obtaining (S10) a data set comprising information about a plurality of room objects, said information comprising: A classification of the objects in the rooms; and each of a set of counts consisting of each respective pair of occurrences involving the same first real object and the same second real object in a room, wherein the first real object and the second real object of each respective pair of occurrences are arranged according to a mutual placement relationship from a predefined set of one or more mutual placement relationships; information about the relative positioning of the first real object and the second real object in each occurrence of each pair; and and - executing (S20) computer program instructions, said instructions comprising: - attempting to identify in said dataset one or more first pairs, each pair comprising said input 3D object and a respective positioned 3D object of said input 3D scene; if no first pair is identified, attempting to identify in the dataset one or more second pairs each including only one of the input 3D object and the respective positioned 3D object of the input 3D scene and an equivalent object of the other, wherein a given real object is an equivalent object of another given real object when said given real object has the same classification as said other given real object, said given real object being closest in size to said other given real object in the set consisting of each real object of the same classification present in the dataset; when the first pair is not identified and the second pair is not identified, attempting to identify in the dataset one or more third pairs, each comprising an equivalent object of the input 3D object of the input 3D scene and an equivalent object of each positioned 3D object; 5. A step of causing - outputting (S30) one or more pairs in the set of each identified pair and the count of the one or more identified pairs, said outputting being based on the set of each identified pair's count of each identified pair in the set of each identified pair; - determining (S40), for each output pair, a position of each of the input 3D objects in the input 3D scene based on the information on the relative positioning of the first real object and the second real object in each occurrence of the output pair; - positioning (S50) the input 3D object according to the respective positions determined for one of the one or more output pairs; A method for providing the above.

2. The method of claim 1 , wherein the predetermined set of one or more mutual placement relationships includes an adjacent relationship and / or a vertical overlap relationship.

3. The method of claim 1 or 2, wherein the pairs of the data set comprise pairs including a decorative object and a furniture object.

4. The method of claim 1 , wherein the pairs of the data set include pairs that include a decorative object and another decorative object.

5. The step of determining (S40) the respective positions of the output pairs includes: - relative positioning (S41) of the input 3D object with the other object of the pair in the input 3D scene based on the information on the relative positioning of the first real object and the second real object in each of the output occurrences of the pair; Snapping the relatively positioned input 3D object onto an element of the 3D scene (S42); The method according to claim 1 , comprising:

6. 6. The method of claim 5, wherein, for each output pair, the information regarding the relative positioning of the first real object and the second real object in each occurrence of the output pair comprises a respective relative positioning for each occurrence, and the method further comprises fitting parameters of a Gaussian mixture model to the relative positioning of objects in the occurrences of the pair prior to the relative positioning for each output pair (S41), the relative positioning (S41) comprising sampling in the distribution represented by the Gaussian mixture model for the fitted parameters.

7. 7. A method as claimed in any preceding claim, wherein each of the output pairs has a count higher than a predetermined number.

8. The method of claim 1 , wherein each of the rooms in the dataset is of the same type as the room represented by the input 3D scene.

9. 9. The method of claim 1, wherein for each output pair, the respective position of the input 3D object in the input 3D scene determined for the output pair does not have a conflict with any other 3D object of the plurality of positioned 3D objects for the input 3D scene.

10. 10. The method of claim 1, wherein the one or more output pairs comprise several output pairs, and the method comprises, prior to the positioning, user selection of the one of the one or more output pairs in which the input 3D object is to be positioned.

11. 10. The method of claim 1, wherein the one or more output pairs comprises a number of output pairs, the method further comprising prior to positioning, ranking the output pairs according to the count of the output pairs in the dataset, and the output pair to which the input 3D object is positioned is the pair with the highest ranking.

12. The method according to any one of claims 1 to 11, wherein the method further comprises, after the positioning (S50) of the input 3D object, displaying (S60) the positioned input 3D object.

13. A computer program comprising instructions for carrying out the method according to any one of claims 1 to 12.

14. A computer readable storage medium having recorded thereon the computer program of claim 13.

15. 14. A system comprising a processor coupled to a memory, the memory having stored thereon a computer program as claimed in claim 13.

Images