Locating input 3D object in input 3D scene

Abstract

The present disclosure particularly relates to a computer-implemented method for automatically locating an input 3D object representing a real object in an input 3D scene representing a room. The method includes obtaining a data set including information about objects in a plurality of rooms. The method includes executing computer program instructions to attempt to identify a first, second, and / or third pair. The method includes outputting one or more identified pairs in a set composed of each identified pair and a count of the one or more identified pairs. The method includes, for each output pair, determining a corresponding position of the input 3D object in the input 3D scene. The method includes locating the input 3D object according to the corresponding location determined corresponding to one of the pair of the one or more outputs. The method improves the positioning of an input 3D object in an input 3D scene.

Description

Technical Field

[0001] The present disclosure relates to the field of computer programs and systems, and more particularly, to a method, system and program for automatically positioning an input 3D object representing a real object in an input 3D scene representing a room. Background Art

[0002] Many systems and programs are offered on the market for the design, engineering and manufacturing of objects. CAD is an abbreviation for Computer-Aided Design, for example, it relates to software solutions for designing objects. CAE is an abbreviation for Computer-Aided Engineering, for example, it relates to software solutions for simulating the physical behavior of future products. CAM is an abbreviation for Computer-Aided Manufacturing, for example, it relates to software solutions for defining manufacturing processes and operations. In such computer-aided design systems, graphical user interfaces play an important role in terms of technical efficiency. These technologies can be embedded in a Product Lifecycle Management (PLM) system. PLM refers to a business strategy that helps companies share product data, apply common processes, and leverage corporate knowledge to develop products from concept to the end of their life across the concept of an extended enterprise. The PLM solution provided by Dassault Systèmes (trademarks CATIA, SIMULIA, DELMIA and ENOVIA) provides an engineering center (organizing product engineering knowledge), a manufacturing center (managing manufacturing engineering knowledge) and an enterprise center (realizing enterprise integration and connecting to the engineering center and the manufacturing center). The entire system provides an open object model that links products, processes and resources to achieve dynamic, knowledge-based product creation and decision support, thereby driving optimized product definition, manufacturing preparation, production and service.

[0003] In this context, 3D scene completion applications are being developed. These applications are generally intended to create, manipulate and furnish 3D scenes, in particular (but not exclusively) for use with touch-sensitive devices such as smartphones or tablets. One of the tasks of these applications is to calculate a suitable position for a given 3D object to be added to a partially furnished 3D scene.

[0004] To solve this problem, most existing solutions include a step of predicting the position of new objects to be added to the 3D scene. Referring to the list of prior art references below, among these methods, several use deep learning algorithms such as references [1, 2, 3], others use data-driven and optimization techniques such as references [4, 5], and others use process-based methods that rely on manually designed rules such as reference [6].

[0005] The methods described in references [1] and [2] use a neural network to predict the positions of previously predicted objects to be added to a 3D scene. In reference [1], the neural network takes as input the category of the object to be added, a top-down view image of the room, and some spatial constraints that the added object must obey (e.g., to add a TV, the constraint is that it must be placed in front of the bed). This method does not work well if the constraints are not appropriate or cannot be satisfied (e.g., if the bed is in front of a window or a wardrobe, then the TV cannot be placed in front of the bed). In reference [2], the neural network takes as input the category of the object to be added and a top-down view image of the room. Both methods in references [1] and [2] use a second neural network to predict the orientation of the object to be added.

[0006] The method described in [3] uses a Transformer neural network to complete 3D scenes. The Transformer predicts the features of the object to be added to the scene, and four attribute extractors (MLP networks) predict its class, position, orientation, and size in precise order. If the class of the object is constrained, its position, orientation, and size can be predicted. Then, the object belonging to the given class and with the closest size is selected from the catalog and added to the scene at the predicted position and orientation.

[0007] The methods described in references [1, 2, 3] only consider the class of the object and not the 3D model itself, which increases the limitations of their use. For example, depending on how finely the class is defined, it may not be able to distinguish a table lamp from a floor lamp, a wall lamp, or a ceiling lamp, and therefore predict inappropriate positions for these objects. In addition, they all rely on neural networks, which may limit integration and deployment in applications. In fact, deep learning-based algorithms require the deployment of specific infrastructure.

[0008] The method described in reference [4] uses a genetic algorithm to optimize the layout of 3D furniture pieces in a 3D room according to manually defined interior design KPIs (e.g., movement or alignment). This method uses manually defined rules and KPIs and can take a long time to compute. This method is therefore not suitable for end-user applications and is not suitable for furniture catalogs that may evolve as new categories or objects are added.

[0009] The method described in [5] uses a graph-based approach to furnish a 3D room with a given set of 3D furniture pieces. It groups objects together and computes the relative positions of objects within a group using a template. The template is obtained using a data-driven approach based on the relative positions of furniture classes in a dataset of 3D scenes. The positions of the groups in the 3D room are computed using an optimization algorithm. Because it considers the furniture classes for template computation, it relies on accurate annotations of the furniture.

[0010] Finally, reference [6] proposes a streamlined approach for 3D scene generation based on hand-crafted rules. Specifically, each 3D furniture in the dataset is accurately annotated with its class, size, and class-dependent location rules (e.g., “on the floor” for a sofa, or “on an object” for a plate). Templates are manually designed for groups of furniture classes that are common in indoor scenes (e.g., four chairs around a table). Then, several groups of objects are iteratively sampled and placed in the 3D room. This method relies heavily on hand-crafted rules and templates defined at the furniture class scale, which suffers from the same shortcomings as previous methods.

[0011] List of prior art references:

[0012] [1]Wang,K.,Lin,Y.,Weissmann,B.,Savva,M.,Chang,AX,&Ritchie,D.(2019).PlanIT: planning and instantiating indoor scenes with relation graph and spatial prior networks.ACM Trans.Graph.,38,132:1-132:15.

[0013] [2] RITCHIE, Daniel, WANG, Kai, et LIN, Yu-an. Fast and flexible indoorscene synthesis via deep convolutional generative models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.p.6182-6190.

[0014] [3]Despoina Paschalidou,Amlan Kar,Maria Shugrina,Karsten Kreis,Andreas Geiger,Sanja Fidler.2021.ATISS:Autoregressive Transformers for IndoorScene Synthesis.NeurIPS2021.

[0015] [4]Peter Kan and Hannes Kaufmann.2017.Automated Interior Design Usinga Genetic Algorithm.In Proceedings of VRST2017,Gothenburg,Sweden,November2017,10pages.

[0016] [5]Yang L.,Zhang S.,Zhang S.,Xie W.,Fu H.(2020).Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extraction.arXiv:2002.00328v2

[0017] [6]Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Roozbeh Mottaghi.2022.ProcTHOR:Large-Scale Embodied AI Using ProceduralGeneration.NeurIPS2022.

[0018] [7]Document EP4075316 A1.

[0019] [8]Scikit-learn library implementations of Gaussian Mixture Models and silhouette scores.https: / / scikit-learn.org / stable / modules / generated / sklearn.

[0020] mixture.GaussianMixture.html

[0021] https: / / scikit-learn.org / stable / modules / generated

[0022] / sklearn.metrics.silhouette_score.html

[0023] Against this background, there remains a need for improved solutions for localizing input 3D objects in an input 3D scene. Summary of the invention

[0024] Therefore, the present disclosure provides a computer-implemented method for automatically positioning an input 3D object representing a real object in an input 3D scene representing a room. The input 3D scene includes a plurality of positioned 3D objects. The plurality of positioned 3D objects include positioned furniture 3D objects, each of which represents a real piece of furniture. The method includes obtaining a data set, the data set including information about objects in a plurality of rooms. The information includes classifications of the objects in the plurality of rooms. The information includes counts, each of which is a count in a set consisting of each occurrence of a corresponding pair of the same first real object and the same second real object in the room. The first real object and the second real object in each occurrence of each corresponding pair are arranged according to a mutual arrangement relationship from one or more preset sets of mutual arrangement relationships. The information includes information about the relative positioning of the first real object and the second real object in each occurrence of each pair.

[0025] The method comprises executing computer program instructions, which cause: attempting to identify one or more first pairs in the data set, each of which comprises the input 3D object and a corresponding positioned 3D object in the input 3D scene. The computer program instructions also cause: when no first pair is identified, attempting to identify one or more second pairs in the data set, each of which comprises only one of the input 3D object and the corresponding positioned 3D object in the input 3D scene, and an equivalent object of the other. When a given real object has the same classification as another given real object and the given real object is closest in size to the other given real object in a set of real objects of the same classification present in the data set, the given real object is an equivalent object of the other given real object. The computer program instructions also cause: when no first pair is identified and no second pair is identified, attempting to identify one or more third pairs in the data set, each of which comprises an equivalent object of the input 3D object and an equivalent object of the corresponding positioned 3D object in the input 3D scene.

[0026] The method includes outputting one or more identified pairs and counts of the one or more identified pairs in the set consisting of each identified pair. The output is based on the set consisting of each count, each count corresponding to an identified pair in the set consisting of each identified pair. The method includes determining, corresponding to each output pair, a corresponding position of the input 3D object in the input 3D scene based on information about the relative positioning of the first real object and the second real object in each occurrence of the output pair. The method includes positioning the input 3D object according to the corresponding position determined corresponding to one of the one or more output pairs.

[0027] The method may include one or more of the following:

[0028] - the preset one or more sets of mutual arrangement relationships include an adjacency relationship and / or a vertical stacking relationship;

[0029] - said pairs of said data set include: pairs comprising decorative objects and furniture objects;

[0030] - said pairs of said data set comprise: pairs comprising a decorative object and another decorative object;

[0031] -Determining the corresponding position for each output pair comprises:

[0032] o Based on the information about the relative positioning of the first real object and the second real object in each occurrence of the output pair, relatively positioning the input 3D scene with another object in the pair

[0033] Subject; and

[0034] o snapping the relatively positioned input 3D object to an element of the 3D scene;

[0035] - For each output pair, the information about the relative positioning of the first real object and the second real object each time it appears in the output pair includes the relative positioning corresponding to each occurrence. The method also includes, before performing relative positioning for each output pair, fitting Gaussian mixture model parameters to the relative positioning of the objects appearing in the pair; the relative positioning includes sampling in the distribution represented by the Gaussian mixture model of the fitted parameters.

[0036] - Each output pair has a count higher than a preset number.

[0037] - Each room of the data set is of the same type as the room represented by the input 3D scene.

[0038] - For each output pair, the corresponding position of the input 3D object determined for the output pair in the input 3D scene does not conflict with any other 3D object of the plurality of positioned 3D objects in the input 3D scene.

[0039] - the one or more output pairs include several output pairs. The method further includes: before positioning, a user selecting one of the one or more output pairs from which to position the input 3D object.

[0040] - the one or more output pairs include a plurality of output pairs. The method further comprises: prior to positioning, ranking the output pairs according to counts of the output pairs in the data set. The output pair from which the input 3D object is positioned is the pair with the highest ranking; and / or

[0041] - The method further comprises: after positioning the input 3D object, displaying the positioned input 3D object.

[0042] The present disclosure also provides a computer program comprising instructions for executing the method.

[0043] The present disclosure also provides a computer-readable storage medium on which the computer program is recorded.

[0044] The present disclosure also provides a system, which includes a processor coupled to a memory, wherein the memory has the computer program recorded thereon. The system may include a graphical user interface.

[0045] The present disclosure also provides a device, which includes a data storage medium on which the computer program is recorded. The device can form or be used as a non-transitory computer-readable medium, such as in SaaS (Software as a Service) or other servers, or cloud-based platforms, etc. The device can alternatively include a processor coupled to the data storage medium. Thus, the device can form a computer system in whole or in part (for example, the device is a subsystem of the entire system). The system can also include a graphical user interface coupled to the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Non-limiting examples will now be described with reference to the accompanying drawings, in which:

[0047] Figure 1 A flow chart showing an example of the method;

[0048] Figure 2 An example of a graph is shown;

[0049] Figure 3 A flow chart showing examples of offline and online phases;

[0050] Figure 4 An example of the occurrence of pairs is shown;

[0051] Figure 5 An example of counting is shown;

[0052] Figure 6 An example of the determined corresponding position is shown;

[0053] Figures 7 to 9 An example of fitting a Gaussian mixture model is shown;

[0054] Fig.10 An example of the ranking of the output pairs is shown;

[0055] Figures 11 to 22 An example of positioning of an input 3D object is shown;

[0056] Fig.23 An example of a graphical user interface of the system is shown; and

[0057] Fig.24 An example of a system is shown. DETAILED DESCRIPTION

[0058] Reference Figure 1A flowchart is provided, which proposes a computer-implemented method for automatically positioning an input 3D object representing a real object in an input 3D scene representing a room. The input 3D scene includes a plurality of positioned 3D objects. The plurality of positioned 3D objects include a positioned furniture 3D object, each furniture 3D object representing a real piece of furniture. The method includes S10: obtaining a data set, the data set including information about a plurality of objects in a plurality of rooms. The information includes a classification of the plurality of objects in the plurality of rooms. The information includes counts. Each count is a count in a set consisting of each occurrence of a corresponding pair of the same first real object and the same second real object in the room. Each occurrence of the first real object and the second real object in each corresponding pair is arranged according to a mutual arrangement relationship from one or more preset mutual arrangement relationship sets. The information includes information about the relative positioning of the first real object and the second real object in each occurrence of each pair.

[0059] The method comprises S20: executing computer program instructions, causing: attempting to identify one or more first pairs in the data set, each first pair comprising an input 3D object and a corresponding positioned 3D object in the input 3D scene. When no first pair is identified, executing computer program instructions causes: attempting to identify one or more second pairs in the data set, each second pair comprising only one of the input 3D object and the corresponding positioned 3D object in the input 3D scene, and an equivalent object of the other. When a given real object has the same classification as another given real object, and in a set consisting of each real object of the same classification present in the data set, the given real object is closest in size to the other given real object, the given real object is an equivalent object of the other given real object. When no first pair is identified and no second pair is identified, executing computer program instructions causes: attempting to identify one or more third pairs in the data set, each third pair comprising an equivalent object of the input 3D object and an equivalent object of the corresponding positioned 3D object in the input 3D scene.

[0060] The method comprises S30: outputting one or more identified pairs in a set consisting of each identified pair and counts of the one or more identified pairs. The output is based on a set consisting of each count, each count corresponding to an identified pair in the set consisting of each identified pair. The method comprises S40: for each output pair, determining a corresponding position of an input 3D object in an input 3D scene based on information about the relative positioning of a first real object and a second real object in each occurrence of the output pair. The method comprises S50: locating the input 3D object according to the corresponding position determined corresponding to one of the one or more output pairs.

[0061] Such an approach forms an improved solution for localizing an input 3D object in an input 3D scene.

[0062] In particular, the method allows to calculate the suitable position of the input 3D object within the input 3D scene based on the position of the furniture 3D object in the input 3D scene. In particular, the calculated position is suitable for physical laws and interior design rules. In fact, the counting of these pairs in the data set allows the statistical extraction of these physical laws and interior design rules from the 3D scene included in the data set. The claimed method allows to position the input 3D object in a way that complies with the physical constraints of the object, which are statistically determined on the 3D scene of the data set. For example, the method allows to calculate the positioning of the bedside lamp on the top surface of the nightstand, the positioning of the cushion on the top of the bed, or the positioning of the painting on the wall above the headboard. This is particularly useful for (but not limited to) touch screen devices that have difficulty handling 3D environments.

[0063] Furthermore, the claimed method is data driven and does not rely on neural networks. This has advantages for integrating it into existing applications that use current infrastructure. It is also fast to reach conclusions and can be used instantaneously from the user's perspective (position calculation takes only about 1 second). In fact, the method identifies pairs by simply iterating over the pairs in the data set and can output these pairs based solely on statistical information of the counts. Therefore, the method is based solely on identifying similar pairs and comparing the number of occurrences of each pair (since it is based on quantity, this comparison is fast). These statistical considerations are less computationally intensive (and therefore faster) than using neural networks, which are necessarily more computationally intensive to implement. Therefore, the claimed method is more efficient and requires fewer computer resources. This is an advantage because the method must be fast so that insertion appears instantaneous to the end-user application.

[0064] Moreover, the claimed method allows, where possible, to consider adding an accurate 3D model to the 3D scene, without abstracting it to its classification. In fact, the executed computer program instructions allow first considering pairs of identical input 3D objects. Furthermore, the claimed method also allows such abstraction if necessary. In fact, when a first pair is not identified, the executed computer program instructions allow considering a second pair of equivalent objects and a third pair of equivalent objects. Thus, the claimed method allows an adaptive search for similar pairs, depending on whether the input 3D objects are classified or not, which improves the relevance of the calculated positioning.

[0065] The method is computer-implemented. This means that the steps (or substantially all steps) of the method are performed by at least one computer or any similar system. Thus, the steps of the method may be performed fully or semi-automatically by a computer. In an example, the triggering of at least some of the steps of the method may be performed by user-computer interaction. The required level of user-computer interaction may depend on the level of automaticity foreseen and balanced with the need to implement the user's wishes. In an example, the level may be user-defined and / or pre-defined.

[0066] For example, the method, before executing S20, may include a step of selecting an input 3D object to be located. The selection may be made from a virtual catalog containing multiple potential input 3D objects. The virtual catalog may be updated by the object manufacturer (e.g., each time a new product is launched). For example, the virtual catalog may be stored in a database (e.g., available online), and each manufacturer may upload the objects they manufacture to the catalog. The selection may be performed by a user, such as a user who is currently designing a room. The selection may be performed in the virtual catalog in any manner. For example, a user may use an object reference to select an object in a catalog. Alternatively or additionally, the method may include displaying objects in the catalog (e.g., filtering a classification of objects), and the user may select one of the displayed objects (e.g., by performing a user interaction thereon, such as touching with a finger or stylus when the object is displayed on a touch screen or touchpad, or clicking a mouse when the user uses a mouse).

[0067] A typical example of a computer-implemented method is to perform the method using a system suitable for this purpose. The system may include a processor and a graphical user interface (GUI) coupled to a memory, on which a computer program including instructions for performing the method is recorded. The memory may also store a database. The memory is any hardware suitable for such storage, and may include several physically different components (e.g., one for the program and perhaps one for the database). Optionally, the system may include a touch screen or touch pad. The touch screen or touch pad may be coupled to the processor and the memory. A user may perform user interaction on the touch screen or touch pad. The method may display a GUI (e.g., including a 3D scene) on the touch screen or touch pad.

[0068] Data sets can be stored in a database. "Database" refers to any collection of data (i.e., information) organized for search and retrieval (e.g., a relational database, e.g., based on a predetermined structured language such as SQL). When stored on a memory, a database allows a computer to perform rapid search and retrieval. In fact, the structure of a database facilitates the storage, retrieval, modification, and deletion of data in conjunction with various data processing operations. A database can consist of a file or a set of files, which can be subdivided into records, each of which consists of one or more fields. A field is the basic unit of data storage. Users can retrieve data primarily through queries. Using keywords and sorting commands, users can quickly search, rearrange, group, and select fields from many records to retrieve or create reports on specific data aggregations, according to the rules of the database management system used.

[0069] The method generally operates on modeled (3D) objects. A modeled object is any object defined by data stored, for example, in a database. By extension, what is expressed by "modeled object" is the data itself. Depending on the type of system, the modeled object may be defined by different types of data. The system may actually be any combination of a CAD system, a CAE system, a CAM system, a PDM system and / or a PLM system. In these different systems, the modeled object is defined by the corresponding data. Accordingly, what is referred to may be said to be a CAD object, a PLM object, a PDM object, a CAE object, a CAM object, a CAD data, a PLM data, a PDM data, a CAM data, a CAE data. However, these systems are not mutually exclusive, since the modeled object may be defined by data corresponding to any combination of these systems. Therefore, it is very likely that a system is both a CAD and a PLM system.

[0070] In addition, a CAD system means any system that is at least suitable for designing a modeled object based on a graphical representation of the modeled object (e.g., CATIA). In this case, the data defining the modeled object includes data that allows the representation of the modeled object. For example, a CAD system can use edges or lines, and in some cases faces or surfaces, to provide representations of CAD modeled objects. Lines, edges, or surfaces can be represented in various ways, for example, non-uniform rational B-splines (NURBS). Specifically, a CAD file contains specifications, from which geometric figures can be generated, thereby enabling the representation to be generated. The specifications of a modeled object can be stored in a single CAD file, or in multiple CAD files.

[0071] In the context of CAD, a modeled object may typically be a 3D modeled object, e.g., representing a real object that is placed (or to be placed) in a real room (e.g., a furniture object or a decorative object). "3D modeled object" means any object that is modeled by data that allows its 3D representation. The 3D representation allows the part to be viewed from various angles. For example, when a 3D modeled object is represented in 3D, the 3D modeled object is manipulated or rotated about any of its axes or about any axis in the screen displaying the representation. This obviously excludes 2D icons, which are not modeled in 3D. Any 3D modeled object may be a CAD object, e.g., including a mesh or a B-rep (e.g., from calculations defined in a geometry feature history).

[0072] Each 3D object may represent the geometry of a real object positioned (or to be positioned) in a real room. The real object may be manufactured in the real world after its virtual design is completed (e.g., using a CAD software solution or a CAD system). The method takes into account a classification of objects, and each object (real or its 3D representation) may belong to a class of objects in the classification. The classification may include a category for each class of objects that is typically present in a room. The dataset includes a classification of objects in multiple rooms (e.g., the dataset may include a label for each object present in each room, the label indicating the category to which the object belongs). The input 3D object may also belong to a category of the same classification. The classification may include a furniture object category and a decorative object category. Each room (in the dataset and represented by the input 3D scene) may include one or more furniture objects. Each room may also include one or more decorative objects. The input 3D object may be a decorative object, and the method may position this decorative object relative to one of the other 3D objects (decorative object or furniture object) in the 3D scene.

[0073] In an example, the room represented by the room in the input 3D scene and / or dataset may be an indoor room. For example, the room represented by the room in the input 3D scene and / or dataset may include a room in a residence (e.g., a house or apartment), such as a kitchen, a bathroom, a bedroom, a living room, a garage, a laundry room, a loft, an office (e.g., personal or shared), a conference room, a children's room, a nursery, a hallway, a dining room, and / or a library (the list may include other types of rooms). Alternatively or additionally, the room represented by the room in the input 3D scene and / or dataset may include other indoor rooms, such as a factory, a museum, and / or a theater. Alternatively or additionally, the room represented by the room in the input 3D scene and / or dataset may include an outdoor scene, such as a garden, a terrace, or an amusement park.

[0074] Any furniture object here can have a furnishing function in the room where they are placed. For example, the classification can include any combination of one or more chair classes, one or more lamp classes, one or more cabinet classes, one or more shelf classes, one or more sofa classes, one or more table classes, one or more bed classes, one or more sideboard classes, one or more nightstand classes, one or more desk classes, and / or one or more wardrobe classes. For example, the classification can include any combination of corresponding classes of the following objects: partitions, walls, air conditioners, armchairs, wardrobes, bars, base cabinets, mirrors, single sinks, double sinks, sinks, storage rooms, bathtubs, bookshelves, sideboards, bunk beds, ceiling lights, children's beds, fireplace tools, clothing organizers, shoe organizers, coffee tables, kitchen countertops, desks, chairs, tables, beds, double beds, dressers, dryers, electrical switches, televisions, electronics and entertainment, home theaters, faucets, fireplaces, fitness equipment, floor lamps, freezers, headboards, high cabinets, home automation, intercoms , home alarm, hood, island, small chair, lighting, microwave, nursery, changing table, bed, crib, pergola, patio, playground, pipes, swimming pool, radiator, stove, refrigerator, shelves, shower column, shower door, shower, side table, single bed, sofa, stool, table lamp, toilet, media furniture, closet, wall lamp, wall shelf, washing machine, bench, sun lounge, oven, dishwasher, bedside table, vehicle, mailbox, tree, front door, security door, patio door, screen door, simple interior door, double interior door, sliding door, interior door, interior opening, garage door, service door, simple window, double windows, simple sliding windows, double sliding windows, basement windows, fixed windows, skylights, stairs, outdoor activities, cushions, wall stickers, dressers, wood-burning stoves, oven cabinets, under-sink cabinets, sliding storage, corner cabinets, security detectors, openings, fountains, bridges, wells, columns, water heaters, furnaces, boilers, barbecues, gardening, patio heaters, spiral stairs, straight stairs, four-turn stairs, double four-turn stairs, kitchen equipment, folding doors, sliding windows, bay windows, railings, platforms, cabinets, mattresses, wine cellars, chandeliers, meridians, footstools, display cabinets, lounge chairs, office Chair, Bed bench, Outdoor sofa, Outdoor chair, Outdoor bench, Outdoor table, Pivoting patio door, Sliding patio door, Bathroom mirror, Children's desk, Children's bedroom furniture, Gate, Garden lounge, Straight sofa, Corner sofa, Modular sofa, Armless chair, Four poster bed, Mezzanine bed, Floor mirror, Wall mirror, Kitchen accessories, Baby bathtub, Bathroom base cabinet, Bathroom column, Bathroom vanity unit, Cradle, Garden lighting, Balcony, Front, Door, Double door, Double glazed door, Drawer, Glass door, Cabinet shelf, Infill corner, Wall panelling, Freestanding infill strip, Wall panelling edge, Roof window and / or Half height cabinet.

[0075] Any decorative object herein may have a decorative function in the room in which they are placed. For example, the classification may include any combination of one or more accessory classes, one or more plant classes, one or more book classes, one or more frame classes, one or more kitchen accessory classes, one or more mat classes, one or more lamp classes, one or more curtain classes, one or more vase classes, one or more carpet classes, one or more mirror classes, and one or more classes corresponding to electronic objects (e.g., refrigerators, freezers, and / or washing machines). For example, the classification may include any combination of the respective classes of the following objects: accessories, mirrors, blinds, ceiling lights, fireplace tools, curtains, decorative accessories, plugs and meters, electronic equipment and entertainment equipment, hi-fi, fences, floor lamps, smoke detectors, lighting equipment, decorations, people, plants and flowers, carpets, table lamps, toys, wall decorations, wall lamps, small appliances, aquariums, hangers, vases, children's activities, beams, towel racks, bags, screens, trash cans, frames, candles, candlesticks, paintings, books, children's mobile phones, altimeters, weathervanes, statues and sculptures, clocks, posters, swings, sandboxes, trampolines, kitchen utensils, tableware, pet supplies, decorations objects, chandeliers, tapestries, towel dryers, sleds, hammocks, carpet fabrics, hardwood, wallpaper, paint, vinyl, tiles, mosaics, stones, slates, bricks, concrete, marble, linoleum, shingles, fiberboard, wall coverings, pebbles, mats, bathroom mirrors, wainscots, decorative tiles, decorative panels, self-service items, filler strips, safety products, floor mirrors, wall mirrors, bathroom accessories, office supplies, children's decorations, outdoor decorations, garden lighting, beach umbrellas, gravel, paint, workbenches, bases, boxes, support legs, wall tiles, joints, operations, wall strips, decorative strips, quartz, resources, cornices, countertop edges, gutters, wall strip edges, cover plates and / or handles.

[0076] A room in the plurality of rooms may include (i.e., grouping together all objects in each room) objects belonging to a plurality of categories in the classification (the categories are all represented in the dataset). For example, a room in the plurality of rooms may include at least one object belonging to each category in the classification (in which case each category in the classification is represented). The purpose of determining the classification may be to determine one category for each type of object in the different rooms in the dataset. The input 3D scene may also include objects belonging to another number of categories in the classification (the number being less than the number of categories represented in the plurality of rooms in the dataset, but, for example, being equal on average to the number of categories represented in each room in the dataset). For example, each object of the input 3D scene may belong to one category represented in the dataset.

[0077] The method may be included in a real room design (i.e., effective arrangement) process, which may include, after performing the method, using the input 3D scene to illustrate a room to be arranged (e.g., from scratch, or by adding objects represented by the input 3D objects to a room that is already furnished and / or decorated). For example, the illustration may be shown to a user, such as an owner of a house in which the room is located. The illustration may include an image and / or video of the input 3D object determined to be located within the input 3D scene. The determined image and / or video may be used by the user to decide whether to acquire the object, and may assist the user's selection by displaying the object in the room. During the real room design process, the method may be repeated to place different objects in the input 3D scene. Repeating the method may be used to show a complete virtual interior of a room (with several objects positioned using the method).

[0078] Alternatively or additionally (e.g., prior to the illustration), the real room design process may include populating an input 3D scene representing a room with one or more decorative objects. Populating may include, for each decorative object, repeatedly performing a method of positioning a corresponding input 3D object representing the decorative object in the input 3D scene. The real room design process allows for the creation of richer, more enjoyable environments (for animation, advertising, and / or generating virtual environments, e.g., for simulations). The real room design process may be used to generate virtual environments (e.g., a metaverse). The real room design process may be included in a general process that may include repeating the real room design process for several input 3D scenes, thereby populating a set of input 3D scenes with decorative objects.

[0079] Alternatively or additionally (e.g., after population), the real room design process may include evaluating one or more layout metrics on the room represented by the input 3D scene. The one or more layout metrics may include a flow metric, a brightness metric, an accessibility metric (e.g., indicating the ability to use functional furniture), and / or an ergonomic metric. The real room design process may include using one or more of these evaluated metrics to ensure ease of movement in the room, accessibility of certain objects, safety, and / or a certain level of ergonomics in the room (e.g., a room to be built later).

[0080] Alternatively or additionally, the real room design process may include, after executing the method, physically arranging the (i.e., real) room so that its design matches the input 3D scene. For example, the room (without the objects represented by the input 3D scene) may already exist in the real world, and the real room design process may include positioning the real object represented by the input 3D object in the already existing room (i.e., in the real world). The real object may be positioned according to the position calculated by the method. When the method is repeated to position different 3D objects, the real room design process may include positioning different real objects in the room. Alternatively, when the method is executed, the room may not exist. In this case, the real room design process may include building the room according to the provided 3D scene (i.e., starting from scratch, by placing objects in the provided 3D scene). Since the method improves the positioning of the 3D object in the 3D scene, the method also improves the construction of the room corresponding to the 3D scene, thereby improving the production efficiency of the real room design process.

[0081] The method may include the step of forming a data set from the plurality of rooms (data set forming step). In this case, obtaining S10 may include the data set forming step. The data set forming step may be simple and fast. For example, the data set forming step may include a single loop through the data set, counting and recording the relative positions of each pair of objects. The formation of the data set may be fully automated (i.e., without human intervention).

[0082] The data set forming step may include determining information about the objects in the plurality of rooms (i.e., classification, counts, and / or information about the relative positions in each occurrence of each pair). For example, the data set forming step may include counting the number of occurrences of the pairs in the plurality of rooms. After the data set forming step, obtaining S10 may include recording the calculated data set in a database, and step S20 may be performed by accessing the recorded data set. The plurality of rooms may be recorded in, for example, the same database or another database. The calculation of the data set may be accomplished by querying the recorded plurality of rooms.

[0083] Alternatively, the method may include retrieving a data set in a database. In such a case, the data set may optionally be obtained according to the above-mentioned data set forming step (before performing the method). In this case, obtaining S10 may include providing access to the data set in the database. The method may use the access to identify the first pair, the second pair and / or the third pair in the data set (during step S20).

[0084] The calculation of the data set (i.e., the data set forming step) may be performed during an offline phase. In this case, the calculated data set may be recorded in a database, for example, during this offline phase. For example, the data set may be calculated by a server having access to information of a plurality of rooms (e.g., a 3D representation of these rooms). Then, steps S20 to S50 (and step S10, when the data set has been calculated) may be performed during an online phase. For example, these steps may be performed by a client computer having access to the data set calculated in the offline phase.

[0085] The method may include updating the dataset (e.g., periodically). For example, a manufacturer may add new objects to the catalog, e.g., by adding new rooms (including these new objects) to the database, and the method may update the dataset to include these new objects. In this case, the updating may include determining new pairs including these new objects. The method may then include counting the number of occurrences of these new pairs and including the number of occurrences of these new pairs (and their relative positions) in the dataset in the dataset. Efficiency is improved because the dataset is only updated for new objects to be added (without recalculating statistics for existing object pairs).

[0086] The objects and rooms in the dataset may exist in the real world (either already existing when the dataset was computed or appearing in the future). For example, the rooms may be real rooms that actually exist in the real world (in terms of layout), and objects may be placed in these real rooms as specified by the information in the dataset. Multiple rooms may include rooms that have been designed (e.g., by an interior designer) and then realized in the real world (i.e., multiple rooms correspond to virtually designed rooms that have been or can be replicated in people's homes). In the example, each room in the dataset is of the same type as the room represented by the input 3D scene. For example, all rooms in the dataset may be a kitchen, bathroom, bedroom, living room, garage, laundry room, attic, office (e.g., individual or shared), conference room, children's room, baby room, hallway, dining room, or library (the list may include other types of rooms). It allows for more precise locations to be provided and increases the robustness of the method.

[0087] The classification of the objects in the plurality of rooms may include the class (i.e., category) to which the objects belong. The category may be one of the categories listed above for each furniture or decorative object. The classification may be determined during the computation of the data set. For example, the computation may include determining the classification of the objects in the plurality of rooms based on a predetermined list of object classes. The determination of the classification may include identifying to which class each object in each room belongs.

[0088] The information in the data set also includes counts of pairs. Each count can be the number of elements in a set consisting of each occurrence of a pair of the same first real object and the same second real object in a room. The same real object refers to an object that has the same properties as the other objects that appear each time. For example, the objects can have the same geometric structure (e.g., external envelope, texture and / or color) in the mutual occurrence of the pair and / or can be the same product manufactured by the same manufacturer (e.g., can have the same product number). The corresponding set can include one or more occurrences of the corresponding pair. The corresponding set can include the number of occurrences of the pair in the same room, and / or the number of occurrences in multiple different rooms (i.e., one for each corresponding room). The information includes the corresponding counts for each pair (in the same room and different rooms). The count is the number of occurrences of the corresponding pair (e.g., when multiple rooms contain n occurrences of the corresponding pair, the value of the count is n, where n is an integer). The pairs in the data set can include one or more pairs, each pair including decorative objects and furniture objects (the decorative objects and furniture objects are arranged according to the mutual arrangement relationship in one or more rooms of the multiple rooms). Alternatively or additionally, the pairs in the data set may include one or more pairs, each pair including a decorative object and another decorative object (the two decorative objects are arranged according to a mutual arrangement relationship in one or more rooms of the plurality of rooms).

[0089] For each occurrence of each pair, the mutual arrangement relationship between the first real object and the second real object is one of one or more mutual arrangement relationships in a predetermined set. Each mutual arrangement relationship specifies a specific type of spatial arrangement between the first and second real objects in the room (i.e., according to the position and orientation of the first and second real objects). The spatial arrangement between the objects can be divided into categories (e.g., in front, sideways, or one above the other), and the predetermined set can include a mutual arrangement relationship corresponding to each category.

[0090] In an example, the occurrences of pairs of the same room may be represented in a graph, which may include nodes and edges representing room objects, each edge linking two nodes of a pair. Each edge may represent the corresponding occurrences of the two nodes linked by the edge. Each edge may include a label indicating the mutual arrangement relationship between the two objects represented by the nodes linked by the edge. The calculation of counts may be based on these graphs. For example, the method may count the number of occurrences of the same pair in multiple rooms by browsing graphs of multiple rooms. The data set may include a corresponding graph for each of the multiple rooms. In this case, the calculation of counts may be based on these graphs. Alternatively, the graphs may be recorded in a database, and the method may perform the calculation of counts by accessing the database.

[0091] In an example, the predetermined set may include an adjacency relationship. When two objects are spaced less than a predetermined distance (e.g., 5 cm) from each other, the two objects may be arranged in an adjacency relationship. For example, the distance may be the distance between the bounding boxes of the two objects in a 3D scene representing a room (e.g., the distance defined along the axis of one of the two objects). Alternatively or additionally, the predetermined set may include a vertical stacking relationship. When two objects are placed one on top of the other, they may be arranged according to an adjacency relationship (e.g., a lamp on a ceiling above a table).

[0092] The information of the data set also includes information about the relative position of the real object in each occurrence of each pair. The information may include, for each occurrence, the corresponding relative position of the real object. The relative positioning may be in the form of a vector having coordinates representing the distance between the first object and the second object in different directions of the coordinate system (e.g., between the bounding boxes representing the two objects or between the centers of these bounding boxes). Alternatively, the information may be general information about all occurrences. For example, the information may include an average value of the relative positions of the real objects in all occurrences (e.g., the average value of the vector).

[0093] The computer program instructions executed in step S20 can be recorded on a memory. The computer program instructions are instructions for executing the marked pairs in the data set. The computer program instructions can be a data block representing an algorithm and can generate different sequences of steps depending on the situation, in particular depending on whether the "when" condition is verified. Therefore, the exact steps of the method depend on the specific situation. In one example, the method executes all branches of the algorithm at least once (i.e., all attempts to identify). Attempting to identify means starting a process, the operation of which and its final result depend on the specific situation. For example, the final result can be the output of the identified pair (if it exists), otherwise, if not, no operation can be performed or a "fail" message can be output.

[0094] Execution S20 may include providing one or more pairs among all pairs of the data set (the one or more pairs are one or more pairs identified during the execution of the computer program instructions). The computer program instructions result in three consecutive recognition attempts: the first recognition attempt searches for pairs with the same object, the second recognition attempt searches only for one of the two objects with the pair (and the equivalent of the second object), and the third recognition attempt searches for the equivalent of the pair. The computer program instructions may include instructions for each of these recognition attempts (although not all of these recognition attempts are always performed). Execution S20 begins with the execution of the first recognition attempt. When at least one first pair is identified, the second and third recognition attempts are not performed, and the identified pair is the first pair identified. Otherwise, the second recognition attempt is performed. In this case, when at least one second pair is identified, the third recognition attempt is not performed, and the identified pair is the second pair identified. Otherwise, the third recognition attempt is performed, and the identified pair is the third pair identified (the data set may always include at least one third pair).

[0095] The execution of output S30 is based on the pairs identified during execution S20. The pairs of output are among the pairs identified. For example, the pairs of output may include all pairs of identification. Or, the pairs of output may include each pair of identification whose count is higher than a predetermined number. In this case, each pair of output may have a count higher than the predetermined number (for example, higher than 5 or 10 and / or lower than a predetermined number of 15 or 20). In this case, output S30 may include filtering the pairs of identification based on the count of the pairs of identification, so as to only output the pairs whose count is higher than the predetermined number. Alternatively, the number of pairs of output may be predetermined, and output S30 may include selecting a predetermined number of pairs from the pairs of identification (the pairs of output are the selected pairs). In this case, the predetermined number of pairs may be selected based on the count of the pairs of identification. For example, output S30 may include ranking the pairs according to the count of the pairs of identification, and selecting the pairs of the predetermined number of rankings first (for example, the k pairs of rankings first, wherein k is an integer, for example, greater than 1 and / or less than 10, for example, equal to 5) (the pairs of outputs are the k pairs of rankings first).

[0096] After output S30, the method includes S40: for each output pair, determine the corresponding position of the input 3D object. For each output pair, the determined corresponding position may be different. For each output pair, the determined corresponding position may allow the object to be positioned in the same manner as each occurrence of the output pair (e.g., average). For example, when the output pair is the first pair, the input 3D object may be the same as one of the 3D objects of the pair, and the 3D scene may include a given object that is the same as another object in the pair. In this case, the determined respective positions may allow the input 3D object to be positioned relative to the given object as two objects that appear in pairs. The same method may also be applied when the output pair is the second or third pair with an equivalent object (for the input 3D object, the given object, or the two objects). The output S30 and the determination S40 steps may be fast. For example, the method may complete the output S30 and the determination S40 steps in as little as 1 second. When the positioning S50 is performed automatically, the method may calculate the position of the input 3D object (ie, including performing the outputting S30 and determining S40 steps) within one second in total.

[0097] After determining S40, the method includes S50 positioning of the input 3D object. The positioning S50 of the input 3D object is performed according to the corresponding position determined for one of the one or more output pairs. When the method outputs only one pair, the method can accurately position the input 3D object according to the relative positioning determined for the only output pair. Alternatively, the method can output several pairs. In this case, the method can include, before positioning S50, selecting this pair (among the output pairs) from which the object is positioned. For example, the selection can be random. Alternatively, the pair can be selected by the user. In this case, the selection can include performing a user interaction for selecting one of the displayed positions. For example, the method can include displaying a set of different icons or graphisms, each icon or graphism is associated with a corresponding position, and the user can select by clicking on one of the displayed icons and performing a user interaction thereon (e.g., for example, touching with a finger or stylus when the object is displayed on a touch screen or touchpad, or clicking a mouse when the user uses a mouse). Alternatively, the selection can also be performed according to a position specified by the user (e.g., also based on touch or click). In this case, the selected location may be the location closest to the user-specified location.

[0098] Alternatively, the pair may be selected automatically. For example, the output pairs may be ranked (e.g., according to their counts), and the output pair of the input 3D object may be located therefrom, which may be the output pair with the highest ranking. In this case, the method may further include, before locating S50, ranking the output pairs according to the counts of the output pairs in the data set, and automatically selecting the output pair with a higher ranking (i.e., a higher count) (the automatically selected pair is the pair from which the input 3D object is located). When at least two pairs have the highest ranking, the method may randomly select one of them.

[0099] After positioning S50, the method may include S60 displaying the positioned input 3D object (i.e., in the input 3D scene, together with other 3D objects of the input 3D scene). In an example, the method may repeat the positioning S50 and displaying S60 steps. For example, the method may repeat positioning S50 each time the user selects a new position for the output centered input 3D object. For example, the method may repeat positioning S50 each time the user selects a new icon (when performing a position selection through a displayed set of icons) or moves the cursor to a new position (when performing a selection using a cursor). In this case, the method may repeat displaying S60 the input 3D object at the newly selected position.

[0100] In an example, for each output pair, determining S40 can be performed in two steps. In the first step, determining S40 can include relative positioning of the input 3D object with the other object of the pair in the input 3D scene based on information about the relative positioning of the first real object and the second real object for each occurrence in the output pair. For example, the method can relatively position the input 3D object as an average of all occurrences of the output pair. In this case, the method can include determining an average relative position of the two objects for all occurrences in the output pair (e.g., considering the average of each axis of the coordinate system), and can relatively position the input 3D object with the other object of the pair in the input 3D scene based on the average relative position.

[0101] Alternatively, when the information about the relative positioning of the first real object and the second real object for each occurrence in the output pair includes the relative positioning corresponding to each occurrence, the method may further include fitting Gaussian mixture model parameters to the relative positioning of the objects appearing in the pair before relative positioning each output pair. For example, a Gaussian mixture model may include corresponding parameters for each coordinate of a coordinate system (the coordinate system is defined relative to one of the objects of the pair, for example, an object already present in the scene is used to calculate the position of an object to be added, which is the same for each occurrence). In this case, the relative positioning may include sampling in a distribution represented by a Gaussian mixture model of the fitted parameters (the relative position of the input 3D object corresponding to the corresponding position sampled in the distribution). Sampling allows the variability of the data set to be taken into account. One advantage of a Gaussian mixture model over simple position averaging is the identification of several Gaussian components (such as Figure 7 and 8 ). It also allows new positions to be provided for the input 3D objects. For example, the method can repeat the sampling, thereby providing new relative positions for the input 3D objects at each repetition (the new relative positions are in the above distribution).

[0102] In a second step, determining S40 may include snapping the relatively positioned input 3D object to an element of the 3D scene. The snapping may be performed as known in the art. The snapping may include automatically pasting the input 3D object to an element of the 3D scene while remaining within the relative position determined in the previous step S41. For example, the snapping may include determining an element of the 3D scene on which the input 3D object is snapped, and then snapping the input 3D object to this determined element. The determined element may depend on the category of the input 3D object and may be the closest positioned input 3D object. For example, when the input 3D object is a ceiling lamp, the determined element may be the ceiling, or, when the input 3D object is a mural object (such as a wall lamp, frame, wall mirror), the determined element may be the nearest wall. In all other cases, the method may snap the inserted object to the nearest horizontal surface (e.g., the top of a floor, shelf, or furniture shelf) to account for gravity, modifying only the z coordinate of the input 3D object. The snapping enhances the realism of the provided position.

[0103] In an example, for each output pair, the corresponding position of the input 3D object in the input 3D scene determined for the output pair may not conflict with any other 3D object in the plurality of positioned 3D objects of the input 3D scene. For example, the method may include verifying that the intersection between the input 3D object and all other 3D objects in the input 3D scene (e.g., except for the object relative to which the input 3D object is positioned) is zero. Alternatively, each 3D object of the input 3D scene may be represented by a corresponding bounding box, and the method may include verifying that the intersection between the bounding box representing the input 3D object and the bounding box representing any other 3D object in the input 3D scene is zero. The bounding box of the 3D object may be a box with a minimum metric (area or volume) in which all points of the 3D object are located. To this end, the method may include filtering the identified pairs (at step S20) to discard all pairs for which intersections are identified. The method may perform the filtering before ranking the identified pairs, and may ignore the discarded pairs, so that only pairs providing non-intersecting positions of the input 3D objects can exist in the output pairs.

[0104] refer to Figures 2 to 24 , we now discuss how to implement this method.

[0105] The method calculates suitable positions for input 3D objects to be added to the 3D scene. The method calculates the positions of so-called decorative objects (different from functional objects). The position of decorative objects depends on other objects (i.e. functional objects and decorative objects) already present in the room (e.g. a table lamp is located on a table / desk / nightstand).

[0106] The method is based on statistics computed over a dataset of 3D (e.g., indoor) scenes (representing multiple rooms) and their corresponding extracted scene graphs, as defined in EP4075316 A1 (reference [7]), the contents of which are incorporated herein by reference. The method uses the frequent positions of decorative objects relative to other objects in the room to compute suitable positions for new 3D decorative furniture to be added to the 3D room.

[0107] The method can be performed in two stages. The first stage can be an offline stage for computing statistics for the rest of the pipeline on a given 3D scene dataset. In this stage, the method can use a scene graph of the 3D scenes in the dataset (e.g., the graph discussed in reference [7]) to compute the occurrences of decoration-function object pairs and optionally decoration-decoration object pairs organized by category. The method can also collect the relative positions of each pair seen in the dataset into a relative position dictionary.

[0108] The second stage can be an online stage for calculating the position of a given 3D decoration object (i.e., input 3D object) to be added to the 3D scene (i.e., input 3D scene). In this step, the method first includes selecting potential "anchors" among other 3D objects that already exist in the 3D scene. For each of these candidate anchors, the method can include calculating the position based on a relative position dictionary. These positions are then refined to make them physically valid and ranked. The added object is then inserted into the first position in the 3D scene, or, through user interaction, into a position selected by the user among all output positions.

[0109] Now let's discuss the concept of a 3D scene. A 3D scene can include a 3D layout (i.e., the geometry of the environment: the set of walls and boundaries, their positions and dimensions), a set of 3D objects (i.e., a set of CAD models, their size, position and orientation in 3D space) with semantic annotations, such as their type (e.g., chair, table, sofa, ...), model ID and function type ("functional", "decorative" or both), and optionally a set of 3D openings (i.e., doors and windows in the environment). These 3D openings can also be considered 3D objects, etc.

[0110] The method considers the classification of 3D objects between two classes: functional and decorative. Functional objects are room-defining objects and are usually placed in a room relative to the room's architecture (walls and openings). They are usually larger and important to the use of the room (e.g., a bed in a bedroom or a sofa in a living room). Decorative objects are not room-defining objects and are used for decoration and creating atmosphere without changing the functionality of the room. Decorative objects are usually placed in a room relative to other objects (e.g., cushions on a sofa or a lamp on a nightstand). Some objects are considered both functional and decorative (e.g., lamps and mirrors). The method can focus on the automatic positioning of decorative objects within a partially furnished room.

[0111] Each 3D scene can be associated with a corresponding scene graph. These scene graphs are discussed, for example, in reference [7]. A scene graph represents the spatial relationships between 3D elements in a 3D scene (3D elements include walls, openings, and objects). A scene graph contains the basic spatial relationships between elements in the scene that represent the most meaningful and functional constraints.

[0112] The structure of the scene graph can be as follows. The scene graph can include a list of nodes, each node corresponding to a 3D element in the room. Each node can include the following attributes: node ID, category of the 3D element, 3D element 3D bounding box (length, width and height), 3D element transformation matrix (to obtain the absolute position and orientation of the 3D element in the 3D room), the corresponding 3D model of each 3D element, 3D element macro category (optional), in-edge list, out-edge list, parent and / or superstructure tags (if any) (e.g., 'is_hub', 'is_spoke', 'is_chain'). The scene graph can include a list of edges to represent the relationship between 3D elements in the room. Each edge can include the following attributes: edge ID, ID of the start node, ID of the end node, direction (e.g., 'front', 'back', 'left', 'right', 'support'), optional tag (e.g., 'contains'), function (optional), distance tag, exact value of the distance between the bounding box of the start node and the bounding box of the end node of the edge and / or superstructure tag (if any) (e.g., 'is_spoke', 'is_chain'). For example, if the minimum distance between the bounding box of the start node of the edge and the bounding box of the end node of the edge is less than a "neighboring" threshold (e.g., 5 cm), the distance label can be "adjacent". For example, if the minimum distance between the bounding box of the start node of the edge and the bounding box of the end node of the edge is greater than the "neighboring" threshold and less than another "close" threshold (e.g., 50 cm), the distance label can be "close". For example, if the minimum distance between the bounding box of the start node of the edge and the bounding box of the end node of the edge is greater than the "close" threshold, the distance label can be "distance".

[0113] Figure 2An example of a diagram 100 of a room is shown. In this example, the properties are as follows. For "node5", the properties are "category": "Double beds", "macro-category": "bed", "transformation_matrix": [0, 1, 0, 0, -1, 0, 0, 0, 0, 1, 0, -13302.5, -5203.2, 455.0, 1], "bounding_box": [2000, 1800, 1000], #mm, "3D_model": #Ref3DModel, "ingoing_edges": [edge1], "outgoing_edges": [edge2], "parents" ;

[0114] Figure 3 A flow chart showing an example of an offline phase S110 and an online phase 120 is shown.

[0115] The offline phase S110 involves the calculation of the data set (statistical calculation). The offline phase S110 corresponds to the calculation of various statistical files used for positioning. All these statistical information are calculated based on a representative data set of 3D scenes (each scene has its corresponding scene graph). The statistical analysis of the data set allows the extraction of the number of occurrences of decoration-functional object pairs and decoration-decoration object pairs (optional). Based on the number of occurrences of these pairs, the method may include calculating the frequency of occurrence of each pair. The occurrence and frequency of each pair are organized by object category so that the accurate 3D model can be abstracted into its category when needed. Alternatively, the method may also include extracting the relative positions of each pair of objects seen in the data set and saving them in a relative position dictionary organized by object category and model. Optionally, the method includes fitting a Gaussian mixture model to the relative positions of each pair of objects and saving the parameters of these distributions for each pair in a specific file.

[0116] The online phase S120 involves the calculation of the position when a new decorative 3D object (input 3D object) is added to the 3D scene. The online phase S120 takes as input the 3D scene, the decorative 3D object to be added to the 3D scene, and the statistical file (calculated data set) calculated in the offline phase. The method is based on a statistical file with the number of occurrences and frequencies of paired objects, including selecting anchor candidates for the object to be added among the 3D objects already present in the 3D scene. For each of these candidate anchors, the method calculates the position based on a relative position dictionary (or, if present, by sampling from the distribution of the corresponding Gaussian mixture model saved in an optional file). These positions are then optimized (snapping it to its anchor object, or filtering it if it conflicts with other 3D objects of the 3D scene) to make it physically valid and ranked. The component outputs a ranked list of the positions of the 3D decorative objects in the 3D scene. The added decorative 3D object is then inserted into the 3D scene at the first position of the list, or, by user interaction, at a position selected by the user among all output positions.

[0117] The offline phase S110 is now discussed in more detail. All the steps detailed in this subsection can be run at time intervals or on different data sets to update the statistics files to the latest or most representative data at a given moment. For example, the method can regenerate these files periodically (e.g., monthly) to take into account updates in the 3D object database (e.g., after adding new 3D objects to the catalog discussed earlier).

[0118] The offline phase S110 comprises computing pairwise occurrence statistics S111. Based on a representative dataset of 3D indoor scenes and their corresponding scene graphs defined above, the method comprises extracting a large number of statistics on the number of occurrences of objects in these items and the number of pairwise object occurrences.

[0119] A representative data set can be created to include only certain types of rooms (e.g., only bedrooms or only living rooms) or a combination of several room types.

[0120] For each project, based on its scene graph, calculation S111 considers the "adjacent" and "supporting" edges between functional objects and decoration objects and optionally between two decoration objects to count the associated functional-decoration object pairs and optional decoration-decoration object pairs. Calculation S111 formally names these pairs by anchor-decoration object pairs, where the anchor can be either a functional object or an optional decoration object. Calculation S111 aggregates the occurrences of all pairs over the dataset into count files organized by decoration category.

[0121] Figure 4An example of the occurrence of pairs is shown. In this example, object 5 101 and object 6 103 are functional, and object 7 104 and object 8 105 are decorative objects. The pairings including objects 5 and 7 and objects 6 and 8 are considered to be pairs of related functional-decorative objects.

[0122] The structure of the calculated file is as follows:

[0123]

[0124] Based on the first count file, calculation S111 includes deriving a second file of frequencies of occurrence of pairs of anchor-decoration objects. For each pair of objects in the count file, calculation S111 includes deriving the counts of paired occurrences as frequencies. In detail, for the "total" key, calculation S111 includes dividing the value by the sum of all "total" values ​​for that decoration model ID. For the anchor key corresponding to the model ID, calculation S111 includes dividing the count by the "total" key value for that particular anchor category. Therefore, calculation S111 includes obtaining a frequency value between 0 and 1. The sum of all "total" fields for a given decoration model ID is equal to 1. The sum of all model IDs associated with it from the same anchor category is also equal to 1.

[0125] Frequency files are organized in the same way as count files. Figure 5 An example of a frequency file obtained for model ID "8901" is shown.

[0126] The idea behind this frequency file is to identify other objects that a decorated object most often decorates, based on object categories and object models. Figure 5 For example, the decoration object with ID "8901" has 75% of its occurrences in the dataset as decoration shelves, and there are only models for decoration shelves "1747", "80" and "585".

[0127] In addition to object categories to object model IDs, geometric compatibility can also be considered. In fact, for a certain sofa model, the method is able to identify the most geometrically compatible cushion models based on their frequency of occurrence in the dataset.

[0128] Then, the offline phase S110 comprises calculating S112 relative position statistics. Based on the same representative data set as the statistics of the number of paired occurrences, the calculation S112 comprises extracting information about the relative positions of the objects in these pairs. The relative position in this method refers to the position of the decoration object relative to the anchor object it is decorating, i.e. its position in the coordinate system of the anchor object.

[0129] For each occurrence of a pair of objects counted in the previous step S111, calculation S112 includes saving the relative position of the decoration object relative to the anchor object as a transformation matrix. In order to save memory space, this transformation matrix can be reduced to 4 values: x, y, z coordinates and an orientation value in the anchor object coordinate system. Therefore, calculation S112, corresponding to each anchor-decoration pair, obtains a list of relative positions of the decoration object relative to its anchor. These positions are stored in a relative position file with the following structure:

[0130]

[0131] Figure 6 An example of the determined corresponding positions is shown. In this example, the corresponding positions determined for the model "4852" are indicated.

[0132] Now, the fitting of the Gaussian mixture model S113 is discussed. This step is optional. For example, when there are enough pairs of anchor-decoration objects, the offline stage S110 may include the fitting S113 step. Fitting S113 may include fitting a Gaussian mixture model (GMM) to the relative positions of the objects in these pairs.

[0133] This step S113 allows fitting a mixture of Gaussian distributions to the relative positions of the objects, thereby sampling from them, rather than just selecting the positions seen in the relative position file obtained from the representative dataset. In addition, since this method may only consider the bounding box of the 3D object, but not its precise 3D shape, fitting the GMM can better understand the anchor 3D model.

[0134] Figure 7 A first example of fitting a Gaussian mixture model is shown. The figure shows the position of the lamp (point 110) relative to the bedside table model (volume 112). When the GMM is fitted to the position, the method obtains two 3D Gaussian components 114, indicating that the bedside table has a hole in the middle where the lamp can be placed. Figure 8 A second example of fitting of a Gaussian mixture model is shown. The figure shows the position of the cushions (points 120) relative to the sofa model (body 122). When the GMM is fitted to the position, the method obtains two 3D Gaussian components 124, indicating that the sofa has an L-shape.

[0135] Specifically, for each pair of anchor-decoration objects that has sufficient occurrences (at least 10 in our case), the method performs the following steps.

[0136] In the first step, the method consists in determining the number of Gaussian components that best fit our data, here referring to the relative positions of the decorative objects with respect to the anchors. To this end, the method can be implemented using the Gaussian mixture model fitting [8] of the scikit-learn library. Given a dataset and a number of Gaussian components, the parameters of each component, i.e., the weight, mean, and covariance of each component, are estimated. The method iteratively fits the GMM to the data by iterating the number of Gaussian components from 1 to 10. The method also calculates the silhouette score of each model with different numbers of components based on the scikit-learn library [8] and retains the model with the best silhouette score (i.e., the one with the maximum silhouette score).

[0137] In a second step, the method includes fitting a GMM with the previously determined number of components. The method may retain only components with weights > 0.05 (5%) to avoid components formed by outliers in the data set. The method may save the parameters (weight, mean and covariance) of each component in a list.

[0138] This process is also applied to the aggregation of all decoration objects of the same category for a given anchor, to gather information about the anchor even if it has been decorated by various decoration objects of the same category (whereas each decoration object considered individually may represent only a few occurrences). This method can save the GMM parameters in another file organized in the same way as the previously calculated statistics file.

[0139] Fig. 9 An example of model "4975" is shown. The GMM for the positions of "all" curtains that decorate "4975" has two components, while the GMM for the "15915" curtains has three components.

[0140] The online phase S120 is now discussed in more detail.

[0141] The online stage S120 comprises determining S121 candidate anchors. Given a 3D scene partially equipped with 3D objects, a 3D decoration object to be added to the scene, and a frequency file calculated on a representative dataset, the method attempts to identify potential anchors for the inserted 3D decoration object (i.e., the functional object to be decorated). The model ID of the decoration object to be added is represented by deco_ID and deco_cat to indicate its category.

[0142] Determining S121 candidate anchors comprises executing computer program instructions that result in attempting to identify pairs. In detail, the computer program instructions may comprise instructions for performing the following steps to create a list of candidate anchors:

[0143] - Initialize an empty list of candidate anchors;

[0144] - Check if deco_ID is in the primary key of the frequency file. If not, this means that the object is not present in the dataset from which the frequency file was generated. This can happen when adding an object that has been added to the database of 3D objects after the frequency file has been calculated. If the model is unknown, deco_ID will take the value of the model ID of the nearest object from the deco_cat category.

[0145] -For each 3D object present in the scene:

[0146] ○ Consider obj_ID to be the model ID of the 3D object and obj_cat to be its category;

[0147] ○ If obj_ID-deco_ID pair exists in the frequency file, add the 3D object in the candidate anchor list.

[0148] ○ Otherwise, if the pair does not appear in the frequency file, but there are pairs of other decoration objects with obj_ID and deco_cat category, select the pair whose decoration object size is closest to the object to be added to the scene. The decoration model ID of this pair is represented by closest_deco_ID. Then, the 3D object is added to the candidate anchor list with the information of the closest pair.

[0149] ○ Otherwise, if the pair does not appear in the frequency file, determine the object in the obj_cat object that is closest in size to the 3D object and indicate its model ID by closest_obj_ID.

[0150] ■ If the closest_obj_ID-deco_ID pair exists in the frequency file, the 3D object is added to the candidate anchor list with the information of the closest pair.

[0151] ■ Otherwise, if the pair does not exist, but there are pairs with closest_obj_ID and other decoration objects of category deco_cat, select from these pairs the one that is closest in size to the decoration object to be added to the scene. The decoration model ID of the pair is represented by closest_deco_ID. Then, the 3D object is added to the candidate anchor list with the information of the closest pair.

[0152] ■ Otherwise, consider that the considered 3D object cannot be an anchor and move to the next 3D object in the 3D scene.

[0153] Therefore, after executing the computer program instructions, a list of candidate anchors of the 3D decoration object to be added in the 3D scene is obtained. Each element of the candidate anchor list refers to an object already in the 3D scene, and is itself a list containing the following:

[0154] -idx: index of the 3D object in the 3D scene;

[0155] -freq: frequency, taken from the frequency file corresponding to the pair, used to add the 3D object to the candidate list (according to the step of adding 3D objects);

[0156] -ref_deco_model: the closest_deco_ID (if used), otherwise None;

[0157] - ref_anchor_model: the closest_obj_ID (if used), otherwise None;

[0158] After identifying the pairs, determining S121 candidate anchors includes outputting the pairs in the identified pairs. The output can be based on a ranking according to the count of the identified pairs. For example, the method can include first ranking the obtained anchor candidate list from highest to lowest according to its frequency (i.e., the second value of the sublist). Then, the method can rank it again so that the candidate anchors that appear in pairs with the original decoration model ID in the frequency file are first obtained, and added to the 3D scene, i.e., the candidate anchors for which ref_deco_model is "None". Fig.10 An example of a ranked list of candidate anchors obtained by the method using these computer program instructions is shown.

[0159] If no 3D objects of the 3D scene are added to the candidate anchor list, the decoration object will be inserted at a default location in the 3D scene (e.g., the center of the room or a free space in the room visible on the user's screen).

[0160] Once the list of candidate anchors is obtained, the online stage S120 includes calculating S122 the positions of the pairs output (i.e., for each output pair, determining the corresponding position of the input 3D object in the input 3D scene). Calculating S122 the positions includes calculating the 3D position of the decoration object to insert for each candidate anchor in the obtained list.

[0161] Specifically, for each candidate anchor in the previously calculated list, the method may execute computer program instructions for performing:

[0162] i. If ref_deco_model is "None" (i.e., the deco_ID of the selected pair is the original model ID of the decoration model to be added to the 3D scene):

[0163] a. If ref_anchor_model is "None" (i.e., the selected pair is obj_ID-deco_ID): randomly select a relative position from the "rel_transfos" list of obj_ID-deco_ID pairs in the relative positions file.

[0164] It is then applied to the inserted decoration object relative to the anchor candidate to obtain the absolute position of the inserted decoration object (as a transformation matrix).

[0165] b. Otherwise, when ref_anchor_model is not "None" (i.e., the selected pair is ref_anchor_model-deco_ID): randomly select a relative position from the "rel_transfos" list of the pair ref_anchor_model-deco_ID in the relative position file. Then, apply it to the inserted decoration object relative to the anchor candidate to obtain the absolute position of the inserted decoration object (as a transformation matrix).

[0166] ii. Otherwise, when ref_deco_model is not "None" (i.e., the selected pair does not contain the original model ID deco_ID of the decoration model to be added to the 3D scene, but the closest one in the same category):

[0167] a. If ref_anchor_model is "None" (i.e., the selected pair is obj_ID - ref_deco_model):

[0168] A relative position is randomly selected from the "rel_transfos" list of pairs obj_ID-ref_deco_model in the relative position file. Then, it is applied to the inserted decoration object relative to the anchor candidate to obtain the absolute position of the inserted decoration object (as a transformation matrix).

[0169] b. Otherwise, when ref_anchor_model is not "None" (that is, the selected pair is

[0170] ref_anchor_model-ref_deco_model): Randomly select relative positions from the "rel_transfos" list of the pair ref_anchor_model-ref_deco_model in the relative positions file.

[0171] It is then applied to the inserted decoration object relative to the anchor candidate to obtain the absolute position of the inserted decoration object (as a transformation matrix).

[0172] Alternatively, instead of selecting the 3D position directly from the relative position file, the method may use the GMM parameters in the GMM parameter file (if these GMM parameters have been calculated in the offline phase) to sample the 3D position. Alternatively, the method may sample the 3D position in each Gaussian component of the GMM. Alternatively, the method may select the position closest to the center of the GMM component from the relative position file instead of sampling directly from it. These different alternatives lead to different behaviors and different levels of 3D positioning changes.

[0173] Now let's discuss the verification of no intersection (collision management). Once the absolute 3D position of the decorative object to be inserted is obtained, the method may include verifying that it will not collide with other objects in the 3D scene. Since the method may only consider the bounding box of the 3D object, the method may not consider collisions with the anchors of the inserted object. In fact, some collisions may not be a problem in the sense of bounding boxes (for example, the bounding box of the cushions on the sofa will be contained in the bounding box of the sofa, but they are valid positions).

[0174] Specifically, for each object in the 3D scene (excluding the anchor of the inserted object), the method may include calculating the volume of the intersection of its bounding box and the bounding box of the inserted object. The method may also include calculating the volume of their union. The method may then include using the intersection-over-union (IoU) ratio of the intersection to determine whether to consider the collision as a conflict. If the IoU ratio is greater than a threshold (0.05 in our implementation), the method may include considering the collision as a conflict and may include ignoring the solution.

[0175] Alternatively, in case of a conflict, the method may comprise, according to the case of insertion (ia, ib, ii.a. or ii.b.), selecting another position from the "rel_transfos" list of its corresponding pair. Alternatively, if the method has already calculated the GMM parameters in an offline phase, the method may comprise, for example, sampling a new position from another component.

[0176] Fixing will now be discussed in detail. In cases ib, ii.a. and ii.b., the method uses additional objects than those in the 3D scene or the 3D objects to be added, i.e., ref_anchor_model or ref_deco_model or both, resulting in the calculation of an imperfect 3D position of the object to be inserted relative to its anchor. In practice, the inserted object may float or collide with its anchor. To realign the inserted object, the method can use the "fix" function to refine and correct its 3D position.

[0177] The Pin function takes as input an inserted object, an origin on the bounding box of that object, and a direction to find the surface on which to pin the inserted object.

[0178] This method can distinguish three cases of fixation:

[0179] - When the inserted object is a chandelier, this method can snap it to the ceiling: the top of the chandelier bounding box can be placed against the surface of the ceiling, and the z coordinate of the chandelier is modified accordingly. In this case, the origin is the center of the top of the chandelier bounding box, and the direction is upwards (towards the ceiling).

[0180] - When the inserted object is a wall painting object (e.g. wall lamp, frame, wall mirror), this method can fix the object to the nearest wall: the back of the wall painting object can be placed against the surface of the wall to modify the x and y axes of the wall painting object. In this case, the origin is the center of the back of the wall painting object's bounding box, and the direction is the normal of this side of the bounding box. The first wall found in this direction may be the wall to which the wall painting object will be attached.

[0181] - In all other cases, the method can pin the inserted object to the nearest horizontal surface to account for gravity, modifying only the z coordinate of the inserted object. In this case, the origin is the center of the bottom face of the inserted object's bounding box, and the method can consider two directions. One direction is downward (towards the floor) and the other direction is upward (towards the ceiling). The method can collect the nearest horizontal surface for each direction and finally perform pinning to the closest surface between these two directions.

[0182] This also allows inserting 3D position corrections of objects in cases where the quality of the dataset used to calculate the statistics file in the offline phase is not good enough (the user may have misplaced the object in his project).

[0183] The last two steps ensure that the calculated 3D position of the inserted decoration object is physically valid (no collisions, no floating objects).

[0184] Reference Figures 11 to 22 , we now discuss an example of localization of an input 3D object.

[0185] exist Figures 11 to 18 In each of the graphs, two different objects belonging to the same category are located in two different bedrooms by this method. For example, Fig.11A first bedroom 210 is shown, where a first lamp 211 is positioned by the method, and a second bedroom 220, where a second lamp 221 is positioned by the method. For each bedroom, the figure shows different relative positions calculated by the method. For the first lamp 211 in the first bedroom 210, the method provides three corresponding positions 212, 213 and 214 (each corresponding to a respective output pair). Therefore, the user can choose whether to position the lamp between the three corresponding positions 212, 213, 214. For the second lamp 221 in the second bedroom 210, the method provides four corresponding positions 222, 223, 224 and 225 (due to the presence of an additional table in the second bedroom 210, the pairs output by the method include additional pairs). Therefore, the user can choose whether to position the lamp between the four determined corresponding positions 222, 223, 224 and 225.

[0186] Fig.12 Examples of positioning of vertical stacking relationships are shown. In these examples, the method recognizes that the data set includes occurrences of a pair including lamps 231 and 241 (added in the bedroom) arranged according to a vertical stacking relationship with other 3D objects of the bedroom (a bed or a bedroom table). Therefore, the method calculates the positions of lamps 231 and 241 above these objects.

[0187] Similarly, Figures 13 to 18 It shows the positions calculated by this method for the same bedroom discussed previously when the input 3D objects are respectively a cushion, a plant, a frame, a mirror, a carpet or a curtain.

[0188] exist Figures 19 to 21 In this case, two different objects are located in two different living rooms. Fig.19 A first living room 310 with a first lamp 311 added and a second living room 320 with a second lamp 321 added are shown. For each living room, the figure shows different relative positions 312 calculated by the method. Thus, the user can choose whether to position the lamp between these different relative positions 312. Similarly, Fig. 20 and 21 Shown are the positions calculated by this method for the same living room when the input 3D objects are mats or frames, respectively.

[0189] Fig. 22 Shown is the position calculated by this method for a living room comprising different shelves when the input 3D object is a book. In this case, the method allows calculating the position on different shelf levels.

[0190] Fig.23 An example of a GUI for a system is shown, where the system is a CAD system.

[0191] GUI 2100 can be a typical CAD-like interface, with standard menu bars 2110, 2120, and bottom and side toolbars 2140, 2150. Such menus and toolbars contain a set of user-selectable icons, each associated with one or more operations or functions, as known in the art. Some of these icons are associated with software tools, suitable for editing and / or working on the 3D modeling object 2000 displayed in GUI 2100. Software tools can be grouped into workbenches. Each workbench includes a subset of software tools. In particular, one of the workbenches is an editing workbench, suitable for editing geometric features of modeling product 2000. In operation, a designer can, for example, pre-select a portion of object 2000, then initiate an operation (e.g., change size, color, etc.) or edit geometric constraints by selecting an appropriate icon. For example, a typical CAD operation is punching or folding modeling of a 3D modeling object displayed on the screen. GUI can, for example, display data 2500 related to the displayed product 2000. In the example of the figure, data 2500 displayed as a "feature tree" and its 3D representation 2000 relate to a brake assembly including a brake caliper and a brake disc. The GUI may also display various types of graphical tools 2130, 2070, 2080, such as for facilitating 3D orientation of an object, for triggering an operational simulation of an edited product, or for presenting various properties of the displayed product 2000. The cursor 2060 may be controlled by a haptic device to allow the user to interact with the graphical tools.

[0192] Fig.24 An example of a system is shown, where the system is a client computer system, such as a user's workstation.

[0193] The client computer of this example includes a central processing unit (CPU) 1010 connected to an internal communication bus 1000, a random access memory (RAM) 1070 also connected to the bus. The client computer is further provided with a graphics processing unit (GPU) 1110, which is associated with a video random access memory (Video RAM) 1100 connected to the bus. Video RAM 1100 is also known in the art as a frame buffer. A mass storage device controller 1020 manages access to a mass storage device (e.g., a hard disk drive 1030). Mass storage devices suitable for tangibly embodying computer program instructions and data include various forms of non-volatile memory, such as semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks. Any of the foregoing may be supplemented or integrated by a specially designed ASIC (Application Specific Integrated Circuit). A network adapter 1050 is responsible for managing access to a network 1060. The client computer may also include a tactile device 1090, such as a cursor control device, a keyboard, etc. A cursor control device is used in the client computer to allow the user to selectively position the cursor at any desired location on the display 1080. Additionally, the cursor control device allows the user to select various commands and input control signals. The cursor control device includes a plurality of signal generating devices for inputting control signals to the system. Typically, the cursor control device may be a mouse, the buttons of which are used to generate the signals. Alternatively or additionally, the client computer system may include a sensor pad and / or a sensor screen.

[0194] The computer program may include instructions executable by a computer, the instructions including means for causing the above system to perform the method. The program may be recordable on any data storage medium (including the memory of the system). The program may be implemented, for example, in a digital electronic circuit device or computer hardware, firmware, software or a combination thereof. The program may be implemented as a device, for example, a product tangibly embodied in a machine-readable storage device to be executed by a programmable processor. The method steps may be performed by executing an instruction program by a programmable processor to perform the functions of the method by operating on input data and generating output. Therefore, the processor may be programmable and coupled to receive data and instructions from a data storage system, at least one input device and at least one output device, and to send data and instructions to a data storage system, at least one input device and at least one output device. The application program may be implemented in a high-level program or object-oriented programming language, or, if desired, in an assembly or machine language. In any case, the language may be a compiled language or an interpreted language. The program may be a complete installer or updater. The application of the program on the system in any case results in instructions for performing the method. The computer program may alternatively be stored and executed on a server in a cloud computing environment, which communicates with one or more clients via a network. In such a case, the processing unit executes the instructions included in the program, thereby causing the method to be performed in a cloud computing environment.

Claims

1. A computer-implemented method for automatically positioning an input 3D object representing a real object in an input 3D scene representing a room, the input 3D scene comprising a plurality of positioned 3D objects, the plurality of positioned 3D objects comprising positioned furniture 3D objects, each of the furniture 3D objects representing a real piece of furniture, the method comprising: Obtain (S10) a data set, the data set comprising information about objects in a plurality of rooms, the information comprising: o classification of said objects in said plurality of rooms; o counts, each of the counts being a count in a set consisting of each occurrence of a corresponding pair of the same first real object and the same second real object in the room, wherein each occurrence of the first real object and the second real object in each corresponding pair is arranged according to a mutual arrangement relationship from one or more preset mutual arrangement relationship sets; and o information about the relative positioning of the first real object to the second real object in each occurrence of each pair; ● Executing (S20) computer program instructions, causing: o attempting to identify one or more first pairs in the dataset, each first pair comprising the input 3D object and a corresponding positioned 3D object in the input 3D scene; o when no first pair is identified, attempting to identify one or more second pairs in the dataset, each of the second pairs comprising only one of the input 3D object and the corresponding positioned 3D object in the input 3D scene, and an equivalent object of the other, wherein the given real object is an equivalent object of the other given real object when the given real object has the same classification as the other given real object and the given real object is closest in size to the other given real object in a set of real objects of the same classification present in the dataset; o when no first pair is identified and no second pair is identified, attempting to identify one or more third pairs in the data set, each of the third pairs comprising an equivalent of the input 3D object and an equivalent of a corresponding located 3D object in the input 3D scene; Output (S30) one or more identified pairs and counts of the one or more identified pairs in the set consisting of each identified pair, the output being based on the set consisting of each count, each count corresponding to a respective identified pair in the set consisting of each identified pair; ● for each output pair, determining (S40) a corresponding position of the input 3D object in the input 3D scene based on information about the relative positioning of the first real object and the second real object in each occurrence of the output pair; and - Positioning (S50) the input 3D object according to the corresponding position determined for one of the pairs of the one or more outputs.

2. The method according to claim 1, wherein: The preset one or more sets of mutual arrangement relationships include an adjacency relationship and / or a vertical stacking relationship.

3. The method according to claim 1 or 2, wherein: The pairs in the data set include pairs including decorative objects and furniture objects.

4. The method according to any one of claims 1 to 3, wherein: The pairs in the data set include pairs including a decorative object and another decorative object.

5. The method according to any one of claims 1 to 4, wherein: Determining (S40) the corresponding position corresponding to each output pair includes: Based on the information about the relative positioning of the first real object and the second real object in each occurrence of the output pair, relatively positioning (S41) the input 3D object with the other object of the pair in the input 3D scene; and - Fixing (S42) the relatively positioned input 3D object to an element of the 3D scene.

6. The method according to claim 5, wherein: Corresponding to each output pair, information about the relative positioning of the first real object and the second real object that appear each time in the output pair includes the relative positioning corresponding to each appearance. The method also includes, before performing relative positioning (S41) corresponding to each output pair, fitting Gaussian mixture model parameters for the relative positioning of the objects appearing in the pair, wherein the relative positioning (S41) includes sampling in the distribution represented by the Gaussian mixture model of the fitted parameters.

7. The method according to any one of claims 1 to 6, wherein: Each output pair has a count above a preset number.

8. The method according to any one of claims 1 to 7, wherein: Each room in the dataset is of the same type as the room represented by the input 3D scene.

9. The method according to any one of claims 1 to 8, wherein: For each output pair, the corresponding position of the input 3D object determined for the output pair in the input 3D scene does not conflict with any other 3D object in the plurality of positioned 3D objects in the input 3D scene.

10. The method according to any one of claims 1 to 9, wherein: The one or more output pairs include several output pairs, and the method further includes: before positioning, a user selecting one of the one or more output pairs from which to position the input 3D object.

11. The method according to any one of claims 1 to 9, wherein: The one or more output pairs include several output pairs, and the method further includes: before positioning, ranking the output pairs according to the counts of the output pairs in the data set, the output pair used to locate the input 3D object is the pair with the highest ranking.

12. The method according to any one of claims 1 to 11, wherein: The method further includes, after positioning ( S50 ) the input 3D object, displaying ( S60 ) the positioned input 3D object.

13. A computer program comprising instructions for executing the method of any one of claims 1 to 12.

14. A computer-readable storage medium having recorded thereon the computer program according to claim 13.

15. A system comprising a processor coupled to a memory having recorded thereon the computer program of claim 13.

Images