Training scenario generation method and apparatus, electronic device, readable storage medium, and computer device

Abstract

The present application relates to the technical field of three-dimensional modeling, and provides a training scenario generation method and apparatus, an electronic device, a readable storage medium, and a computer device. The training scenario generation method comprises: obtaining an object model library; on the basis of scenario information of a scenario model, determining a target object model in the object model library by means of a domain randomization algorithm, the target object model being matched with the scenario information; and configuring the target object model into the scenario model to generate a training scenario file.

Description

Training scenario generation method, device, electronic device, readable storage medium and computer device

[0001] This application claims priority to the Chinese patent applications filed with the State Intellectual Property Office of China on July 25, 2023, with application number "202310921901.3" and application name "Training scenario generation method, device, electronic device and readable storage medium" and application number "202310919728.3" and application name "Model configuration method, device, electronic device, program product and storage medium", and application number "202310919628.0" and application name "Model configuration method, device, electronic device, program product and storage medium", the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of three-dimensional modeling technology, and more specifically, to a training scene generation method, device, electronic device, readable storage medium, and computer device. Background Art

[0003] With the continuous advancement of robotics technology, the application of robots has gradually expanded from the industrial sector to non-deterministic tasks in complex household scenarios. Because household scenarios are highly unstructured, large-scale training and testing in simulation scenarios is a mainstream technical approach to improving the generalization and autonomous operation capabilities of service robots in highly unstructured environments.

[0004] Summary of the Invention

[0005] This application aims to solve one of the technical problems existing in the prior art or related technologies.

[0006] To this end, the first aspect of this application proposes a training scenario generation method.

[0007] The second aspect of this application proposes a model configuration method.

[0008] The third aspect of this application proposes a model configuration method.

[0009] The fourth aspect of the present application proposes a training scenario generating device.

[0010] The fifth aspect of the present application proposes a training scenario generating device.

[0011] The sixth aspect of the present application proposes a model configuration device.

[0012] The seventh aspect of the present application proposes a model configuration device.

[0013] The eighth aspect of the present application proposes a model configuration device.

[0014] A ninth aspect of the present application proposes a readable storage medium.

[0015] A tenth aspect of the present application proposes a computer program product.

[0016] An eleventh aspect of the present application provides an electronic device.

[0017] A twelfth aspect of the present application proposes a computer device.

[0018] In view of this, according to the first aspect of the present application, a training scene generation method is proposed, including: obtaining an object model library; determining the target object model in the object model library through a random algorithm based on the scene information of the scene model, and the target object model matches the scene information; configuring the target object model into the scene model to generate a training scene file.

[0019] According to the second aspect of the present application, a model configuration method is proposed, including: obtaining an object bounding box corresponding to an object model; determining a target anchor point in the object model based on the first coordinate information of the object bounding box and the category information of the object model; and configuring the object model to a scene model based on the target anchor point.

[0020] According to a third aspect of the present application, a model configuration method is proposed, including: obtaining regional information of a placement area of ​​a carrier model, the placement area being an area in the carrier model used to place objects; determining a target area in the placement area through a random algorithm based on the regional information; configuring a first object model to the target area of ​​the carrier model; and returning to the step of determining the target area in the placement area through a random algorithm based on the regional information when the positional relationship between the first object model and the second object model meets preset conditions, the second object model being the object model that has been configured in the carrier model.

[0021] According to the fourth aspect of the present application, a training scene generation device is proposed, including: an acquisition module for acquiring an object model library; a determination module for determining a target object model in the object model library through a domain random algorithm based on scene information of the scene model, wherein the target object model matches the scene information; and a configuration module for configuring the target object model into the scene model to generate a training scene file.

[0022] According to the fifth aspect of the present application, a model configuration device is proposed, including: a memory, in which programs or instructions are stored; a processor, which executes the programs or instructions stored in the memory to implement the steps of the training scene generation method in any of the above-mentioned technical solutions, thereby having all the beneficial technical effects of the training scene generation method in any of the above-mentioned technical solutions, and will not be elaborated on here.

[0023] According to the sixth aspect of the present application, a model configuration device is proposed, including: an acquisition module for acquiring an object bounding box corresponding to an object model; a determination module for determining a target anchor point in the object model based on the first coordinate information of the object bounding box and the category information of the object model; and a configuration module for configuring the object model to a scene model based on the target anchor point.

[0024] According to the seventh aspect of the present application, a model configuration device is proposed, including: an acquisition module for acquiring regional information of a placement area of ​​a carrier model; a determination module for determining a target area in the placement area through a random algorithm based on the regional information, where the placement area is an area in the carrier model for placing objects; a configuration module for configuring a first object model to the target area of ​​the carrier model; and an execution module for returning to the step of determining the target area in the placement area through a random algorithm based on the regional information when the positional relationship between the first object model and the second object model meets a preset condition.

[0025] According to the eighth aspect of the present application, a model configuration device is proposed, comprising: a memory, in which a program or instruction is stored; a processor, which executes the program or instruction stored in the memory to implement the steps of the model configuration method in any technical solution in the first aspect, thereby having all the beneficial technical effects of the model configuration method in any technical solution in the first aspect above, and no further details will be given here.

[0026] According to a ninth aspect of the present application, a readable storage medium is provided. The readable storage medium stores a program or instruction. When the program or instruction is executed by a processor, the program or instruction implements the steps of the method described in any of the above technical solutions. Therefore, all the beneficial technical effects of any of the above technical solutions are achieved, and no further details are given here.

[0027] According to the tenth aspect of the present application, a computer program product is provided. When executed by a processor, the computer program product implements the steps of the method in any of the above technical solutions. Therefore, it has all the beneficial technical effects of the method in any of the above technical solutions, and no further details will be given here.

[0028] According to the eleventh aspect of the present application, an electronic device is proposed, comprising: a training scene generation method as in any of the above-mentioned technical solutions, and / or a model configuration device as in any of the above-mentioned technical solutions, and / or a readable storage medium as in any of the above-mentioned technical solutions, and / or a computer program product as in any of the above-mentioned technical solutions, and thus having all the beneficial technical effects of the training scene generation method as in any of the above-mentioned technical solutions, and / or a model configuration device as in any of the above-mentioned technical solutions, and / or a readable storage medium as in any of the above-mentioned technical solutions, and / or a computer program product as in any of the above-mentioned technical solutions, which will not be elaborated on here.

[0029] The twelfth aspect of the present application proposes a computer device, which includes: a processor and a memory, the memory storing programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the training scene generation method in any of the above-mentioned technical solutions and / or the model configuration method in any of the above-mentioned technical solutions are implemented, thereby having the technical effects that can be achieved by the training scene generation method in any of the above-mentioned technical solutions and / or the model configuration method in any of the above-mentioned technical solutions. To avoid repetition, they will not be repeated here.

[0030] In the technical solution of this application, when a scene model is obtained, the scene information corresponding to the scene model is obtained, and based on this information, a target object model that matches the scene information is randomly selected from the object model library according to a random algorithm. By combining the randomly extracted target object model with the scene model, a training scene file with strong generalization and randomization capabilities can be obtained.

[0031] In the technical solution of this application, when an object model to be configured in a scene model is obtained, the object bounding box corresponding to the object model is obtained, and based on the first coordinate information of the object bounding box and the category information of the object model, the target anchor point corresponding to the object bounding box can be determined. The object model is then configured in the scene model based on the target anchor point, which can realize automatic placement of the object model in the scene model, eliminating the need for manual adjustment of the modeling origin of the object model and the scene model, and simplifying the operation process of configuring the object model in the scene model.

[0032] In the technical solution of the present application, in the process of building a simulation scene of a robot, the object models are randomly configured in the placement area of ​​the carrier model. After the random configuration, it is detected whether the positional relationship between the first object model currently configured in the carrier model and the second object model already configured in the carrier model meets the preset conditions, so as to determine whether the target area where the first object model is located is reasonable. If the target area where the first object model is located is unreasonable, the target area is randomly re-selected in the placement area in a timely manner, thereby reducing the possibility of object models in unreasonable placement positions in the constructed simulation scene, and there is no need to manually adjust the constructed simulation scene, which further simplifies the process of building the simulation.

[0033] Additional aspects and advantages of the present application will become apparent in the following description or may be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0035] FIG1 shows one of the schematic flow charts of the training scenario generation method provided in some embodiments of the present application;

[0036] FIG2 shows a second schematic flow chart of a training scenario generation method provided in some embodiments of the present application;

[0037] FIG3 shows a third schematic flow chart of a training scenario generation method provided in some embodiments of the present application;

[0038] FIG4 shows a fourth schematic flow chart of a training scenario generation method provided in some embodiments of the present application;

[0039] FIG5 shows a fifth schematic flow chart of a training scenario generation method provided in some embodiments of the present application;

[0040] FIG6 shows a schematic diagram of model metadata provided by some embodiments of the present application;

[0041] FIG7 shows a schematic diagram of scene metadata provided by some embodiments of the present application;

[0042] FIG8 shows a sixth schematic flow chart of a training scenario generation method provided in some embodiments of the present application;

[0043] FIG9 shows one of the structural block diagrams of the training scenario generation device provided in some embodiments of the present application;

[0044] FIG10 shows a second structural block diagram of a training scenario generating device provided in some embodiments of the present application;

[0045] FIG11 shows one of the structural block diagrams of an electronic device provided in some embodiments of the present application;

[0046] FIG12 shows one of the schematic flow charts of the model configuration method provided in some embodiments of the present application;

[0047] FIG13 shows a second schematic flow chart of the model configuration method provided in some embodiments of the present application;

[0048] FIG14 shows a third schematic flow chart of the model configuration method provided in some embodiments of the present application;

[0049] FIG15 shows a fourth schematic flow chart of the model configuration method provided in some embodiments of the present application;

[0050] FIG16 is a schematic diagram showing an object bounding box provided in some embodiments of the present application;

[0051] FIG17 shows a fifth schematic flow chart of the model configuration method provided in some embodiments of the present application;

[0052] FIG18 shows one of the structural block diagrams of the model configuration device provided in some embodiments of the present application;

[0053] FIG19 shows a sixth schematic flow chart of the model configuration method provided in some embodiments of the present application;

[0054] FIG20 shows a seventh schematic flow chart of the model configuration method provided in some embodiments of the present application;

[0055] FIG21 shows a two-dimensional schematic diagram of a first object model and a carrier model provided by some embodiments of the present application;

[0056] FIG22 shows a schematic diagram of a first line segment provided in some embodiments of the present application;

[0057] FIG23 shows an eighth schematic flow chart of the model configuration method provided in some embodiments of the present application;

[0058] FIG24 shows a ninth schematic flow chart of a model configuration method provided in some embodiments of the present application;

[0059] FIG25 shows a tenth schematic flow chart of a model configuration method provided in some embodiments of the present application;

[0060] FIG26 shows a second structural block diagram of the model configuration device provided in some embodiments of the present application;

[0061] FIG27 shows a third structural block diagram of a model configuration device provided in some embodiments of the present application;

[0062] FIG28 shows a second structural block diagram of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0063] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the features of this embodiment and the embodiments can be combined with each other.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0065] The following describes the training scenario generation method, device, electronic device, readable storage medium and computer device according to some embodiments of the present application with reference to Figures 1 to 11.

[0066] In related technologies, most simulation scenes are built using standard 3D modeling software, and the output files are in the formats required by various simulation software. However, these software ignores the semantic segmentation related to service robot operations and the requirements of tasks such as physical interaction with objects. These software only converts according to a fixed modeling environment and lacks the generalized domain randomness capability.

[0067] Therefore, according to one embodiment of the present application, as shown in FIG1 , a training scenario generation method is proposed, including:

[0068] Step 102: Obtain an object model library;

[0069] Step 104, determining a target object model in an object model library using a domain random algorithm based on the scene information of the scene model, wherein the target object model matches the scene information;

[0070] Step 106: configure the target object model into the scene model to generate a training scene file.

[0071] In an embodiment of the present application, a training scene generation method is proposed for building a simulation training scene for a robot. By obtaining an object model library and parsing the scene information in the scene model, a target object model that matches the object model library can be determined. The target object model is a model found in the object model library based on a random algorithm. The target object model is then configured into the scene model to generate a training scene file. The target object model is found in the object model library through a random algorithm, and the found target object model is combined with the scene model to generate a corresponding training scene file, thereby improving the diversity of the training scene file. Among them, the object model library pre-stores the three-dimensional model of the object and its metadata information. After the object model library is established, the three-dimensional object model therein can be queried and used based on the object metadata in the object model library.

[0072] It should be noted that after the scene model is generated, reasonable physical parameters are added to the target object model in the scene model according to the object parameter information pre-stored in the database corresponding to the individual examples of the model.

[0073] Specifically, the robot can be a household sweeping robot, a shopping mall guide robot, a restaurant delivery robot, etc. Robots also include various types of service robots, such as household service robots.

[0074] In this embodiment, the object model library stores object models from different sources. The object model library constructed by the object models from different sources enables object models from multiple different sources to be selected when constructing a training scene file.

[0075] Exemplarily, the object models in the object model library can be obtained by forward modeling, by reverse engineering scanning of models, or by three-dimensional reconstruction.

[0076] In this embodiment, the scene model is the model required to construct the training scene file and serves as the carrier for the remaining object models. Scene information includes pre-stored information within the scene model, including but not limited to scene category information, scene size information, and scene function information. This scene information can be used to determine the relevant information about the object model to be placed within the scene model, thereby identifying multiple object models in the object model library that can be placed within the scene model. A rule-based algorithm is then used to extract the target object model from the multiple object models found based on the scene information.

[0077] For example, the scene model can be directly downloaded from the server, or constructed by modeling.

[0078] In the embodiments of the present application, when a scene model is obtained, the scene information corresponding to the scene model is obtained, and based on this information, a target object model that matches the scene information is randomly selected from the object model library according to a random algorithm. By combining the randomly extracted target object model with the scene model, a training scene file with strong generalization and randomization capabilities can be obtained.

[0079] As shown in FIG2 , in some embodiments, optionally, the target object model includes a first object model and a second object model, and the first object model is capable of carrying the second object model. Determining the target object model in the object model library using a domain random algorithm based on scene information of the scene model includes:

[0080] Step 202: extract a first model list from an object model library based on the scene information;

[0081] Step 204 , extracting individual instances of the first object model in the first model list using a domain randomization algorithm;

[0082] Step 206, obtaining object category information of the first object model;

[0083] Step 208: extract a second model list from the object model library based on the object category information;

[0084] Step 210 : extract individual instances of the second object model in the second model list by using a domain random algorithm.

[0085] In the embodiments of the present application, a domain randomization algorithm is used to search, based on scene information, for a first object model placed directly within the scene model, as well as a second object model placed within the first object model, within the scene model file. By randomly extracting the first object model that matches the scene information, and then randomly extracting the second object model that matches the object category information of the first object model, a large number of object models within the scene model are extracted using the domain randomization algorithm, further improving the generalization randomization capability of the resulting training scene file.

[0086] For example, the first object model may be a “table” model, and the individual instance of the first object model is a “dining table of xx model”; the second object model may be a “cup” model, and the individual instance of the second object model is a “wine glass of xx size”.

[0087] In this embodiment, since the scene model is the scene model of the object model in the training scene file, the object model that can be directly configured in the scene model can be determined through the scene information. The object models that can be configured in the scene model are stored in the form of a first model list to facilitate subsequent retrieval and search.

[0088] In this embodiment, the scene models in the first model list are all object models that can be directly configured into the scene model. At this time, the first object model is randomly extracted from the first model list through the domain random algorithm, and the number of the first object model is at least one.

[0089] For example, the scene model includes a house model, and the scene information includes room information and dimensions within the house model, such as a "kitchen" area. Using this scene information, object models that can be placed in the "kitchen" area can be found in an object model library. Based on the found object models, a first model list is generated. The models in the first model list include a "stove" model and a "microwave oven" model. A domain randomization algorithm is used to extract the first object model from the first model list. The number of first object models is positively correlated with the dimension information in the scene information.

[0090] In this embodiment, after randomly extracting a first object model from the first model list, object category information for the first object model is obtained. This object category information can represent the specific classification of the first object model. Using the object category information of the first object model, a second model list is retrieved from the object model library. The object models in the second model list are all object models that can be placed next to the first object model. In this case, a domain random algorithm is used to randomly extract at least one second object model from the second model list.

[0091] In this embodiment, after the second model list is extracted based on the object category information, a correspondence is established between the second model list and the first object model, so that the corresponding second object model can be found in the second model list through the domain random algorithm.

[0092] Exemplarily, the first object model is a “dining table” model, and the models in the second model list may include a “dinner plate” model, a “cutlery” model, and the like.

[0093] Exemplarily, the first object model is a “refrigerator” model, and the models in the second model list may include a “can” model, a “fruit” model, and the like.

[0094] In an embodiment of the present application, based on the scene information in the scene model, a first model list that can be directly placed in the scene model can be extracted from the object model library, and the first object model in the first model list can be randomly extracted. Then, based on the object category information of the first object model, a second model list in the object model library can be extracted, and the second object model in the second model list can be randomly extracted. This enables multi-level random extraction of object models in the object model library, and ensures that the extracted first object model placed in the scene model and the second object model placed in the first object model are consistent with common sense. While improving the generalization randomness of the generated training scene file, it also ensures the rationality of the simulation scene corresponding to the training scene file.

[0095] As shown in FIG3 , in some embodiments, optionally, the target object model includes a first object model, and configuring the target object model into a scene model to generate a training scene file includes:

[0096] Step 302, obtaining the object placement area of ​​the scene model;

[0097] Step 304: extracting a target area from the object placement area using a rule-based judgment algorithm based on the object category information;

[0098] Step 306: Place the first object model in the target area.

[0099] In the embodiment of the present application, a rule-based algorithm is used to determine a target area within the scene model placement area for placing a first object model, and the first object model is placed within the target area. By randomly selecting the target area within the scene model object placement area for placing the first object model, the generalization randomness of the generated training scene file is further improved.

[0100] In this embodiment, the placement area is the area where the scene model can place the first object model, the target area is at least a part of the placement area, and the target area is an area randomly screened in the placement area based on the object category information of the first object model, ensuring that the target area is a more suitable area for placing the first object model.

[0101] For example, if the scene model is a house scene model, the placement area includes a bedroom area, a kitchen area, etc., and the target area includes a part of the bedroom area, a part of the kitchen area, etc.

[0102] In an embodiment of the present application, after determining the object placement area in the scene model, based on the category information of the first object model, the target area in the placement area is randomly extracted, so that the first object model can be randomly placed in the scene model within a reasonable range. While ensuring the random ability of the generated training scene file, the rationality of the simulation scene can also be guaranteed.

[0103] As shown in FIG4 , in some embodiments, optionally, the target object model includes a second object model. After the first object model is placed in the target area, the following steps are further included:

[0104] Step 402: Acquire the accommodation area information of the first object model;

[0105] Step 404: Determine the configuration position and configuration posture of the second object model in the first object model using a random algorithm based on the accommodation area information;

[0106] Step 406: Arrange the second object model onto the first object model according to the arrangement posture and arrangement position.

[0107] In an embodiment of the present application, when the target object model includes a second object model and a first object model has already been configured within the scene model, the configuration position of the second object model within the first object model and the configuration posture of the first object model within the first object model can be randomly selected based on the accommodation area information of the first object model. The second object model is then placed within the first object model based on the configuration posture and configuration position.

[0108] In this embodiment, the accommodation area information of the first object model includes the outer contour coordinates of the first object model. According to the outer contour coordinates, the configuration posture and configuration position of the second object model in the accommodation area can be determined. The second object model is rotated by the configuration posture, and the second object model is moved by the configuration position.

[0109] Exemplarily, the configuration posture includes but is not limited to the orientation of the second object model.

[0110] In an embodiment of the present application, when configuring the second object model to the first object model, since the accommodation area of ​​the first object model is small and the types of second object models that can be placed in the accommodation area are relatively fixed, the configuration posture and configuration position of the second object model can be randomly selected based on the accommodation area information, which simplifies the process of configuring the second object model to the first object model and improves the random ability.

[0111] In some embodiments, optionally, the number of second object models is at least one; after configuring the second object model to the first object model according to the configuration posture and configuration position, it also includes: based on the number of said second object models being at least two, in the case where there is an overlapping area between at least two second object models, returning to execute the step of determining the configuration position and configuration posture of the second object model in the first object model through a random algorithm according to the accommodation area information.

[0112] In an embodiment of the present application, when there are multiple second object models configured in the same first object model, whether there is interference between the two second object models is determined by detecting whether there is overlap between the two second object models. If it is determined that there is interference between the two second object models, it is determined that the placement positions of the two second object models in the first object model are unreasonable, and the process of randomly configuring the second object model's position and configuration posture is returned to reduce the occurrence of unreasonable placement of the second object models in the constructed simulation scene in the first object model.

[0113] For example, the first object model is a “dining table” model, the second object model is a “dinner plate” model, and the third object model may be a “rice bowl” model, a “chopsticks” model, a “spoon” model, or the like.

[0114] In an embodiment of the present application, after configuring the first object model for the second object model, the reasonableness of the placement of the second object model is determined by judging whether there is overlap between multiple second object models, thereby avoiding interference between multiple second object models, which may cause errors in the constructed simulation scene.

[0115] As shown in FIG5 , in some embodiments, optionally, obtaining an object model library includes:

[0116] Step 502: Acquire an object model and first model information of the object model;

[0117] Step 504: Generate model metadata corresponding to the object model based on the first model information;

[0118] Step 506: Build an object model library based on the model metadata.

[0119] The present embodiment provides a process for constructing an object model library. After acquiring an object model, model metadata corresponding to the object model is generated based on the object model's model information. The object model library is constructed using model metadata, meaning that all model data in the object model library is stored in the form of model metadata, making it easier to extract object models. This construction of the object model library facilitates querying metadata corresponding to randomly generated individual model instances during the execution of the scene generation algorithm.

[0120] In an embodiment of the present application, the model metadata includes not only the object model itself, but also the first model information corresponding to the object model, ensuring that the object models in the constructed object model library all correspond to the corresponding first model information, which facilitates the search and extraction of the corresponding object model in the constructed object model library.

[0121] In the embodiment of the present application, the first model information is model information of the object model, which includes but is not limited to information such as the object category and object name of the object model.

[0122] In an embodiment of the present application, after the object model is acquired, the object model and its corresponding first model information are stored in the object model database in the format of model metadata, which can facilitate subsequent extraction of the object model and its corresponding metadata in the object model library.

[0123] In some embodiments, optionally, model metadata corresponding to the object model is generated based on the first model information, including: generating a model label corresponding to the object model based on the first model information; configuring the model label to the object model to generate model metadata; wherein the model label includes at least one of the following: a name label, a category label, a subcategory label, a volume label, an anchor point label, a bias label, a size label, and a physics label.

[0124] In this embodiment of the present application, the model metadata includes not only the object model itself but also model tags determined based on the first model information. The model tags include, but are not limited to, at least one of a name tag, a category tag, a volume tag, and a physical parameter tag. These model tags enable rapid search for the corresponding object model in the object model library.

[0125] Exemplarily, the model labels corresponding to the object model include: a name label “Chinese dining table abc”, a category label “table”, and a subtype label “dining table”.

[0126] It should be noted that the volume tag can store the object model's bounding box information, which can be used to calculate the maximum cubic volume occupied by the object model. The physical parameter tag includes the physical properties of the object corresponding to the object model, ensuring the authenticity of the robot's physical interactions in the simulation scene when the object model is configured.

[0127] As shown in Figure 6, model metadata includes the object model's name tag, type tag, subcategory tag, volume tag (bounding box tag), anchor tag, offset tag, size tag, and physics tag. The physics tags include: mass tag, inertia tag, maximum penetration velocity, maximum impulse force, rebound coefficient, static friction coefficient, average value of dynamic friction coefficient, and the random range of each tag.

[0128] In the embodiment of the present application, by setting corresponding model tags in the object model, the generated model metadata includes various model tags, thereby enabling the rapid search for matching object models in the object model library through the model tags.

[0129] In some embodiments, optionally, after obtaining the object model, the method further includes: calculating an object bounding box corresponding to the object model; and adjusting the object model based on the object bounding box so that the model parameters of the object model match the preset parameters. The object models in the object model library in the embodiments of the present application come from different sources, and the model parameters of the object models from different sources are different. By unifying the model parameters of the object models from different sources, the process of configuring the object models in the scene model can be automated, reducing the manual operations required in the process of configuring the object models in the scene model.

[0130] For example, the expression of the object bounding box is as follows: aabb ={x min y min z min x max y max z max};

[0131] Among them, X aabb is the coordinate information of the object bounding box, x min is the minimum coordinate value of the object's bounding box on the x-axis of its own coordinate system, min is the minimum coordinate value of the object's bounding box on the y-axis of its own coordinate system, min is the minimum coordinate value of the object's bounding box on the z axis of its own coordinate system, x max is the maximum coordinate value of the object's bounding box on the x-axis in its own coordinate system, max is the maximum coordinate value of the object's bounding box on the y-axis of its own coordinate system,max The maximum coordinate value of the object's bounding box on the z-axis in its own coordinate system.

[0132] In this embodiment, each model vertex in the object model is traversed to obtain the vertex coordinates corresponding to each model vertex. Through the vertex coordinates of multiple model vertices, the maximum coordinate value and the minimum coordinate value in the x-axis, y-axis, and z-axis directions of the object modeling coordinate system can be determined, and the object bounding box can be constructed through the obtained maximum coordinate value and minimum coordinate value.

[0133] In the embodiment of the present application, by obtaining the vertex coordinates of the model vertices of the object model, the object bounding box corresponding to the object model can be automatically constructed, further simplifying the process of configuring the object model to the scene model.

[0134] In an embodiment of the present application, by unifying the model parameters of object models from different sources to match the preset parameters, the model parameters in the object model library are made relatively unified, which facilitates the subsequent configuration of the extracted object model to the scene model, and can realize the process of automatically configuring the object model to the scene model.

[0135] In some embodiments, optionally, the model parameters include a model construction origin, the preset parameters include a target position of the construction origin, and the object model is adjusted based on the object bounding box, including: moving the construction origin of the object model to the target position based on the coordinate information of the object bounding box.

[0136] In the embodiment of the present application, the construction origins of object models from different sources are different. In order to facilitate the subsequent automatic placement of the object models into the scene model, the different construction origins need to be unified.

[0137] In this embodiment, the coordinate information of the object's bounding box is the 3D coordinates in its own coordinate system, which is the reference coordinate system for the object model's construction origin. By combining the coordinate information of the object's bounding box with the coordinate information of the target position, the construction origins of different object models can be uniformly processed to the target position.

[0138] For example, the target position of the construction origin of the object model is the geometric center of the object bounding box, which requires each vertex v in the object model to be i =[x i y i z i ] T All are processed according to the following relationship:

[0139] Among them, v i is the vertex of the object model, x iis the coordinate of the vertex of the object model on the x-axis, y i The coordinates of the vertices of the object model on the y-axis, z i is the coordinate of the vertex of the object model on the z axis, T is the coordinate dimension, where x max is the maximum coordinate value on the x-axis, x min is the minimum coordinate value on the x-axis, y max is the maximum coordinate value on the y-axis, y min is the minimum coordinate value on the y-axis, z max is the maximum coordinate value on the z-axis, z min is the minimum coordinate value on the z-axis.

[0140] In an embodiment of the present application, the coordinate information of the object bounding box of the object model and the target position of the construction origin are obtained, so that the construction origin of the object model is uniformly moved to the target position, so that each object model in the object model library can be at the same construction origin, which facilitates the subsequent process of configuring the object model to the scene model.

[0141] In some embodiments, optionally, the model parameters include a scaling ratio, the preset parameters include a preset ratio, and based on at least two object bounding boxes, at least two object models are adjusted, including: based on the coordinate information and preset size parameters of the object bounding box, the scaling ratio of the object model is adjusted to the preset ratio.

[0142] In the embodiment of the present application, object models from different sources have different scaling ratios. In order to facilitate the subsequent automatic placement of the object models into the scene model, the different scaling ratios need to be unified into a preset ratio.

[0143] In this embodiment, the preset size parameter may be a size parameter in one or more dimensions.

[0144] Exemplarily, the preset size parameters include a preset height, a preset width, a preset length, and the like.

[0145] In this embodiment, the scaling ratio of each object model is set according to the coordinate information and preset size parameters of the object bounding boxes corresponding to the multiple object models.

[0146] For example, the unit of the preset size parameter is centimeters, and the average height of the object models of the same category is 15 centimeters, then the scaling ratio of the object models of the same category is adjusted by the following relationship: z =15 / (z max -z min );

[0147] Among them, s z is the adjusted scale ratio, 15 is the average height of the object model of this category, z maxis the maximum coordinate value of the object model on the z axis, min It is the minimum coordinate value of the object model on the z-axis.

[0148] In an embodiment of the present application, the scaling ratio of the object model is adjusted to a preset ratio based on the coordinate information of the object bounding box of the object model and the preset size parameters, so that the scaling ratio of the object model is unified, which facilitates the subsequent process of configuring the object model to the scene model.

[0149] In some embodiments, considering the impact of lighting conditions on perception algorithms based on computer vision, optionally, before configuring the target object model into the scene model and generating a training scene file, it also includes: configuring the light source parameters of the scene model through a random algorithm; wherein the light source parameters include at least one of the following: light source position, light source size, light source lumen, and light source color.

[0150] In an embodiment of the present application, after configuring the target object model to the scene model, it also includes: configuring the scene model with light source parameters including any one of light source position, light source size, light source lumen, and light source color, and realizing the rendering of light and shadow in the robot training scene through a real-time ray tracing algorithm to improve the authenticity of the training simulation scene.

[0151] In some embodiments, optionally, after configuring the target object model into the scene model and generating the training scene file, it also includes: obtaining the target object model configured into the scene model, and second model information of the scene model; based on the second model information, generating scene metadata of the training scene; wherein the second model information includes at least one of the following: position information, category information, name information, size information, and posture information.

[0152] In an embodiment of the present application, after the target object model is configured to the scene model and the training scene file is generated, the scene metadata is constructed by reading the second model information of the scene model and the target object model. The scene metadata can reflect the positional relationship between each model in the training scene, the size parameters of each model, etc. Through the scene metadata, any model in the training scene file can be quickly located, which facilitates the subsequent automatic replacement of the model in the training scene file.

[0153] As shown in Figure 7, the scene metadata includes the room names in the scene model, such as the living room and kitchen. Each room has an object model tag and a building model tag. The object model tags indicate the type, quantity, size, and positioning information of the object models in the room. Object model tags include furniture model tags, door and window model tags, and so on. For example, the door and window model tags can indicate the size of the doors and windows. The building model tags are determined based on the second model information of the scene model. The building model tags include wall model tags, floor model tags, and roof corner model tags, which are used to describe the entire room outline.

[0154] Among them, recording door and window model labels can facilitate the automatic replacement of doors and windows in the simulation scene of the same apartment type, providing a training environment for tasks such as opening and closing doors and windows.

[0155] For example, the first-level menu in the configuration file is the room name, such as: living room, kitchen, bedroom, bathroom, dining room, etc., and the second-level menu is the parameters corresponding to each room, such as: furniture type, door and window size, house wall, house floor, house roof, room height, etc.

[0156] In one embodiment of the present application, as shown in FIG8 , a training scenario generation method is proposed, including:

[0157] Step 802: import the three-dimensional apartment grid and render the apartment model with random materials according to the building structure label;

[0158] The apartment model is the scene model in any of the above embodiments.

[0159] Step 804: Determine the range of each room based on the ground grid, query the database and randomly select furniture models corresponding to the room type, and import them into the room;

[0160] The range of each room is the scene information of the house model in any of the above embodiments, and the furniture model is the first object model in any of the above embodiments.

[0161] Step 806: After randomly selecting a certain type of furniture model, adjust the coordinates of the furniture in the room according to the object bounding box, anchor point and offset of the furniture;

[0162] In which, a certain type of furniture is the first object model in any of the above embodiments. After the first object model is converted to the scene model, the position of the first object model in the room is adjusted according to the object bounding box and anchor point of the first object model, as well as the corresponding bias parameters. The anchor point is updated according to the construction origin of the first object model, and the bias parameter is used to offset the initial construction origin when updating the construction origin.

[0163] Step 808: Read the structured random file and parse the carrier model and the type and quantity of random items to be placed;

[0164] The carrier model is the first object model in any of the above embodiments, and the random items to be placed are the second object model in any of the above embodiments. The structured random file is a model file stored in a structured manner.

[0165] Step 810: Retrieve a list of items that meet the conditions for adding items from the model database, and randomly select an item model from the list;

[0166] The model database is the object model library in any of the above embodiments.

[0167] Step 812, randomly selecting the position and orientation of the object according to the placement area of ​​the carrier model;

[0168] The carrier model is the first object model in any of the above embodiments, the object position is the configuration position in any of the above embodiments, and the orientation is the configuration posture in any of the above embodiments;

[0169] Step 814: Determine whether the item is within the carrier area. If it is not, return to step 812; if it is, return to step 816.

[0170] Step 816, determine whether the object interferes with other objects on the carrier. If it is, return to step 812; if it is not, go to step 818;

[0171] Step 818, add the item to the carrier's list of items;

[0172] The list of items carried by the carrier is the list in the scene metadata in any of the above embodiments.

[0173] Step 820, adding item collision volume, random mass and random center of mass;

[0174] Among them, the object collision volume, random mass, and random center of mass are the physical parameters configured for the model in the training scene file.

[0175] Step 822, calculating the inertia tensor based on the random mass and center of mass;

[0176] Step 824: Generate a simulation scenario, assign a unique identifier, and wait for calling.

[0177] In the technical solution of this application, when a scene model is obtained, the scene information corresponding to the scene model is obtained, and based on this information, a target object model that matches the scene information is randomly selected from the object model library according to a random algorithm. By combining the randomly extracted target object model with the scene model, a training scene file with strong generalization and randomization capabilities can be obtained.

[0178] In one embodiment of the present application, as shown in FIG9 , a training scenario generating apparatus 900 is proposed, including:

[0179] An acquisition module 902 is used to acquire an object model library;

[0180] A determination module 904 is configured to determine a target object model in an object model library using a domain random algorithm based on the scene information of the scene model, wherein the target object model matches the scene information;

[0181] The configuration module 906 is used to configure the target object model into the scene model to generate a training scene file.

[0182] In an embodiment of the present application, a training scene generation device is proposed for building a simulation training scene for a robot. By obtaining an object model library and parsing the scene information in the scene model, a target object model that matches the object model library can be determined. The target object model is a model found in the object model library based on a random algorithm. The target object model is then configured into the scene model to generate a training scene file. The target object model is searched for in the object model library using a random algorithm, and the found target object model is combined with the scene model to generate a corresponding training scene file, thereby improving the generalization and randomization capabilities of the training scene file.

[0183] In this embodiment, the object model library stores object models from different sources. The object model library constructed by the object models from different sources enables object models from multiple different sources to be selected when constructing a training scene file.

[0184] In this embodiment, the scene model is the model required to construct the training scene file and serves as the carrier for the remaining object models. Scene information includes information pre-stored in the scene model, including but not limited to scene category information, scene size information, scene function information, and so on. This scene information can be used to determine the relevant information about the object model to be placed in the scene model, thereby determining the multiple object models in the object model library that can be placed in the scene model. A random algorithm is then used to extract the target object model from the multiple object models found based on the scene information.

[0185] In the embodiments of the present application, when a scene model is obtained, the scene information corresponding to the scene model is obtained, and based on this information, a target object model that matches the scene information is randomly selected from the object model library according to a random algorithm. By combining the randomly extracted target object model with the scene model, a training scene file with strong generalization and randomization capabilities can be obtained.

[0186] In some embodiments, optionally, the target object model includes a first object model and a second object model, and the first object model can carry the second object model; the training scene generation apparatus 900 further includes:

[0187] An extraction module, configured to extract a first model list from an object model library based on scene information;

[0188] An extraction module, configured to extract individual instances of the first object model in the first model list by using a domain random algorithm;

[0189] An acquisition module 902 is configured to acquire object category information of a first object model;

[0190] An extraction module, configured to extract a second model list from the object model library based on the object category information;

[0191] The extraction module is used to extract individual instances of the second object model in the second model list by using a domain random algorithm.

[0192] In the embodiments of the present application, a domain randomization algorithm is used to locate, within a scene model file, first object models directly placed within the scene model, as well as second object models placed within the first object model, based on scene information. By randomly extracting first object models that match the scene information, and then randomly extracting second object models that match the object category information of the first object model, a large number of object models within the scene model are extracted using the randomization algorithm, further improving the generalization randomization capability of the resulting training scene file.

[0193] In an embodiment of the present application, based on the scene information in the scene model, a first model list that can be directly placed in the scene model can be extracted from the object model library, and the first object model in the first model list can be randomly extracted. Then, based on the object category information of the first object model, a second model list in the object model library can be extracted, and the second object model in the second model list can be randomly extracted. This enables multi-level random extraction of object models in the object model library, and ensures that the extracted first object model placed in the scene model and the second object model placed in the first object model are consistent with common sense. While improving the generalization randomness of the generated training scene file, it also ensures the rationality of the simulation scene corresponding to the training scene file.

[0194] In some embodiments, optionally, the acquisition module 902 is configured to acquire an object placement area of ​​the scene model;

[0195] An extraction module is used to extract the target area in the object placement area through a rule judgment algorithm based on object category information;

[0196] The configuration module 906 is configured to configure the first object model to the target area.

[0197] In the embodiment of the present application, a rule-based algorithm is used to determine a target area within the scene model placement area for placing a first object model, and the first object model is placed within the target area. By randomly selecting the target area within the scene model object placement area for placing the first object model, the generalization randomness of the generated training scene file is further improved.

[0198] In an embodiment of the present application, after determining the object placement area in the scene model, based on the category information of the first object model, the target area in the placement area is randomly extracted, so that the first object model can be randomly placed in the scene model within a reasonable range. While ensuring the random ability of the generated training scene file, the rationality of the simulation scene can also be guaranteed.

[0199] In some embodiments, optionally, the acquisition module 902 is configured to acquire accommodation area information of the first object model;

[0200] A determination module 904 is configured to determine a configuration position and a configuration posture of the second object model in the first object model by a random algorithm according to the accommodation area information;

[0201] The configuration module 906 is configured to configure the second object model to the first object model according to the configuration posture and configuration position.

[0202] In an embodiment of the present application, when the target object model includes a second object model and a first object model has already been configured within the scene model, the configuration position of the second object model within the first object model and the configuration posture of the first object model within the first object model can be randomly selected based on the accommodation area information of the first object model. The second object model is then placed within the first object model based on the configuration posture and configuration position.

[0203] In an embodiment of the present application, when configuring the second object model to the first object model, since the accommodation area of ​​the first object model is small and the types of second object models that can be placed in the accommodation area are relatively fixed, the configuration posture and configuration position of the second object model can be randomly selected based on the accommodation area information, which simplifies the process of configuring the second object model to the first object model and improves the random ability.

[0204] In some embodiments, optionally, the number of the second object model is at least one;

[0205] The training scenario generating device 900 further includes:

[0206] The execution module is used to return to the step of determining the configuration position and configuration posture of the second object model in the first object model by a random algorithm based on the accommodation area information, based on the number of the second object models being at least two and the presence of an overlapping area between the at least two second object models.

[0207] In an embodiment of the present application, after configuring the first object model for the second object model, the reasonableness of the placement of the second object model is determined by judging whether there is overlap between multiple second object models, thereby avoiding interference between multiple second object models, which may cause errors in the constructed simulation scene.

[0208] In some embodiments, optionally, the acquisition module 902 is configured to acquire an object model and first model information of the object model;

[0209] The training scenario generating device 900 includes:

[0210] A generating module, configured to generate model metadata corresponding to the object model based on the first model information;

[0211] A construction module is used to build an object model library based on model metadata.

[0212] In an embodiment of the present application, after the object model is acquired, the object model and its corresponding first model information are stored in the object model library in the format of model metadata, which can facilitate subsequent extraction of the object model from the object model library.

[0213] In some embodiments, optionally, a generating module is configured to generate a model label corresponding to the object model according to the first model information;

[0214] A generation module is used to configure model tags into the object model to generate model metadata; wherein the model tags include at least one of the following: name tag, category tag, subcategory tag, volume tag, anchor tag, offset tag, size tag, and physics tag.

[0215] In the embodiment of the present application, by setting corresponding model tags in the object model, the generated model metadata includes various model tags, thereby enabling the rapid search for matching object models in the object model library through the model tags.

[0216] In some embodiments, optionally, the training scenario generating apparatus 900 further includes:

[0217] A construction module is used to calculate the object bounding box corresponding to the object model;

[0218] The adjustment module is used to adjust the object model based on the object bounding box so that the model parameters of the object model match the preset parameters.

[0219] In an embodiment of the present application, by unifying the model parameters of object models from different sources to match the preset parameters, the model parameters in the object model library are made relatively unified, which facilitates the subsequent configuration of the extracted object model to the scene model, and can realize the process of automatically configuring the object model to the scene model.

[0220] In some embodiments, optionally, the model parameters include a model construction origin, the preset parameters include a target location of the construction origin, and the training scenario generation device 900 further includes:

[0221] The moving module is used to move the construction origin of the object model to the target position based on the coordinate information of the object bounding box.

[0222] In the embodiment of the present application, the construction origins of object models from different sources are different. In order to facilitate the subsequent automatic placement of the object models into the scene model, the different construction origins need to be unified.

[0223] In an embodiment of the present application, the coordinate information of the object bounding box of the object model and the target position of the construction origin are obtained, so that the construction origins of multiple object models are uniformly moved to the target position, so that each object model in the object model library can be at the same construction origin, which facilitates the subsequent process of configuring the object model to the scene model.

[0224] In some embodiments, optionally, the model parameters include a scaling ratio, the preset parameters include a preset ratio, and the training scenario generating apparatus 900 further includes:

[0225] The adjustment module is used to adjust the scaling ratio of the object model to a preset ratio based on the coordinate information of the object bounding box and the preset size parameters.

[0226] In the embodiment of the present application, object models from different sources have different scaling ratios. In order to facilitate the subsequent automatic placement of the object models into the scene model, the different scaling ratios need to be unified.

[0227] In an embodiment of the present application, the scaling ratio of the object model is adjusted to a preset ratio based on the coordinate information of the object bounding box of the object model and the preset size parameters, so that the scaling ratio of the object model is unified, which facilitates the subsequent process of configuring the object model to the scene model.

[0228] In some embodiments, optionally, the configuration module 906 is used to configure light source parameters for the scene model through a random algorithm; wherein the light source parameters include at least one of the following: light source position, light source size, light source lumen, and light source color.

[0229] In an embodiment of the present application, after configuring the target object model to the scene model, it also includes: configuring the scene model with light source parameters including any one of light source position, light source size, light source lumen, and light source color, and realizing the rendering of light and shadow in the robot training scene through a real-time ray tracing algorithm to improve the authenticity of the training simulation scene.

[0230] In some embodiments, optionally, the acquisition module 902 is configured to acquire a target object model configured into the scene model and second model information of the scene model;

[0231] A generation module is used to generate scene metadata of the training scene based on the second model information; wherein the second model information includes at least one of the following: position information, category information, name information, size information, and posture information.

[0232] In an embodiment of the present application, after the target object model is configured to the scene model and the training scene file is generated, the scene metadata is constructed by reading the second model information of the scene model and the target object model. The scene metadata can reflect the positional relationship between each model in the training scene, the size parameters of each model, etc. Through the scene metadata, any model in the training scene file can be quickly located, which facilitates the subsequent automatic replacement of the model in the training scene file.

[0233] In one embodiment of the present application, as shown in Figure 10, a training scene generation device 1000 is proposed, including: a processor 1002 and a memory 1004, in which a program or instruction is stored; the processor 1002 executes the program or instruction stored in the memory 1004 to implement the steps of the training scene generation method in any of the above embodiments, and thus has all the beneficial technical effects of the training scene generation method in any of the above embodiments, which will not be elaborated here.

[0234] In one embodiment of the present application, a readable storage medium is provided. The readable storage medium stores a program or instructions. When executed by a processor, the program or instructions implement the steps of the training scenario generation method described in any of the above embodiments. Therefore, the method has all the beneficial technical effects of the training scenario generation method described in any of the above embodiments, and no further details will be given here.

[0235] In one embodiment of the present application, a computer program product is proposed. When the computer program product is executed by a processor, it implements the steps of the training scenario generation method in any of the above embodiments, and thus has all the beneficial technical effects of the training scenario generation method in any of the above embodiments, which will not be further elaborated here.

[0236] In one embodiment of the present application, as shown in Figure 11, an electronic device 1100 is proposed, including: a training scene generation device 1000 as in any of the above embodiments, and / or a readable storage medium 1102 in any of the above embodiments, and / or a computer program product 1104 in any of the above embodiments, and thus has all the beneficial technical effects of the training scene generation device 1000 in any of the above embodiments, and / or the readable storage medium 1102 in any of the above embodiments, and / or the computer program product 1104 in any of the above embodiments, and no further details will be given here.

[0237] The following describes a model configuration method and a model configuration device according to some embodiments of the present application with reference to Figures 12 to 18.

[0238] In related technologies, since the modeling origins of the massive three-dimensional models from different sources on the market cannot be unified, when constructing simulation scenes for home robots, they all rely on manual vision for placement, resulting in high labor and time costs.

[0239] 1. A model configuration method, comprising:

[0240] Get the object bounding box corresponding to the object model;

[0241] determining a target anchor point in the object model according to the first coordinate information of the object bounding box and the category information of the object model;

[0242] Based on the target anchor point, the object model is configured to a scene model.

[0243] 2. The model configuration method according to clause 1, wherein determining the target anchor point in the object model based on the first coordinate information of the object bounding box and the category information of the object model comprises:

[0244] Determining a target plane where the target anchor point is located in the object bounding box according to the category information;

[0245] The target anchor point is determined according to the first coordinate information and the target plane.

[0246] 3. The model configuration method according to clause 2, wherein determining the target anchor point based on the first coordinate information and the target plane comprises:

[0247] determining an offset parameter according to the first coordinate information and the target plane;

[0248] The modeling origin of the object model is offset by using the offset parameter to obtain the target anchor point.

[0249] 4. The model configuration method according to clause 3, wherein configuring the object model to the scene model based on the target anchor point comprises:

[0250] Determining second coordinate information of a preset position in the scene model, wherein the preset position is used to configure the object model;

[0251] Obtaining scaling parameters and rotation matrix of the object model;

[0252] Determining third coordinate information according to the scaling parameter, the rotation matrix, the bias parameter, and the second coordinate information;

[0253] The object model is configured in the scene model according to the third coordinate information.

[0254] 5. The model configuration method according to clause 4, wherein determining the second coordinate information of the preset position in the scene model comprises:

[0255] extracting the preset position within a target area in the scene model by a random algorithm, wherein the target area is associated with the category information;

[0256] Obtain the second coordinate information of the preset position within the placement area.

[0257] 6. The model configuration method according to any one of clauses 2 to 5, wherein determining the target plane where the target anchor point is located in the object bounding box based on the category information comprises:

[0258] The target plane is determined according to the category information and the preset corresponding relationship.

[0259] 7. The model configuration method according to any one of clauses 2 to 6, wherein, before configuring the object model to the scene model based on the target anchor point, the method further comprises:

[0260] Acquire image data of the scene model through a depth camera;

[0261] Extracting a pixel point set of the scene model according to the image data, wherein the pixel point set includes pixel points of an outer contour of the scene model;

[0262] Determining fourth coordinate information according to the pixel point set and the camera intrinsic parameter of the depth camera;

[0263] An object placement area in the scene model is determined according to the fourth coordinate information.

[0264] 8. The model configuration method according to any one of clauses 1 to 7, wherein, before configuring the object model to the scene model based on the target anchor point, the method further comprises:

[0265] Obtaining modeling parameters of the scene model;

[0266] Determining the accommodation space in the scene model using the modeling parameters;

[0267] An object placement area in the scene model is determined according to the fifth coordinate information of the accommodation space.

[0268] 9. The model configuration method according to any one of clauses 1 to 8, wherein obtaining an object bounding box corresponding to the object model comprises:

[0269] Obtaining fifth coordinate information of a model vertex in the object model;

[0270] The object bounding box is constructed based on the fifth coordinate information.

[0271] 10. The model configuration method according to any one of clauses 1 to 9, wherein the scene model comprises at least one of the following: a house model, a furniture model.

[0272] 11. A model configuration device, comprising:

[0273] An acquisition module is used to obtain the object bounding box corresponding to the object model;

[0274] a determination module, configured to determine a target anchor point in the object model based on the first coordinate information of the object bounding box and the category information of the object model;

[0275] A configuration module is used to configure the object model to a scene model based on the target anchor point.

[0276] Therefore, according to one embodiment of the present application, as shown in FIG12 , a model configuration method is proposed, including:

[0277] Step 1202, obtaining the object bounding box corresponding to the object model;

[0278] Step 1204: determining a target anchor point in the object model based on the first coordinate information of the object bounding box and the category information of the object model;

[0279] Step 1206: Based on the target anchor point, the object model is configured into the scene model.

[0280] In an embodiment of the present application, a model configuration method is proposed for configuring an object model into a scene model, thereby completing the construction of a simulation scene for a robot. During the process of configuring the object model into the scene model, by updating the modeling origin of the object model to the target anchor point, the object model can be automatically placed into the scene model based on the target anchor point, eliminating the need for manual configuration of the object model into the scene model, thereby simplifying the process of building the simulation scene.

[0281] Exemplarily, the embodiments of the present application set different target anchor points for different object models. For example, the target anchor point of the ceiling light model is set to the top of the model. Then, after determining the position and orientation of the ceiling light model in the scene model, it can be automatically placed in the scene model, and ensure that the automatically placed ceiling light model complies with the placement rules of the objects corresponding to the model.

[0282] In this embodiment, the object model can be a 3D model obtained by modeling the object, a 3D model downloaded from a server, or a 3D model obtained by 3D reconstruction of another model. The object bounding box is the minimum bounding box of the object model in its own coordinate system, where the own coordinate system is the reference coordinate system of the object model's modeling origin. The first coordinate information of the object bounding box is the 3D coordinate in the own coordinate system.

[0283] For example, the expression of the object bounding box is as follows: aabb ={x min y min z min x max y max z max};

[0284] Among them, X aabb is the first coordinate information of the object bounding box, x min is the minimum coordinate value of the object's bounding box on the x-axis of its own coordinate system, min is the minimum coordinate value of the object's bounding box on the y-axis of its own coordinate system, min is the minimum coordinate value of the object's bounding box on the z axis of its own coordinate system, x max is the maximum coordinate value of the object's bounding box on the x-axis in its own coordinate system, max is the maximum coordinate value of the object's bounding box on the y-axis of its own coordinate system, max The maximum coordinate value of the object's bounding box on the z-axis in its own coordinate system.

[0285] In this embodiment, the category information of the object model is the category information of the object corresponding to the model. When constructing a model database of object models, the category information of each object model is stored accordingly.

[0286] For example, the model name of the object model in the model database is “Chinese dining table abc”, the category in the category information is “table”, and the subtype under the category is “dining table”.

[0287] In an embodiment of the present application, when an object model to be configured within a scene model is obtained, the object bounding box corresponding to the object model is obtained, and based on the first coordinate information of the object bounding box and the category information of the object model, the target anchor point corresponding to the object bounding box can be determined. The object model is then configured within the scene model based on the target anchor point, enabling automated placement of the object model within the scene model without the need for manual adjustment of the modeling origins of the object model and the scene model, thus simplifying the process of configuring the object model within the scene model.

[0288] As shown in FIG13 , in some embodiments, optionally, determining a target anchor point in the object model according to the first coordinate information of the object bounding box and the category information of the object model includes:

[0289] Step 1302: determining the target plane where the target anchor point is located in the object bounding box according to the category information;

[0290] Step 1304 , obtaining first coordinate information of the object bounding box, where the first coordinate information is coordinate information of the object model's modeling origin in reference coordinates;

[0291] Step 1306: Determine the target anchor point based on the first coordinate information and the target plane.

[0292] In the embodiments of the present application, a target plane of the target anchor point in the object bounding box is selected according to the category information of the object model, and a specific process of determining the target anchor point according to the target plane and the first coordinate information is proposed, thereby ensuring that the target anchor point matches the category information of the object model, thereby ensuring that the object model configured in the scene model based on the target anchor point conforms to the actual situation.

[0293] In this embodiment, to avoid the problem of abnormal overlap between the object model and the scene model when assigning the object model to the scene model based on the target anchor point, it is necessary to set the target anchor point on the outer surface of the object bounding box. The object bounding box can be a hexahedron, that is, the object bounding box includes six planes, and the target plane is one of the six planes of the object bounding box.

[0294] In this embodiment, when object models of different categories are configured into a scene model, different points are required to connect with the scene model. Therefore, based on the category information of the object model, the target anchor point can be selected in the target plane of the object bounding box corresponding to the object model.

[0295] In this embodiment, after the target plane is determined, since the first coordinate information is the coordinates of the object bounding box in the reference coordinate system of the construction origin, the corresponding target anchor point can be determined according to the first coordinate information and the target plane.

[0296] In an embodiment of the present application, the target plane where the target anchor point is located in the object bounding box corresponding to the object model can be determined through the category information of the object model, and based on the target plane and the first coordinate information of the object bounding box in the construction origin reference coordinate system, the target anchor point of the target model can be accurately determined, ensuring that in the process of configuring the object model to the scene model based on the target anchor point, it can be ensured that the contact point between the object model and the scene model complies with conventional rules.

[0297] As shown in FIG14 , in some embodiments, optionally, determining a target anchor point according to the first coordinate information and the target plane includes:

[0298] Step 1402: determining an offset parameter based on the first coordinate information and the target plane;

[0299] Step 1404 : offset the modeling origin of the object model using the offset parameter to obtain a target anchor point.

[0300] The embodiment of the present application provides a specific process for determining a target anchor point based on a target plane and first coordinate information, thereby ensuring the accuracy of the obtained target anchor point.

[0301] In this embodiment, the modeling origin is the origin of the object model during the modeling process. By performing offset processing on the modeling origin, the target anchor point of the object model can be obtained. The offset parameter is used to offset the modeling origin of the object model. Since the first coordinate information is the coordinates of the object bounding box in the reference coordinate system of the modeling origin, the offset parameter corresponding to the target anchor point in the target plane can be obtained based on the first coordinate information, thereby offsetting the modeling origin.

[0302] For example, when the target plane is the bottom surface of the target bounding box, the bias parameter is calculated by the following relationship:

[0303] Among them, X offset is the bias parameter, x min is the minimum coordinate value of the object's bounding box on the x-axis of its own coordinate system, minis the minimum coordinate value of the object's bounding box on the y-axis of its own coordinate system, min is the minimum coordinate value of the object's bounding box on the z axis of its own coordinate system, x max is the maximum coordinate value of the object's bounding box on the x-axis in its own coordinate system, max The maximum coordinate value of the object's bounding box on the y-axis in its own coordinate system.

[0304] For example, when the target plane is the back surface of the target bounding box, the bias parameter is calculated by the following relationship:

[0305] Among them, X offset is the bias parameter, x min is the minimum coordinate value of the object's bounding box on the x-axis of its own coordinate system, min is the minimum coordinate value of the object's bounding box on the y-axis of its own coordinate system, min is the minimum coordinate value of the object's bounding box on the z axis of its own coordinate system, max is the maximum coordinate value of the object's bounding box on the y-axis of its own coordinate system, max The maximum coordinate value of the object's bounding box on the z-axis in its own coordinate system.

[0306] In an embodiment of the present application, after determining the target plane, the corresponding relationship formula for calculating the offset parameter can be selected, and then the first coordinate information is substituted into the relationship formula to accurately obtain the offset parameter. The modeling origin is offset by the offset parameter to obtain the target anchor point.

[0307] As shown in FIG15 , in some embodiments, optionally, configuring the object model to the scene model based on the target anchor point includes:

[0308] Step 1502: determining second coordinate information of a preset position in the scene model, where the preset position is used to configure the object model;

[0309] Step 1504, obtaining the scaling parameters and rotation matrix of the object model;

[0310] Step 1506, determining third coordinate information according to the scaling parameter, the rotation matrix, the offset parameter, and the second coordinate information;

[0311] Step 1508: configure the object model into the scene model according to the second coordinate information.

[0312] In the embodiments of the present application, since the orientation of the object model may need to be adjusted and the size of the object model may need to be scaled when the object model is configured in the scene model, the target anchor point also needs to be adjusted accordingly when performing the above adjustments. The second coordinate information of the preset position is calculated based on the bias parameter and the adjustment parameter for adjusting the object model, and the third coordinate information of the object model in the scene model can be determined. The object model is then configured in the scene model accordingly, thereby achieving automatic configuration of the object model after adjustment in the scene model.

[0313] In this embodiment, the preset position may be the desired position of the object model in the scene model, and the second coordinate information is the coordinate information of the preset position in the scene model. The second coordinate information is the coordinate information in the world coordinate system of the scene model. Since the third coordinate information is the coordinate information obtained by calculating and processing the second coordinate information, the third coordinate information is also the coordinate information in the world coordinate system of the scene model.

[0314] In this embodiment, the scaling parameter includes a scaling ratio, by which the object model can be scaled. The rotation matrix is ​​used to adjust the orientation of the object model.

[0315] For example, when constructing a simulation scene of a robot, the scaling parameters and the rotation matrix can be determined by a random algorithm, so that the constructed simulation scene has randomness.

[0316] Exemplarily, the third coordinate information is calculated using the following relationship:

[0317] in, is the third coordinate information, X pos is the second coordinate information, R is the rotation matrix, S is the scaling parameter, X offset is the bias parameter.

[0318] In an embodiment of the present application, when the object model needs to be rotated and scaled during the process of configuring the object model to the scene model, the third coordinate information of the preset position is corrected through the rotation matrix, scaling parameters and bias parameters to ensure that the actual position of the object model configured in the scene model matches the preset position.

[0319] In some embodiments, optionally, determining the second coordinate information of the preset position in the scene model includes: extracting the preset position in the target area in the scene model through a random algorithm, where the target area is associated with the category information; and obtaining the second coordinate information of the preset position in the placement area.

[0320] In the embodiments of the present application, when building a simulation scene for a robot, it is necessary to ensure that the constructed simulation scene has a certain degree of randomness, so that the repetition between multiple simulation scenes is low, thereby improving the training effect of the home server based on the simulation scene. Therefore, when configuring the object model to the scene model, it is necessary to randomize the preset position of the object model in the scene model to ensure that the constructed simulation model has a certain degree of randomness.

[0321] In this embodiment, the target area is at least a portion of the object placement area of ​​the scene model. The target area is an area obtained by screening the object placement area based on the category information of the object model, that is, a suitable area of ​​the object model corresponding to the category information in the object placement area.

[0322] For example, the scene model is a house model, and the object placement areas include the various room areas within the house model, such as the "kitchen," "bedroom," "living room," and "bathroom." If the object model is a "bed," the target area is the "bedroom," and the random location is any location within the "bedroom." If the object model is a "washing machine," the target area is the "bathroom," and the random location is any location within the "bathroom."

[0323] For example, if the scene model is a refrigerator model, the object placement area includes various areas within the refrigerator model where objects can be placed, such as the "refrigerated" area, the "freezer" area, the "top" area, and the "panel" area. If the object model is a "vase" model, the target area is the "top" area, and the random position is any location within the "top" area. If the object model is a "beverage bottle" model, the target area is the "refrigerated" area, and the random position is any location within the "refrigerated" area.

[0324] In this embodiment, the second coordinate information is three-dimensional coordinate information of a preset position, and the three-dimensional coordinate information can be read after the preset position is determined.

[0325] In an embodiment of the present application, in the process of building a simulation scene for a robot, a preset position can be selected within the target area based on the category information of the object model that needs to be configured in the scene model, thereby ensuring that the constructed simulation scene is random and reducing the repetition between the multiple constructed simulation scenes.

[0326] In some embodiments, optionally, determining a target plane where the target anchor point is located in the object bounding box according to the category information includes: determining the target plane according to the category information and a preset correspondence relationship.

[0327] In this embodiment, the preset corresponding relationship is the corresponding relationship between the category information of the object model and the target plane.

[0328] As shown in FIG16 , the object bounding box is a hexahedron. By unfolding the object bounding box, six planes can be obtained, namely, the top plane, the bottom plane, the left side plane, the right side plane, the front side plane, and the back side plane. The target plane is any one of the six planes.

[0329] For example, the object model is a "sofa" model, the category information is "sofa", and the "sofa" is usually placed against a wall, so the target plane is set to the back of the object bounding box corresponding to the "sofa" model.

[0330] For example, the object model is a "crystal lamp" model, the category information is a "ceiling light", and a "ceiling light" is usually hung on the ceiling, so the target plane is set to the top surface of the object bounding box corresponding to the "ceiling light" model.

[0331] For example, the object model is a "chair" model, and the category information is "tables and chairs". "Tables and chairs" are usually placed on the ground, so the target plane is set to the bottom surface of the object bounding box corresponding to the "chair" model.

[0332] In the embodiment of the present application, by presetting the correspondence relationship and category information, the target plane in the object bounding box can be found to ensure the matching degree between the target plane and the category information.

[0333] As shown in FIG17 , in some embodiments, optionally, before configuring the object model to the scene model based on the target anchor point, the process further includes:

[0334] Step 1702: Acquire image data of the scene model through a depth camera;

[0335] Step 1704: extracting a pixel point set of the scene model based on the image data, where the pixel point set includes pixel points of the outer contour of the scene model;

[0336] Step 1706: Determine fourth coordinate information based on the pixel set and the camera intrinsic parameters of the depth camera;

[0337] Step 1708: Determine the object placement area in the scene model based on the fourth coordinate information.

[0338] In an embodiment of the present application, image data of a scene model can be acquired through a simulated depth camera, and based on depth recognition of the image data, an object placement area in the scene model that can be used to configure an object model can be identified, thereby facilitating subsequent selection of a target area in the object placement area and random extraction of a preset position in the target area.

[0339] In this embodiment, the depth camera can be a virtual RGB-D (color depth) camera, which can simulate the image data of the scene model. The image data includes a complete carrier model. The algorithm extracts the pixel point set of the outer contour in the image data. Then, based on the pixel point set and the depth camera that collected the image data, the fourth coordinate information of the object placement area is calculated.

[0340] The fourth coordinate information is a coordinate in a two-dimensional coordinate system.

[0341] It should be noted that the above process of determining the object placement area of ​​the scene model is applicable to scene models whose outer surfaces are used for placing objects, such as plate models, house models, etc.

[0342] Exemplarily, the calculation relationship of the fourth coordinate information is as follows:

[0343] Among them, x i is the set of fourth coordinate information, z i is the depth value of the pixel in the pixel set, p i is the pixel point in the pixel set, and M is the camera intrinsic parameter.

[0344] In an embodiment of the present application, when the scene model is a model whose surface is used to place objects, the object placement area where objects can be placed in the scene model can be determined by performing image recognition on the image data collected by the depth camera, thereby simplifying the process of determining the object placement area.

[0345] In some embodiments, optionally, before configuring the object model to the scene model based on the target anchor point, it also includes: obtaining modeling parameters of the scene model; determining the accommodation space in the scene model through the modeling parameters; and determining the object placement area in the scene model based on the fifth coordinate information of the accommodation space.

[0346] In an embodiment of the present application, the fifth coordinate information of the accommodation space in the scene model can be determined through the modeling parameters of the scene model, and based on the fifth coordinate information, the accommodation space is determined as an object placement area where objects can be placed.

[0347] In this embodiment, when there is a storage space inside the scene model, the grid information in the modeling parameters is obtained. The grid information can be a two-dimensional list. Each row and column of the two-dimensional list contains a spatial bounding box corresponding to the internal storage space of the row and column. Each spatial bounding box corresponds to a storage space. By obtaining the fifth coordinate information of each spatial bounding box, the corresponding object placement area can be determined. The object placement area corresponds one-to-one to the control bounding box.

[0348] In an embodiment of the present application, when the scene model is a multi-layer scene model, the accommodation space inside the scene model is determined according to the modeling parameters, and the object placement area is obtained according to the fifth coordinate information of the accommodation space, thereby ensuring the integrity of the obtained object placement area and avoiding missing the model area in the scene model that can be used to place objects.

[0349] In some embodiments, optionally, obtaining an object bounding box corresponding to the object model includes: obtaining fifth coordinate information of a model vertex in the object model; and constructing the object bounding box based on the fifth coordinate information.

[0350] In the embodiment of the present application, by traversing the fifth coordinate information of the model vertices of the object model, an object bounding box corresponding to the object model can be constructed, thereby simplifying the process of constructing the object bounding box.

[0351] In this embodiment, each model vertex in the object model is traversed to obtain the fifth coordinate information corresponding to each model vertex. Through the fifth coordinate information of multiple model vertices, the maximum coordinate value and the minimum coordinate value in the x-axis, y-axis, and z-axis directions of the object modeling coordinate system can be determined, and the object bounding box can be constructed through the obtained maximum coordinate value and minimum coordinate value.

[0352] In the embodiment of the present application, by obtaining the fifth coordinate information of the model vertices of the object model, the object bounding box corresponding to the object model can be automatically constructed, further simplifying the process of configuring the object model to the scene model.

[0353] In some embodiments, optionally, the scene model includes at least one of the following: a house model, a furniture model.

[0354] For example, when the scene model is a house model, the object models include furniture models, electrical appliance models, etc. For example, the object models include a sofa model, a refrigerator model, etc.

[0355] For example, if the scene model is a furniture model, the object models include small furniture models. For example, if the furniture model is a dining table model, the object models may include a dinner plate model, a cutlery model, etc. If the furniture model is a refrigerator model, the object models may include a dinner plate model, a water cup model, etc.

[0356] In one embodiment of the present application, as shown in FIG18 , a model configuration device 1800 is provided, including:

[0357] An acquisition module 1802 is used to acquire an object bounding box corresponding to the object model;

[0358] a determination module 1804 for determining a target anchor point in the object model based on the first coordinate information of the object bounding box and the category information of the object model;

[0359] The configuration module 1806 is used to configure the object model to the scene model based on the target anchor point.

[0360] In an embodiment of the present application, a model configuration device is proposed for configuring an object model into a scene model, thereby completing the construction of a simulation scene for a robot. During the process of configuring the object model into the scene model, by updating the modeling origin of the object model to the target anchor point, the object model can be automatically placed into the scene model based on the target anchor point, eliminating the need for manual configuration of the object model into the scene model, thereby simplifying the process of building the simulation scene.

[0361] In this embodiment, the object model can be a 3D model obtained by modeling the object, a 3D model downloaded from a server, or a 3D model obtained by 3D reconstruction of another model. The object bounding box is the minimum bounding box of the object model in its own coordinate system, where the own coordinate system is the reference coordinate system of the object model's modeling origin. The first coordinate information of the object bounding box is the 3D coordinate in the own coordinate system.

[0362] In this embodiment, the category information of the object model is the category information of the object corresponding to the model. When constructing a model database of object models, the category information of each object model is stored accordingly.

[0363] In an embodiment of the present application, when an object model to be configured within a scene model is obtained, the object bounding box corresponding to the object model is obtained, and based on the first coordinate information of the object bounding box and the category information of the object model, the target anchor point corresponding to the object bounding box can be determined. The object model is then configured within the scene model based on the target anchor point, enabling automated placement of the object model within the scene model without the need for manual adjustment of the modeling origins of the object model and the scene model, thus simplifying the process of configuring the object model within the scene model.

[0364] In some embodiments, optionally, the determination module 1804 is configured to determine a target plane where the target anchor point is located in the object bounding box according to the category information;

[0365] The determination module 1804 is configured to determine a target anchor point according to the first coordinate information and the target plane.

[0366] In the embodiments of the present application, a target plane of the target anchor point in the object bounding box is selected according to the category information of the object model, and a specific process of determining the target anchor point according to the target plane and the first coordinate information is proposed, thereby ensuring that the target anchor point matches the category information of the object model, thereby ensuring that the object model configured in the scene model based on the target anchor point conforms to the actual situation.

[0367] In an embodiment of the present application, the target plane where the target anchor point is located in the object bounding box corresponding to the object model can be determined through the category information of the object model, and based on the target plane and the first coordinate information of the object bounding box in the construction origin reference coordinate system, the target anchor point of the target model can be accurately determined, ensuring that in the process of configuring the object model to the scene model based on the target anchor point, it can be ensured that the contact point between the object model and the scene model complies with conventional rules.

[0368] In some embodiments, optionally, the determination module 1804 is configured to determine an offset parameter based on the first coordinate information and the target plane;

[0369] The model configuration device 1800 further includes:

[0370] The processing module is used to perform offset processing on the modeling origin of the object model through the offset parameter to obtain the target anchor point.

[0371] The embodiment of the present application provides a specific process for determining a target anchor point based on a target plane and first coordinate information, thereby ensuring the accuracy of the obtained target anchor point.

[0372] In an embodiment of the present application, after determining the target plane, the corresponding relationship formula for calculating the offset parameter can be selected, and then the first coordinate information is substituted into the relationship formula to accurately obtain the offset parameter. The modeling origin is offset by the offset parameter to obtain the target anchor point.

[0373] In some embodiments, optionally, the determining module 1804 is configured to determine second coordinate information of a preset position in the scene model, where the preset position is used to configure the object model;

[0374] An acquisition module 1802 is used to acquire scaling parameters and rotation matrix of the object model;

[0375] A determination module 1804 is configured to determine third coordinate information based on the scaling parameter, the rotation matrix, the offset parameter, and the second coordinate information;

[0376] The configuration module 1806 is used to configure the object model into the scene model according to the third coordinate information.

[0377] In the embodiments of the present application, since the orientation of the object model may need to be adjusted and the size of the object model may need to be scaled when the object model is configured in the scene model, the target anchor point also needs to be adjusted accordingly when performing the above adjustments. The second coordinate information of the preset position is calculated based on the bias parameter and the adjustment parameter for adjusting the object model, and the third coordinate information of the object model in the scene model can be determined. The object model is then configured in the scene model accordingly, thereby achieving automatic configuration of the object model after adjustment in the scene model.

[0378] In an embodiment of the present application, when the object model needs to be rotated and scaled during the process of configuring the object model to the scene model, the third coordinate information of the preset position is corrected through the rotation matrix, scaling parameters and bias parameters to ensure that the actual position of the object model configured in the scene model matches the preset position.

[0379] In some embodiments, optionally, the model configuration device 1800 further includes:

[0380] An extraction module, configured to extract a preset position within a target area in the scene model by using a random algorithm, wherein the target area is associated with category information;

[0381] The acquisition module 1802 is used to obtain the second coordinate information of the preset position in the placement area.

[0382] In the embodiments of the present application, when building a simulation scene for a robot, it is necessary to ensure that the constructed simulation scene has a certain degree of randomness, so that the repetition between multiple simulation scenes is low, thereby improving the training effect of the home server based on the simulation scene. Therefore, when configuring the object model to the scene model, it is necessary to randomize the preset position of the object model in the scene model to ensure that the constructed simulation model has a certain degree of randomness.

[0383] In an embodiment of the present application, in the process of building a simulation scene for a robot, a preset position can be selected within the target area based on the category information of the object model that needs to be configured in the scene model, thereby ensuring that the constructed simulation scene is random and reducing the repetition between the multiple constructed simulation scenes.

[0384] In some embodiments, optionally, the determination module 1804 is configured to determine a target plane according to the category information and a preset correspondence relationship.

[0385] In the embodiment of the present application, by presetting the correspondence relationship and category information, the target plane in the object bounding box can be found to ensure the matching degree between the target plane and the category information.

[0386] In some embodiments, optionally, the acquisition module 1802 is configured to acquire image data of the scene model through a depth camera;

[0387] An extraction module, configured to extract a pixel point set of a scene model based on the image data, wherein the pixel point set includes pixel points of an outer contour of the scene model;

[0388] A determination module 1804 is configured to determine fourth coordinate information based on the pixel point set and the camera intrinsic parameters of the depth camera;

[0389] The determination module 1804 is configured to determine an object placement area in the scene model according to the fourth coordinate information.

[0390] In an embodiment of the present application, image data of a scene model can be acquired through a simulated depth camera, and based on depth recognition of the image data, an object placement area in the scene model that can be used to configure an object model can be identified, thereby facilitating subsequent selection of a target area in the object placement area and random extraction of a preset position in the target area.

[0391] In an embodiment of the present application, when the scene model is a model whose surface is used to place objects, the object placement area where objects can be placed in the scene model can be determined by performing image recognition on the image data collected by the depth camera, thereby simplifying the process of determining the object placement area.

[0392] In some embodiments, optionally, the acquisition module 1802 is configured to acquire modeling parameters of the scene model;

[0393] A determination module 1804 is configured to determine the accommodation space in the scene model using modeling parameters;

[0394] The determination module 1804 is configured to determine an object placement area in the scene model according to the fifth coordinate information of the accommodation space.

[0395] In an embodiment of the present application, the fifth coordinate information of the accommodation space in the scene model can be determined through the modeling parameters of the scene model, and based on the fifth coordinate information, the accommodation space is determined as an object placement area where objects can be placed.

[0396] In an embodiment of the present application, when the scene model is a multi-layer scene model, the accommodation space inside the scene model is determined according to the modeling parameters, and the object placement area is obtained according to the fifth coordinate information of the accommodation space, thereby ensuring the integrity of the obtained object placement area and avoiding missing the model area in the scene model that can be used to place objects.

[0397] In some embodiments, optionally, the acquisition module 1802 is configured to acquire fifth coordinate information of a vertex in the object model;

[0398] The model configuration device 1800 further includes:

[0399] The construction module is used to construct the object bounding box based on the fifth coordinate information.

[0400] In the embodiment of the present application, by traversing the fifth coordinate information of the model vertices of the object model, an object bounding box corresponding to the object model can be constructed, thereby simplifying the process of constructing the object bounding box.

[0401] In the embodiment of the present application, by obtaining the fifth coordinate information of the model vertices of the object model, the object bounding box corresponding to the object model can be automatically constructed, further simplifying the process of configuring the object model to the scene model.

[0402] In some embodiments, optionally, the scene model includes at least one of the following: a house model, a furniture model.

[0403] The following describes a model configuration method, a model configuration device, a readable storage medium, a computer program product, and an electronic device according to some embodiments of the present application with reference to Figures 19 to 28.

[0404] The construction of a home simulation scene for a service robot not only requires certain visual realism, but also needs to consider the rationality of the layout of objects in the scene.

[0405] 1. A model configuration method, comprising:

[0406] Acquire area information of a placement area of ​​the carrier model, where the placement area is an area in the carrier model for placing objects;

[0407] Determining a target area in the placement area by a random algorithm based on the area information;

[0408] arranging a first object model to the target area of ​​the carrier model;

[0409] When the positional relationship between the first object model and the second object model meets the preset conditions, the step of determining the target area in the placement area through a random algorithm based on the area information is returned to, and the second object model is an object model that has been configured in the carrier model.

[0410] 2. The model configuration method according to clause 1, wherein the carrier model comprises a house model, and the first object model comprises a first furniture model;

[0411] The determining the target area in the placement area by a random algorithm according to the area information includes:

[0412] Obtaining an object bounding box of the first furniture model;

[0413] Determining at least two sub-areas in the placement area according to the area information and the object bounding box, wherein the at least two sub-areas can both accommodate the first object model;

[0414] The target region in the at least two sub-regions is determined by a random algorithm.

[0415] 3. The model configuration method according to clause 2, wherein determining at least two sub-areas in the placement area based on the area information and the object bounding box comprises:

[0416] Determining, based on the area information, at least two first line segments in the placement area, where the first line segments match walls in the house model;

[0417] determining, according to the object bounding box, a second line segment in the first furniture model, where the second line segment is related to a preset placement posture of the first furniture model;

[0418] determining a target line segment among the at least two first line segments according to sizes of the at least two first line segments and a size of the second line segment, wherein the size of the target line segment is larger than the size of the second line segment;

[0419] The at least two sub-areas are determined based on the first coordinate information of the target line segment.

[0420] 4. The model configuration method according to clause 3, wherein after configuring the first object model to the target area of ​​the carrier model, the method further comprises:

[0421] determining a translation direction of the first furniture model according to the first coordinate information and second coordinate information, wherein the second coordinate information is coordinate information of a coordinate point located in the placement area;

[0422] The first furniture model is moved along the translation direction until the distance between the first furniture model and the second line segment is greater than a first preset distance.

[0423] 5. The model configuration method according to clause 3 or 4, wherein the preset condition includes at least one of the following:

[0424] In the case where the second object model includes a door and window model, the distance between the first furniture model and the door and window model is less than a second preset distance;

[0425] In the case where the second object model includes a second furniture model, there is an overlapping area between the first furniture model and the second furniture model;

[0426] In a case where the second object model includes a wall model, a distance between the first furniture model and the wall model is less than a third preset distance.

[0427] 6. The model configuration method according to any one of clauses 1 to 5, wherein the carrier model includes a third furniture model, and the volume of the first object model is smaller than a preset volume;

[0428] The determining the target area in the placement area by a random algorithm according to the area information includes:

[0429] Determining a preset orientation and a preset position of the first object model by a random algorithm according to the area information, wherein the preset position is located within the placement area;

[0430] The target area in the third furniture model is determined according to the preset orientation and the preset position.

[0431] 7. The model configuration method according to clause 6, wherein before configuring the first object model to the target area of ​​the carrier model, the method further comprises:

[0432] Determining the type and placement quantity of objects corresponding to the carrier model according to the carrier information of the carrier model;

[0433] Based on the object type, screening at least two third object models in the object model library;

[0434] generating an object model list according to the at least two third object models and the placement quantity;

[0435] The first object model is selected from the object model list by a random algorithm, and the placement quantity corresponding to the first object model is determined.

[0436] 8. The model configuration method according to clause 7, wherein the preset conditions include: the first object model and the second object model overlap, and / or the first object model is not within the target area.

[0437] 9. The model configuration method according to any one of clauses 1 to 8, wherein after configuring the first object model to the target area of ​​the carrier model, the method further comprises:

[0438] The model tag of the first object model is configured within the model tag of the carrier model.

[0439] 10. The model configuration method according to any one of clauses 1 to 8, wherein after configuring the first object model to the target area of ​​the carrier model, the method further comprises:

[0440] configuring physical parameters of the first object model through a random algorithm;

[0441] The physical parameters include at least one of the following: mass, center of mass, inertia tensor, friction coefficient, and volume.

[0442] 11. The model configuration method according to clause 10, wherein configuring the physical parameters of the first object model using a random algorithm comprises:

[0443] In a case where the physical parameters include the mass, the center of mass, and the inertia tensor, determining a target mass by a random algorithm within a preset range;

[0444] determining a target inertia tensor based on the target mass;

[0445] When the target center of mass is determined, the target inertia tensor is updated based on the target center of mass.

[0446] 12. A model configuration device, comprising:

[0447] An acquisition module, used to obtain the area information of the placement area of ​​the carrier model;

[0448] A determination module, configured to determine a target area in the placement area by a random algorithm based on the area information, where the placement area is an area in the carrier model for placing objects;

[0449] a configuration module, configured to configure the first object model to the target area of ​​the carrier model;

[0450] An execution module is used to return to the step of determining the target area in the placement area through a random algorithm based on the area information when the positional relationship between the first object model and the second object model meets a preset condition, and the second object model is an object model that has been configured in the carrier model.

[0451] Therefore, according to one embodiment of the present application, as shown in FIG19 , a model configuration method is proposed, including:

[0452] Step 1902: Acquire regional information of a placement area of ​​the carrier model, where the placement area is an area in the carrier model used for placing objects;

[0453] Step 1904: Determine the target area in the placement area using a random algorithm based on the area information;

[0454] Step 1906 , placing the first object model in the target area of ​​the carrier model;

[0455] Step 1908: When the positional relationship between the first object model and the second object model satisfies a preset condition, the process returns to the step of determining the target area in the placement area by a random algorithm based on the area information.

[0456] The second object model is an object model that has been configured in the carrier model.

[0457] In an embodiment of the present application, a model configuration method is proposed for configuring an object model into a carrier model, thereby completing the construction of a simulation scene for a robot. In the process of configuring the object model into the carrier model, a target area for placing the first object model in the placement area of ​​the carrier model is determined by a random algorithm, and the first object model is configured into the target area. After the first object model is configured into the target area, the positional relationship between the first object model and the second object model is detected to determine whether the layout of the first object model in the carrier model is reasonable. If the layout is unreasonable, the step of randomly selecting the target area in the placement area is returned to execute, thereby reducing the occurrence of unreasonable layout of the first object model in the constructed simulation scene.

[0458] Specifically, the robot can be a household sweeping robot, a shopping mall guide robot, a restaurant delivery robot, etc. Robots also include various types of service robots, such as household service robots.

[0459] In this embodiment, the first object model is the object model that currently needs to be configured in the carrier model, and the second object model is the object model that has already been configured in the carrier model, wherein the first object model, the second object model and the carrier model can be three-dimensional models obtained by modeling the object, or can be three-dimensional models downloaded from the server, or can be three-dimensional models obtained by three-dimensional reconstruction of other models.

[0460] It should be noted that the carrier model can be a site model such as a house, or a model of furniture.

[0461] For example, when the carrier model is a house model, the first object model includes furniture models, electrical appliance models, etc. For example, the first object model is a sofa model, a refrigerator model, etc.

[0462] For example, if the carrier model is a furniture model, the first object model includes a small furniture item model. For example, if the furniture model is a dining table model, the first object model may include a dinner plate model, a cutlery model, etc. If the furniture model is a refrigerator model, the first object model may include a dinner plate model, a water cup model, etc.

[0463] In this embodiment, the placement area is the area where the carrier model can place the object model, the target area is at least a part of the object placement area, and the target area is an area randomly screened in the placement area based on the model information of the first object model to ensure that the target area is a more suitable area for placing the first object model.

[0464] Exemplarily, the target area is an area determined based on object category information of the first object model.

[0465] Exemplarily, the carrier model is a refrigerator model, and the placement area includes the top surface of the refrigerator model and the inner cavity of the refrigerator model.

[0466] Exemplarily, the carrier model is a dinner plate model, and the discharge area is the accommodation space in the dinner plate model.

[0467] It should be noted that the first object model is the object model that currently needs to be configured into the carrier model, while the second object model is the object model that is fixed in the target area within the carrier model. In other words, the second object model is the model that is already located in the carrier model. After the first object model is configured, it is marked as the second object model.

[0468] In this embodiment, after the first object model is placed on the carrier model according to the target area, the positional relationship between the first object model and the second object model is detected to determine whether the positional relationship meets the preset conditions. If the positional relationship meets the preset conditions, the target area where the first object model is located is determined to be unreasonable. In this case, it is necessary to randomly select a new target area within the placement area until the positional relationship between the first object and the second object no longer meets the preset conditions. If the positional relationship does not meet the preset conditions, the target area where the first object model is located is determined to be reasonable, and the placement of the first object model is completed.

[0469] In an embodiment of the present application, in the process of building a simulation scene of a robot, object models are randomly configured in the placement area of ​​the carrier model. After the random configuration, it is detected whether the positional relationship between the first object model currently configured in the carrier model and the second object model that has been configured in the carrier model meets the preset conditions, so as to determine whether the target area where the first object model is located is reasonable. If the target area where the first object model is located is unreasonable, the target area is randomly re-selected within the placement area in a timely manner, thereby reducing the possibility of object models in unreasonable placement positions in the constructed simulation scene, and eliminating the need for manual adjustment of the constructed simulation scene, further simplifying the process of building the simulation.

[0470] As shown in FIG20 , in some embodiments, optionally, the carrier model includes a house model, and the first object model includes a first furniture model; determining a target area in the placement area using a random algorithm based on the area information includes:

[0471] Step 2002, obtaining the object bounding box of the first furniture model;

[0472] Step 2004: Determine at least two sub-regions in the placement region based on the region information and the object bounding box, where both sub-regions can accommodate the first object model.

[0473] Step 2006: Determine a target area in at least two sub-areas using a random algorithm.

[0474] In an embodiment of the present application, when the carrier model is a house model and the first object model includes a first furniture model, multiple sub-areas in the placement area are determined through the object bounding box of the first furniture model and the area information of the placement area, and then a target area is selected from the multiple sub-areas through a random algorithm to ensure that the target area can accommodate the first furniture model, while also ensuring the randomness of the placement position of the first furniture model.

[0475] In this embodiment, when it is necessary to select a target area within the placement area for a first furniture model, the object bounding box corresponding to the first furniture model is obtained. Based on the object bounding box and the area information of the placement area, a sub-area within the placement area that can accommodate the first furniture model is determined. Multiple sub-areas within the placement area can be used to place the first furniture model, so it is necessary to select a sub-area from the multiple sub-areas as the target area. To ensure randomness in the placement of the first furniture model, the target area is selected from the multiple sub-areas using a random algorithm.

[0476] In this embodiment, the object bounding box is the minimum bounding box of the first furniture model in its own coordinates, and the own coordinate system is the reference coordinate system of the modeling origin of the first furniture model. Since the object bounding box is the minimum bounding box of the first furniture model, multiple sub-areas that can accommodate the first furniture model can be determined within the placement area based on the object bounding box, wherein each sub-area can accommodate the object bounding box, so each sub-area can be used to place the first furniture model.

[0477] For example, in the process of screening at least two sub-regions through the object bounding box and region information, the object bounding box can be converted into a planar quadrilateral, and the placement area can also be converted into a planar figure based on the region information. By processing the object bounding box and the placement area in a two-dimensional plane dimension, the corresponding target area can be accurately obtained while reducing the amount of data processing.

[0478] In an embodiment of the present application, a corresponding object bounding box is constructed for the first furniture model, and at least two corresponding sub-areas can be screened in the placement area through the object bounding box to ensure that the screened sub-areas can stably accommodate the object bounding box. A target area is selected from at least two sub-areas through a random algorithm to ensure that the target area can be used to place the first furniture model, thereby reducing the possibility of interference between the first furniture model and the house model.

[0479] In some embodiments, optionally, at least two sub-areas in the placement area are determined based on the area information and the object bounding box, including: determining at least two first line segments in the placement area based on the area information, the first line segments matching the walls in the house model; determining a second line segment in the first furniture model based on the object bounding box, the second line segment being related to a preset placement posture of the first furniture model; determining a target line segment in the at least two first line segments based on the sizes of the at least two first line segments and the sizes of the second line segments, the size of the target line segment being larger than the size of the second line segment; and determining at least two sub-areas based on the first coordinate information of the target line segment.

[0480] In an embodiment of the present application, based on the size relationship between the first line segment corresponding to the wall of the placement area and the second line segment in the first furniture model, it is accurately judged whether the sub-area where the wall corresponding to the first line segment is located can be used to place the first furniture model, thereby ensuring that multiple sub-areas determined in the placement area can accommodate the first furniture model.

[0481] In this embodiment, the region information includes coordinate information of the placement area. A first plane figure of the placement area can be obtained using the region information. The first plane figure includes multiple second line segments, each of which corresponds to a wall in the house model. Based on the object bounding box of the first furniture model, a second plane figure corresponding to the first furniture model can be determined. The second plane figure includes multiple third line segments. A second line segment of the multiple third line segments is selected based on the placement posture of the first furniture model. The second line segment must extend parallel or approximately parallel to the first line segment. The second line segment is a line segment on the side wall that must be close to a wall in the house model when the first furniture model is placed.

[0482] As shown in FIG21 , the first furniture model is a refrigerator model, and the object bounding box corresponding to the refrigerator model is converted into a second plane figure 2102, which is a quadrilateral. Furthermore, the house model is converted into a first plane figure 2104 based on the area information of the placement area.

[0483] In this embodiment, the size of the first line segment is compared with the size of the second line segment to determine a target line segment whose size is larger than the size of the second line segment. The target line segment is a line segment among the multiple first line segments whose size is larger than the size of the second line segment. The area where the target line segment is located can accommodate multiple sub-areas of the first furniture model. Specifically, if the size of the second line segment is smaller than the size of the first line segment, it is determined that the area where the second line segment is located can accommodate the first furniture model. Then, based on the coordinate information of the second line segment, multiple sub-areas are determined.

[0484] As shown in Figure 22, first line segment 2202 is a line segment on the side wall of the first furniture model. First line segment 2202 is a wall line segment. When the size of second line segment 2204 is smaller than the size of first line segment 2202, it is determined that there are multiple sub-areas 2206 along the extension direction of first line segment 2202 that can accommodate the first furniture model.

[0485] Exemplarily, the first coordinate information of the first line segment is a two-dimensional coordinate, and the first coordinate information of the first line segment is (x, y). According to the maximum and minimum values ​​of x and y, a range area can be determined, and at least two sub-areas can be generated in the range area.

[0486] In an embodiment of the present application, by comparing the sizes of the second line segment with the multiple first line segments in sequence, the area where the target line segment whose size is larger than the second line segment among the multiple first line segments is located is used to determine at least two sub-areas that can accommodate the first furniture model, thereby ensuring that the at least two sub-areas obtained can both accommodate the first furniture model.

[0487] As shown in FIG23 , in some embodiments, optionally, after arranging the first object model in the target area of ​​the carrier model, the following steps are included:

[0488] Step 2302: Determine the translation direction of the first furniture model based on the first coordinate information and the second coordinate information, where the second coordinate information is the coordinate information of the coordinate point located in the placement area.

[0489] Step 2304: Move the first furniture model along the translation direction until the distance between the first furniture model and the second line segment is greater than a first preset distance.

[0490] In an embodiment of the present application, after the first furniture model is placed in the target area, the translation direction for correcting the position of the first furniture model is determined using the first coordinate information of the target line segment and the second coordinate information of the coordinate point within the placement area. The position of the first furniture model can be corrected according to this translation direction, ensuring that the first furniture model remains stable within the placement area and preventing interference between the first furniture model and the wall of the carrier model.

[0491] In this embodiment, the second coordinate information is the coordinate information of the coordinate point in the placement area, so the translation direction of the first furniture model moving into the placement area can be determined based on the first coordinate information and the second coordinate information.

[0492] As shown in FIG21 , the second coordinate information may be the coordinate information of the geometric center point 2106 of the placement area, and the first coordinate information may be the coordinate information of the first line segment 2108 . The translation direction may be determined by the first coordinate information and the second coordinate information.

[0493] Exemplarily, the first preset distance ranges from 5 cm to 15 cm.

[0494] In an embodiment of the present application, the first coordinate information and the second coordinate information can be used to determine the translation direction of the first furniture model after it is configured on the house model, and the first furniture model can be moved according to the translation direction so that the first furniture model can be inside the house model, avoiding interference between the first furniture model and the house model.

[0495] In some embodiments, optionally, the preset conditions include at least one of the following: when the second object model includes a door and window model, the distance between the first furniture model and the door and window model is less than a second preset distance; when the second object model includes a second furniture model, there is an overlapping area between the first furniture model and the second furniture model; when the second object model includes a wall model, the distance between the first furniture model and the wall model is less than a third preset distance.

[0496] In an embodiment of the present application, when the first object model is a first furniture model and the carrier model is a house model, by judging whether the first furniture model overlaps with the second furniture model that has been configured in the house model, judging whether the first furniture model is too close to the door and window model in the house model, and judging whether the first furniture model is too close to the wall model in the house model, it is determined whether the positional relationship between the first object model and the second object model meets the preset conditions, thereby determining whether the placement of the first furniture model is reasonable.

[0497] In this embodiment, the door and window model is a sub-model in the house model. When the distance between the first furniture model and the door and window model is less than the second preset distance, it is determined that the first furniture model is too close to the door and window model, and therefore the placement of the first furniture model is unreasonable.

[0498] As shown in Figure 21, for example, when generating the first plane graphic according to the area information of the placement area, the door and window models 2110 in the house model are also shown in the first plane graphic. For example, the door and window models in the house model are shown by quadrilaterals.

[0499] In this embodiment, the second furniture model is other furniture models that have been configured in the house model. After the first furniture model is configured in the house model, there is an overlapping part between the first furniture model and the other second furniture models. It is determined that the first furniture model has interfered with the second furniture model, so the placement of the first furniture model is unreasonable.

[0500] In this embodiment, the wall model is a sub-model in the house model. When the distance between the first furniture model and the wall model is less than the third preset distance, it is determined that the first furniture model is too close to the wall model, and therefore the placement of the first furniture model is unreasonable.

[0501] In an embodiment of the present application, by comparing the positional relationship between the first furniture model and the door and window model, the wall model, and the second furniture model that has been configured in the house model, it is possible to accurately determine whether the current position of the first furniture model is reasonable. If the position of the first furniture model is unreasonable, the target area can be randomly reselected in a timely manner.

[0502] As shown in FIG. 24 , in some embodiments, optionally, the carrier model includes a third furniture model, and the volume of the first object model is smaller than a preset volume; determining a target area in the placement area by a random algorithm based on the area information includes:

[0503] Step 2402: Determine a preset orientation and a preset position of the first object model using a random algorithm based on the area information, wherein the preset position is within the placement area.

[0504] Step 2404: Determine a target area in the third furniture model according to the preset orientation and the preset position.

[0505] In an embodiment of the present application, when the carrier model is a third furniture model, the first object model is a smaller model. Based on the regional information of the placement area of ​​the third furniture model, a preset orientation of the first object model and a preset position of the first object model within the third furniture model can be randomly selected. Based on the preset orientation and position, the first object model is placed within the target area.

[0506] In this embodiment, when the carrier model is the third furniture model, the area information includes the outer contour coordinates of the placement area. According to the outer contour coordinates, the preset orientation and preset position of the first object model in the placement area can be determined, the first object model is rotated according to the preset orientation, and the position of the first object model in the placement area is moved according to the preset position.

[0507] In an embodiment of the present application, when the carrier model is a furniture model, since the placement area in the furniture model is small and the types of first object models that can be placed in the placement area are relatively fixed, the preset orientation and preset position of the first object model can be directly randomly selected, thereby further simplifying the process of configuring the first object model to the third furniture model.

[0508] In some embodiments, optionally, before configuring the first object model to the target area of ​​the carrier model, the method includes: determining the object type and placement quantity corresponding to the carrier model based on the carrier information of the carrier model; screening at least two third object models in the object model library based on the model type; generating an object model list based on the at least two third object models and the placement quantity; selecting the first object model from the object model list through a random algorithm, and determining the placement quantity corresponding to the first object model.

[0509] In the embodiment of the present application, the carrier information can be used to determine the type of objects that the carrier model can carry, as well as the number of object models corresponding to the object type placed on the carrier model.

[0510] It should be noted that the carrier model is stored in the form of a structured random file, such as json format or xml format. By retrieving the structured random file, the object type corresponding to the carrier information of the carrier model and the placement quantity corresponding to the object type can be parsed. For example, if the object type is fruit, the placement quantity is 2 to 5.

[0511] Exemplarily, the carrier information is the carrier type. For example, if the carrier model is a dining table model, the carrier type is a table. It can be determined that the object types corresponding to the dining table model include tableware, pots, etc. The number of tableware placed is 4 to 10, and the number of pots placed is 2 to 5.

[0512] In this embodiment of the present application, after determining the object type corresponding to the object model that can be placed on the carrier model, at least two matching third object models are selected from the object model library based on the object type. For example, if the object type is tableware, the object model library will search for corresponding rice bowl models, dinner plate models, chopstick models, etc., and each model can be multiple or one.

[0513] In the embodiment of the present application, after finding at least two third object models, an object model list is constructed based on the third object models and the corresponding placement quantities of the third object models. The object model list is shown in Table 1 below:

[0514] Table 1

[0515] In this embodiment of the present application, after constructing the object model list, a random algorithm is used to randomly select the corresponding placement quantity and the third object model from the object model list. For example, if a dinner plate model is randomly selected in Table 1, then the object models to be placed in the target area are determined to be two dinner plate models.

[0516] In an embodiment of the present application, the object type and placement quantity of the object models that can be placed on the carrier model can be found based on the carrier information, and the corresponding third object model can be found accordingly, and an object model list is constructed. The first object model placed in the carrier model and the placement quantity corresponding to the first object model are randomly selected from the object model list through a random algorithm to ensure that the random first object model matches the carrier model, and the placement quantity of the first object models can be stably placed in the target area of ​​the carrier model.

[0517] In some embodiments, optionally, the preset condition includes: the first object model overlaps with the second object model, and / or the first object model is not within the target area.

[0518] In an embodiment of the present application, if the first object model overlaps with the second object model that has been configured in the third furniture model, it is determined that the first object model has interfered with other object models, and thus it is determined that the placement of the first object model is unreasonable. Therefore, the process of randomly selecting the target area is returned at this time.

[0519] Exemplarily, an interference detection algorithm is used to detect whether a first object model interferes with a second object model. The interference detection algorithm may be a ray-guided method. If there are multiple second object models, it is necessary to sequentially check whether there is overlap between the first object model and each second object model, that is, whether the first object model interferes with a second object model already positioned within the target area.

[0520] In this embodiment of the present application, if the first object model is detected to be outside the target area of ​​the carrier model, it is determined that the placement of the first object model is unreasonable, and the process of randomly selecting the target area is therefore returned to. In this embodiment, if the carrier model is a third furniture model and the first object model is an object model smaller than a preset volume, that is, the first object model is a model of a small household item, then the second object model is a model of another small household item that has been placed in the third furniture model.

[0521] For example, the third furniture model is a “dining table” model, the first object model is a “dinner plate” model, and the second object model may be a “rice bowl” model, a “chopsticks” model, a “spoon” model, or the like.

[0522] In an embodiment of the present application, after configuring the first object model for the third furniture model, it is determined whether the placement of the first object model is reasonable by judging whether there is overlap between the first object model and the second object model, and whether the first object model is located in the target area, so as to avoid interference between the first object model and the second object model, which may cause errors in the constructed simulation scene.

[0523] In some embodiments, optionally, after configuring the first object model to the target area of ​​the carrier model, the method further includes: configuring the model tag of the first object model to the model tag of the carrier model.

[0524] In the embodiment of the present application, after the first object model is configured on the carrier model, the tag of the first object model is added to the model tag of the carrier model. When using the generated simulation scene, the model tag of the carrier model can be read to determine the model tag of the first object model placed in the carrier model.

[0525] It should be noted that the model label includes information such as object name, object type, object physical parameters, object volume, etc.

[0526] In some embodiments, optionally, after configuring the first object model to the target area of ​​the carrier model, it also includes: configuring physical parameters of the first object model through a random algorithm; wherein the physical parameters include at least one of the following: mass, center of mass, inertia tensor, friction coefficient, and volume.

[0527] In an embodiment of the present application, after configuring the first object model to the carrier model, corresponding physical parameters are configured for the first object model so that the constructed simulation scene not only includes various object models, but also each object model has simulated physical parameters.

[0528] In this embodiment, after the first object model is configured on the carrier model, the volume of the first object model is added, which may be a collision volume. The mass, center of mass, friction coefficient, and inertia tensor of the first object model also need to be randomly selected.

[0529] For example, the mass and friction coefficient of the first object model can be determined within a selection range based on the object type and material information of the first object model, and a target mass and friction coefficient can be selected within the selection range using a random algorithm. The center of mass and inertia tensor can be calculated based on the target mass.

[0530] In an embodiment of the present application, after the configuration and placement of the first object model is completed, the usability of the simulation scene is improved by configuring the physical parameters of the first object model.

[0531] In some embodiments, optionally, physical parameters of the first object model are configured through a random algorithm, including: when the physical parameters include mass, center of mass and inertia tensor, determining the target mass through a random algorithm within a preset range; determining the target inertia tensor based on the target mass; and when the target center of mass is determined, updating the target inertia tensor based on the target center of mass.

[0532] In an embodiment of the present application, since the inertia tensor in the physical parameters is associated with the mass and the center of mass, the corresponding target inertia tensor is determined by randomly selecting a target mass within a preset range, and after determining the target center of mass, the target inertia tensor is updated based on the target center of mass.

[0533] Exemplarily, the target inertia tensor is calculated from the target mass by the following relationship: o =I com +mD 2

[0534] Among them, I o is the target inertia tensor of the first object model, the inertia tensor matrix of the first object model in the reference coordinate system centered at the construction origin, that is, the inertia tensor matrix in the origin coordinate system is constructed, I com is the inertia tensor matrix of the first object model in the reference coordinate system with its center of mass as the center, that is, the inertia tensor matrix in the center of mass coordinate system, m is the mass of the first object model, and D is the translation distance between the construction origin coordinate system and the center of mass coordinate system.

[0535] Exemplarily, the target inertia tensor is updated based on the target center of mass through the following relationship.

[0536] in, is the updated target inertia tensor, I com is the target inertia tensor before updating, m is the mass of the first object model, D is the translation distance between the construction origin coordinate system and the center of mass coordinate system before updating, To construct the translation distance between the origin coordinate system and the updated center of mass coordinate system, I xx is the target inertia tensor in the two x-axis directions of the two coordinate systems, I xy , I xz , I yx , I yy is the target inertia tensor in the two y-axis directions of the two coordinate systems, I yz is the target inertia tensor in the y-axis and z-axis directions of the two coordinate systems, I zx is the target inertia tensor in the z-axis and x-axis directions of the two coordinate systems, I zyis the target inertia tensor in the z-axis and y-axis directions of the two coordinate systems, I zz is the target inertia tensor in the two z-axis directions of the two coordinate systems.

[0537] In an embodiment of the present application, when the mass and center of mass of the first object model change, the target inertia tensor can be adjusted accordingly through the updated mass and center of mass to avoid anomalies between the configured multiple object parameters.

[0538] In one embodiment of the present application, as shown in FIG25 , a model configuration method is proposed, including:

[0539] Step 2502: Import the 3D apartment grid and render the apartment model with random materials according to the building structure labels;

[0540] The apartment model is the house model in any of the above embodiments.

[0541] Step 2504: Determine the range of each room based on the ground grid, query the database and randomly select furniture models corresponding to the room type, and import them into the room;

[0542] The range of each room is the area information of the object placement area of ​​the house model in any of the above embodiments, and the furniture model is the first furniture model in any of the above embodiments.

[0543] Step 2506: After randomly selecting a certain type of furniture model, adjust the coordinates of the furniture in the room according to the object bounding box, anchor point, and offset of the furniture;

[0544] Among them, the selected type of furniture is the first furniture model in any of the above embodiments. After the first furniture model is configured to the house model, the position of the first furniture model in the room is adjusted according to the object bounding box and anchor point of the first furniture model, and the corresponding bias parameters. The anchor point is updated according to the construction origin of the first furniture model, and the bias parameter is used to offset the initial construction origin when updating the construction origin.

[0545] Step 2508: Read the structured random file, parse the carrier model and the type and quantity of random items to be placed;

[0546] The carrier model is the third furniture model in any of the above embodiments, and the random item to be placed is the first object model smaller than the preset volume in any of the above embodiments. The structured random file is a structured storage model file.

[0547] Step 2510: Retrieve a list of items that meet the conditions for adding items from the model database, and randomly select an item model from the list;

[0548] The model database is a pre-built model database that includes constructed object models, object models downloaded from the server, and object models obtained by 3D reconstruction. The item list is a list of object models, and the object models in the object model list are all models that can be placed on the third furniture model.

[0549] Step 2512, randomly selecting the position and orientation of the object according to the placement area of ​​the carrier model;

[0550] The carrier model is the third furniture model in any of the above embodiments, the position of the item is the preset position in any of the above embodiments, and the orientation is the preset orientation in any of the above embodiments;

[0551] Step 2514: Determine whether the item is within the carrier area. If it is not, return to step 2512; if it is, return to step 2516.

[0552] Step 2516: Determine whether the object interferes with other objects on the carrier. If so, return to step 2512; if not, return to step 2518.

[0553] Step 2518, add the item to the carrier's list of items;

[0554] The carrier carries an item list for storing model information of object models placed in the furniture model, so as to facilitate determining the item information placed in the furniture model after the simulation scene is constructed.

[0555] Step 2520, add item collision volume, random mass and random center of mass;

[0556] The object collision volume, random mass, and random center of mass are respectively the volume, mass, and center of mass of the physical parameters in any of the above embodiments.

[0557] Step 2522, calculating the inertia tensor based on the random mass and center of mass;

[0558] Step 2524: Generate a simulation scenario, assign a unique identifier, and wait for calling.

[0559] In an embodiment of the present application, in the process of building a simulation scene of a robot, object models are randomly configured in the placement area of ​​the carrier model. After the random configuration, it is detected whether the positional relationship between the first object model currently configured in the carrier model and the second object model that has been configured in the carrier model meets the preset conditions, so as to determine whether the target area where the first object model is located is reasonable. If the target area where the first object model is located is unreasonable, the target area is randomly re-selected within the placement area in a timely manner, thereby reducing the possibility of object models in unreasonable placement positions in the constructed simulation scene, and eliminating the need for manual adjustment of the constructed simulation scene, further simplifying the process of building the simulation.

[0560] In one embodiment of the present application, as shown in FIG26 , a model configuration device 2600 is proposed, including:

[0561] An acquisition module 2602 is used to obtain area information of a placement area of ​​a carrier model;

[0562] A determination module 2604 is configured to determine a target area in a placement area using a random algorithm based on the area information, where the placement area is an area in the carrier model for placing objects;

[0563] a configuration module 2606 for configuring the first object model to a target area of ​​the carrier model;

[0564] Execution module 2608 is used to return to the step of determining the target area in the placement area through a random algorithm based on the area information when the positional relationship between the first object model and the second object model meets the preset conditions, and the second object model is an object model that has been configured in the carrier model.

[0565] In the technical solution of the present application, in the process of building a simulation scene of a robot, the object models are randomly configured in the placement area of ​​the carrier model. After the random configuration, it is detected whether the positional relationship between the first object model currently configured in the carrier model and the second object model already configured in the carrier model meets the preset conditions, so as to determine whether the target area where the first object model is located is reasonable. If the target area where the first object model is located is unreasonable, the target area is randomly re-selected in the placement area in a timely manner, thereby reducing the possibility of object models in unreasonable placement positions in the constructed simulation scene, and there is no need to manually adjust the constructed simulation scene, which further simplifies the process of building the simulation.

[0566] In some embodiments, optionally, the carrier model includes a house model, and the first object model includes a first furniture model;

[0567] An acquisition module 2602 is configured to acquire an object bounding box of the first furniture model;

[0568] A determination module 2604 is configured to determine, based on the region information and the object bounding box, at least two sub-regions in the placement region, where both sub-regions can accommodate the first object model.

[0569] The determination module 2604 is configured to determine a target region in the at least two sub-regions by using a random algorithm.

[0570] In the technical solution of the present application, when the carrier model is a house model and the first object model includes a first furniture model, multiple sub-areas in the placement area are determined through the object bounding box of the first furniture model and the area information of the placement area, and then a target area is selected from the multiple sub-areas through a random algorithm to ensure that the target area can accommodate the first furniture model, while also ensuring the randomness of the placement position of the first furniture model.

[0571] In the technical solution of the present application, a corresponding object bounding box is constructed for the first furniture model, and at least two corresponding sub-areas can be screened in the placement area through the object bounding box to ensure that the screened sub-areas can stably accommodate the object bounding box. A target area is selected from at least two sub-areas through a random algorithm to ensure that the target area can be used to place the first furniture model, thereby reducing the possibility of interference between the first furniture model and the house model.

[0572] In some embodiments, optionally, the determination module 2604 is configured to determine, based on the area information, at least two first line segments in the placement area, where the first line segments match the walls in the house model;

[0573] A determination module 2604 is configured to determine a second line segment in the first furniture model based on the object bounding box, where the second line segment is related to a preset placement posture of the first furniture model;

[0574] A determining module 2604 is configured to determine a target line segment among the at least two first line segments based on the sizes of the at least two first line segments and the sizes of the second line segments, wherein the size of the target line segment is larger than the size of the second line segment;

[0575] The determination module 2604 is configured to determine at least two sub-regions based on the first coordinate information of the target line segment.

[0576] In the technical solution of the present application, the size relationship between the first line segment corresponding to the wall of the placement area and the second line segment in the first furniture model is used to accurately judge whether the sub-area where the wall corresponding to the first line segment is located can be used to place the first furniture model, thereby ensuring that multiple sub-areas determined in the placement area can accommodate the first furniture model.

[0577] In the technical solution of the present application, by comparing the sizes of the second line segment with multiple first line segments in sequence, the area where the target line segment whose size is larger than the second line segment among the multiple first line segments is located is used to determine at least two sub-areas that can accommodate the first furniture model, thereby ensuring that the at least two sub-areas obtained can both accommodate the first furniture model.

[0578] In some embodiments, optionally, the determination module 2604 is configured to determine the translation direction of the first furniture model based on the first coordinate information and the second coordinate information, where the second coordinate information is the coordinate information of the coordinate point located in the placement area;

[0579] The model configuration device 2600 further includes:

[0580] The moving module is used to move the first furniture model along the translation direction until the distance between the first furniture model and the second line segment is greater than a first preset distance.

[0581] In the technical solution of this application, after the first furniture model is placed in the target area, the translation direction for correcting the position of the first furniture model is determined using the first coordinate information of the target line segment and the second coordinate information of the coordinate point within the placement area. The position of the first furniture model can be corrected according to this translation direction, ensuring that the first furniture model remains stable within the placement area and preventing interference between the first furniture model and the wall of the carrier model.

[0582] In the technical solution of the present application, the first coordinate information and the second coordinate information can be used to determine the translation direction of the first furniture model after it is configured on the house model, and the first furniture model can be moved according to the translation direction, so that the first furniture model can be inside the house model, avoiding interference between the first furniture model and the house model.

[0583] In some embodiments, optionally, the preset conditions include at least one of the following: when the second object model includes a door and window model, the distance between the first furniture model and the door and window model is less than a second preset distance; when the second object model includes a second furniture model, there is an overlapping area between the first furniture model and the second furniture model; when the second object model includes a wall model, the distance between the first furniture model and the wall model is less than a third preset distance.

[0584] In the technical solution of the present application, when the first object model is a first furniture model and the carrier model is a house model, by judging whether the first furniture model overlaps with the second furniture model that has been configured in the house model, and judging whether the first furniture model is too close to the door and window model in the house model, and judging whether the first furniture model is too close to the wall model in the house model, it is determined whether the positional relationship between the first object model and the second object model meets the preset conditions, thereby determining whether the placement of the first furniture model is reasonable.

[0585] In the technical solution of the present application, by comparing the positional relationship between the first furniture model and the door and window model, the wall model, and the second furniture model that has been configured in the house model, it is possible to accurately judge whether the current position of the first furniture model is reasonable. If the position of the first furniture model is unreasonable, the target area can be randomly reselected in a timely manner.

[0586] In some embodiments, optionally, the carrier model includes a third furniture model, and the volume of the first object model is smaller than a preset volume;

[0587] A determination module 2604 is configured to determine a preset orientation and a preset position of the first object model using a random algorithm based on the area information, wherein the preset position is within the placement area;

[0588] The determination module 2604 is configured to determine a target area in the third furniture model according to a preset orientation and a preset position.

[0589] In the technical solution of this application, when the carrier model is a third furniture model, the first object model is a smaller model. Based on the regional information of the placement area of ​​the third furniture model, a preset orientation of the first object model and a preset position of the first object model within the third furniture model can be randomly selected. Based on the preset orientation and position, the first object model is placed within the target area.

[0590] In the technical solution of the present application, when the carrier model is a furniture model, since the placement area in the furniture model is small and the types of first object models that can be placed in the placement area are relatively fixed, the preset orientation and preset position of the first object model can be directly randomly selected, thereby further simplifying the process of configuring the first object model to the third furniture model.

[0591] In some embodiments, optionally, the determination module 2604 is configured to determine the type and placement quantity of objects corresponding to the carrier model based on the carrier information of the carrier model;

[0592] The model configuration device 2600 further includes:

[0593] A selection module, configured to screen at least two third object models in the object model library based on the model type;

[0594] A generating module, configured to generate an object model list according to at least two third object models and a placement quantity;

[0595] A selection module, configured to select a first object model from the object model list by using a random algorithm;

[0596] The determination module 2604 is used to determine the placement quantity corresponding to the first object model.

[0597] In an embodiment of the present application, the object type and placement quantity of the object models that can be placed on the carrier model can be found based on the carrier information, and the corresponding third object model is found accordingly, and an object model list is constructed. A random algorithm is used to randomly select a first object model from the object model list to be placed in the carrier model, as well as the placement quantity corresponding to the first object model, to ensure that the random first object model matches the carrier model and that the placement quantity of the first object model can be stably placed in the target area of ​​the carrier model. In some embodiments, optionally, the preset conditions include: the first object model overlaps with the second object model, and / or the first object model is not within the target area.

[0598] In the technical solution of the present application, after configuring the first object model for the third furniture model, it is determined whether the placement of the first object model is reasonable by judging whether there is overlap between the first object model and the second object model, and whether the first object model is located in the target area, so as to avoid interference between the first object model and the second object model, which may cause errors in the constructed simulation scene.

[0599] In some embodiments, optionally, the configuration module 2606 is configured to configure the model tag of the first object model into the model tag of the carrier model.

[0600] In the embodiment of the present application, after the first object model is configured on the carrier model, the tag of the first object model is added to the model tag of the carrier model. When using the generated simulation scene, the model tag of the carrier model can be read to determine the model tag of the first object model placed in the carrier model.

[0601] In some embodiments, optionally, the configuration module 2606 is used to configure physical parameters of the first object model through a random algorithm; wherein the physical parameters include at least one of the following: mass, center of mass, inertia tensor, friction coefficient, and volume.

[0602] In the technical solution of the present application, after the first object model is configured to the carrier model, the corresponding physical parameters are configured for the first object model, so that the constructed simulation scene not only includes various object models, but also each object model has simulated physical parameters.

[0603] In some embodiments, optionally, the determination module 2604 is configured to determine the target mass by a random algorithm within a preset range when the physical parameters include mass, center of mass, and inertia tensor;

[0604] A determination module 2604 is configured to determine a target inertia tensor based on the target mass;

[0605] The determination module 2604 is configured to update the target inertia tensor based on the target center of mass when the target center of mass is determined.

[0606] In an embodiment of the present application, since the inertia tensor in the physical parameters is associated with the mass and the center of mass, the corresponding target inertia tensor is determined by randomly selecting a target mass within a preset range, and after determining the target center of mass, the target inertia tensor is updated based on the target center of mass.

[0607] In the technical solution of the present application, when the mass and center of mass of the first object model change, the target inertia tensor can be adjusted accordingly through the updated mass and center of mass to avoid anomalies between the parameters of multiple configured objects.

[0608] In some embodiments, optionally, the model configuration device 2600 further includes:

[0609] An extraction module, used to extract the placement area in the carrier model;

[0610] A determination module 2604 is configured to determine a list of object models corresponding to the placement area based on the category information of the placement area;

[0611] The extraction module is used to extract the first object model from the object model list through a random algorithm.

[0612] In the technical solution of this application, based on the category information corresponding to the placement area, the object models that can be placed in the placement area can be determined and stored in the form of a list. A random algorithm is used to randomly extract the first object model from the object model list, thereby realizing the use of a random method to build a simulation scene for a home robot.

[0613] In one embodiment of the present application, as shown in Figure 27, a model configuration device 2700 is proposed, including: a processor 2702 and a memory 2704, in which a program or instruction is stored; the processor 2702 executes the program or instruction stored in the memory 2704 to implement the steps of the model configuration method in any of the above embodiments, and thus has all the beneficial technical effects of the model configuration method in any of the above embodiments, which will not be elaborated here.

[0614] In one embodiment of the present application, a readable storage medium is provided. The readable storage medium stores a program or instructions. When executed by a processor, the program or instructions implement the steps of the method described in any of the above embodiments. Therefore, all the beneficial technical effects of the method described in any of the above embodiments are achieved, and further details are omitted here.

[0615] In one embodiment of the present application, a computer program product is proposed. When the computer program product is executed by a processor, it implements the steps of the method in any of the above embodiments, and thus has all the beneficial technical effects of the method in any of the above embodiments, which will not be further elaborated here.

[0616] In one embodiment of the present application, as shown in Figure 28, an electronic device 2800 is proposed, including: a model configuration device 2700 as in any of the above embodiments, and / or a readable storage medium 2802 in any of the above embodiments, and / or a computer program product 2804 in any of the above embodiments, and thus has all the beneficial technical effects of the model configuration device 2700 in any of the above embodiments, and / or the readable storage medium 2802 in any of the above embodiments, and / or the computer program product 2804 in any of the above embodiments, and no further details will be given here.

[0617] In one embodiment of the present application, a computer device is proposed, which includes: a processor and a memory, the memory storing programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the training scenario generation method in any of the above embodiments and / or the model configuration method in any of the above embodiments are implemented, thereby having the technical effects that can be achieved by the training scenario generation method in any of the above embodiments and / or the model configuration method in any of the above embodiments. To avoid repetition, they will not be described here.

[0618] It should be clarified that in the claims, specification and drawings of this application, the term "plurality" refers to two or more. Unless otherwise clearly defined, the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the drawings. It is only for the purpose of more conveniently describing this application and making the description process simpler, and is not intended to indicate or imply that the device or element referred to must have the specific orientation described, be constructed and operated in a specific orientation. Therefore, these descriptions cannot be understood as limitations on this application. The terms "connect", "install", "fix" and the like should be understood in a broad sense. For example, "connection" can be a fixed connection between multiple objects, or a detachable connection between multiple objects, or an integral connection; it can be a direct connection between multiple objects, or an indirect connection between multiple objects through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood based on the specific circumstances of the above data.

[0619] In the claims, specification, and drawings of this application, the terms "one embodiment," "some embodiments," "a specific embodiment," and the like mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in the embodiment or example of this application. In the claims, specification, and drawings of this application, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0620] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A training scenario generation method, wherein: include: Get the object model library; According to the scene information of the scene model, a target object model in the object model library is determined by a domain random algorithm, and the target object model matches the scene information; The target object model is configured into the scene model to generate a training scene file.

2. The training scenario generation method according to claim 1, wherein: The target object model includes a first object model and a second object model, and the first object model can carry the second object model; The step of determining the target object model in the object model library by a domain random algorithm according to the scene information of the scene model comprises: Based on the scene information, extracting a first model list in the object model library; Extracting individual instances of the first object model in the first model list by a domain random algorithm; Acquiring object category information of the first object model; Based on the object category information, extracting a second model list in the object model library; By using a domain random algorithm, individual instances of the second object model in the second model list are extracted.

3. The training scenario generation method according to claim 1 or 2, wherein: The target object model includes a first object model, and configuring the target object model into the scene model to generate the training scene file includes: Obtaining an object placement area of ​​the scene model; According to the object category information, a target area in the object placement area is extracted by a rule judgment algorithm; The first object model is configured to the target area.

4. The training scenario generation method according to claim 3, wherein: The target object model includes a second object model, and after configuring the first object model to the target area, the method further includes: Acquiring accommodation area information of the first object model; Determining, according to the accommodation area information, a configuration position and a configuration posture of the second object model in the first object model by a random algorithm; The second object model is configured to the first object model according to the configuration posture and the configuration position.

5. The training scenario generation method according to claim 4, wherein: The number of the second object model is at least one; After configuring the second object model to the first object model according to the configuration posture and the configuration position, the method further includes: Based on the fact that the number of the second object models is at least two, and in the case that there is an overlapping area between at least two of the second object models, return to the step of determining the configuration position and configuration posture of the second object model in the first object model by a random algorithm based on the accommodating area information.

6. The training scenario generation method according to any one of claims 1 to 5, wherein: The obtaining of the object model library comprises: Acquire an object model and first model information of the object model; generating model metadata corresponding to the object model according to the first model information; The object model library is constructed based on the model metadata.

7. The training scenario generation method according to claim 6, wherein: The step of generating model metadata corresponding to the object model according to the first model information includes: generating a model label corresponding to the object model according to the first model information; Configuring the model tag into the object model to generate the model metadata; The model label includes at least one of the following: a name label, a category label, a subcategory label, a volume label, an anchor point label, a bias label, a size label, and a physics label.

8. The training scenario generation method according to claim 6 or 7, wherein: After obtaining the object model, the method further includes: Calculating an object bounding box corresponding to the object model; Based on the object bounding box, the object model is adjusted so that model parameters of the object model match preset parameters.

9. The training scenario generation method according to claim 8, wherein: The model parameters include a model construction origin, the preset parameters include a target position of the construction origin, and the object model is adjusted based on the object bounding box, including: Based on the coordinate information of the object bounding box, the construction origin of the object model is moved to the target position.

10. The training scenario generation method according to claim 8, wherein: The model parameter includes a scaling ratio, the preset parameter includes a preset ratio, and the adjusting the object model based on the object bounding box includes: Based on the coordinate information and preset size parameters of the object bounding box, the scaling ratio of the object model is adjusted to the preset ratio.

11. The training scenario generation method according to any one of claims 1 to 10, wherein: Before configuring the target object model into the scene model and generating the training scene file, the method further includes: Configuring light source parameters for the scene model by a random algorithm; The light source parameters include at least one of the following: light source position, light source size, light source lumen, and light source color.

12. The training scenario generation method according to any one of claims 1 to 11, wherein: After configuring the target object model into the scene model and generating a training scene file, the method further includes: Acquire the target object model configured in the scene model and second model information of the scene model; Based on the second model information, generating scene metadata of the training scene; The second model information includes at least one of the following: position information, category information, name information, size information, and posture information.

13. A training scene generation device, wherein: include: An acquisition module, used to acquire an object model library; A determination module, configured to determine a target object model in the object model library by a domain random algorithm according to the scene information of the scene model, wherein the target object model matches the scene information; The configuration module is used to configure the target object model into the scene model to generate a training scene file.

14. A training scene generation device, wherein: include: A memory on which programs or instructions are stored; A processor, configured to implement the steps of the training scenario generation method as described in any one of claims 1 to 12 when executing the program or instructions.

15. A readable storage medium having a program or instruction stored thereon, wherein: When the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

16. A computer program product, wherein: When the computer program product is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

17. An electronic device, wherein: include: The training scene generating device as claimed in claim 13 or 14; and / or The readable storage medium according to claim 15; and / or A computer program product as claimed in claim 16.

18. A computer device, wherein: include: A processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the training scenario generation method as described in any one of claims 1 to 12 are implemented.

Images