Industrial scene generation method, storage medium and equipment

Abstract

The invention discloses an industrial scene generation method, a storage medium and equipment, and the method comprises the following steps: carrying out the parameterization and distinguishing of all objects in an industrial scene, and then putting the objects into a preset diffusion model for training, and obtaining a distribution set of all objects; wherein collision avoidance constraints are added into the diffusion model, and it is ensured that all objects in the generated scene do not penetrate through the model; adding an industrial layout specification constraint in the diffusion model to ensure that the layout of each object follows a specific function partition and a safety specification; and adding agent reachable and interactable constraints into the diffusion model to evaluate whether the path between the agent and the position where the agent needs to reach is reachable and operable. The industrial scene layout generation model is obtained through training of the steps, compared with traditional manual modeling or rule-based programmed generation, the method can automatically generate large-scale scenes with diversified styles with extremely high efficiency, and rich data sources are provided for intelligent research.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and particularly relates to an industrial scene generation method, a storage medium and equipment. BACKGROUND

[0002] With the rapid development of embodied AI and robotics, the training of agents capable of physical interaction with the environment becomes crucial. Training these agents (such as industrial robots, automated guided vehicles) requires massive, diverse, and high-quality three-dimensional scene data. However, in the industrial field, obtaining such data faces great challenges: first, the collection of real industrial scenes is costly, complex, and time-consuming, making it difficult to scale; second, manually creating three-dimensional industrial scenes requires specialized domain knowledge and a large amount of manual labor, resulting in low efficiency and limited scene diversity.

[0003] Existing three-dimensional scene generation techniques mainly focus on indoor home environments, such as methods based on generative adversarial networks (GANs), variational autoencoders (VAEs), or autoregressive models (such as ATISS). Although these methods can generate visually realistic scenes, they have some inherent limitations: Lack of physical realism: generated scenes often have objects that penetrate each other, float in space, or have unreasonable layouts, which are physically impossible and cannot be directly used in simulation environments that require accurate physical simulation.

[0004] Lack of interactivity: existing methods mainly focus on the static layout of objects and do not consider the interaction requirements of agents with objects in the scene, such as the need for sufficient operating space for robots to grasp parts or the need for clear paths for automated guided vehicles to reach target points.

[0005] Domain mismatch: public datasets (such as 3D-FRONT) and related models are all oriented towards home scenes, which are completely unsuitable for industrial environments (such as factories, warehouses) in terms of object categories, layout rules, and spatial constraints, and cannot generate industrial scenes containing conveyor belts, shelves, robotic arms, safety fences, and other specific objects.

[0006] Therefore, how to efficiently and automatically generate large-scale, diverse three-dimensional industrial scene data that meets the requirements of physical realism and agent interactivity is a technical problem that needs to be solved in the current embodied AI field. SUMMARY

[0007] The present application aims to provide an industrial scene generation method, a storage medium, and equipment to solve the problems raised in the background.

[0008] To achieve the above object, the application adopts one technical solution: a kind of industrial scene generation method, storage medium and equipment, the parameters of each object existing in industrial scene are parameterized and distinguished; The parameterized objects are put into the preset diffusion model for training to obtain the distribution set X_0 of each object in the diffusion model; Collision avoidance constraints are added in the diffusion model to ensure that each object in the generated scene does not appear in the simulation environment; Industrial layout specification constraints are added in the diffusion model to ensure that the layout of each object follows specific functional zoning and safety specifications; Agent accessibility and interactable constraints are added in the diffusion model to evaluate whether the path between the positions that the agent needs to reach is accessible and operable; The industrial scene layout generation model is obtained through the training of the above steps; Multi-modal data input by the user is obtained, which is used to describe the target scene, and the target scene is the target three-dimensional scene that the user expects to obtain; The multi-modal data is input into the industrial scene layout generation model to obtain the industrial scene layout that the user wants.

[0009] Preferably, each object in the industrial scene is parameterized and represented by a multi-dimensional vector, which includes the center position information of the object, the three-dimensional size information of the object, the rotation orientation information of the object around the vertical axis, the category information of the object, and the geometric shape feature information of the object.

[0010] Preferably, in the preset diffusion model, noise needs to be added to the distribution set X0: a predefined Markov chain is used to gradually add Gaussian noise to it within T time steps to obtain the noise-added data at any time step t ; Then, the distribution set X0 is denoised: a deep neural network is trained to enable it to predict the added noise according to the current time step t, the noise-added scene data , and the optional scene layout plan F as conditions. The optimization objective of the training is to minimize the mean square error between the predicted noise and the real noise: , so as to obtain a new distribution set .

[0011] Preferably, when denoising the distribution set X0, a guide function is added for sampling, which is used to evaluate whether the industrial scene layout predicted in the current step satisfies at least one of the collision avoidance constraints, the industrial layout specification constraints, and the agent accessibility and interactable constraints.

[0012] The guided function is preferably The corrected sample mean As follows: , Where and are the mean and variance of the sampling process of the diffusion model respectively, is the guided intensity.

[0013] Preferably, The collision avoidance constraint function The function is specifically expressed as: , Wherein, , are the three-dimensional bounding boxes calculated from the positions, sizes and orientations of two different objects in .

[0014] Preferably, The industrial layout constraint function The function is specifically expressed as: , Wherein F is the input scene layout plan, which defines the allowed placement area of different categories of objects , is an indicator function, which is 1 when the position of the object is beyond the area where it should be, and 0 otherwise.

[0015] Preferably, The agent reachable and interactable constraint function The function is specifically expressed as: , Wherein, is the index set of all interactable objects in the scene, The function is used to calculate the path cost, and returns a very large value if the path is not available, represents the position of the starting action point of the agent, represents the position of the interaction point with the object.

[0016] The present scheme also provides a storage medium having executable instructions stored thereon, which instructions are executed by a processor to cause the processor to execute the method for generating an industrial scene.

[0017] The scheme also provides a device including a processor and a memory, wherein the memory stores a program; when the program stored in the memory is executed by the processor, the processor executes the generation method of the industrial scene in the scheme.

[0018] The present application has the following advantages: the present application is the first scene generation method specially for the industrial field and for the purpose of generating embodied intelligent training data. By adding collision avoidance constraints, industrial layout specification constraints and agent accessibility and interactability constraints in the special industrial dataset, and then training, the present application can generate high-quality and high-fidelity three-dimensional scenes conforming to the characteristics of industrial environments (such as object categories and layout specifications). Compared with traditional manual modeling or rule-based programmed generation, the generation method based on the diffusion model can automatically generate large-scale and diversified scenes with high efficiency, providing a rich data source for embodied intelligence research. In the present application, a three-dimensional industrial scene is modeled as a set of unordered parameterized objects containing position, size, orientation, industrial-specific categories and geometric shape features, and a denoising diffusion model is used to learn the internal complex distribution. In the generation (denoising) process of the diffusion model, a composite constraint mechanism for embodied intelligence training is introduced, especially the combination of physical collision, industrial layout and agent accessibility and interactability guidance, which realizes the leap from “looking real” to “effective in use”. By introducing a collision avoidance guidance function, the present application can significantly reduce or even eliminate the object penetration problem in the generated scene, ensuring the physical rationality of the scene and enabling it to be seamlessly imported into mainstream physics simulation engines. By introducing an industrial layout constraint function, the layout of each object is ensured to follow the specific functional zoning and safety specifications in the industrial scene. By introducing an agent accessibility and interactability constraint function, the generated scene is not only visually reasonable, but also functionally effective for agent training. The generated scene layout naturally considers the movement path and operation space of the agent, greatly improving the effectiveness of the training data. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of the generation method of the present application. DETAILED DESCRIPTION

[0020] To make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the drawings and detailed descriptions will be used to clearly explain the spirits of the present application. Any person skilled in the art can make changes and modifications to the technology taught by the present application without departing from the spirits and scopes of the present application.

[0021] The illustrative embodiments of the present application and their description serve the purpose of explanations only. In addition, the same or similar components / elements are used with the same or similar reference numerals in the drawings and embodiments, which are used to represent the same or similar parts.

[0022] As to the "first", "second", and the like used in the present application, they are not intended to particularly refer to the order or sequence, nor to limit the present application. They are merely used to distinguish the elements or operations described with the same technical terms.

[0023] As to the directional terms used in the present application, such as "up", "down", "left", "right", "front", "back", and the like, they are merely used with reference to the directions of the drawings. Therefore, the directional terms used are used to explain, not to limit the present creation.

[0024] As to the "comprise", "include", "have", "contain", and the like used in the present application, they are all open terms, that is, they mean comprising but not limited to.

[0025] As to the "and / or" used in the present application, it includes any or all combinations of the described things.

[0026] As to the "plurality" in the present application, it includes "two" and "more than two"; as to the "multiple groups" in the present application, it includes "two groups" and "more than two groups".

[0027] As to the terms "approximately", "about", and the like used in the present application, they are used to modify any quantity or error that can be slightly changed, but the slight changes or errors do not change the nature. Generally, the slight changes or errors modified by such terms can be 20% in some embodiments, 10% in some embodiments, 5% or other values in some embodiments. Those skilled in the art should understand that the aforementioned values can be adjusted according to actual needs, and are not limited thereto.

[0028] All terms used herein (including technical and scientific terms) have meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present specification, and should not be interpreted in an idealized or overly formal manner.

[0029] When expressions such as "at least one of A, B, and C" are used, they should generally be interpreted in accordance with the meaning commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). When expressions such as "at least one of A, B, or C" are used, they should generally be interpreted in accordance with the meaning commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, or C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). A person skilled in the art should also understand that any conjunction and / or phrase that substantially arbitrarily indicates two or more optional items, whether in the specification, claims, or drawings, should be understood to indicate the possibility of including one of these items, either of these items, or both items. For example, the phrase “A or B” should be understood as including the possibility of “A” or “B”, or “A and B”.

[0030] Example: See Figure 1 A method for generating industrial scenes is proposed. This method trains a diffusion model on the layout of objects in the industrial scene to automatically generate large-scale and diverse scenes with extremely high efficiency, providing a rich data source for embodied intelligence research. The specific generation method is as follows: We collect various objects contained in an industrial scene, such as machine tools (lathes, milling machines, drilling machines, grinding machines, cutting machines, etc.), shelves, conveyor belts, robotic arms, and material boxes. Assuming a 3D industrial scene S consists of N objects, we represent it as an unordered set X0 = ... o2, ..., o N Among them, each object o i Represented by a multidimensional vector parameterization: o i =[l i s i theta i c i f i ], in: It is the center position of the object in the scene coordinate system; s i These are the three-dimensional dimensions (length, width, and height) of the object. theta i It is the orientation of an object's rotation about its vertical axis, using (cos...) sin () indicates to avoid the problem of periodicity of angles; c i This is the category label of the object, using one-hot encoding, where C is the total number of predefined object categories in the industrial scene (e.g., machine tools, shelves, conveyor belts, robotic arms, material boxes, etc.). f i It is an encoding of the geometric shape features of an object. This encoding is extracted from an industrial 3D model library through a pre-trained variational autoencoder (VAE) and is able to characterize the fine geometric shape of the object.

[0031] Next, a denoising diffusion model (such as the DIffuScene model) is used to learn the complex data distribution of the above parameterized scene set. In the denoising diffusion model, the distribution set X0 needs to be denoised: Gaussian noise is gradually added to it over T time steps using a predefined Markov chain to obtain the denoised data at any time step t. , ,in It is noise sampled from a standard normal distribution. These are predefined parameters related to noise scheduling (e.g.) , ,in (This is the noise variance). When t is large enough, It approximates pure Gaussian noise; Then, the distribution set X0 is denoised by training a deep neural network. This enables it to adjust the scene data based on the current time step t and the added noise. And, using the optional scene layout plan F as a condition, predict the added noise. The optimization objective of training is to minimize the mean square error between the predicted noise and the actual noise. This results in a new distribution set. , A one-dimensional U-Net structure with a self-attention mechanism can be used.

[0032] After training, from random noise Initially, through a pre-trained network By iterating T times in reverse denoising, a completely new set of scene parameters is finally generated. To ensure that the generated scene is not only reasonable in terms of data distribution, but also meets the requirements of embodied intelligence training at the physical and interactive levels, a guided sampling mechanism is innovatively introduced. That is, in each denoising step t, the original prediction is not sampled, but a gradient term is added to it. This gradient comes from one or more guided functions. These functions are used to evaluate the "clean" scenario predicted in the current step. Does it meet specific constraints? Corrected sample mean as follows: , in, and These are the mean and variance in the sampling process of the standard diffusion model. It is the guiding strength (which is represented by a scaling factor that can be adjusted according to the generation requirements).

[0033] in, Including Including collision avoidance constraint functions Industrial layout constraint functions reachability and interactivity constraint functions of intelligent agents At least one of the three; In this scheme, the intersection-over-union (IoU) ratio of the object's 3D bounding box is used to approximate the collision degree. The collision avoidance constraint function is specifically expressed as follows: , in, yes medium object o i The position, size, and orientation of the calculated 3D bounding box; Yes, yes medium object The position, size, and orientation of the calculated 3D bounding box; Among these, the layout of objects must follow specific functional zoning and safety regulations. For example, heavy equipment cannot be placed in pedestrian walkways, and material boxes should be placed in designated areas on shelves. Industrial layout constraints are specifically manifested as follows: , This bootstrapping function penalizes objects that violate predefined industrial layout constraints, where F is the input scene layout planar graph that defines different categories of objects c. i Permitted placement area , It is an indicator function, when the object The function value is 1 when the position is outside the area where its category should be located; otherwise, it is 0. The agent reachability and interactivity constraint function is central to ensuring the scene is useful for embodied intelligence training. This function evaluates whether key interactive objects in the scene (such as consoles, switches, and workpieces to be grasped) are reachable and operable for a standard-sized agent (such as a mobile robot). Specifically, we construct a two-dimensional walkable map in the scene based on the agent's size. Then, for each object requiring interaction... We calculate the shortest path from a starting point to its interaction region (e.g., 0.5 meters in front of an object). If the path is blocked by other objects, a penalty is applied, specifically expressed as follows: , in, It is a set of indices for all interactive objects in the scene. The function is used to calculate path cost; if the path is impassable, it returns a very large value. This indicates the position of the agent's initial action point. This function indicates the location of the interaction points with the object, prompting the model to generate a layout with sufficient passageways and operational space. Finally, through sampling and shape retrieval, we transform the generated object parameters into a 3D industrial scene with a realistic geometric model, which can be directly used in simulators for training and testing embodied intelligence algorithms.

[0034] In addition, this solution also provides a storage medium on which executable instructions are stored, which, when executed by a processor, cause the processor to execute the method for generating industrial scenes in this solution.

[0035] This solution also provides a device that includes a processor and a memory, wherein the memory stores a program; when the program stored in the memory is executed by the processor, the processor executes the method for generating industrial scenes in this solution.

[0036] When using the model trained by this method, users can obtain a realistic and effective industrial scene layout that conforms to physical rules, industrial layout requirements, and the accessibility and interactivity requirements of intelligent agents by inputting multimodal data of the desired target 3D scene into the model (multimodal data refers to information expressed or perceived from multiple modalities, usually including visual, auditory, text and other forms of information).

[0037] The diffusion-based industrial layout generation method described in this solution can automatically generate large-scale and diverse scenes with extremely high efficiency compared to traditional manual modeling or rule-based procedural generation, providing a rich data source for embodied intelligence research.

[0038] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for generating an industrial scene, characterized in that: Includes the following steps: Parameterize and distinguish the various objects in the industrial scene; Each parameterized object is fed into a pre-defined diffusion model for training to obtain the distribution set X0 of each object. Add collision avoidance constraints to the diffusion model to ensure that objects in the generated scene do not clip through the simulation environment. Industrial layout constraints are incorporated into the diffusion model to ensure that the layout of each object follows specific functional zoning and safety standards. Incorporate agent reachability and interactivity constraints into the diffusion model to evaluate whether the path between the agent and its required location is reachable and operable; The industrial scene layout generation model is obtained through training using the above steps. Acquire multimodal data input by the user, the multimodal data being used to describe the target scene, the target scene being the target 3D scene that the user expects to obtain; Multimodal data is input into the industrial scene layout generation model to obtain the industrial scene layout desired by the user.

2. The method for generating an industrial scene according to claim 1, characterized in that: Each object in the industrial scenario is represented by a multidimensional vector parameterization, which includes the object's center position information, the object's three-dimensional size information, the object's rotational orientation information about the vertical axis, the object's category information, and the object's geometric shape feature information.

3. The method for generating an industrial scene according to claim 1, characterized in that: In the predefined diffusion model, the distribution set X0 needs to be noise-added: Gaussian noise is gradually added to it over T time steps using a predefined Markov chain to obtain the noisy data at any time step t. ; Then, the distribution set X0 is denoised by training a deep neural network. This enables it to adjust the scene data based on the current time step t and the added noise. And, using the optional scene layout plan F as a condition, predict the added noise. The optimization objective of training is to minimize the mean square error between the predicted noise and the actual noise. Thus, a new distribution set is obtained. .

4. The method for generating an industrial scene according to claim 3, characterized in that: During the denoising process of the distribution set X0, a guiding function is added. Sampling was then performed. This is used to evaluate whether the industrial scenario layout predicted in the current step satisfies at least one of the following constraints: collision avoidance constraint, industrial layout specification constraint, and agent reachability and interactivity constraint.

5. The method for generating an industrial scene according to claim 4, characterized in that: via guiding function Corrected sample mean as follows: ， in and These represent the mean and variance during the sampling process of the diffusion model, respectively. It refers to the guiding strength.

6. The method for generating an industrial scene according to claim 4, characterized in that: Including collision avoidance constraint functions The function is specifically manifested as follows: ， in, , They are The three-dimensional bounding boxes are calculated based on the position, size, and orientation of two different objects.

7. The method for generating an industrial scene according to claim 4, characterized in that: Including industrial layout constraint functions The function is specifically manifested as follows: ， Where F is the input scene layout plan, which defines the allowed placement areas for different types of objects. , It is an indicator function, when the object The function value is 1 when the position is outside the area where its category should be located, and 0 otherwise.

8. The method for generating an industrial scene according to claim 4, characterized in that: Including agent reachability and interactivity constraint functions The function is specifically manifested as follows: ， in, It is a set of indices for all interactive objects in the scene. The function is used to calculate path cost; if the path is impassable, it returns a very large value. This indicates the position of the agent's initial action point. Indicates the location of the point of interaction with the object.

9. A storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1-7.

10. A device, characterized in that: It includes a processor and a memory, wherein the memory stores a program; when the program stored in the memory is executed by the processor, the processor performs the method of any one of claims 1-7.

Images