Training set construction method for noise robustness enhancement and related apparatus

Abstract

The application provides a training set construction method for noise robustness enhancement and related devices, the method comprising: obtaining a target multi-view image of a target object and a target object mask image in a robot noise operation scene; determining a reference particle set according to the target multi-view image and the target object mask image; performing parameter configuration and noise injection on the reference particle set to obtain a target particle model a; screening the target particle model a to obtain a target particle model b, and inputting the target particle model b into a differentiable particle engine for noise scene simulation to obtain b groups of target noise data; screening the b groups of target noise data to obtain c groups of target noise data; and determining a target training set according to the c groups of target noise data and original noise-free data. Through controllable noise injection, physical simulation and effectiveness screening, a training set containing high-quality noise samples is constructed, and the noise robustness of the model is improved.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method and apparatus for constructing training sets for enhancing noise robustness. Background Technology

[0002] With the widespread application of robotic automation technology, vision-guided robot operation has become a core technology in industrial sorting, material handling, and precision assembly. This technology mainly relies on deep learning models to achieve target recognition, pose estimation, and action decision-making. Currently, most related models are trained using clean, interference-free datasets. While these models can achieve high accuracy in ideal environments, data collected during real robot operations is highly susceptible to interference from multiple factors such as sensor noise, environmental disturbances, and mechanical vibrations, making it difficult to guarantee data quality. Models trained on clean data have extremely poor noise robustness and are prone to problems such as recognition errors, positioning deviations, and operational failures when faced with noisy inputs.

[0003] Existing data augmentation methods often involve simply adding random noise and blindly expanding the sample size. Not only are the noise types limited and the perturbation methods crude, which are seriously inconsistent with the noise distribution patterns in real-world scenarios, but they also introduce a large amount of low-quality and invalid data, significantly reducing the training efficiency and inference accuracy of the model.

[0004] Therefore, how to construct a training set containing high-quality noise samples to improve the noise robustness of deep learning models in real interference environments is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a training set construction method and related apparatus for enhancing noise robustness. By performing directional noise perturbation and physical compliance verification on the particle properties of the target object, and then using a differentiable particle engine to complete the noise scene simulation and task effectiveness screening, the effective noise samples are finally fused with the original noise-free data to construct a training set containing high-quality noise samples, thereby improving the noise robustness of the deep learning model in real interference environments.

[0006] In a first aspect, embodiments of this application provide a method for constructing a training set for noise robustness enhancement, the method comprising: Acquire the target multi-view image and target object mask image of the target object in the robot noise-adding operation scene; A reference particle set is determined based on the target multi-view image and the target object mask image; the reference particle set is used to characterize the geometry of the target object. The reference particle set is configured with parameters and noise is injected to obtain a target particle models; where a is an integer greater than 1. The noise rationality screening of the a target particle models is performed to obtain b target particle models; b is a positive integer less than or equal to a. The b target particle models are respectively input into a preset differentiable particle engine to simulate a noise scene, resulting in b sets of target noise data; The target noise data in group b is filtered for task effectiveness to obtain target noise data in group c; c is a positive integer less than or equal to b. The target training set is determined based on the c sets of target noise data and the preset original noise-free data.

[0007] Secondly, embodiments of this application provide a training set construction apparatus for enhancing noise robustness. The apparatus includes an acquisition module, a first determination module, a processing module, a first filtering module, a noise simulation module, a second filtering module, and a second determination module, wherein: The acquisition module is used to acquire the target multi-view image and the target object mask image of the target object in the robot noise-adding operation scene; The first determining module is used to determine a reference particle set based on the target multi-view image and the target object mask image; the reference particle set is used to characterize the geometric shape of the target object; The processing module is used to configure parameters and inject noise into the reference particle set to obtain a target particle models; where a is an integer greater than 1. The first screening module is used to perform noise rationality screening on the a target particle models to obtain b target particle models; b is a positive integer less than or equal to a. The noise simulation module is used to input the b target particle models into a preset differentiable particle engine to simulate noise scenes and obtain b sets of target noise data. The second filtering module is used to filter the b groups of target noise data for task effectiveness to obtain c groups of target noise data; c is a positive integer less than or equal to b; The second determining module is used to determine the target training set based on the c groups of target noise data and the preset original noise-free data.

[0008] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing steps in any method of the first aspect of this application.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in any method of the first aspect of this application.

[0010] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any method of the first aspect of this application. The computer program product may be a software installation package.

[0011] By implementing the embodiments of this application, the particle properties of the target object can be subjected to directional noise perturbation and physical compliance verification. Then, a differentiable particle engine is used to complete the noise scene simulation and task effectiveness screening. Finally, the effective noise samples are fused with the original noise-free data to construct a training set containing high-quality noise samples, thereby improving the noise robustness of the deep learning model in real interference environments. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a system architecture diagram of a training set construction system provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a differentiable simulation layer provided in an embodiment of this application; Figure 3 This is an application scenario diagram of a training set construction system provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 5 This is a flowchart illustrating a training set construction method for enhancing noise robustness provided in an embodiment of this application. Figure 6 This is a schematic diagram of a process for determining a reference particle set provided in an embodiment of this application; Figure 7 This is a schematic diagram of a task validity screening process provided in an embodiment of this application; Figure 8This is a block diagram of the functional modules of a training set construction device for enhancing noise robustness, provided in an embodiment of this application. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0015] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0016] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0017] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0018] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0019] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] With the widespread application of robotic automation technology, vision-guided robot operation has become a core technology in industrial sorting, material handling, and precision assembly. This technology mainly relies on deep learning models to achieve target recognition, pose estimation, and action decision-making. Currently, most related models are trained using clean, interference-free datasets. While these models can achieve high accuracy in ideal environments, data collected during real robot operations is highly susceptible to interference from multiple factors such as sensor noise, environmental disturbances, and mechanical vibrations, making it difficult to guarantee data quality. Models trained on clean data have extremely poor noise robustness and are prone to problems such as recognition errors, positioning deviations, and operational failures when faced with noisy inputs.

[0021] Existing data augmentation methods often involve simply adding random noise and blindly expanding the sample size. Not only are the noise types limited and the perturbation methods crude, which are seriously inconsistent with the noise distribution patterns in real-world scenarios, but they also introduce a large amount of low-quality and invalid data, significantly reducing the training efficiency and inference accuracy of the model.

[0022] Therefore, how to construct a training set containing high-quality noise samples to improve the noise robustness of deep learning models in real interference environments is an urgent problem to be solved.

[0023] To address the aforementioned issues, this application provides a training set construction method and related apparatus for enhancing noise robustness. First, a multi-view image and a mask image of the target object in a noisy robot operation scenario are acquired. A reference particle set is determined based on the multi-view image and the mask image, representing the geometric shape of the target object. Then, parameters are configured and noise is injected into the reference particle set to obtain *a* target particle models, where *a* is an integer greater than 1. Next, the *a* target particle models are subjected to noise rationality screening to obtain *b* target particle models, where *b* is a positive integer less than or equal to *a*. Then, the *b* target particle models are input into a preset differentiable particle engine for noise scene simulation to obtain *b* sets of target noise data. The *b* sets of target noise data are subjected to task effectiveness screening to obtain *c* sets of target noise data, where *c* is a positive integer less than or equal to *b*. Finally, a target training set is determined based on the *c* sets of target noise data and preset original noise-free data.

[0024] As can be seen, by performing directional noise perturbation and physical compliance verification on the particle properties of the target object, and then using a differentiable particle engine to complete the noise scene simulation and task effectiveness screening, the effective noise samples are finally fused with the original noise-free data to construct a training set containing high-quality noise samples, thereby improving the noise robustness of the deep learning model in real interference environments.

[0025] For easier understanding, please refer to Figure 1 , Figure 1 This is a system architecture diagram of a training set construction system provided in an embodiment of this application. The training set construction system includes a data acquisition layer, a particle modeling layer, a differentiable simulation layer, and a training set integration layer. Each layer progresses from top to bottom and interacts bidirectionally with data to jointly complete the entire process of constructing a noise-robust training set. The specific layer functions are as follows: The data acquisition layer is responsible for synchronously acquiring multi-source heterogeneous data of the robot operation scene, including RGBD images, RGB auxiliary images, robot joint motion data, end effector force sensor data, etc. It also integrates noise recognition function to classify and label sensor noise, environmental noise and mechanical vibration noise in the scene, and output standardized raw dataset with noise labels.

[0026] Among them, the particle modeling layer can generate a set of particles that accurately fit the geometry of the target object based on the image data output by the data acquisition layer through multi-view point cloud reconstruction, improved K-nearest neighbor Chamfer loss optimization, etc.; and configure physical attributes such as three-dimensional position, three-axis velocity, and equivalent mass for the particles to construct a complete particle system state; then perform targeted noise injection according to noise labels, and complete particle attribute calibration by combining physical rule verification to output a target particle model that conforms to the real noise scene.

[0027] Among them, the differentiable simulation layer has a built-in dynamic simulation engine based on MLS-MPM, which integrates a two-way mapping process between particles and meshes, loads operating parameters such as simulation time step, gravitational acceleration, and environmental boundary conditions, and performs robot operation interaction simulation in a noisy scene on the target particle model; through physical rule constraints and interaction effectiveness verification, target noise data that conforms to physical laws and is valuable for robust training of the model are selected.

[0028] The training set integration layer can merge the simulation data output by the differentiable simulation layer with the original noise-free data, and convert the data into a tensor format compatible with deep learning frameworks such as TensorFlow and PyTorch through a data format standardization module. Finally, data quality screening is performed to remove invalid data with physical inconsistencies and motion anomalies, generating a high-quality and robust robot operation training set.

[0029] It is evident that by using particle modeling, targeted noise injection, and differentiable physical simulation, high-quality training data covering multiple real-world noise scenarios can be efficiently generated, significantly improving the robustness, generalization ability, and task success rate of the robot's operation model in complex interference environments.

[0030] For easier understanding, please refer to Figure 2 , Figure 2 This is a schematic diagram of a differentiable simulation layer provided in an embodiment of this application. The differentiable simulation layer includes, but is not limited to, a differentiable particle engine, a physical rule constraint module, and a task validity verification module, which are not specifically limited here. The differentiable particle engine uses the moving least squares matter point method to construct a high-precision dynamic simulation environment, supporting the deduction of physical behaviors such as motion, collision, and deformation of noisy particle models. Through a bidirectional mapping process of Particle to Grid (P2G), Grid Operation (GO), and Grid to Particle (G2P), it achieves temporal iterative updates of particle states, accurately reproducing the interaction process between objects and robots in noisy scenarios. The physical rule constraint module has a built-in complete physical rule system, covering core rules such as gravitational acceleration, inter-particle friction, collision detection and response, and environmental boundary constraints. During the simulation, it verifies the particle motion state in real time, avoiding non-physical phenomena such as particle penetration and hyper-distance motion, ensuring the realism and reliability of the noisy scenario simulation. The task validity verification module can utilize the differentiable characteristics of the differentiable particle engine to quantitatively analyze the impact of noise on robot operation tasks: evaluate core indicators such as robot grasping success rate and trajectory tracking accuracy in noisy scenarios, and determine whether noise causes the operation target to be unachievable; screen out noise enhancement scenarios that are valuable for improving the robustness of the model, and eliminate meaningless extreme noise scenarios.

[0031] It is evident that by accurately simulating the interaction between the robot and the target object in a real-world noise environment, the physical compliance of the simulation data is ensured, and the noise enhancement scenarios most valuable for model training are effectively selected, significantly improving the quality of the training set.

[0032] For easier understanding, please refer to Figure 3 , Figure 3This is an application scenario diagram of a training set construction system provided in an embodiment of this application. In this system, the robot inputs the collected raw dataset (including RGBD images, robot motion data, force sensor data, and other multi-source operational data) into the training set construction system. The training set construction system performs noise enhancement and quality screening on the raw data through processes such as particle modeling, targeted noise injection, differentiable physical simulation, and data integration. Finally, it outputs a high-quality target training set for training a noise-robust robot control model, thereby improving the robot's operational stability and task success rate in real and complex interference scenarios.

[0033] The following is combined with Figure 4 The electronic devices in the embodiments of this application will be described. Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 4 As shown, the electronic device includes one or more processors, a memory, a communication interface, and one or more programs. The processor is connected to the memory and the communication interface via an internal communication bus.

[0034] The processor can be used for: Acquire the target multi-view image and target object mask image of the target object in the robot noise-adding operation scene; A reference particle set is determined based on the target multi-view image and the target object mask image; the reference particle set is used to characterize the geometry of the target object. The reference particle set is configured with parameters and noise is injected to obtain a target particle models; where a is an integer greater than 1. The noise rationality screening of the a target particle models is performed to obtain b target particle models; b is a positive integer less than or equal to a. The b target particle models are respectively input into a preset differentiable particle engine to simulate a noise scene, resulting in b sets of target noise data; The target noise data in group b is filtered for task effectiveness to obtain target noise data in group c; c is a positive integer less than or equal to b. The target training set is determined based on the c sets of target noise data and the preset original noise-free data.

[0035] The one or more programs are stored in the aforementioned memory and configured to be executed by the aforementioned processor, and the one or more programs include instructions for performing any step in the above method embodiments.

[0036] The processor can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, cells, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication unit can be a communication interface, transceiver, transceiver circuit, etc., and the storage unit can be a memory.

[0037] The memory can be volatile or non-volatile, or a combination of both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0038] It is understood that the electronic device may include more or fewer structural elements than those shown in the block diagram above, such as a power module, physical buttons, a Wi-Fi module, a speaker, a Bluetooth module, sensors, a display module, etc., without limitation. It is understood that the electronic device may incorporate elements such as... Figure 1 The system architecture described above.

[0039] After understanding the software and hardware architecture of this application, the following will be combined with... Figure 5 This application describes a method for constructing a training set to enhance noise robustness in its embodiments. Figure 5 This is a flowchart illustrating a training set construction method for enhancing noise robustness according to an embodiment of this application, specifically including the following steps: Step S501: Obtain the target multi-view image and target object mask image of the target object in the robot noise-adding operation scene.

[0040] The specific steps for obtaining the target multi-view image and target object mask image of the target object in the robot noise-adding operation scene include: A1. Retrieve the multimodal sensors and noise simulation equipment corresponding to the robot's noiseless operation scenario; A2. Using the noise simulation device, a reference noise is generated and applied to the robot's noiseless operation scenario to obtain the robot's noisy operation scenario; A3. Acquire original multi-view images of the target object in the robot's noisy operation scenario using the multimodal sensor; A4. Perform foreground segmentation processing on the original multi-view image to obtain the original object mask image; A5. Perform distortion correction and multi-view registration on the original multi-view image and the original object mask image respectively to obtain the target multi-view image and the target object mask image.

[0041] In a specific embodiment, firstly, the calibrated multimodal sensors for a noise-free robot operation scenario are retrieved. These multimodal sensors include, but are not limited to, a main RGBD camera, an auxiliary RGB camera, and a force / torque sensor. The noise-free robot operation scenario refers to a scenario where the robot performs its tasks in an ideal, standard environment free from external interference, sensor errors, and environmental fluctuations. Then, noise simulation equipment, including a controllable light source, a mechanical vibration platform, and signal interference devices, is retrieved simultaneously to generate reproducible standard noise. At the same time, timing synchronization between the multimodal sensors and the noise simulation equipment is achieved to ensure strict alignment between noise application and image acquisition.

[0042] Then, reference noise is generated according to preset parameters using a noise simulation device, including noise from changes in illumination, mechanical vibration, and sensor depth. The reference noise is then stably applied to the robot's noiseless operation scenario, keeping the poses of objects in the scenario and the robot's movements consistent with the noiseless scenario, thus obtaining a noisy robot operation scenario. This scenario has reproducible, quantifiable, and real physical distribution noise characteristics.

[0043] Next, original multi-view images of the target object in the noisy robot operation scenario are acquired using multimodal sensors. These original multi-view images contain the complete appearance and spatial information of the target object under noise interference. The target object is the work object that is subjected to actions such as grasping, pushing, placing, tidying, and tilting in the robot operation task.

[0044] Then, a preset foreground segmentation algorithm is used to process the original multi-view image, identifying and preserving the target object region while setting the background to invalid or zero values, thereby obtaining the original object mask image to accurately mark the effective area of ​​the object. Next, preset camera calibration parameters are used to correct the original multi-view image and the original object mask image to eliminate camera imaging errors such as radial and tangential distortion, ensuring the geometric authenticity of the image. Then, feature matching and spatial transformation are performed on the corrected image to strictly align the position, scale, and pose of the object from different perspectives, thereby obtaining the target multi-view image and the target object mask image.

[0045] It is evident that the noise simulation equipment can controllably generate realistic noise scenes. Combined with multi-view image acquisition, foreground segmentation, and registration correction, it provides accurate, distortion-free, and high-quality image input with realistic noise labels for subsequent particle modeling, ensuring the scene realism and modeling accuracy of the training set data.

[0046] Step S502: Determine a reference particle set based on the target multi-view image and the target object mask image.

[0047] For easier understanding, please refer to Figure 6 , Figure 6 This is a flowchart illustrating a process for determining a reference particle set according to an embodiment of this application. The reference particle set is used to characterize the geometry of the target object. The specific steps for determining the reference particle set based on the target multi-view image and the target object mask image include: B1. Determine the particle generation region corresponding to the target object mask image; B2. Based on a preset particle generation function, using the target multi-view image as data input, generate particles within the particle generation area to obtain an initial particle set; B3. Optimize the spatial distribution of each particle in the initial particle set according to the preset loss function to obtain the reference particle set.

[0048] In a specific embodiment, firstly, based on the target object mask image, the effective foreground region belonging to the target object in the target mask image is determined as the particle generation region, and the background region is excluded to ensure that the particles are generated only within the object outline and are consistent with the object's geometry.

[0049] Then, based on the preset particle generation function, the target multi-view image is used as data input, and spatial sampling is performed within the particle generation region to extract the initial particle set. =(x, y, z). Where x, y, and z are the three-dimensional spatial coordinates of the particle, which preliminarily characterize the surface geometry of the target object.

[0050] Next, the spatial distribution of each particle in the initial particle set is optimized according to a preset loss function to obtain a reference particle set. The loss function can be an improved K-nearest neighbor Chamfer loss function, as shown below:

[0051] in, This represents the improved K-nearest neighbor Chamfer loss value, used to quantify the spatial difference between the particle reprojection point set and the mask sampling point set; This represents the 2D point set of particle reprojection, that is, the set of two-dimensional coordinate points of the particle reprojected onto the target multi-view image plane; This represents a particle reprojection point, i.e., a single two-dimensional coordinate point in the particle reprojection 2D point set; This represents the set of mask sampling points, which is the set of two-dimensional coordinate points uniformly sampled from the mask image of the target object. This represents a mask sampling point, that is, a single two-dimensional coordinate point in the set of mask sampling points; The number of nearest neighbors (K can be 80); For the K nearest neighbors in the corresponding point set, when the outer summation traversal... In hour, Refers to the set of sampling points from the mask Selected from the current point The K nearest points are found when the outer summation traversal is performed. In hour, Refers to the set of reprojected points from particles Selected from the current point The K nearest points; The square of the two-dimensional Euclidean distance is used to calculate the reprojection point of a single particle. With a single mask sampling point The spatial distance between them.

[0052] As can be seen, by accurately defining the particle generation region, generating an initial particle set based on multi-view images, and optimizing the particle spatial distribution through a loss function, a reference particle set that accurately fits the geometry of the target object is efficiently generated, laying a high-precision particle modeling foundation for subsequent noise injection and dynamic simulation.

[0053] Step S503: Configure parameters and inject noise into the reference particle set to obtain a target particle models.

[0054] Where a is an integer greater than 1, the steps of configuring parameters and injecting noise into the reference particle set to obtain a target particle models include: C1. Configure the parameters of each particle in the reference particle set to obtain the target particle set; the parameters of each particle in the target particle set include: three-dimensional spatial position coordinates, three-axis motion velocity, and particle mass; C2. Obtain the constraint relationships between all particles in the target particle set to obtain a constraint relationship set; C3. Determine the initial particle state set based on the target particle set, the constraint relationship set, and the preset system friction coefficient; C4. Perform noise injection a times on the initial particle state set and perform physical rationality screening to obtain a target particle state sets. C5. Determine the model of the a target particles based on the preset particle dynamics equations and the set of states of the a target particles.

[0055] In a specific embodiment, firstly, parameters are configured for each particle in the reference particle set (i.e., physical properties are assigned to each particle), where a particle can be defined as... , This represents the three-dimensional spatial coordinates of the particle, that is, the initial spatial position of the particle in the preset coordinate system; This represents the three-axis velocity of a particle, that is, the velocity of the particle in... The initial state of the three-directional motion speed can be configured according to the characteristics of the robot's operation task (e.g., the initial speed of a static grasping task is set to 0, and the initial speed of a dynamic handling task is set to the object's motion speed). The particle mass represents the total mass of the particles. It can be evenly distributed according to the total mass of the target object and the total number of particles to ensure that the total particle mass is consistent with the actual mass of the object.

[0056] Then, obtain the constraint relationships between all particles in the target particle set to obtain the constraint relationship set. This process involves using a neighborhood search algorithm (such as the K-nearest neighbor algorithm) to determine the neighboring particles of each particle based on the spatial distribution of the target particle set, thus establishing constraint relationships between them. The constraint type can be configured according to the physical characteristics of the target object. For example, if the target object is a rigid object (such as a metal workpiece), fixed connection constraints are configured to limit the relative positions and angles between particles, ensuring that the object's shape does not deform. If the target object is a deformable object (such as fabric or rope), spring-damping constraints are configured to simulate the object's tensile and bending characteristics using elastic and damping coefficients. Finally, an initial particle state set is determined based on the target particle set, the constraint relationship set, and a preset system friction coefficient. This initial particle state set can be defined as... , Represents the set of target particles; Represents a set of constraint relationships; This represents the system friction coefficient, which includes the inter-particle friction coefficient and the particle-environment contact friction coefficient.

[0057] Next, for the initial set of particle states, according to the preset noise distribution model (such as Gaussian distribution or uniform distribution), the position, velocity and other parameters of the particles are injected with noise a times independently, and the physical rationality of the particle state after each noise injection is screened to obtain a set of target particle states.

[0058] Then, the particle motion state is deduced using preset particle dynamics equations, that is, the trajectory and deformation state of the particles under the action of forces are calculated. This allows for independent dynamic simulation of each set of target particle states, generating *a* target particle models. Each target particle model can fully characterize the physical motion characteristics of the target object in a corresponding noise scene. The particle dynamics equations can be... , express The particle state at any given time (including position, velocity, and mass); express The particle state at any given moment; express The robot's actions at any given moment (such as gripper position and thrust); This represents a set of physical parameters (including object rigidity, coefficient of friction, viscosity, etc.) to ensure that the motion of particles after noise injection conforms to physical laws.

[0059] It is evident that by configuring complete physical properties for reference particles, establishing inter-particle constraint relationships, performing targeted noise injection and physical rationality screening, and combining particle dynamics equations to generate target particle models that conform to real physical laws, a high-precision and high-fidelity particle modeling foundation is provided for subsequent differentiable simulation and training set construction.

[0060] The specific steps of injecting noise a times into the initial particle state set and performing physical rationality screening to obtain a target particle state sets include: D1. Perform noise injection a times on the initial particle state set to obtain a reference particle state sets; the noise type corresponding to each noise injection in the a noise injections includes at least one of the following: sensor noise, environmental noise, and mechanical noise. D2. Obtain the target physics rules corresponding to the differentiable particle engine; D3. Filter all particles that conform to the target physical rules in each of the a reference particle state sets to obtain the a target particle state sets.

[0061] In a specific embodiment, firstly, for each noise injection, at least one of the three preset types of noise (sensor noise, environmental noise, and mechanical noise) is selected for combined injection to ensure the diversity of the a sets of reference particle states: single noise injection: such as injecting only sensor noise to simulate camera measurement error scenarios; composite noise injection: such as injecting mechanical noise and environmental noise simultaneously to simulate complex scenarios of robot joint vibration superimposed with environmental parameter fluctuations.

[0062] If sensor noise is selected: the three-dimensional spatial coordinates of the particle Inject Gaussian noise The three-dimensional spatial coordinates after injecting Gaussian noise are: , .in, (same principle) , ); A random variable representing Gaussian noise; ( ) indicates that the mean is 0 and the variance is . Gaussian distribution; Noise intensity (values ​​range from 0.01 to 0.05).

[0063] If environmental noise is selected: random perturbations are injected into particle physics parameters, such as particle mass. System friction coefficient .in, The random disturbance coefficient (with a value of 0-0.2) controls the fluctuation range of physical parameters.

[0064] If mechanical noise is selected: the triaxial velocity of the particle Adding random fluctuations, the triaxial motion velocity after noise injection is: .in, (same principle) , ); The values ​​are uniformly distributed random numbers, ranging from -0.02 to 0.02.

[0065] Then, the target physics rules corresponding to the differentiable particle engine are obtained. These target physics rules include general physics law constraints, particle system-specific constraints, and robot operation scenario constraints. Among them, general physics law constraints include the law of conservation of mass and Newton's laws of motion; particle system-specific constraints include the validity constraints of inter-particle constraint relationships and the constraints of particle-environment contact friction; and robot operation scenario constraints include the particle velocity not exceeding the robot's maximum motion speed and the particle position not exceeding the geometric contour range of the target object.

[0066] Next, each particle in each reference particle state set is individually verified, with verification dimensions including: parameter validity verification, position rationality verification, and velocity rationality verification. Among these, parameter validity verification includes checking the particle mass. >0, system friction coefficient >0, and the total mass of the particles deviates from the actual total mass of the target object by no more than 5%; Position rationality verification: the three-dimensional spatial position coordinates of the particles. , The particle did not exceed the geometric contour of the target object; velocity rationality verification: particle's three-axis motion velocity. It does not exceed the preset physical limits (e.g., the speed of a rigid object does not exceed the maximum speed of the robot's joints).

[0067] Then, all abnormal particles that do not conform to the target physical rules in the set of a reference particle states are screened out and removed. The set of reference particle states after removal is then modified (e.g., the particle mass is adjusted proportionally) to satisfy the constraint conditions, thus obtaining a set of target particle states.

[0068] As can be seen, by injecting multiple types of differentiated noise into the initial particle state and completing the effectiveness screening based on the physical rules of the differentiable particle engine, the target particle state set covering the real noise scene is efficiently generated while ensuring the physical compliance of the particle state. This provides high-fidelity and diversified noise enhancement data for subsequent simulation and training set construction.

[0069] Step S504: Perform noise rationality screening on the a target particle models to obtain b target particle models.

[0070] Where b is a positive integer less than or equal to a. The pre-defined Chamfer distance formula can be used to measure the shape similarity between each of the a target particle models and the target object. This involves extracting the corresponding particle point sets from each of the a target particle models, with each target particle model corresponding to a set of noisy particle points P. The original noise-free particle point set Q is then extracted from the original noise-free data of the target object and used as a benchmark for measuring shape similarity. The Chamfer distance formula is shown below:

[0071] in, The Chamfer distance value represents the core quantitative indicator for measuring the shape similarity between the noisy particle point set P and the original noise-free particle point set Q. This represents the total number of particles in the noisy particle set P; This represents the total number of particles in the original noise-free particle point set Q; This represents a single particle in a noisy particle point set P; This represents a single particle in the original noise-free particle point set Q.

[0072] Step S505: Input the b target particle models into a preset differentiable particle engine to simulate the noise scene and obtain b sets of target noise data.

[0073] The specific steps of inputting the b target particle models into a preset differentiable particle engine to simulate a noise scene and obtain b sets of target noise data include: E1. Obtain the target operating parameters corresponding to the differentiable particle engine; the target operating parameters include: simulation time step, gravitational acceleration, environmental boundary conditions, and the target action sequence of the robot in the robot noise-adding operation scenario; E2. Load the target running parameters into the differentiable particle engine, and use the differentiable particle engine to simulate noise scenarios for the b target particle models respectively, to obtain the b sets of target noise data.

[0074] In a specific embodiment, firstly, the target operating parameters corresponding to the differentiable particle engine are obtained. These target operating parameters include: simulation time step, gravitational acceleration, environmental boundary conditions, and the target action sequence of the robot in the robot noise-adding operation scenario.

[0075] Next, the target operating parameters are loaded into the differentiable particle engine for initialization. Then, b target particle models are input into the differentiable particle engine one by one. This differentiable particle engine, based on the moving least squares matter point method, updates the particle state through three iterative steps: particle-to-mesh (P2G), mesh operation (GO), and mesh-to-particle (G2P) to simulate the noisy scene. Throughout the simulation, particle state data for each target particle model is collected in real-time at a preset sampling frequency. This particle state data includes at least the particle's three-dimensional spatial coordinates, three-axis motion velocity, deformation gradient, and mesh node momentum, resulting in b sets of particle state data. The time-series processing of each particle state data in the b sets of particle state data yields b sets of target noise data.

[0076] First, Particle-to-Mesh (P2G): Updating deformation gradients and mesh momentum can transfer particle deformation information, mass, and velocity to background mesh nodes, realizing the mapping of particle properties to the mesh.

[0077] The deformation gradient update formula is shown below:

[0078] in, Represents particles exist The deformation gradient at time step; Represents the identity matrix; Indicates the simulation time step; Represents particles exist The velocity gradient at any given time; Represents particles exist The deformation gradient at time t.

[0079] The grid momentum update formula is shown below:

[0080] in, Represents grid nodes exist Momentum at any given moment; Represents particles For grid nodes Interpolation weights; Represents particles The quality; Represents particles exist The speed of time; Represents particles exist Affine velocity at any given moment; Represents grid nodes The coordinates; Represents particles exist The location at any given moment.

[0081] Then, mesh operations (GO): normalize the mesh velocity and handle boundary conditions. The velocity can be calculated based on the momentum of the mesh nodes, and physical boundary constraints can be applied to ensure that the simulation results conform to the real working conditions.

[0082] The formula for normalizing the grid velocity is shown below:

[0083] in, Represents grid nodes exist Normalized velocity at any given moment; Represents grid nodes exist The cumulative quality of time.

[0084] The boundary condition handling formula is shown below:

[0085] in, Represents grid nodes exist The final velocity at any given moment; This represents the preset boundary condition handling function.

[0086] Next, mesh-to-particle (G2P): updating particle velocity and position, which can map the processed mesh node velocity back to the particle, completing the iterative update of the particle motion state.

[0087] The particle velocity update formula is shown below:

[0088] in, Represents particles exist The speed of time; Represents grid nodes exist The final velocity at any given moment.

[0089] The particle position update formula is as follows:

[0090] in, Represents particles exist The location at any given moment.

[0091] It is evident that by configuring simulation parameters that fit the real-world scenario, and utilizing a differentiable particle engine to perform high-precision noise scene dynamics simulation on the target particle model, target noise data that conforms to physical laws and covers multiple types of noise scenarios can be efficiently generated, providing high-fidelity simulation data support for the subsequent construction of the training set.

[0092] Step S506: Perform task validity screening on the target noise data in group b to obtain target noise data in group c.

[0093] For easier understanding, please refer to Figure 7 , Figure 7 This is a flowchart illustrating a task validity screening process provided in an embodiment of this application, where c is a positive integer less than or equal to b. The specific steps for screening the b groups of target noise data to obtain c groups of target noise data include: F1. Obtain the preset reward function and the reward threshold corresponding to the reward function; F2. Based on the reward function, calculate b reward values ​​for each group of target noise data and the target action sequence in the b groups of target noise data; each target noise data corresponds to one reward value; F3. Based on the b reward values, select target noise data from the b groups of target noise data that satisfy the reward value being greater than or equal to the reward threshold to obtain the c groups of target noise data.

[0094] In a specific embodiment, firstly, a preset reward function and its corresponding reward threshold are obtained. This reward function is used to quantify and evaluate the completion quality of robot operation tasks in noisy scenarios. Then, based on the reward function, b reward values ​​are calculated for each set of target noise data and target action sequence in b sets of target noise data. Each set of target noise data corresponds to one reward value; a higher reward value indicates a higher success rate for the corresponding operation task in noisy scenarios. The reward function is as follows:

[0095] in, This represents the ground truth reward value, used to measure task achievement. Its value ranges from (0, 1). The closer the ground truth reward value is to 1, the better the particle state in the noisy scene matches the task objective. When the task is completed, the corresponding noisy scene data is considered valid; otherwise, it is discarded. This represents the real-time intermediate state of a particle in a noisy environment when the robot is executing the target action sequence. This represents the current target action sequence of the robot, which are the specific operation command parameters executed by the robot, such as the opening and closing degree of the gripper, translation speed, and rotation angle of the grasping action, or the magnitude and direction of the pushing force of the pushing action. This represents the distance weighting coefficient (values ​​range from 1 to 5). Represents the distance metric function; The particle state in a noisy scene is the final set of particle states after noise is injected by a differentiable particle engine. The target state represents a predefined set of particle baseline states when the task is successful. For example, the set of positions of object particles relative to the gripper when the gripping is successful, the set of docking postures of part particles when they are assembled, and the set of target area distributions of material particles when sorting is completed.

[0096] It is evident that by using a pre-defined reward function to quantitatively filter noisy scene data, invalid noise data that does not contribute to the task is accurately removed, while high-value, effective noise enhancement data is retained, significantly improving the quality of subsequent training sets and the training efficiency and robustness of the robot control model.

[0097] Step S507: Determine the target training set based on the c group of target noise data and the preset original noise-free data.

[0098] Specifically, the pre-defined raw noise-free data can be standardized to unify its data structure, field types, and storage format, making it completely consistent with the format of the target noise data in group c, thus generating a standardized noise-free sample set. Then, the standardized noise-free sample set and the target noise data in group c are integrated, reusing the annotation logic of the raw noise-free data. The original annotations (such as "successfully captured," "assembled in place") are synchronously mapped to the target noise data of the corresponding operation scenario, forming a consistent integrated dataset. Finally, the integrated dataset is converted to a format compatible with deep learning frameworks, and the training and validation sets are randomly divided in an 8:2 ratio to ensure that the divided data is free of redundancy and missing data. The integrity and consistency of the divided training and validation subsets are then checked, and outlier data is removed, ultimately outputting the target training set for deep learning model training.

[0099] It is evident that generating high-quality target training sets through format standardization and quality verification can effectively improve the training efficiency, convergence speed, and noise robustness of deep learning models.

[0100] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0102] When dividing each function into modules according to its corresponding function. Figure 8 This is a functional block diagram of a training set construction device for noise robustness enhancement provided in an embodiment of this application. The training set construction device 800 for noise robustness enhancement includes an acquisition module 810, a first determination module 820, a processing module 830, a first screening module 840, a noise simulation module 850, a second screening module 860, and a second determination module 870, wherein: The acquisition module 810 is used to acquire the target multi-view image and the target object mask image of the target object in the robot noise-adding operation scene. The first determining module 820 is used to determine a reference particle set based on the target multi-view image and the target object mask image; the reference particle set is used to characterize the geometric shape of the target object; The processing module 830 is used to configure parameters and inject noise into the reference particle set to obtain a target particle models; where a is an integer greater than 1. The first screening module 840 is used to perform noise rationality screening on the a target particle models to obtain b target particle models; b is a positive integer less than or equal to a. The noise simulation module 850 is used to input the b target particle models into a preset differentiable particle engine to simulate the noise scene and obtain b sets of target noise data. The second filtering module 860 is used to filter the b groups of target noise data for task effectiveness to obtain c groups of target noise data; c is a positive integer less than or equal to b; The second determining module 870 is used to determine the target training set based on the c group of target noise data and the preset original noise-free data.

[0103] Optionally, in acquiring the target multi-view image and target mask image of the target object in the robot noise-adding operation scenario, the acquisition module 810 is specifically used for: Retrieve multimodal sensors and noise simulation devices corresponding to the robot's noiseless operation scenario; The noise simulation device generates reference noise and applies it to the robot's noiseless operation scenario to obtain the robot's noisy operation scenario. The multimodal sensor is used to acquire original multi-view images of the target object in the robot's noisy operation scenario; The original multi-view image is subjected to foreground segmentation processing to obtain the original object mask image; Distortion correction and multi-view registration are performed on the original multi-view image and the original object mask image respectively to obtain the target multi-view image and the target object mask image.

[0104] Optionally, in determining the reference particle set based on the target multi-view image and the target object mask image, the first determining module 820 is specifically used for: Determine the particle generation region corresponding to the target object mask image; Based on a preset particle generation function, using the target multi-view image as data input, particles are generated within the particle generation area to obtain an initial particle set; The spatial distribution of each particle in the initial particle set is optimized according to a preset loss function to obtain the reference particle set.

[0105] Optionally, in the process of configuring parameters and injecting noise into the reference particle set to obtain a target particle models, the processing module 830 is specifically used for: The parameters of each particle in the reference particle set are configured to obtain the target particle set; the parameters of each particle in the target particle set include: three-dimensional spatial position coordinates, three-axis motion velocity, and particle mass; Obtain the constraint relationships between all particles in the target particle set to obtain a constraint relationship set; The initial particle state set is determined based on the target particle set, the constraint relationship set, and the preset system friction coefficient; Each initial particle state set is subjected to a noise injection and physical rationality screening to obtain a target particle state sets. The model of the a target particles is determined based on the preset particle dynamics equations and the set of states of the a target particles.

[0106] Optionally, in the process of injecting noise a times and performing physical plausibility screening on the initial particle state set to obtain a target particle state sets, the processing module 830 is further specifically used for: Each initial particle state set is subjected to a noise injections to obtain a reference particle state sets; the noise type corresponding to each of the a noise injections includes at least one of the following: sensor noise, environmental noise, and mechanical noise. Obtain the target physics rules corresponding to the differentiable particle engine; All particles conforming to the target physical rules in each of the a reference particle state sets are filtered to obtain the a target particle state sets.

[0107] Optionally, in the step of inputting the b target particle models into a preset differentiable particle engine to simulate a noise scene and obtain b sets of target noise data, the noise simulation module 850 is specifically used for: Obtain the target operating parameters corresponding to the differentiable particle engine; the target operating parameters include: simulation time step, gravitational acceleration, environmental boundary conditions, and the target action sequence of the robot in the robot noise-adding operation scenario; The target operating parameters are loaded into the differentiable particle engine, and the differentiable particle engine is used to simulate noise scenarios for the b target particle models respectively to obtain the b sets of target noise data.

[0108] Optionally, in the process of filtering the target noise data in group b for task effectiveness to obtain target noise data in group c, the second filtering module 860 is specifically used for: Obtain the preset reward function and the reward threshold corresponding to the reward function; Based on the reward function, b reward values ​​are calculated for each group of target noise data and the target action sequence in the b groups of target noise data; each target noise data corresponds to one reward value. Based on the b reward values, target noise data that satisfy the reward threshold among the b groups of target noise data are selected, and the c groups of target noise data are obtained.

[0109] As can be seen, by performing directional noise perturbation and physical compliance verification on the particle properties of the target object, and then using a differentiable particle engine to complete the noise scene simulation and task effectiveness screening, the effective noise samples are finally fused with the original noise-free data to construct a training set containing high-quality noise samples, thereby improving the noise robustness of the deep learning model in real interference environments.

[0110] It should be noted that the specific implementation of each operation can be described in the corresponding description of the method embodiments shown above. The training set construction device 800 for enhancing noise robustness can be used to execute the method embodiments of this application, and will not be described again here.

[0111] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0112] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0113] It should be noted that, for the sake of simplicity, the above embodiments are all described as a series of actions. Those skilled in the art should understand that this application is not limited to the described order of actions, as some steps in the embodiments of this application can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions, steps, modules, or units involved are not necessarily essential to the embodiments of this application.

[0114] In the above embodiments, the descriptions of each embodiment in this application have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0116] The steps of the methods or algorithms described in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Furthermore, the ASIC can reside in a terminal device or management device. Alternatively, the processor and storage medium can exist as discrete components in the terminal device or management device.

[0117] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0118] The modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on the processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented through a software program that runs on the processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0119] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for constructing a training set for noise robustness enhancement, characterized in that, The method includes: Acquire the target multi-view image and target object mask image of the target object in the robot noise-adding operation scene; A reference particle set is determined based on the target multi-view image and the target object mask image; the reference particle set is used to characterize the geometry of the target object. The reference particle set is configured with parameters and noise is injected to obtain a target particle models; where a is an integer greater than 1. The noise rationality screening of the a target particle models is performed to obtain b target particle models; b is a positive integer less than or equal to a. The b target particle models are respectively input into a preset differentiable particle engine to simulate a noise scene, resulting in b sets of target noise data; The target noise data in group b is filtered for task effectiveness to obtain target noise data in group c; c is a positive integer less than or equal to b. The target training set is determined based on the c sets of target noise data and the preset original noise-free data.

2. The method as described in claim 1, characterized in that, The acquisition of the target multi-view image and target object mask image in the robot noise-adding operation scene includes: Retrieve multimodal sensors and noise simulation devices corresponding to the robot's noiseless operation scenario; The noise simulation device generates reference noise and applies it to the robot's noiseless operation scenario to obtain the robot's noisy operation scenario. The multimodal sensor is used to acquire original multi-view images of the target object in the robot's noisy operation scenario; The original multi-view image is subjected to foreground segmentation processing to obtain the original object mask image; Distortion correction and multi-view registration are performed on the original multi-view image and the original object mask image respectively to obtain the target multi-view image and the target object mask image.

3. The method as described in claim 2, characterized in that, Determining the reference particle set based on the target multi-view image and the target object mask image includes: Determine the particle generation region corresponding to the target object mask image; Based on a preset particle generation function, using the target multi-view image as data input, particles are generated within the particle generation area to obtain an initial particle set; The spatial distribution of each particle in the initial particle set is optimized according to a preset loss function to obtain the reference particle set.

4. The method as described in claim 3, characterized in that, The process of configuring parameters and injecting noise into the reference particle set to obtain a target particle models includes: The parameters of each particle in the reference particle set are configured to obtain the target particle set; the parameters of each particle in the target particle set include: three-dimensional spatial position coordinates, three-axis motion velocity, and particle mass; Obtain the constraint relationships between all particles in the target particle set to obtain a constraint relationship set; The initial particle state set is determined based on the target particle set, the constraint relationship set, and the preset system friction coefficient; Each initial particle state set is subjected to a noise injection and physical rationality screening to obtain a target particle state sets. The model of the a target particles is determined based on the preset particle dynamics equations and the set of states of the a target particles.

5. The method as described in claim 4, characterized in that, The process of injecting noise a times into the initial particle state set and then performing physical plausibility screening to obtain a target particle state sets includes: Each initial particle state set is subjected to a noise injections to obtain a reference particle state sets; the noise type corresponding to each of the a noise injections includes at least one of the following: sensor noise, environmental noise, and mechanical noise. Obtain the target physics rules corresponding to the differentiable particle engine; All particles conforming to the target physical rules in each of the a reference particle state sets are filtered to obtain the a target particle state sets.

6. The method as described in claim 4, characterized in that, The process involves inputting the b target particle models into a preset differentiable particle engine to simulate a noise scene, resulting in b sets of target noise data, including: Obtain the target operating parameters corresponding to the differentiable particle engine; the target operating parameters include: simulation time step, gravitational acceleration, environmental boundary conditions, and the target action sequence of the robot in the robot noise-adding operation scenario; The target operating parameters are loaded into the differentiable particle engine, and the differentiable particle engine is used to simulate noise scenarios for the b target particle models respectively to obtain the b sets of target noise data.

7. The method as described in claim 6, characterized in that, The step of filtering the target noise data in group b for task effectiveness to obtain target noise data in group c includes: Obtain the preset reward function and the reward threshold corresponding to the reward function; Based on the reward function, b reward values ​​are calculated for each group of target noise data and the target action sequence in the b groups of target noise data; each target noise data corresponds to one reward value. Based on the b reward values, target noise data that satisfy the reward threshold among the b groups of target noise data are selected, and the c groups of target noise data are obtained.

8. A training set construction apparatus for enhancing noise robustness, characterized in that, The device includes an acquisition module, a first determination module, a processing module, a first filtering module, a noise simulation module, a second filtering module, and a second determination module, wherein: The acquisition module is used to acquire the target multi-view image and the target object mask image of the target object in the robot noise-adding operation scene; The first determining module is used to determine a reference particle set based on the target multi-view image and the target object mask image; the reference particle set is used to characterize the geometric shape of the target object; The processing module is used to configure parameters and inject noise into the reference particle set to obtain a target particle models; where a is an integer greater than 1. The first screening module is used to perform noise rationality screening on the a target particle models to obtain b target particle models; b is a positive integer less than or equal to a. The noise simulation module is used to input the b target particle models into a preset differentiable particle engine to simulate noise scenes and obtain b sets of target noise data. The second filtering module is used to filter the b groups of target noise data for task effectiveness to obtain c groups of target noise data; c is a positive integer less than or equal to b; The second determining module is used to determine the target training set based on the c groups of target noise data and the preset original noise-free data.

9. An electronic device, characterized in that, include: Processor, memory, communication interface, and one or more programs; The one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-7.

Images