A humanoid robot and a disturbance point cloud processing method in an industrial scene and related equipment

Abstract

The application discloses a humanoid robot and a disturbance point cloud processing method and related equipment in an industrial scene. The method comprises the following steps: obtaining disturbance point cloud data and industrial object type information in an industrial scene, and calculating a dynamic distortion degree of the disturbance point cloud data; obtaining an operation task type of the humanoid robot, determining a diffusion time step according to the dynamic distortion degree and the operation task type through a preset diffusion step mapping table; and processing the disturbance point cloud data through a denoising diffusion model according to the diffusion time step to obtain denoised point cloud data, wherein the denoising diffusion model adopts a loss function comprising a geometric constraint term, an industrial feature matching term and a robot operation precision constraint term to calculate a loss value in a training process. The embodiment of the application can effectively suppress point cloud disturbance in an industrial scene, and ensure that the processing result of the disturbance point cloud can retain key features and adapt to robot operation requirements. The application can be widely applied to the field of industrial automation technology.

Description

Technical Field

[0001] This application relates to the field of industrial automation technology, and in particular to a humanoid robot and a method and related equipment for processing disturbed point clouds in industrial scenarios. Background Technology

[0002] In the field of industrial automation, humanoid robots, with their flexible movement and environmental adaptability, are increasingly being applied to assembly, quality inspection, and material handling. Humanoid robots typically rely on sensors such as LiDAR and cameras to collect point cloud data, enabling them to perceive and manipulate their surroundings and target objects. However, electromagnetic interference, vibration, and occlusion in industrial environments can cause disturbances in the point cloud, leading to problems such as coordinate shifts and the injection of outliers, resulting in recognition errors or operational inaccuracies.

[0003] To ensure the validity of point cloud data, it is necessary to repair disturbed point clouds. However, existing methods for processing disturbed point cloud data may result in the loss of key feature details such as planes and holes in industrial objects, affecting operational accuracy; moreover, the processed point cloud data is difficult to directly adapt to the operational needs of robots, resulting in insufficient engineering practicality.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose a method and related equipment for processing perturbed point clouds in humanoid robots and industrial scenarios. By dynamically sensing perturbed point clouds in industrial scenarios and combining robot operation task types and prior industrial object features, a loss function that integrates kinematic constraints is constructed to train a denoising diffusion model, thereby achieving real-time processing of perturbed point clouds. This effectively suppresses point cloud perturbations in industrial scenarios, ensuring that the processing results of perturbed point clouds retain key features and adapt to robot operation requirements, while also having a fast processing speed.

[0006] To achieve the above objectives, one aspect of this application proposes a method for processing perturbed point clouds in a humanoid robot and industrial scenarios, the method comprising:

[0007] Acquire disturbed point cloud data and industrial object type information in an industrial scenario, and calculate the dynamic distortion of the disturbed point cloud data based on the industrial object type;

[0008] The operation task type of the humanoid robot is obtained, and the diffusion time step is determined by a preset diffusion step size mapping table based on the dynamic distortion and the operation task type.

[0009] Based on the diffusion time step, the perturbed point cloud data is processed by the trained denoising diffusion model to obtain denoised point cloud data. The denoising diffusion model uses a loss function that includes geometric constraints, industrial feature matching terms, and robot operation accuracy constraints to calculate the loss value during training.

[0010] In some embodiments, calculating the dynamic distortion of the disturbed point cloud data based on the industrial article type includes:

[0011] The neighborhood range is determined based on the industrial object type information, and several neighboring points of the target point in the disturbed point cloud data are selected based on the neighborhood range to construct the local neighborhood of the target point.

[0012] Fit the optimal plane of the local neighborhood of each target point, and calculate the distance from each target point to the corresponding optimal plane using the following formula:

[0013]

[0014] in, The distance from the target point to the corresponding optimal plane. Let be the normal vector of the optimal plane. The point cloud coordinates of the target point are... The center coordinates of the optimal plane are... This is the pose error compensation coefficient. This is the end effector pose error compensation term for the humanoid robot;

[0015] The dynamic distortion of the perturbed point cloud data is determined based on the motion consistency characteristics of the perturbed point cloud data in the time series and the distance of each target point to the corresponding optimal plane.

[0016] In some embodiments, the optimal plane for fitting the local neighborhood of each of the target points includes:

[0017] Obtain the sensor calibration parameters of the humanoid robot, and correct the point cloud coordinates of each point in the local neighborhood based on the sensor calibration parameters;

[0018] A point cloud matrix is ​​constructed based on the corrected local neighborhood, and the normal vector of the optimal plane is calculated using the singular value decomposition method.

[0019] The direction of the normal vector of the optimal plane is constrained based on the normal vector of the reference surface in the standard model of the industrial object.

[0020] In some embodiments, determining the diffusion time step based on the dynamic distortion and the operation task type using a preset diffusion step size mapping table includes:

[0021] The diffusion step size mapping table is constructed based on historical experience data;

[0022] Based on the dynamic distortion and the operation task type, the corresponding basic diffusion step size is found in the diffusion step size mapping table;

[0023] The basic diffusion step size is adjusted based on the real-time operational accuracy feedback information of the humanoid robot to obtain the diffusion time step size.

[0024] In some embodiments, processing the disturbed point cloud data using a trained denoising diffusion model according to the diffusion time step includes:

[0025] The number of forward noise additions and the number of reverse noise removals are determined based on the diffusion time step.

[0026] The disturbed point cloud data is subjected to forward noise addition using the following formula:

[0027]

[0028]

[0029] in, for Perturbed point cloud data after adding noise The number of forward noise additions. This is the cumulative noise attenuation factor. For noise scale, Standard Gaussian noise;

[0030] The denoised perturbation point cloud data and the reverse denoising count are used as inputs to the denoising diffusion model to obtain the denoised point cloud data. The denoised point cloud data that exceeds the range is truncated according to the workspace constraints of the humanoid robot.

[0031] In some embodiments, the expression for the loss function is as follows:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] in, Let the loss function be... For the geometric constraint term, For the industrial feature matching item, This refers to the robot's operational accuracy constraint term. As the first weighting coefficient, This is the second weighting coefficient. The third weighting coefficient, This is the plane fitting error. To maintain the error of surface curvature, The normal vector of the standard model plane. For point cloud coordinates, The center coordinates of the standard model plane, For point cloud curvature, For standard model curvature, For point cloud feature vectors, For template features, This represents the deviation of the tool's center point.

[0040] In some embodiments, the perturbed point cloud processing method further includes:

[0041] The object localization result is determined based on the denoised point cloud data;

[0042] The humanoid robot performs operational tasks based on the object positioning results, and collects the actual operational errors during the execution.

[0043] The pose error compensation coefficient and the diffusion step size mapping table are optimized based on the actual operational error.

[0044] To achieve the above objectives, another aspect of this application proposes a humanoid robot and a perturbation point cloud processing device for industrial scenarios, the device comprising:

[0045] The dynamic distortion estimation module is used to acquire disturbed point cloud data and industrial object type information in industrial scenarios, and calculate the dynamic distortion of the disturbed point cloud data according to the industrial object type.

[0046] The diffusion step size mapping module is used to obtain the operation task type of the humanoid robot, and determine the diffusion time step size through a preset diffusion step size mapping table based on the dynamic distortion and the operation task type.

[0047] The perturbation point cloud processing module is used to process the perturbation point cloud data according to the diffusion time step and through a trained denoising diffusion model to obtain denoised point cloud data. The denoising diffusion model uses a loss function that includes geometric constraint terms, industrial feature matching terms, and robot operation accuracy constraint terms to calculate the loss value during the training process.

[0048] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the methods described above.

[0049] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0050] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods described above.

[0051] The embodiments of this application include at least the following beneficial effects: This application provides a humanoid robot and a method and related equipment for processing perturbed point clouds in industrial scenarios. This solution calculates the dynamic distortion of perturbed point clouds by combining industrial object type information, which can fully consider the geometric characteristics of different types of industrial objects, thereby achieving more accurate distortion estimation in industrial scenarios and providing a reasonable basis for subsequent diffusion time step matching; by mapping the dynamic distortion and operation task type to the corresponding diffusion time step through a diffusion step mapping table, the number of denoising steps of the diffusion model can be dynamically adjusted to meet the different requirements for accuracy and efficiency in various application scenarios, and realize the processing of different types of operations. Adaptive matching for tasks is achieved. By introducing geometric constraints, industrial feature matching terms, and robot operation accuracy constraints into the loss function of the denoising diffusion model, the practicality of the denoising results in humanoid robots and industrial scenarios is improved. Among them, the geometric constraint term can make the overall shape of the denoised point cloud data conform to the geometric features of the standard model, the industrial feature matching term can improve the matching degree between key industrial features and templates, meet the needs of high-precision operation tasks, and the robot operation accuracy constraint term optimizes the denoising results based on the positioning error of the robot end effector, making it more in line with the actual execution path, so that the processed point cloud can be directly used for robot operation, thus improving practicality. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the steps of a method for processing disturbed point clouds in an industrial setting using a humanoid robot, as provided in an embodiment of this application.

[0053] Figure 2 This is a schematic diagram of bolt hole deviation values ​​before perturbation point cloud processing provided in the embodiments of this application;

[0054] Figure 3 This is a schematic diagram of the bolt hole deviation values ​​after perturbation point cloud processing provided in the embodiments of this application;

[0055] Figure 4 This is a schematic diagram of the loss function optimization process provided in the embodiments of this application;

[0056] Figure 5 This is a schematic diagram of the structure of a humanoid robot and a disturbance point cloud processing device in an industrial setting, provided in an embodiment of this application.

[0057] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0059] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0060] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0062] This application provides a method for processing disturbed point clouds in a humanoid robot and industrial scenarios, relating to the field of information technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited thereto. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a method for processing disturbed point clouds in a humanoid robot and industrial scenarios, but is not limited to the above forms.

[0063] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0064] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0065] Figure 1 This is an optional flowchart of a method for processing disturbed point clouds in a humanoid robot and industrial scenarios, provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S103.

[0066] S101. Obtain disturbed point cloud data and industrial object type information in industrial scenarios, and calculate the dynamic distortion of the disturbed point cloud data according to the type of industrial object.

[0067] Specifically, in industrial settings, point cloud data collected by sensors may be disturbed due to factors such as electromagnetic interference, vibration, and obstruction, resulting in distortion characteristics such as coordinate shifts and outlier injection. In this embodiment, the disturbed point cloud data to be processed can be acquired by sensors such as LiDAR and 3D cameras mounted on the humanoid robot body in the industrial setting, or by sensor systems deployed in its surrounding environment. Based on actual production planning, operational information, or visual recognition results, the type information of the target industrial object in the current point cloud data can be obtained. For example, by reading the target workpiece type of the current robot task to obtain industrial object type information, it is possible to determine whether the current object is a shaft, plate, or box-type structure, providing prior structural data for subsequent distortion estimation.

[0068] In some embodiments, calculating the dynamic distortion of the disturbed point cloud data based on the type of industrial article includes:

[0069] S1011. Determine the neighborhood range based on the industrial object type information, and select several neighboring points of the target point in the disturbed point cloud data to construct the local neighborhood of the target point based on the neighborhood range.

[0070] S1012. Fit the optimal plane of the local neighborhood of each target point, and calculate the distance from each target point to the corresponding optimal plane using the following formula:

[0071]

[0072] in, Let be the distance from the target point to the corresponding optimal plane. The normal vector of the optimal plane. The point cloud coordinates of the target point, The center coordinates of the optimal plane, This is the pose error compensation coefficient. This is the pose error compensation term for the end effector of a humanoid robot.

[0073] S1013. Determine the dynamic distortion of the perturbed point cloud data based on the motion consistency characteristics of the perturbed point cloud data in the time series and the distance of each target point to the corresponding optimal plane.

[0074] Specifically, in this embodiment, the distortion of each target point is determined by fitting the optimal plane of the local neighborhood of each target point in the perturbed point cloud data and calculating the distance from the target point to the optimal plane. Finally, the overall dynamic distortion of the current perturbed point cloud data is determined based on the distortion of all target points.

[0075] In industrial environments, the structural features of different objects vary significantly. Using a fixed neighborhood size may lead to insufficient fitting accuracy and feature loss. Therefore, this embodiment sets adaptive neighborhood ranges for different types of industrial objects, selecting different numbers of neighboring points to construct local neighborhoods. For example, when the target industrial object is a plate-like object, which typically has a large planar structure, the number of neighborhood points K can be set to a larger value of 20 or more to enhance the stability of planar fitting. For shaft-like objects, which typically have obvious curved surface features, the number of neighborhood points K can be set to a smaller value of around 10 to preserve surface details.

[0076] Meanwhile, to further improve the accuracy of distortion estimation, this embodiment introduces a robot end effector pose error compensation term when calculating the distance from the point cloud to the optimal plane. To correct point cloud offsets caused by robot vibration, and based on this, combined with the motion consistency characteristics of point cloud in the time series dimension (such as the motion trajectory of part conveyor belt), false disturbance points in dynamic scenes are filtered out. The deviation between each target point in the disturbed point cloud and its local fitting plane is statistically analyzed, and finally the overall dynamic distortion of the current point cloud data is estimated to provide data support for subsequent processing operations such as diffusion time step matching.

[0077] In some embodiments, fitting the optimal plane of the local neighborhood of each target point includes:

[0078] S10121. Obtain the sensor calibration parameters of the humanoid robot, and correct the point cloud coordinates of each point in the local neighborhood based on the sensor calibration parameters;

[0079] S10122. Construct a point cloud matrix based on the corrected local neighborhood, and calculate the normal vector of the optimal plane using the singular value decomposition method.

[0080] S10123. Apply directional constraints to the normal vector of the optimal plane based on the normal vector of the reference surface in the standard model of the industrial object.

[0081] Specifically, in this embodiment, to fit the optimal plane, it is first necessary to obtain the calibration parameters of the robot sensor (including the intrinsic parameter matrix K and the extrinsic parameter matrix T) to correct the point cloud coordinates of each point. Subsequently, a point cloud matrix is ​​constructed based on the corrected point cloud coordinates, and conventional fitting algorithms such as singular value decomposition are used to calculate the optimal plane normal vector corresponding to the local neighborhood.

[0082] To improve the stability of the fitting direction, this embodiment further incorporates planar feature information (such as the assembly reference surface normal vector) from the standard model of the industrial object to constrain the direction of the fitting result, ensuring that the deviation angle does not exceed a preset threshold (such as 5°) to avoid subsequent error accumulation.

[0083] S102. Obtain the operation task type of the humanoid robot, and determine the diffusion time step according to the dynamic distortion and operation task type through the preset diffusion step mapping table.

[0084] Specifically, this embodiment first obtains the operation task type of the humanoid robot, such as assembly, coarse positioning, dynamic handling, etc. Different task types have different requirements for point cloud processing accuracy and processing efficiency. For example, high-precision assembly pays more attention to whether key features can be preserved, while coarse positioning tasks pay more attention to processing speed.

[0085] Then, based on the dynamic distortion value estimated above, together with the current task type, it is used as an input parameter to look up a preset diffusion step size mapping table to determine the diffusion time step. For example, for tasks with high precision requirements or high dynamic distortion, a longer diffusion time step will be mapped to ensure that noise is removed cleanly. For coarse localization tasks or low dynamic distortion, a shorter diffusion time step will be mapped to reduce the inference time of the subsequent diffusion model and improve real-time performance.

[0086] In some embodiments, the diffusion time step is determined by a preset diffusion step size mapping table based on the dynamic distortion and the type of operation task, including:

[0087] S1021. Construct a diffusion step size mapping table based on historical experience data;

[0088] S1022. Find the corresponding basic diffusion step size in the diffusion step size mapping table according to the dynamic distortion and the operation task type;

[0089] S1023. Adjust the basic diffusion step size based on the real-time operation accuracy feedback information of the humanoid robot to obtain the diffusion time step size.

[0090] Specifically, in this embodiment, a three-dimensional mapping table is first constructed based on a large amount of industrial field data and robot operation experience, jointly mapping dynamic distortion and operation task type to diffusion step size. During actual processing, the corresponding base diffusion step size is retrieved from this mapping table based on the currently estimated dynamic distortion value and the type of operation task performed by the robot. For example, when the dynamic distortion D = 0.1 mm and a high-precision assembly task is performed, a base diffusion step size λ of 15 and a step size coefficient of 1.2 can be matched.

[0091] Building upon this, this embodiment further incorporates feedback information on the current operational accuracy of the humanoid robot, such as real-time operational errors. When detected When the value exceeds the threshold, the basic diffusion step size is adjusted according to the step size coefficient to obtain the final diffusion time step. For example, when the basic diffusion step size λ=15 (step size coefficient is 1.2), when the value exceeds the threshold, the basic diffusion step size is adjusted according to the step size coefficient to obtain the final diffusion time step. =0.12mm. When the value exceeds the preset threshold, λ can be dynamically adjusted to 18.

[0092] S103. Based on the diffusion time step, the disturbed point cloud data is processed by the trained denoising diffusion model to obtain denoised point cloud data. The denoising diffusion model uses a loss function that includes geometric constraints, industrial feature matching terms, and robot operation accuracy constraints to calculate the loss value during training.

[0093] In some embodiments, the perturbed point cloud data is processed using a trained denoising diffusion model according to the diffusion time step, including:

[0094] S1031. Determine the number of forward noise additions and reverse noise removals based on the diffusion time step;

[0095] S1032. Perform forward noise addition on the disturbed point cloud data using the following formula:

[0096]

[0097]

[0098] in, for Perturbed point cloud data after adding noise For the number of forward noise additions, This is the cumulative noise attenuation factor. For noise scale, Standard Gaussian noise;

[0099] S1033. The noise-added perturbed point cloud data and the number of reverse denoising attempts are used as inputs to the denoising diffusion model to obtain denoised point cloud data. The denoised point cloud data that exceeds the range is truncated according to the workspace constraints of the humanoid robot.

[0100] Specifically, in this embodiment, based on the aforementioned diffusion time step, the number of noise additions and denoising operations required for the point cloud in the diffusion model are determined, and forward noise addition processing is performed on the original disturbed point cloud data accordingly. After noise addition processing, the noise-added point cloud data is used as the input to the denoising diffusion model, and necessary point cloud truncation is performed in conjunction with the robot's workspace range to ensure that the final output point cloud data is compatible with the robot's operation. In practical application scenarios, the bolt hole deviation identification results before and after disturbance processing are as follows: Figure 2 and Figure 3 As shown, this effectively improves the accuracy of key assembly features.

[0101] In some embodiments, the loss function is expressed as follows:

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] in, For loss function, For geometric constraint terms, For industrial feature matching items, For robot operation accuracy constraints, As the first weighting coefficient, This is the second weighting coefficient. The third weighting coefficient, This is the plane fitting error. To maintain the error of surface curvature, The normal vector of the standard model plane. For point cloud coordinates, The center coordinates of the standard model plane, For point cloud curvature, For standard model curvature, For point cloud feature vectors, For template features, This represents the deviation of the tool's center point.

[0110] Specifically, in the training process of the model in this embodiment, to enhance the model's adaptability to industrial scenarios, geometric constraint terms, industrial feature matching terms, and robot operation accuracy constraint terms are introduced to construct the loss function, among which the geometric constraint term... Used to ensure that the overall shape of the point cloud is consistent with the standard model; industrial feature matching item The positional accuracy of key features such as holes and corners is constrained (error ≤ 0.05mm) using the SIFT (Scale Invariant Feature Transform) feature matching algorithm; robot operation accuracy constraints. The processing results are correlated with the end effector positioning error to ensure that the processed point cloud can be directly used for robot operation. Clean point clouds from industrial scenes (including CAD model feature annotations for parts) can be used for training data to enhance the model's learning ability for features such as bolt holes and assembly reference surfaces. In practical applications, the loss value decreases from 0 to 50,000 training iterations as follows... Figure 4 As shown.

[0111] In some embodiments, the perturbed point cloud processing method further includes:

[0112] S104. Determine the object localization result based on the denoised point cloud data;

[0113] S105. The humanoid robot performs operational tasks based on the object positioning results, and the actual operational errors during the execution are collected.

[0114] S106. Optimize the pose error compensation coefficient and diffusion step size mapping table based on the actual operation error.

[0115] Specifically, in this embodiment, the target object is located and identified based on the denoised point cloud data, and the humanoid robot performs corresponding tasks such as grasping and assembly. During the execution process, operational error information, such as the offset of the end effector, assembly gap, and grasping deviation, can be collected in real time as external indicators for evaluating the quality of point cloud processing.

[0116] After obtaining the aforementioned error information, it can be used to correct the pose error compensation coefficients used in the dynamic distortion estimation process and to update the relevant parameters in the diffusion step size mapping table, thereby achieving closed-loop optimization of the point cloud perception-operation processing flow. Through this feedback mechanism, the system can gradually accumulate parameter experience in repetitive tasks, continuously improving the point cloud processing effect.

[0117] The present invention will be further described below with reference to a specific embodiment.

[0118] Taking the assembly of automotive transmission housing as an example, humanoid robots need to use LiDAR to collect point clouds of the transmission housing, identify the bolt hole positions, and tighten the bolts. Vibrations in the industrial environment can cause the point cloud to fluctuate by ±0.1mm, which may lead to deviations in the hole position recognition.

[0119] First, the degree of dynamic distortion is assessed. Since the dynamic gearbox housing is a box-type part, the number of neighborhood points is set to K=20; the pose error of the robot's end effector is then collected. compensation coefficient Then the single-point cloud distortion estimate is:

[0120] ,

[0121] Calculate all points The mean value is used to obtain the degree of dynamic distortion. .

[0122] Then, diffusion step size matching is performed. The operation task is high-precision assembly (step size coefficient 1.2), which is obtained by looking up the diffusion step size mapping table. The base value is λ=12, which, after multiplying by a coefficient, results in λ=14; bolt hole positioning error is detected in real time. Then λ is adjusted to 14 × 1.1 = 15.4 (rounded down to 15).

[0123] Next, denoising is achieved using a denoising diffusion model. During the forward diffusion process, the point cloud of the bolt hole region is divided into blocks (2000 points per block) and calculated in parallel:

[0124] ,

[0125] in The confidence level decreases as the hole location increases (hole edge point). Reduce by 20% to retain details.

[0126] In the reverse denoising process, the denoising diffusion neural network Output denoised point cloud, and simultaneously through The center distance of the constrained bolt holes (design value 100mm) is penalized if the deviation exceeds 0.03mm.

[0127] Finally, closed-loop optimization is performed to process the hole position recognition error. The robot completes the tightening operation, and the actual assembly gap is... (≤0.08mm threshold); record the current parameters, and keep λ=15 and k=0.01 for the next similar task.

[0128] During the training of the denoising diffusion model, the training dataset can be 500 sets of clean point clouds of the gearbox housing (including CAD annotations of 6 bolt holes); the weights of the loss function can be set as follows: α=0.3 (geometry), β=0.4 (features), γ=0.3 (operation); after 60k training iterations, the loss function converges to 0.008, and the bolt hole positioning error stabilizes within 0.02mm, the training can be stopped.

[0129] Please see Figure 5 This application also provides a humanoid robot and a disturbance point cloud processing device for industrial scenarios, which can implement the above-mentioned method. The device includes:

[0130] The dynamic distortion estimation module is used to acquire disturbed point cloud data and industrial object type information in industrial scenarios, and calculate the dynamic distortion of the disturbed point cloud data according to the type of industrial object.

[0131] The diffusion step size mapping module is used to obtain the operation task type of the humanoid robot and determine the diffusion time step based on the dynamic distortion and operation task type through a preset diffusion step size mapping table.

[0132] The perturbation point cloud processing module is used to process the perturbation point cloud data according to the diffusion time step and the trained denoising diffusion model to obtain denoised point cloud data. The denoising diffusion model uses a loss function that includes geometric constraints, industrial feature matching terms, and robot operation accuracy constraints to calculate the loss value during the training process.

[0133] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0135] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0137] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0138] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application.

[0139] The input / output interface 903 is used to implement information input and output;

[0140] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0141] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0142] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0143] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0144] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0145] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0146] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0147] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0148] This application provides a humanoid robot and a method, apparatus, electronic device, storage medium, and program product for processing perturbed point clouds in industrial scenarios. By combining information on the type of industrial objects, it calculates the dynamic distortion of the perturbed point cloud, fully considering the geometric characteristics of different types of industrial objects. This allows for more accurate distortion estimation in industrial scenarios, providing a reasonable basis for subsequent diffusion time step matching. By mapping the dynamic distortion and operation task type to the corresponding diffusion time step through a diffusion step mapping table, it can dynamically adjust the number of denoising steps in the diffusion model, meeting the different accuracy and efficiency requirements of various application scenarios and enabling processing of different types of operations. Adaptive matching for tasks: By introducing geometric constraints, industrial feature matching terms, and robot operation accuracy constraints into the loss function of the denoising diffusion model, the practicality of the denoising results in humanoid robots and industrial scenarios is improved. Among them, the geometric constraint term can make the overall shape of the denoised point cloud data conform to the geometric features of the standard model, the industrial feature matching term can improve the matching degree between key industrial features and templates, meet the needs of high-precision operation tasks, and the robot operation accuracy constraint term optimizes the denoising results based on the positioning error of the robot end effector, making it more in line with the actual execution path, so that the processed point cloud can be directly used for robot operation, thus improving practicality.

[0149] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0150] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0151] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0153] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0154] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0156] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0158] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for processing disturbed point clouds in a humanoid robot and industrial scenarios, characterized in that, include: Acquire disturbed point cloud data and industrial object type information in an industrial scenario, and calculate the dynamic distortion of the disturbed point cloud data based on the industrial object type information; The operation task type of the humanoid robot is obtained, and the diffusion time step is determined by a preset diffusion step size mapping table based on the dynamic distortion and the operation task type. According to the diffusion time step, the perturbed point cloud data is processed by the trained denoising diffusion model to obtain denoised point cloud data. The denoising diffusion model uses a loss function that includes geometric constraint terms, industrial feature matching terms, and robot operation accuracy constraint terms to calculate the loss value during the training process. The calculation of the dynamic distortion of the disturbed point cloud data based on the industrial product type includes: The neighborhood range is determined based on the industrial object type information, and several neighboring points of the target point in the disturbed point cloud data are selected based on the neighborhood range to construct the local neighborhood of the target point. Fit the optimal plane of the local neighborhood of each target point, and calculate the distance from each target point to the corresponding optimal plane using the following formula: in, The distance from the target point to the corresponding optimal plane. Let be the normal vector of the optimal plane. The point cloud coordinates of the target point are... The center coordinates of the optimal plane are... This is the pose error compensation coefficient. This is the end effector pose error compensation term for the humanoid robot; The dynamic distortion of the perturbed point cloud data is determined based on the motion consistency characteristics of the perturbed point cloud data in the time series and the distance of each target point to the corresponding optimal plane. The expression for the loss function is as follows: in, Let the loss function be... For the geometric constraint term, For the industrial feature matching item, This refers to the robot's operational accuracy constraint term. As the first weighting coefficient, This is the second weighting coefficient. The third weighting coefficient, This is the plane fitting error. To maintain the error of surface curvature, The normal vector of the standard model plane. For point cloud coordinates, The center coordinates of the standard model plane, For point cloud curvature, For standard model curvature, For point cloud feature vectors, For template features, This represents the deviation of the tool's center point.

2. The method for processing perturbation point clouds in a humanoid robot and industrial setting according to claim 1, characterized in that, The optimal plane for fitting the local neighborhood of each of the target points includes: Obtain the sensor calibration parameters of the humanoid robot, and correct the point cloud coordinates of each point in the local neighborhood based on the sensor calibration parameters; A point cloud matrix is ​​constructed based on the corrected local neighborhood, and the normal vector of the optimal plane is calculated using the singular value decomposition method. The direction of the normal vector of the optimal plane is constrained based on the normal vector of the reference surface in the standard model of the industrial object.

3. The method for processing perturbation point clouds in a humanoid robot and industrial setting according to claim 1, characterized in that, The step of determining the diffusion time step based on the dynamic distortion and the operation task type using a preset diffusion step size mapping table includes: The diffusion step size mapping table is constructed based on historical experience data; Based on the dynamic distortion and the operation task type, the corresponding basic diffusion step size is found in the diffusion step size mapping table; The basic diffusion step size is adjusted based on the real-time operational accuracy feedback information of the humanoid robot to obtain the diffusion time step size.

4. The method for processing disturbed point clouds in a humanoid robot and industrial setting according to claim 1, characterized in that, The step of processing the disturbed point cloud data using a trained denoising diffusion model based on the diffusion time step includes: The number of forward noise additions and the number of reverse noise removals are determined based on the diffusion time step. The disturbed point cloud data is subjected to forward noise addition using the following formula: in, for Perturbed point cloud data after adding noise The number of forward noise additions. This is the cumulative noise attenuation factor. For noise scale, Standard Gaussian noise; The denoised perturbation point cloud data and the reverse denoising count are used as inputs to the denoising diffusion model to obtain the denoised point cloud data. The denoised point cloud data that exceeds the range is truncated according to the workspace constraints of the humanoid robot.

5. The method for processing disturbed point clouds in a humanoid robot and industrial setting according to claim 1, characterized in that, The perturbation point cloud processing method further includes: The object localization result is determined based on the denoised point cloud data; The humanoid robot performs operational tasks based on the object positioning results, and collects the actual operational errors during the execution. The pose error compensation coefficient and the diffusion step size mapping table are optimized based on the actual operational error.

6. A humanoid robot and a disturbance point cloud processing device for industrial scenarios, characterized in that, The device includes: The dynamic distortion estimation module is used to acquire disturbed point cloud data and industrial object type information in an industrial scenario, and to calculate the dynamic distortion of the disturbed point cloud data based on the industrial object type information. The diffusion step size mapping module is used to obtain the operation task type of the humanoid robot, and determine the diffusion time step size through a preset diffusion step size mapping table based on the dynamic distortion and the operation task type. The perturbation point cloud processing module is used to process the perturbation point cloud data according to the diffusion time step and through a trained denoising diffusion model to obtain denoised point cloud data. The denoising diffusion model uses a loss function that includes geometric constraint terms, industrial feature matching terms and robot operation accuracy constraint terms to calculate the loss value during the training process. The calculation of the dynamic distortion of the disturbed point cloud data based on the industrial product type includes: The neighborhood range is determined based on the industrial object type information, and several neighboring points of the target point in the disturbed point cloud data are selected based on the neighborhood range to construct the local neighborhood of the target point. Fit the optimal plane of the local neighborhood of each target point, and calculate the distance from each target point to the corresponding optimal plane using the following formula: in, The distance from the target point to the corresponding optimal plane. Let be the normal vector of the optimal plane. The point cloud coordinates of the target point are... The center coordinates of the optimal plane are... This is the pose error compensation coefficient. This is the end effector pose error compensation term for the humanoid robot; The dynamic distortion of the perturbed point cloud data is determined based on the motion consistency characteristics of the perturbed point cloud data in the time series and the distance of each target point to the corresponding optimal plane. The expression for the loss function is as follows: in, Let the loss function be... For the geometric constraint term, For the industrial feature matching item, This refers to the robot's operational accuracy constraint term. As the first weighting coefficient, This is the second weighting coefficient. The third weighting coefficient, This is the plane fitting error. To maintain the error of surface curvature, The normal vector of the standard model plane. For point cloud coordinates, The center coordinates of the standard model plane, For point cloud curvature, For standard model curvature, For point cloud feature vectors, For template features, This represents the deviation of the tool's center point.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images