Polishing and spraying path planning method and device based on three-dimensional generative reconstruction

Abstract

This invention belongs to the field of artificial intelligence equipment repair technology, and discloses a method and device for grinding and spraying path planning based on three-dimensional generative reconstruction. The method includes: acquiring sparse point clouds through multi-angle scanning using a camera at the end of a robotic arm; generating a three-dimensional mesh model by employing a three-dimensional Gaussian splashing algorithm, introducing normal consistency constraints and opacity regularization to suppress high reflectivity during optimization; extracting manifold topological features through a graph convolutional network and defining a manifold projection operator; predicting trajectory points using a trajectory manifold consistency model with noise state, time step, and topological features as input, and real-time calling the projection operator to correct the trajectory points to the model surface; injecting a safety potential gradient guidance term to avoid singularities; and performing uncertainty assessment on the generated trajectory, stopping if the threshold is exceeded, otherwise proceeding. This invention achieves path generation with high geometric fit, kinematic safety, and real-time dynamic replanning under complex irregular surfaces, significantly improving repair quality.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence equipment repair technology, and in particular to a grinding and spraying path planning method and apparatus based on three-dimensional generative reconstruction. Background Technology

[0002] Existing tower repair robots mostly adopt a "blind repair" mode: low perception accuracy, traditional lidar or structured light cannot reconstruct sub-millimeter-level corrosion pit textures under strong metal reflection; rigid planning, teaching-based paths based on 2D projection have severe geometric distortions on variable-diameter curved surfaces, resulting in inconsistent grinding overlap rates; weak safety, existing generative path planning often ignores robot kinematic constraints, easily causing joint impacts or collisions near singularities; poor real-time performance, generative methods such as diffusion models are time-consuming and cannot meet the needs of online dynamic replanning.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a grinding and spraying path planning method and apparatus based on three-dimensional generative reconstruction, aiming to solve the technical problems of insufficient perception accuracy of complex high-reflectivity curved surfaces, path distortion of variable diameter curved surfaces, lack of kinematic safety constraints in generative planning, and poor real-time performance.

[0005] To achieve the above objectives, the present invention provides a grinding and spraying path planning method based on three-dimensional generative reconstruction, the grinding and spraying path planning method based on three-dimensional generative reconstruction comprising the following steps: The camera mounted on the end of the robotic arm is used to scan the area to be repaired on the workpiece surface from multiple angles to obtain sparse point cloud data; The sparse point cloud data is reconstructed using a three-dimensional Gaussian splashing algorithm. During the optimization process, normal consistency constraints and opacity regularization are introduced to suppress high reflectivity interference on the workpiece surface, and a three-dimensional mesh model of the area to be repaired is generated. A trajectory manifold consistency model is constructed. A manifold topological feature vector is extracted from the three-dimensional mesh model through a graph convolutional network. A manifold projection operator is defined based on the three-dimensional mesh model. The manifold topological feature vector is used to reflect the local curvature and topological connectivity of the surface. The manifold projection operator is used to constrain spatial points to the surface. In each sampling iteration, the current noise state, time step, and manifold topological feature vector are used as inputs. The trajectory point at the next moment is predicted using the trajectory manifold consistency model. The manifold projection operator is called to perform geometric correction on the predicted trajectory point, and the corrected trajectory point is projected onto the surface of the three-dimensional mesh model. During trajectory generation, a gradient guidance term based on a safety potential field function is injected, wherein the gradient guidance term is used to guide the trajectory point to move towards the target area of ​​the robotic arm when the trajectory point is updated at each step. The uncertainty of the trajectory generated through iteration is quantitatively evaluated. If the evaluation result exceeds the preset threshold, a safety shutdown mechanism is triggered. If the evaluation result does not exceed the preset threshold, the generated trajectory is sent to the robot controller to perform the operation.

[0006] In one embodiment, the process of generating the three-dimensional mesh model includes: Initialize a set of three-dimensional Gaussian spheres, each of which is defined by its center position, covariance matrix, opacity, and spherical harmonic coefficients; The three-dimensional Gaussian sphere is projected onto a two-dimensional image plane using a differentiable rasterizer, and the photometric error between the rendered image and the actual acquired image is calculated for backpropagation optimization. During the backpropagation optimization process, the normal consistency constraint is applied to penalize the deviation between the predicted normal and the geometric normal, and the opacity regularization is applied to eliminate spatial outliers to generate a three-dimensional mesh model.

[0007] In one embodiment, the step of extracting manifold topological feature vectors from the 3D mesh model using a graph convolutional network and defining a manifold projection operator based on the 3D mesh model includes: A dynamic graph convolutional network is used to encode the vertex and edge structure of the three-dimensional mesh model, extract the dense manifold feature field reflecting the local curvature and topological connectivity of the surface, and output the manifold topological feature vector. Construct the manifold projection operator, which is defined as follows:

[0008] in, For manifold projection operators, The current trajectory point to be corrected. From the current trajectory point to the reconstructed surface The signed distance field, For the signed distance field at point The gradient vector at that point, This is the projection step size adjustment coefficient.

[0009] In one embodiment, the method further includes: In each sampling iteration, after the prediction of the trajectory manifold consistency model is completed, the prediction result is immediately input to the manifold projection operator. The trajectory points are geometrically corrected in real time by updating the trajectory points predicted by the trajectory manifold consistency model with the output value of the manifold projection operator. The real-time geometric correction is used to eliminate geometric drift errors in the generation process.

[0010] In one embodiment, the injection of a gradient-guided term based on a safety potential function includes: Define a safe potential field function, wherein the gradient of the safe potential field function points to the high operability space and away from the singularity; During the sampling iteration, the trajectory points are updated and guided by the following formula:

[0011] in, These are the trajectory points after safety guidance and correction. Trajectory manifold consistency model, To ensure safe guidance of step size coefficient, For the safety potential field function At the trajectory point The gradient vector at that point.

[0012] In one embodiment, the uncertainty quantification assessment includes: Multiple candidate trajectories are generated through multiple parallel samplings, and the variance of the distribution of the multiple candidate trajectories is calculated. The calculated variance is compared with a preset safety threshold. When the variance exceeds the preset safety threshold, it is determined that the quality of the current 3D mesh model cannot support safe planning, triggering a safety shutdown mechanism.

[0013] Furthermore, to achieve the above objectives, this invention also proposes a grinding and spraying path planning device based on three-dimensional generative reconstruction. This device is applied to the grinding and spraying path planning method based on three-dimensional generative reconstruction described above. The device includes: The sensing and data acquisition module is used to scan the area to be repaired on the workpiece surface from multiple angles using a camera mounted on the end of the robotic arm to acquire sparse point cloud data. The high-fidelity topography reconstruction module is used to perform three-dimensional reconstruction of the sparse point cloud data using a three-dimensional Gaussian splashing algorithm. During the optimization process, normal consistency constraints and opacity regularization are introduced to suppress high reflectivity interference on the workpiece surface and generate a three-dimensional mesh model of the area to be repaired. The feature extraction and operator construction module is used to construct a trajectory manifold consistency model. It extracts manifold topological feature vectors from the three-dimensional mesh model through a graph convolutional network and defines a manifold projection operator based on the three-dimensional mesh model. The manifold topological feature vectors are used to reflect the local curvature and topological connectivity of the surface, and the manifold projection operator is used to constrain spatial points to the surface. The trajectory generation and projection correction module is used to predict the trajectory point at the next moment in each sampling iteration by taking the current noise state, time step and the manifold topological feature vector as input, using the trajectory manifold consistency model, calling the manifold projection operator to perform geometric correction on the predicted trajectory point, and projecting the corrected trajectory point onto the surface of the three-dimensional mesh model. The safety potential field guidance module is used to inject a gradient guidance term based on the safety potential field function during the trajectory generation process. The gradient guidance term is used to guide the trajectory point to move towards the target area of ​​the robotic arm when the trajectory point is updated at each step. The evaluation and safety control module is used to perform uncertainty quantification evaluation on the trajectory generated through iteration. If the evaluation result exceeds the preset threshold, a safety shutdown mechanism is triggered. If the evaluation result does not exceed the preset threshold, the generated trajectory is sent to the robot controller to perform the operation.

[0014] In one embodiment, the process of generating the three-dimensional mesh model includes: Initialize a set of three-dimensional Gaussian spheres, each of which is defined by its center position, covariance matrix, opacity, and spherical harmonic coefficients; The three-dimensional Gaussian sphere is projected onto a two-dimensional image plane using a differentiable rasterizer, and the photometric error between the rendered image and the actual acquired image is calculated for backpropagation optimization. During the backpropagation optimization process, the normal consistency constraint is applied to penalize the deviation between the predicted normal and the geometric normal, and the opacity regularization is applied to eliminate spatial outliers to generate a three-dimensional mesh model.

[0015] Furthermore, to achieve the above objectives, the present invention also proposes a grinding and spraying path planning device based on three-dimensional generative reconstruction. The grinding and spraying path planning device based on three-dimensional generative reconstruction includes: a memory, a processor, and a grinding and spraying path planning program based on three-dimensional generative reconstruction stored in the memory and executable on the processor. The grinding and spraying path planning program based on three-dimensional generative reconstruction is configured to implement the steps of the grinding and spraying path planning method based on three-dimensional generative reconstruction as described above.

[0016] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a grinding and spraying path planning program based on three-dimensional generative reconstruction. When the grinding and spraying path planning program based on three-dimensional generative reconstruction is executed by a processor, it implements the steps of the grinding and spraying path planning method based on three-dimensional generative reconstruction as described above.

[0017] This invention utilizes a robotic arm's end-effector camera to acquire sparse point clouds through multi-angle scanning; employs a 3D Gaussian splashing algorithm, incorporating normal consistency constraints and opacity regularization to suppress high reflectivity during optimization, generating a 3D mesh model; extracts manifold topological features through a graph convolutional network and defines a manifold projection operator; uses noise state, time step, and topological features as inputs, predicts trajectory points using a trajectory manifold consistency model, and calls the projection operator in real time to correct the trajectory points to the model surface; injects a safe potential field gradient guidance term to avoid singularities; performs uncertainty assessment on the generated trajectory, halting if a threshold is exceeded, otherwise proceeding. This invention achieves path generation with high geometric fit, kinematic safety, and real-time dynamic replanning under complex irregular surfaces, significantly improving repair quality. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the grinding and spraying path planning method based on three-dimensional generative reconstruction of the present invention. Figure 2 This is a comparison of the effects of 3DGS high-fidelity reconstruction and reflection suppression in the grinding and spraying path planning method based on three-dimensional generative reconstruction of the present invention. Figure 3 This is a comparative schematic diagram of path planning under complex curved surfaces in the grinding and spraying path planning method based on three-dimensional generative reconstruction of the present invention. Figure 4 This is a schematic diagram illustrating the rapid trajectory generation using a consistency model in the grinding and spraying path planning method based on three-dimensional generative reconstruction of the present invention. Figure 5 This is a schematic diagram of the safety guidance planning based on the operability field and uncertainty in the grinding and spraying path planning method based on three-dimensional generative reconstruction of the present invention. Figure 6 This is a structural block diagram of the first embodiment of the grinding and spraying path planning device based on three-dimensional generative reconstruction of the present invention.

[0019] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0021] This invention provides a grinding and spraying path planning method based on three-dimensional generative reconstruction, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of a grinding and spraying path planning method based on three-dimensional generative reconstruction according to the present invention.

[0022] In this embodiment, the grinding and spraying path planning method based on three-dimensional generative reconstruction includes the following steps: Step S10: Use a camera mounted on the end of the robotic arm to scan the area to be repaired on the workpiece surface from multiple angles to obtain sparse point cloud data.

[0023] In this embodiment, the execution subject is a grinding and spraying path planning device based on three-dimensional generative reconstruction. The grinding and spraying path planning device based on three-dimensional generative reconstruction has functions such as data processing, data communication and program execution. The grinding and spraying path planning device based on three-dimensional generative reconstruction can be a computer terminal device or other network device, or other devices with similar functions. This embodiment does not limit this.

[0024] It should be noted that existing tower repair robots mostly adopt a "blind repair" mode: low perception accuracy, traditional lidar or structured light cannot reconstruct sub-millimeter-level corrosion pit textures under strong metal reflection; rigid planning, teaching-based paths based on 2D projection have severe geometric distortions on variable-diameter curved surfaces, resulting in inconsistent grinding overlap rates; weak safety, existing generative path planning often ignores robot kinematic constraints, easily causing joint impacts or collisions near singularities; poor real-time performance, generative methods such as diffusion models are time-consuming and cannot meet the needs of online dynamic replanning.

[0025] To address the aforementioned technical issues, this embodiment utilizes a robotic arm's end-effector camera to acquire sparse point clouds through multi-angle scanning; employs a 3D Gaussian splashing algorithm, incorporating normal consistency constraints and opacity regularization to suppress high reflectivity during optimization, generating a 3D mesh model; extracts manifold topological features through a graph convolutional network and defines a manifold projection operator; uses noise state, time step, and topological features as inputs, predicts trajectory points using a trajectory manifold consistency model, and calls the projection operator in real-time to correct the trajectory points to the model surface; injects a safe potential field gradient guidance term to avoid singularities; performs uncertainty assessment on the generated trajectory, halting if the threshold is exceeded, otherwise proceeding. This invention achieves path generation with high geometric fit, kinematic safety, and real-time dynamic replanning under complex irregular surfaces, significantly improving repair quality. Specifically, it can be implemented as follows.

[0026] In this specific implementation, the camera is an RGB-D camera consisting of an RGB camera and a depth sensor. The area to be repaired is the surface of the tower. The sparse point cloud data contains color information and global coordinate reference. Specifically, approximately 30 frames of sparse point cloud data with color and depth information are acquired, and data synchronization and pose recording are performed through the Robot Operating System (ROS).

[0027] Step S20: The sparse point cloud data is reconstructed in three dimensions using a three-dimensional Gaussian splashing algorithm. During the optimization process, normal consistency constraints and opacity regularization are introduced to suppress high reflectivity interference on the workpiece surface, and a three-dimensional mesh model of the area to be repaired is generated.

[0028] In a specific implementation, the process of generating a three-dimensional mesh model includes: initializing a set of three-dimensional Gaussian spheres, each of which is defined by its center position, covariance matrix, opacity, and spherical harmonic coefficients; The three-dimensional Gaussian sphere is projected onto a two-dimensional image plane using a differentiable rasterizer, and the photometric error between the rendered image and the real acquired image is calculated for backpropagation optimization. During the backpropagation optimization process, the normal consistency constraint is applied to penalize the deviation between the predicted normal and the geometric normal, and the opacity regularization is applied to eliminate spatial outliers to generate a three-dimensional mesh model.

[0029] It should be noted that approximately 100,000 anisotropic 3D Gaussian spheres are first initialized. Each Gaussian sphere is defined by its center position μ, covariance matrix Σ, opacity α, and 4th-order spherical harmonic coefficients. Splash rendering is performed using a differentiable rasterizer, and the photometric error between the synthesized image and the real image is calculated. During backpropagation, in addition to photometric loss, normal consistency constraints and opacity regularization are introduced to suppress interference from high metallic reflectivity. The normal consistency constraint is defined as follows: ; Among them, constrained photometric normal With geometric normal Consistency. It reflects the probability that the trajectory point is far away from the singularity and the collider. The predicted normal is derived from the spherical harmonic coefficients. The geometric normal is calculated from the local surface neighborhood of the Gaussian sphere. This constraint forces the surface normal of the radiation field to converge with the underlying geometry, effectively eliminating floating artifacts caused by specular reflection. Opacity regularization, by penalizing the opacity of the spatially outlier Gaussian sphere, focuses the model on the vicinity of the real surface. After approximately 1000 iterations on Jetson Orin (taking about 2 seconds), a sub-millimeter-level 3D mesh model generated by Poisson reconstruction is finally derived, with its average geometric error reduced from 0.5mm in traditional methods to 0.08mm.

[0030] Step S30: Construct a trajectory manifold consistency model, extract manifold topological feature vectors from the three-dimensional mesh model through a graph convolutional network, and define a manifold projection operator based on the three-dimensional mesh model.

[0031] In the specific implementation, a dynamic graph convolutional network is used to encode the vertex and edge structure of the three-dimensional mesh model, extract the dense manifold feature field that reflects the local curvature and topological connectivity of the surface, and output the manifold topological feature vector. Construct the manifold projection operator, which is defined as follows:

[0032] in, For manifold projection operators, The current trajectory point to be corrected. From the current trajectory point to the reconstructed surface The signed distance field, For the signed distance field at point The gradient vector at that point, This is the projection step size adjustment coefficient.

[0033] Step S40: In each sampling iteration, the current noise state, time step, and manifold topological feature vector are used as inputs. The trajectory point at the next moment is predicted using the trajectory manifold consistency model. The manifold projection operator is called to perform geometric correction on the predicted trajectory point, and the corrected trajectory point is projected onto the surface of the three-dimensional mesh model.

[0034] In the specific implementation, in each sampling iteration, after the prediction of the trajectory manifold consistency model is completed, the prediction result is immediately input to the manifold projection operator. The trajectory points are geometrically corrected in real time by updating the trajectory points predicted by the trajectory manifold consistency model with the output value of the manifold projection operator. The real-time geometric correction is used to eliminate geometric drift errors in the generation process.

[0035] It should be noted that the trajectory generation process employs a consistency model framework. In the consistency sampling step, an execution is performed after each update. This ensures that the trajectory always conforms to the manifold surface. The total number of sampling steps N is only 2 steps, and the time for generating a single trajectory is reduced to 75ms, enabling real-time dynamic replanning.

[0036] Step S50: During trajectory generation, a gradient guiding term based on the safety potential field function is injected.

[0037] In the specific implementation, a gradient-guided term based on the safety potential field function is injected, including: Define a safe potential field function, wherein the gradient of the safe potential field function points to the high operability space and away from the singularity; During the sampling iteration, the trajectory points are updated and guided by the following formula:

[0038] in, These are the trajectory points after safety guidance and correction. Trajectory manifold consistency model, To ensure safe guidance of step size coefficient, For the safety potential field function At the trajectory point The gradient vector at that location. The comparison of path planning under complex surfaces in this embodiment can be referenced. Figure 3 As shown, and the schematic diagram of fast trajectory generation using the consistency model, can be referred to. Figure 4 As shown.

[0039] Step S60: Perform uncertainty quantification evaluation on the trajectory generated through iteration. If the evaluation result exceeds the preset threshold, trigger the safety shutdown mechanism. If the evaluation result does not exceed the preset threshold, send the generated trajectory to the robot controller to perform the operation.

[0040] In specific implementation, uncertainty quantification assessment includes generating multiple candidate trajectories through multiple parallel samplings, calculating the variance of the distribution of the multiple candidate trajectories, comparing the calculated variance with a preset safety threshold, and determining that the quality of the current 3D mesh model cannot support safe planning when the variance is greater than the preset safety threshold, triggering a safety shutdown mechanism.

[0041] It should be noted that a Monte Carlo parallel sampling strategy is employed, using the same initial noisy state to generate trajectory points M=20 times, resulting in the trajectory point distribution. The variance σ of each trajectory point is then calculated. 2 If the maximum variance of the joint space exceeds the preset threshold σ limit If the value is 0.015 (normalized value), it is determined that the reconstruction quality of the current 3D mesh model is insufficient to support safety planning due to extreme lighting or other reasons, and the system immediately triggers degradation protection and shutdown alarm; otherwise, the trajectory data is converted into robot motion commands for execution. In this embodiment, the safety guidance planning based on the operability field and uncertainty can refer to... Figure 5 As shown.

[0042] In this embodiment, a sparse point cloud is acquired by multi-angle scanning using a camera at the end effector of a robotic arm. A 3D Gaussian splashing algorithm is employed, incorporating normal consistency constraints and opacity regularization to suppress high reflectivity during optimization, generating a 3D mesh model. Manifold topological features are extracted using a graph convolutional network, and a manifold projection operator is defined. Using noise state, time step, and topological features as input, the trajectory points are predicted using the trajectory manifold consistency model, and the projection operator is called in real time to correct the trajectory points to the model surface. A safe potential field gradient guidance term is injected to avoid singularities. Uncertainty assessment is performed on the generated trajectory; if the uncertainty exceeds a threshold, the process stops; otherwise, execution is initiated. This invention achieves path generation with high geometric fit, kinematic safety, and real-time dynamic replanning under complex irregular surfaces, significantly improving the repair quality.

[0043] Furthermore, this embodiment of the invention also proposes a storage medium storing a grinding and spraying path planning program based on three-dimensional generative reconstruction. When the grinding and spraying path planning program based on three-dimensional generative reconstruction is executed by a processor, it implements the steps of the grinding and spraying path planning method based on three-dimensional generative reconstruction described above.

[0044] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the grinding and spraying path planning device based on three-dimensional generative reconstruction of the present invention.

[0045] like Figure 6 As shown, the grinding and spraying path planning device based on three-dimensional generative reconstruction proposed in this embodiment of the invention includes: The sensing and data acquisition module 10 is used to scan the area to be repaired on the surface of the workpiece from multiple angles using a camera mounted on the end of the robotic arm to acquire sparse point cloud data. The high-fidelity topography reconstruction module 20 is used to perform three-dimensional reconstruction of the sparse point cloud data using a three-dimensional Gaussian splashing algorithm. During the optimization process, normal consistency constraints and opacity regularization are introduced to suppress high reflectivity interference on the workpiece surface and generate a three-dimensional mesh model of the area to be repaired. The feature extraction and operator construction module 30 is used to construct a trajectory manifold consistency model. It extracts manifold topological feature vectors from the three-dimensional mesh model through a graph convolutional network and defines a manifold projection operator based on the three-dimensional mesh model. The manifold topological feature vectors are used to reflect the local curvature and topological connectivity of the surface, and the manifold projection operator is used to constrain spatial points to the surface. The trajectory generation and projection correction module 40 is used to predict the trajectory point at the next moment using the current noise state, time step and the manifold topological feature vector as input in each sampling iteration, call the manifold projection operator to perform geometric correction on the predicted trajectory point, and project the corrected trajectory point onto the surface of the three-dimensional mesh model. The safety potential field guidance module 50 is used to inject a gradient guidance term based on the safety potential field function during the trajectory generation process. The gradient guidance term is used to guide the trajectory point to move towards the target area of ​​the robotic arm when the trajectory point is updated at each step. The evaluation and safety control module 60 is used to perform uncertainty quantification evaluation on the trajectory generated through iteration. If the evaluation result exceeds the preset threshold, a safety shutdown mechanism is triggered. If the evaluation result does not exceed the preset threshold, the generated trajectory is sent to the robot controller to perform the operation.

[0046] In one embodiment, the process of generating the three-dimensional mesh model includes: Initialize a set of three-dimensional Gaussian spheres, each of which is defined by its center position, covariance matrix, opacity, and spherical harmonic coefficients; The three-dimensional Gaussian sphere is projected onto a two-dimensional image plane using a differentiable rasterizer, and the photometric error between the rendered image and the actual acquired image is calculated for backpropagation optimization. During the backpropagation optimization process, the normal consistency constraint is applied to penalize the deviation between the predicted normal and the geometric normal, and the opacity regularization is applied to eliminate spatial outliers to generate a three-dimensional mesh model.

[0047] In this embodiment, a sparse point cloud is acquired by multi-angle scanning using a camera at the end effector of a robotic arm. A 3D Gaussian splashing algorithm is employed, incorporating normal consistency constraints and opacity regularization to suppress high reflectivity during optimization, generating a 3D mesh model. Manifold topological features are extracted using a graph convolutional network, and a manifold projection operator is defined. Using noise state, time step, and topological features as input, the trajectory points are predicted using the trajectory manifold consistency model, and the projection operator is called in real time to correct the trajectory points to the model surface. A safe potential field gradient guidance term is injected to avoid singularities. Uncertainty assessment is performed on the generated trajectory; if the uncertainty exceeds a threshold, the process stops; otherwise, execution is initiated. This invention achieves path generation with high geometric fit, kinematic safety, and real-time dynamic replanning under complex irregular surfaces, significantly improving the repair quality.

[0048] This application embodiment also provides a grinding and spraying path planning device based on three-dimensional generative reconstruction, including a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other through the communication bus. The memory is used to store the grinding and spraying path planning program based on three-dimensional generative reconstruction. When the processor executes the program stored in the memory, it implements the above-mentioned grinding and spraying path planning method based on three-dimensional generative reconstruction.

[0049] The communication bus mentioned in the above-mentioned grinding and spraying path planning equipment based on 3D generative reconstruction can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0050] The communication interface is used for communication between the aforementioned grinding and spraying path planning equipment based on 3D generative reconstruction and other equipment.

[0051] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0052] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0053] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0054] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0055] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0056] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0057] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0058] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0059] In addition, for technical details not described in detail in this embodiment, please refer to the grinding and spraying path planning method based on three-dimensional generative reconstruction provided in any embodiment of the present invention, which will not be repeated here.

[0060] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0061] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0063] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

[0064] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

Claims

1. A grinding and spraying path planning method based on three-dimensional generative reconstruction, characterized in that, The grinding and spraying path planning method based on three-dimensional generative reconstruction includes: The camera mounted on the end of the robotic arm is used to scan the area to be repaired on the workpiece surface from multiple angles to obtain sparse point cloud data; The sparse point cloud data is reconstructed using a three-dimensional Gaussian splashing algorithm. During the optimization process, normal consistency constraints and opacity regularization are introduced to suppress high reflectivity interference on the workpiece surface, and a three-dimensional mesh model of the area to be repaired is generated. A trajectory manifold consistency model is constructed. A manifold topological feature vector is extracted from the three-dimensional mesh model through a graph convolutional network. A manifold projection operator is defined based on the three-dimensional mesh model. The manifold topological feature vector is used to reflect the local curvature and topological connectivity of the surface. The manifold projection operator is used to constrain spatial points to the surface. In each sampling iteration, the current noise state, time step, and manifold topological feature vector are used as inputs. The trajectory point at the next moment is predicted using the trajectory manifold consistency model. The manifold projection operator is called to perform geometric correction on the predicted trajectory point, and the corrected trajectory point is projected onto the surface of the three-dimensional mesh model. During trajectory generation, a gradient guidance term based on a safety potential field function is injected, wherein the gradient guidance term is used to guide the trajectory point to move towards the target area of ​​the robotic arm when the trajectory point is updated at each step. The uncertainty of the trajectory generated through iteration is quantitatively evaluated. If the evaluation result exceeds the preset threshold, a safety shutdown mechanism is triggered. If the evaluation result does not exceed the preset threshold, the generated trajectory is sent to the robot controller to perform the operation.

2. The grinding and spraying path planning method based on three-dimensional generative reconstruction as described in claim 1, characterized in that, The process of generating the three-dimensional mesh model includes: Initialize a set of three-dimensional Gaussian spheres, each of which is defined by its center position, covariance matrix, opacity, and spherical harmonic coefficients; The three-dimensional Gaussian sphere is projected onto a two-dimensional image plane using a differentiable rasterizer, and the photometric error between the rendered image and the actual acquired image is calculated for backpropagation optimization. During the backpropagation optimization process, the normal consistency constraint is applied to penalize the deviation between the predicted normal and the geometric normal, and the opacity regularization is applied to eliminate spatial outliers to generate a three-dimensional mesh model.

3. The grinding and spraying path planning method based on three-dimensional generative reconstruction as described in claim 1, characterized in that, The step of extracting manifold topological feature vectors from the 3D mesh model using a graph convolutional network and defining a manifold projection operator based on the 3D mesh model includes: A dynamic graph convolutional network is used to encode the vertex and edge structure of the three-dimensional mesh model, extract the dense manifold feature field reflecting the local curvature and topological connectivity of the surface, and output the manifold topological feature vector. Construct the manifold projection operator, which is defined as follows: in, For manifold projection operators, The current trajectory point to be corrected. From the current trajectory point to the reconstructed surface The signed distance field, For the signed distance field at point The gradient vector at that point, This is the projection step size adjustment coefficient.

4. The grinding and spraying path planning method based on three-dimensional generative reconstruction as described in claim 1, characterized in that, The method further includes: In each sampling iteration, after the prediction of the trajectory manifold consistency model is completed, the prediction result is immediately input to the manifold projection operator. The trajectory points are geometrically corrected in real time by updating the trajectory points predicted by the trajectory manifold consistency model with the output value of the manifold projection operator. The real-time geometric correction is used to eliminate geometric drift errors in the generation process.

5. The grinding and spraying path planning method based on three-dimensional generative reconstruction as described in claim 1, characterized in that, The injected gradient-guided term based on the safe potential field function includes: Define a safe potential field function, wherein the gradient of the safe potential field function points to the high operability space and away from the singularity; During the sampling iteration, the trajectory points are updated and guided by the following formula: in, These are the trajectory points after safety guidance and correction. Trajectory manifold consistency model, To ensure safe guidance of step size coefficient, For the safety potential field function At the trajectory point The gradient vector at that point.

6. The grinding and spraying path planning method based on three-dimensional generative reconstruction as described in claim 1, characterized in that, The uncertainty quantification assessment includes: Multiple candidate trajectories are generated through multiple parallel samplings, and the variance of the distribution of the multiple candidate trajectories is calculated. The calculated variance is compared with a preset safety threshold. When the variance exceeds the preset safety threshold, it is determined that the quality of the current 3D mesh model cannot support safe planning, triggering a safety shutdown mechanism.

7. A grinding and spraying path planning device based on three-dimensional generative reconstruction, characterized in that, The grinding and spraying path planning device based on three-dimensional generative reconstruction is applied to the grinding and spraying path planning method based on three-dimensional generative reconstruction as described in any one of claims 1 to 6, wherein the device comprises: The sensing and data acquisition module is used to scan the area to be repaired on the workpiece surface from multiple angles using a camera mounted on the end of the robotic arm to acquire sparse point cloud data. The high-fidelity topography reconstruction module is used to perform three-dimensional reconstruction of the sparse point cloud data using a three-dimensional Gaussian splashing algorithm. During the optimization process, normal consistency constraints and opacity regularization are introduced to suppress high reflectivity interference on the workpiece surface and generate a three-dimensional mesh model of the area to be repaired. The feature extraction and operator construction module is used to construct a trajectory manifold consistency model. It extracts manifold topological feature vectors from the three-dimensional mesh model through a graph convolutional network and defines a manifold projection operator based on the three-dimensional mesh model. The manifold topological feature vectors are used to reflect the local curvature and topological connectivity of the surface, and the manifold projection operator is used to constrain spatial points to the surface. The trajectory generation and projection correction module is used to predict the trajectory point at the next moment in each sampling iteration by taking the current noise state, time step and the manifold topological feature vector as input, using the trajectory manifold consistency model, calling the manifold projection operator to perform geometric correction on the predicted trajectory point, and projecting the corrected trajectory point onto the surface of the three-dimensional mesh model. The safety potential field guidance module is used to inject a gradient guidance term based on the safety potential field function during the trajectory generation process. The gradient guidance term is used to guide the trajectory point to move towards the target area of ​​the robotic arm when the trajectory point is updated at each step. The evaluation and safety control module is used to perform uncertainty quantification evaluation on the trajectory generated through iteration. If the evaluation result exceeds the preset threshold, a safety shutdown mechanism is triggered. If the evaluation result does not exceed the preset threshold, the generated trajectory is sent to the robot controller to perform the operation.

8. The grinding and spraying path planning device based on three-dimensional generative reconstruction as described in claim 7, characterized in that, The process of generating the three-dimensional mesh model includes: Initialize a set of three-dimensional Gaussian spheres, each of which is defined by its center position, covariance matrix, opacity, and spherical harmonic coefficients; The three-dimensional Gaussian sphere is projected onto a two-dimensional image plane using a differentiable rasterizer, and the photometric error between the rendered image and the actual acquired image is calculated for backpropagation optimization. During the backpropagation optimization process, the normal consistency constraint is applied to penalize the deviation between the predicted normal and the geometric normal, and the opacity regularization is applied to eliminate spatial outliers to generate a three-dimensional mesh model.

9. A grinding and spraying path planning device based on three-dimensional generative reconstruction, characterized in that, The grinding and spraying path planning device based on three-dimensional generative reconstruction includes: a memory, a processor, and a grinding and spraying path planning program based on three-dimensional generative reconstruction stored in the memory and executable on the processor. The grinding and spraying path planning program based on three-dimensional generative reconstruction is configured to implement the steps of the grinding and spraying path planning method based on three-dimensional generative reconstruction as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a grinding and spraying path planning program based on three-dimensional generative reconstruction. When the grinding and spraying path planning program based on three-dimensional generative reconstruction is executed by the processor, it implements the steps of the grinding and spraying path planning method based on three-dimensional generative reconstruction as described in any one of claims 1 to 7.

Images