Reinforcement learning 3D printing method of multi-degree-of-freedom space

Abstract

A reinforcement learning 3D printing method of a multi-degree-of-freedom space mainly comprises the steps that 1, a three-dimensional model is prepared, scanning data is converted into a voxelization mesh model, and mesh optimization and voxelization processing are conducted; 2, path planning is conducted, specifically, a multi-degree-of-freedom printing path is generated through a space continuous path planning method, and collision and motion range limitation of a mechanical arm is added; 3, learning model training is reinforced, reward functions including printing quality, printing efficiency and mechanical arm movement stability are designed, data are used for model training, and the printing effect is improved through continuous learning and printing parameter optimization; and step 4, system integration and testing: all the parts are integrated, and the printing quality and efficiency are evaluated through printing testing. According to the method, the defects of a traditional path planning method are effectively overcome, and the printing quality and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of 3D printing, and in particular to a reinforcement learning 3D printing method in a multi-degree-of-freedom space. Background Art

[0002] With the continuous development of science and technology, 3D printing technology has been widely used in industrial manufacturing, medical treatment, construction and other fields. There are two main types of traditional 3D printing path planning methods: slice path planning and continuous path planning. The slice path planning method slices the three-dimensional model into two-dimensional layers and generates paths between layers, but its path is not smooth enough and the printing speed is slow. The continuous path planning method generates a continuous printing path with a faster printing speed, but the path planning process is complicated and it is difficult to ensure the printing quality.

[0003] In addition, traditional path planning methods often fail to balance printing quality and efficiency. For example, to improve printing quality, the layer spacing and printing speed need to be reduced, but this will lead to longer printing time and increased material consumption; to improve printing efficiency, the layer spacing and printing speed need to be increased, but this will lead to reduced printing quality. Therefore, how to balance printing quality and efficiency has become the main challenge facing 3D printing path planning. Summary of the invention

[0004] To solve the above problems, the present invention proposes a reinforcement learning 3D printing method in a multi-degree-of-freedom space. Reinforcement learning is a machine learning method that achieves the optimization of the goal by continuously learning the optimization strategy. The present invention applies reinforcement learning technology to 3D printing path planning, designs a comprehensive reward function for printing quality, printing efficiency and robot arm motion stability, and uses data for model training. Through continuous learning, the intelligent agent can adjust the printing parameters according to the printing results to achieve the optimization of the printing effect. The specific steps are as follows, which are characterized by:

[0005] Step 1: 3D model preparation: convert the scan data into a voxelized mesh model, and perform mesh optimization and voxelization;

[0006] Step 2: Path planning: Generate a multi-degree-of-freedom printing path using a spatial continuous path planning method and add collision and motion range restrictions for the robot arm;

[0007] Step 3: Reinforce learning model training, design reward function, and use data for model training;

[0008] Step 4: System integration and testing: Integrate all parts of the system and evaluate the printing quality and efficiency through printing tests.

[0009] The present invention provides a reinforcement learning 3D printing method in a multi-degree-of-freedom space, and the beneficial effects are:

[0010] 1. The present invention designs a comprehensive reward function, which can balance printing quality and efficiency and optimize printing effects. The intelligent agent continuously learns and adjusts printing parameters according to the printing results, which can improve printing speed and efficiency and reduce material consumption while ensuring printing quality.

[0011] 2. The present invention adopts a spatial continuous path planning method, and the generated printing path is smooth and continuous, avoiding sharp edges and too small patches, thereby improving printing accuracy and surface finish.

[0012] 3. The present invention can dynamically adjust printing parameters according to different printing requirements and model characteristics, and has strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of the present invention;

[0014] Figure 2 It is a schematic diagram of the composition of the multi-degree-of-freedom 3D printing platform of the present invention. DETAILED DESCRIPTION

[0015] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0016] The present invention proposes a reinforcement learning 3D printing method in a multi-degree-of-freedom space, and the invention flow chart is as follows: Figure 1 As shown, the steps of the present invention are described in detail below.

[0017] Step 1: 3D model preparation: convert the scan data into a voxelized mesh model, and perform mesh optimization and voxelization;

[0018] Step 1.1, data conversion, convert the scanned point cloud data into STL format, fit the point cloud data into a surface, and generate an STL format file, which consists of a series of triangular facets. The surface shape of the object is described by the vertex coordinates and normal vectors of the triangular facets. For each triangular facet, three vertex coordinates (x 1 ,y 1 ,z 1 ), (x 2 ,y 2 ,z 2 ), (x 3 ,y 3 ,z 3 ) and the corresponding normal vector (n x ,n y ,n z ) to indicate that n x 、n y 、n z are the components of the normal vector in the x, y, and z directions respectively;

[0019] Step 1.2, set the size of the voxel in the voxel model. The size formula is as follows:

[0020] v=k×min(p,f)

[0021] Among them, v is the voxel side length, p is the printing accuracy, f is the minimum feature size of the model, and k is the adjustment coefficient;

[0022] Step 1.3, hole repair, use the hole filling algorithm to fill the holes in the STL mesh, by identifying the triangular facets at the hole boundary, generating new triangular facets on the boundary to fill the hole. Assume that the boundary of the hole is composed of n triangular facets, and their vertex coordinates are (x b1 ,y b1 ,z b1 ), (x b2 ,y b2 ,z b2 ),…,(x bn ,y bn ,z bn ), the subscript b indicates the boundary, 1, 2, ..., n is used to distinguish different vertices, calculate the geometric center of these vertices, and generate new triangles based on the center to fill the holes. The geometric center calculation formula is as follows:

[0023]

[0024] Among them, (x c ,y c ,z c ) is the geometric center coordinate of the hole, and the subscript i is a variable used to distinguish different vertices;

[0025] Step 1.4, patch optimization, evaluates the uniformity of patch distribution by calculating parameters such as patch area and angle to avoid sharp edges and patches that are too small;

[0026] Step 1.5, optimize the mesh and use the mesh smoothing algorithm to smooth the mesh surface. For each vertex v in the mesh i , the new position after smoothing is:

[0027]

[0028] in, is the vertex v i The new position after the mesh smoothing algorithm is processed, w j is the weight of adjacent vertices, and the subscript j is a variable used to distinguish different vertices;

[0029] Step 1.6, network refinement, refine the mesh, re-divide the mesh into smaller tetrahedrons, increase the mesh density, for a larger tetrahedron, divide it into smaller tetrahedrons by connecting the midpoints of its edges; interpolate between two adjacent tetrahedral mesh units, insert new vertices and tetrahedral units, and increase the mesh density;

[0030] Step 1.7, voxelization, uses a tetrahedral mesh generation algorithm to convert the STL mesh into a tetrahedral mesh.

[0031] Step 2: Path planning: Generate a multi-degree-of-freedom printing path using a spatial continuous path planning method and add collision and motion range restrictions for the robot arm;

[0032] Step 2.1, processing constraints, set the angle range of the nozzle collision area to θ collision , the angle between the nozzle at the end of the robot arm and the vertical z-axis is θ robot The best surface quality can be obtained when the nozzle is perpendicular to the printing surface. The best angle for printing the object surface is:

[0033] θ ideal =90°-(90°-θ robot )=θ robot

[0034] In order to avoid nozzle collision, the maximum angle θ allowed is defined in practical applications. max for:

[0035] θ max =90°+θ robot -θ collision

[0036] θ collision Limited to 45°~60°, θ robot Limited to 0°~45°; the diameter of the nozzle outlet is d N , the repeatability error of the nozzle is e, then the layer spacing d is set at 2e-0.8d N within the scope;

[0037] Step 2.2, select the deformation area and transform all the angles in the object model below the maximum angle θ max The surface and its internal voxels are added to the flattened deformation area. Let the angle between a triangular facet on the model surface and the horizontal plane be θ. When θ<θ max When , the patch and its internal voxels are selected for flattening deformation;

[0038] Step 2.3, flattening deformation algorithm, appropriately deform the voxelized grid, and deform the relatively flat surface area into a horizontal surface suitable for planar path planning; let the current iteration number be i, use the iterative algorithm to horizontally deform the patch in the deformed area, define the mapping M as the deformation field of the voxel grid v, and store the new position h(p) of each vertex p in v after deformation in M;

[0039] Step 2.3.1: The result of the previous iteration h i-1 (p) as input;

[0040] Step 2.3.2, traverse the voxel grid h i-1 (p) surface triangles, using Denotes the surface area that meets the flattening deformation condition, using represents the remaining surfaces;

[0041] Step 2.3.3, use the objective function to The patch in is deformed horizontally. The facets in the mesh are then deformed accordingly to ensure the integrity of the model and obtain the new mesh vertex position h i (p); In the function, if all the patches in the flattening are forced, it will usually lead to an unsolvable problem, so only The flattening objective is set as a soft constraint with high weight;

[0042] Step 2.3.4, Detection and its adjacent triangular faces. If all faces are horizontal, stop the iteration and replace them with h i (p) As a result, if there are still patches that are not horizontal, the next iteration is performed. If any surface of the input model does not meet the flattening requirements, the vertex position of the original voxel grid is the iteration result h 0 (p);

[0043] Step 2.4, continuous path generation, select Fermat spiral as the filling method, reduce path segmentation, and make the printing start point and end point close to each other; let the outer contour of the jth area in the i-th slice be r i,j,0 , offset the distance d inwards to form a new contour r i,j,1 , repeat this operation to obtain a series of closed paths r i,j,k ; Each layer of closed paths is stored in a tree-like manner. The closed paths are indexed layer by layer from the outside to the inside, and divided into odd circles and even circles; find the position where the feature changes less in each layer of the path, and determine a point at this position as the connection point connecting adjacent contours; disconnect the path of this layer at this point, and connect the adjacent paths inward to establish a continuous path;

[0044] Step 2.5, use the deformation field to transform the planar continuous path into a multi-degree-of-freedom printing path in space. Suppose the surface contour of the initial model is Ω, and the vertex of the tetrahedron voxel inside it is p i When the surface contour is deformed, the vertices are also displaced. By comparing the positions of the vertices of the tetrahedral mesh before deformation, the position mapping relationship of the flattened deformation, that is, the deformation field M, can be obtained. The inverse mapping of the flattened deformation operation is defined as M -1 , then perform an inverse mapping operation M on the existing plane continuous path P -1 (P), the desired continuous path in space can be obtained;

[0045] Step 2.6, through linear interpolation, map the position of the path point in the original model with the position in the deformed model; after continuous path planning, the printing path can be obtained; since deformation may occur at any position in the printing path, in order to ensure the accuracy of deformation, the sampling rate is set to the nozzle diameter, and the planar printing path after plane slicing is resampled; each sampling point must be located inside or at the boundary of a tetrahedron in the flattened voxel grid; after determining the tetrahedron where the path point is located, linear interpolation is performed according to the vertex position in the corresponding tetrahedron in the original model to obtain the spatial position of the transformed path point;

[0046] Step 2.7, generate support structure based on the STL mesh of the model;

[0047] Step 2.7.1, find all downward-facing triangular facets based on the STL mesh of the model, and then combine these surfaces to form the upper surface of the support structure. The upper surface of the support structure can be determined by the point coordinates of the triangular facet vertex coordinates;

[0048] Step 2.7.2, extract the spatial contour of the outermost edge of the upper surface of the support, project it vertically downward, and construct the side surface of the support structure;

[0049] Step 2.7.3, use the plane where the hot bed is located as the bottom surface of the support structure to form a closed support structure model;

[0050] In step 2.8, a dense grid structure is used on the top surface of the support to ensure the surface quality of the first layer, but this may make the support difficult to peel off and increase the difficulty of post-processing; a sparse grid structure is used in the middle of the support to increase the printing speed and reduce the use of materials. Its pattern is consistent with the conventional 3D printing filling pattern;

[0051] Step 2.9, by setting a reasonable lifting distance between the model and the bottom printing plane, the flatness error of the hot bed can be eliminated during the support printing process, improving the printing effect of the support and model.

[0052] Step 3: Reinforce learning model training, design reward function, and use data for model training;

[0053] Step 3.1, design a comprehensive print quality reward function R quality :

[0054] R quality =k 1 p+k 2 s+k 3 u

[0055] Among them, p is the printing accuracy, s is the surface finish, u is the support-free printing capability, and k 1 , k 2 and k 3 They are the weight coefficients of printing accuracy, surface finish, and support-free printing capability;

[0056] Step 3.2, design printing efficiency reward function R efficiency :

[0057]

[0058] Where T is the expected printing time, M is the expected material consumption, t is the actual printing time, m is the actual material consumption, and k 4 and k 5 are the weight coefficients of printing time and material consumption respectively;

[0059] Step 3.3, design the stability reward function R for the robot arm motion and extruder control stability :

[0060] R stability =k 6 (1-d arm )+k 7 (1-v arm )+k 8 (1-a arm )+k 9 (1-d extruder )+k 10 (1-v extruder )

[0061] Among them, d arm is the deviation of the robot's motion trajectory, v arm is the fluctuation of the robot arm's motion speed, a arm is the fluctuation of the robot arm's motion acceleration, d extruder is the extruder temperature fluctuation, v extruder is the fluctuation of extruder speed, k 6 , k 7 , k 8 , k 9and k 10 They are weight coefficients of robot motion trajectory deviation, robot motion speed fluctuation, robot motion acceleration fluctuation, extruder temperature fluctuation, and extruder speed fluctuation;

[0062] Step 3.4, define the printing parameters of layer thickness, printing speed, filling density, and temperature as the states of reinforcement learning;

[0063] Step 3.5, increasing or decreasing the layer thickness and changing the printing speed are defined as reinforcement learning actions;

[0064] Step 3.6, set the initial printing parameters as the state, initialize the strategy π and the value function Q;

[0065] Step 3.7, select an action a according to the current state s and strategy π;

[0066] Step 3.8, the agent performs action a, the environment adjusts the printing parameters according to the action, and performs 3D printing;

[0067] Step 3.9, calculate the reward based on the printed result;

[0068] Step 3.10, the environment moves to a new state according to the action performed;

[0069] Step 3.11, use the Bellman equation to update the value function Q and policy π.

[0070] Step 4: System integration and testing: Integrate all parts of the system and evaluate the printing quality and efficiency through printing tests. The schematic diagram of the multi-DOF 3D printing platform is shown in the figure. Figure 2 shown.

[0071] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A reinforcement learning 3D printing method in a multi-degree-of-freedom space, the specific steps are as follows, characterized in that: Step 1: 3D model preparation: convert the scan data into a voxelized mesh model, and perform mesh optimization and voxelization; Step 2: Path planning: Generate a multi-degree-of-freedom printing path using a spatial continuous path planning method and add collision and motion range restrictions for the robot arm; Step 3: Reinforce learning model training, design reward function, and use data for model training; Step 4: System integration and testing: Integrate all parts of the system and evaluate the printing quality and efficiency through printing tests.

2. The method for 3D printing in a multi-degree-of-freedom space according to claim 1, characterized in that: The process of 3D model preparation in step 1 is expressed as: Step 1.1, data conversion, convert the scanned point cloud data into STL format, fit the point cloud data into a surface, and generate an STL format file, which consists of a series of triangular facets. The surface shape of the object is described by the vertex coordinates and normal vectors of the triangular facets. For each triangular facet, three vertex coordinates (x1, y1, z1), (x2, y2, z2), (x3, y3, z3) and the corresponding normal vector (n x ,n y ,n z ) to indicate that n x 、n y 、n z are the components of the normal vector in the x, y, and z directions respectively; Step 1.2, set the size of the voxel in the voxel model. The size formula is as follows: v=k×min(p,f) Among them, v is the voxel side length, p is the printing accuracy, f is the minimum feature size of the model, and k is the adjustment coefficient; Step 1.3, hole repair, use the hole filling algorithm to fill the holes in the STL mesh, by identifying the triangular facets at the hole boundary, generating new triangular facets on the boundary to fill the hole. Assume that the boundary of the hole is composed of n triangular facets, and their vertex coordinates are (x b1 ,y b1 ,z b1 ), (x b2 ,y b2 ,z b2 ),…,(x bn ,y bn ,z bn ), the subscript b indicates the boundary, 1, 2, ..., n is used to distinguish different vertices, calculate the geometric center of these vertices, and generate new triangles based on the center to fill the holes. The geometric center calculation formula is as follows: Among them, (x c ,y c ,z c ) is the geometric center coordinate of the hole, and the subscript i is a variable used to distinguish different vertices; Step 1.4, patch optimization, evaluates the uniformity of patch distribution by calculating parameters such as patch area and angle to avoid sharp edges and patches that are too small; Step 1.5, optimize the mesh and use the mesh smoothing algorithm to smooth the mesh surface. For each vertex v in the mesh i , the new position after smoothing is: in, is the vertex v i The new position after the mesh smoothing algorithm is processed, w j is the weight of adjacent vertices, and the subscript j is a variable used to distinguish different vertices; Step 1.6, network refinement, refine the mesh, re-divide the mesh into smaller tetrahedrons, increase the mesh density, for a larger tetrahedron, divide it into smaller tetrahedrons by connecting the midpoints of its edges; interpolate between two adjacent tetrahedral mesh units, insert new vertices and tetrahedral units, and increase the mesh density; Step 1.7, voxelization, uses a tetrahedral mesh generation algorithm to convert the STL mesh into a tetrahedral mesh.

3. The method for 3D printing in a multi-degree-of-freedom space according to claim 1, characterized in that: The path planning process in step 2 is shown as follows: Step 2.1, processing constraints, set the angle range of the nozzle collision area to θ collision , the angle between the nozzle at the end of the robot arm and the vertical z-axis is θ robot The best surface quality can be obtained when the nozzle is perpendicular to the printing surface. The best angle for printing the object surface is: i ideal =90°-(90°-θ robot )=θ robot In order to avoid nozzle collision, the maximum angle θ allowed is defined in practical applications. max for: i max =90°+θ robot -θ collision θ collision Limited to 45°~60°, θ robot Limited to 0°~45°; the diameter of the nozzle outlet is d N , the repeatability error of the nozzle is e, then the layer spacing d is set at 2e-0.8d N within the scope; Step 2.2, select the deformation area and transform all the angles in the object model below the maximum angle θ max The surface and its internal voxels are added to the flattened deformation area. Let the angle between a triangular facet on the model surface and the horizontal plane be θ. When θ<θ max When , the patch and its internal voxels are selected for flattening deformation; Step 2.3, flattening deformation algorithm, appropriately deforming the voxelized grid, deforming the relatively flat surface area into a horizontal plane suitable for planar path planning; Assume that the current iteration number is i, use the iterative algorithm to horizontally deform the patch in the deformation area, define the mapping M as the deformation field of the voxel grid v, and store the new position h(p) of each vertex p in v after deformation in M; Step 2.3.1: The result of the previous iteration h i-1 (p) as input; Step 2.3.2, traverse the voxel grid h i-1 (p) surface triangles, using Denotes the surface area that meets the flattening deformation condition, using represents the remaining surfaces; Step 2.3.3, use the objective function to The patch in is deformed horizontally. The facets in the mesh are then deformed accordingly to ensure the integrity of the model and obtain the new mesh vertex position h i (p); In the function, if all the patches in the flattening are forced, it will usually lead to an unsolvable problem, so only The flattening objective is set as a soft constraint with high weight; Step 2.3.4, Detection and its adjacent triangular faces. If all faces are horizontal, stop the iteration and replace them with h i (p) As a result, if there are still patches that are not horizontal, the next iteration is performed. If any surface of the input model does not meet the flattening requirements, the vertex position of the original voxel grid is the iteration result h0(p); Step 2.4, continuous path generation, select Fermat spiral as the filling method, reduce path segmentation, and make the printing start point and end point close to each other; let the outer contour of the jth area in the i-th slice be r i,j,0 , offset the distance d inwards to form a new contour r i,j,1 , repeat this operation to obtain a series of closed paths r i,j,k ; Each layer of closed paths is stored in a tree-like manner. The closed paths are indexed layer by layer from the outside to the inside, and divided into odd circles and even circles; find the position where the feature changes less in each layer of the path, and determine a point at this position as the connection point connecting adjacent contours; disconnect the path of this layer at this point, and connect the adjacent paths inward to establish a continuous path; Step 2.5, use the deformation field to transform the planar continuous path into a multi-degree-of-freedom printing path in space. Suppose the surface contour of the initial model is Ω, and the vertex of the tetrahedron voxel inside it is p i , when the surface contour is deformed, the vertices are also displaced. By comparing the positions of the vertices of the tetrahedral mesh before deformation, the position mapping relationship of the flattened deformation, i.e., the deformation field M, can be obtained; Define the inverse mapping of the flattening deformation operation as M -1 , then perform an inverse mapping operation M on the existing plane continuous path P -1 (P), the desired continuous path in space can be obtained; Step 2.6, mapping the position of the path point in the original model to the position in the deformed model by linear interpolation; After continuous path planning, the printing path can be obtained; since deformation may occur at any position in the printing path, in order to ensure the accuracy of deformation, the sampling rate is set to the nozzle diameter, and the planar printing path after plane slicing is resampled; Each sampling point must be located inside or at the boundary of a tetrahedron in the flattened voxel grid. After determining the tetrahedron where the path point is located, linear interpolation is performed based on the vertex positions in the corresponding tetrahedron in the original model to obtain the spatial position of the transformed path point. Step 2.7, generate support structure based on the STL mesh of the model; Step 2.7.1, find all downward-facing triangular facets based on the STL mesh of the model, and then combine these surfaces to form the upper surface of the support structure. The upper surface of the support structure can be determined by the point coordinates of the triangular facet vertex coordinates; Step 2.7.2, extract the spatial contour of the outermost edge of the upper surface of the support, project it vertically downward, and construct the side surface of the support structure; Step 2.7.3, use the plane where the hot bed is located as the bottom surface of the support structure to form a closed support structure model; In step 2.8, a dense grid structure is used on the top surface of the support to ensure the surface quality of the first layer, but this may make the support difficult to peel off and increase the difficulty of post-processing; a sparse grid structure is used in the middle of the support to increase the printing speed and reduce the use of materials. Its pattern is consistent with the conventional 3D printing filling pattern; Step 2.9, by setting a reasonable lifting distance between the model and the bottom printing plane, the flatness error of the hot bed can be eliminated during the support printing process, improving the printing effect of the support and model.

4. The method for 3D printing in a multi-DOF space according to claim 1, characterized in that: The process of reinforcement learning model training in step 3 is expressed as follows: Step 3.1, design a comprehensive print quality reward function R quality : R quality =k1p+k2s+k3u Among them, p is the printing accuracy, s is the surface finish, u is the support-free printing capability, k1, k2 and k3 are the weight coefficients of printing accuracy, surface finish and support-free printing capability respectively; Step 3.2, design the printing efficiency reward function R efficiency : Where T is the expected printing time, M is the expected material consumption, t is the actual printing time, m is the actual material consumption, k4 and k5 are the weight coefficients of printing time and material consumption respectively; Step 3.3, design the stability reward function R for the robot arm motion and extruder control stability : R stability =k6(1-d arm )+k7(1-v arm )+k8(1-a arm )+k9(1-d extruder )+k 10 (1-in extruder ) Among them, d arm is the deviation of the robot's motion trajectory, v arm is the fluctuation of the robot arm's motion speed, a arm is the fluctuation of the robot arm's motion acceleration, d extruder is the extruder temperature fluctuation, v extruder is the fluctuation of extruder speed, k6, k7, k8, k9 and k 10 They are weight coefficients of robot motion trajectory deviation, robot motion speed fluctuation, robot motion acceleration fluctuation, extruder temperature fluctuation, and extruder speed fluctuation; Step 3.4, define the printing parameters of layer thickness, printing speed, filling density, and temperature as the states of reinforcement learning; Step 3.5, increasing or decreasing the layer thickness and changing the printing speed are defined as reinforcement learning actions; Step 3.6, set the initial printing parameters as the state, initialize the strategy π and the value function Q; Step 3.7, select an action a according to the current state s and strategy π; Step 3.8, the agent performs action a, the environment adjusts the printing parameters according to the action, and performs 3D printing; Step 3.9, calculate the reward based on the printed result; Step 3.10, the environment moves to a new state according to the action performed; Step 3.11, use the Bellman equation to update the value function Q and policy π.

Images