A Reinforcement Learning 3D Printing Method for Multi-Degree-of-Freedom Space

Abstract

A reinforcement learning 3D printing method for a multi-degree-of-freedom space, the main steps include: Step 1, three-dimensional model preparation, converting the scanned data into a voxelized grid model and performing grid optimization and voxelization processing; Step 2, path planning, using a spatial continuous path planning method to generate a printing path with multiple degrees of freedom and adding collision and motion range limitations of the robotic arm; Step 3, reinforcement learning model training, designing a reward function, including printing quality, printing efficiency, and robotic arm motion stability, and using data for model training, and through continuous learning, optimizing the printing parameters to improve the printing effect; Step 4, system integration and testing, integrating each part of the system, and through printing tests, evaluating the printing quality and efficiency. This method effectively solves the deficiencies of traditional path planning methods and improves the printing quality and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of 3D printing, and particularly to a 3D printing method based on reinforcement learning in a multi-degree-of-freedom space. Background Art

[0002] With the continuous development of technology, 3D printing technology has been widely applied in the fields of industrial manufacturing, medical treatment, architecture, etc. Traditional 3D printing path planning methods are mainly divided into two types: slicing path planning and continuous path planning. The slicing path planning method slices a three-dimensional model into two-dimensional layers and generates paths between layers, but its paths are not smooth enough and the printing speed is slow. The continuous path planning method generates continuous printing paths, with a faster printing speed, but the path planning process is complex and it is difficult to ensure the printing quality.

[0003] In addition, traditional path planning methods often cannot balance printing quality and efficiency. For example, in order to improve printing quality, it is necessary to reduce the layer spacing and printing speed, but this will lead to an extended printing time and increased material consumption; in order to improve printing efficiency, it is necessary to increase the layer spacing and printing speed, but this will lead to a decline in printing quality. Therefore, how to balance printing quality and efficiency has become the main challenge faced by 3D printing path planning. Summary of the Invention

[0004] To solve the above problems, the present invention proposes a 3D printing method based on reinforcement learning in a multi-degree-of-freedom space. Reinforcement learning is a machine learning method that optimizes strategies through continuous learning to achieve the optimization of goals. The present invention applies reinforcement learning technology to 3D printing path planning, designs a comprehensive reward function for printing quality, printing efficiency, and the motion stability of the robotic arm, and uses data for model training. Through continuous learning, the intelligent agent can adjust printing parameters according to the printing results to achieve the optimization of printing effects. The specific steps are as follows, and it is characterized in that:

[0005] Step 1, three-dimensional model preparation, converting the scanned data into a voxelized grid model and performing grid optimization and voxelization;

[0006] Step 2, path planning, using a spatial continuous path planning method to generate a printing path with multiple degrees of freedom and adding collision and motion range restrictions for the robotic arm;

[0007] Step 3, reinforcement learning model training, designing a reward function and using data for model training;

[0008] Step 4, system integration and testing, integrating each part of the system and evaluating printing quality and efficiency through printing tests.

[0009] A 3D printing method based on reinforcement learning in a multi-degree-of-freedom space according to the present invention, beneficial effects: The technical effects of the present invention are as follows:

[0010] 1. The present invention designs a comprehensive reward function, and this method can balance printing quality and efficiency to achieve the optimization of printing effects. The agent continuously learns and adjusts printing parameters according to the printing results, and can improve printing speed and efficiency while ensuring printing quality, and reduce material consumption.

[0011] 2. The present invention adopts a spatial continuous path planning method, and the generated printing path is smooth and continuous, avoiding sharp edges and too small patches, thereby improving printing accuracy and surface finish.

[0012] 3. The present invention can dynamically adjust printing parameters according to different printing requirements and model characteristics, and has strong adaptability. Brief Description of the Drawings

[0013] Figure 1 is a flowchart of the present invention;

[0014] Figure 2 is a schematic diagram of the composition of a multi-degree-of-freedom 3D printing platform of the present invention. Detailed Embodiment

[0015] The present invention will be further described in detail below in conjunction with the drawings and specific embodiments:

[0016] The present invention proposes a reinforcement learning 3D printing method in a multi-degree-of-freedom space. The flowchart of the invention is as Figure 1 shown, and the steps of the present invention will be introduced in detail below.

[0017] Step 1, three-dimensional model preparation, converting the scanned data into a voxelized grid model, and performing grid optimization and voxelization;

[0018] Step 1.1, data conversion, converting the scanned point cloud data into STL format, fitting the point cloud data into a surface, and generating an STL format file, which consists of a series of triangular patches. The surface shape of the object is described by the vertex coordinates and normal vectors of the triangular patches. For each triangular patch, it can be represented by three vertex coordinates (x 1 , y 1 , z 1 ), (x 2 , y 2 , z 2 ), (x 3 , y 3 , z 3 ) and the corresponding normal vector (n x , n y , n z ). n x , n y , n z are the components of the normal vector in the x, y, and z axis directions respectively;

[0019] Step 1.2, set the size of the voxels in the voxel model. The size formula is as follows:

[0020] v = k × min(p, f)

[0021] where v is the side length of the voxel, p is the printing precision, f is the minimum feature size of the model, and k is the adjustment coefficient;

[0022] Step 1.3, hole repair. Use the hole filling algorithm to fill the holes in the STL mesh. By identifying the triangular facets on the hole boundary, generate new triangular facets on the boundary to fill the holes. Suppose the boundary of the hole consists of n triangular facets, and their vertex coordinates are (x b1 , y b1 , z b1 ), (x b2 , y b2 , z b2 ), …, (x bn , y bn , z bn ). The subscript b represents the boundary, and 1, 2, …, n are used to distinguish different vertices. Calculate the geometric center of these vertices, and generate new triangular facets based on the center to fill the holes. The geometric center calculation formula is as follows:

[0023]

[0024] where (x c , y c , z c ) is the geometric center coordinate of the hole, and the subscript i is a variable used to distinguish different vertices;

[0025] Step 1.4, facet optimization. Evaluate the uniformity of the facet distribution by calculating parameters such as the area and angle of the facets, and avoid sharp edges and overly small facets;

[0026] Step 1.5, mesh optimization. Use the mesh smoothing algorithm to smooth the mesh surface. For each vertex v i in the mesh, the new position after smoothing is:

[0027]

[0028] where is the new position of vertex v i after being processed by the mesh smoothing algorithm, w j is the weight of the adjacent vertex, and the subscript j is a variable used to distinguish different vertices;

[0029] Step 1.6, Network refinement: refine the mesh by re - dividing the mesh into smaller tetrahedrons to increase the mesh density. For a larger tetrahedron, it is divided into smaller tetrahedrons by connecting the mid - points of its edges; interpolate between two adjacent tetrahedron mesh elements, insert new vertices and tetrahedron elements to increase the mesh density.

[0030] Step 1.7, Voxelization: use the tetrahedron mesh generation algorithm to convert the STL mesh into a tetrahedron mesh.

[0031] Step 2, Path planning: use the spatial continuous path planning method to generate a multi - degree - of - freedom printing path and add collision and motion range limitations of the robotic arm.

[0032] Step 2.1, Processing constraints: Let the angular range of the nozzle collision area be θ collision , the deflection angle of the nozzle at the end of the robotic arm with respect to the vertical z - axis is θ robot , the best surface quality can be obtained when the nozzle is perpendicular to the printing surface. At this time, the best angle of the printed object surface:

[0033] θ ideal = 90°-(90° - θ robot ) = θ robot

[0034] To avoid nozzle collision, in practical applications, the defined maximum allowable angle θ max is:

[0035] θ max = 90°+θ robot -θ collision

[0036] Limit θ collision between 45° and 60°, and limit θ robot between 0° and 45°; the diameter of the nozzle outlet is d N , the repeat positioning error of the nozzle is e, then set the layer spacing d within the range of 2e - 0.8d N ;

[0037] Step 2.2, Select the deformation area: add all the surfaces and their internal voxels in the object model with angles lower than the maximum angle θ max to the flattened deformation area. Let the angle between a certain triangular patch on the model surface and the horizontal plane be θ. When θ < θ max , this patch and its internal voxels are selected for flattening deformation.

[0038] Step 2.3, Flat deformation algorithm: appropriately deform the voxelized grid, and deform the relatively flat surface area into a horizontal plane suitable for planar path planning. Let the current iteration number be i, and use the iterative algorithm to horizontally deform the patches in the deformation area. Define the mapping M as the deformation field of the voxel grid v, and store the new position h(p) of each vertex p in v after deformation in M.

[0039] Step 2.3.1, Take the result h i-1 (p) of the previous iteration as the input.

[0040] Step 2.3.2, Traverse the surface triangular patches of the voxel grid h i-1 (p), and use to represent the surface area that meets the flat deformation condition, and use to represent the remaining surfaces.

[0041] Step 2.3.3, Use the objective function to horizontally deform the patches in , and the patches in are deformed accordingly to ensure the integrity of the model, and obtain the new grid vertex position h i (p); in the function, if flattening all the patches in forced flattening usually leads to unsolvability, so only set the flattening target of as a soft constraint with high weight.

[0042] Step 2.3.4, Detect and its adjacent triangular patches. If all the patches are horizontal planes, stop the iteration and take h i (p) as the result. If there are still patches that are not horizontal planes, perform the next iteration. If any surface of the input model does not meet the flattening requirements, the vertex position of the original voxel grid is the iteration result h 0 (p).

[0043] Step 2.4, Continuous path generation: Select the Fermat spiral as the filling method to reduce path segmentation and make the printing start and end points close to each other. Let in the i-th layer slice, the outer contour of the j-th region be r i,j,0 , and offset it inward by an equal distance d to form a new contour r i,j,1 . Repeat this operation to obtain a series of closed paths r i,j,k ; The closed paths of each layer are stored in a tree structure. Let the closed paths be indexed layer by layer from the outside to the inside, and divide them into two parts: odd circles and even circles; Find the position with less feature change in each layer of the path, and determine a point at this position as the connection point for connecting adjacent contours; Disconnect the path of this layer at this point and connect it inward to the adjacent path to establish a continuous path.

[0044] Step 2.5: Use the deformation field to convert the planar continuous path into a multi-degree-of-freedom printing path in space. Let the surface contour of the initial model be Ω, and the vertices of the tetrahedral voxels inside it be p i , when the surface contour deforms, the vertices also displace accordingly. By comparing with the positions of the vertices of the tetrahedral mesh before deformation, the position mapping relationship of the flattening deformation, i.e., the deformation field M, can be obtained; define the inverse mapping of the flattening deformation operation as M -1 , then perform the inverse mapping operation M -1 (P) on the existing planar continuous path P to obtain the required continuous path in space;

[0045] Step 2.6: Through linear interpolation, map the positions of the path points in the original model to the positions in the deformed model; after continuous path planning, the printing path can be obtained; since deformation may occur at any position in the printing path, to ensure accurate deformation, set the sampling rate to the nozzle diameter and resample the planar printing path after planar slicing; each sampling point must be inside or on the boundary of a certain tetrahedron in the flattened voxel grid; after determining the tetrahedron where the path point is located, perform linear interpolation according to the vertex positions in the corresponding tetrahedron in the original model to obtain the spatial position of the transformed path point;

[0046] Step 2.7: Generate the support structure according to the STL mesh of the model;

[0047] Step 2.7.1: Find all the triangular facets facing downward according to the STL mesh of the model, and then combine these surfaces to form the upper surface of the support structure. Then the upper surface of the support structure can be determined by the point coordinates of the vertices of the triangular facets;

[0048] Step 2.7.2: Extract the spatial contour of the outermost edge of the support upper surface and project it vertically downward to construct the side surface of the support structure;

[0049] Step 2.7.3: Use the plane where the hot bed is located as the bottom surface of the support structure to form a closed support structure model;

[0050] Step 2.8: Use a dense grid structure for the support top surface to ensure the surface quality of the first layer, but this may cause the support to be not easily peeled off and increase the post-processing difficulty; use a sparse grid structure for the middle part of the support, which can increase the printing speed and reduce the material usage, and its pattern is the same as the conventional 3D printing filling pattern;

[0051] Step 2.9: By setting a reasonable lifting distance between the model and the bottom printing plane, the flatness error of the hot bed can be eliminated during the support printing process, and the printing effects of the support and the model can be improved.

[0052] Step 3, training the reinforcement learning model, designing the reward function, and using the data for model training;

[0053] Step 3.1, designing the comprehensive print quality reward function R quality :

[0054] R quality = k 1 p + k 2 s + k 3 u

[0055] where p is the printing precision, s is the surface finish, u is the ability to print without support, and k 1 , k 2 and k 3 are the weight coefficients of printing precision, surface finish, and the ability to print without support, respectively;

[0056] Step 3.2, designing the print efficiency reward function R efficiency :

[0057]

[0058] where T is the expected printing time, M is the expected material consumption, t is the actual printing time, m is the actual material consumption, and k 4 and k 5 are the weight coefficients of printing time and material consumption, respectively;

[0059] Step 3.3, designing the stability reward function R for the robotic arm movement and extruder control stability :

[0060] R stability = k 6 (1 - d arm ) + k 7 (1 - v arm ) + k 8 (1 - a arm ) + k 9 (1 - d extruder ) + k 10 (1 - v extruder )

[0061] where d arm is the deviation of the robotic arm movement trajectory, v arm is the fluctuation of the robotic arm movement speed, a arm is the fluctuation of the robotic arm movement acceleration, d extruder is the fluctuation of the extruder temperature, v extruder is the fluctuation of the extruder speed, and k 6 , k 7 , k 8 , k 9and k 10 are the weight coefficients of the manipulator motion trajectory deviation, the manipulator motion speed fluctuation, the manipulator motion acceleration fluctuation, the extruder temperature fluctuation, and the extruder speed fluctuation, respectively;

[0062] Step 3.4: Define the layer thickness, printing speed, filling density, and temperature of the printing parameters as the states of the reinforcement learning;

[0063] Step 3.5: Define increasing or decreasing the layer thickness and changing the printing speed as the actions of the reinforcement learning;

[0064] Step 3.6: Set the initial printing parameters as the states, and initialize the policy π and the value function Q;

[0065] Step 3.7: Select an action a according to the current state s and the policy π;

[0066] Step 3.8: The agent executes the action a, and the environment adjusts the printing parameters according to the action and performs 3D printing;

[0067] Step 3.9: Calculate the reward according to the printing result;

[0068] Step 3.10: According to the executed action, the environment transfers to a new state;

[0069] Step 3.11: Use the Bellman equation to update the value function Q and the policy π.

[0070] Step 4: System integration and testing. Integrate each part of the system, conduct printing tests, evaluate the printing quality and efficiency. The schematic diagram of the multi-degree-of-freedom 3D printing platform is as Figure 2 shown.

[0071] The above is only a preferred embodiment of the present invention, and it is not any other form of limitation to the present invention. Any modification or equivalent change made according to the technical essence of the present invention still belongs to the scope protected by the present invention.

Claims

1. A reinforcement learning 3D printing method in a multi-degree-of-freedom space, the specific steps are as follows, characterized in that: Step 1: 3D model preparation: convert the scan data into a voxelized mesh model, and perform mesh optimization and voxelization; Step 2: Path planning: Generate a multi-degree-of-freedom printing path using a spatial continuous path planning method and add collision and motion range restrictions for the robot arm; The path planning process in step 2 is shown as follows: Step 2.1, processing constraints, set the angle range of the nozzle collision area to θ collision , the angle between the nozzle at the end of the robot arm and the vertical z-axis is θ robot The best surface quality can be obtained when the nozzle is perpendicular to the printing surface. The best angle for printing the object surface is: i ideal =90°-(90°-θ robot )=θ robot In order to avoid nozzle collision, the maximum angle θ allowed is defined in practical applications. max for: i max =90°+θ robot -θ collision θ collision Limited to 45°~60°, θ robot Limited to 0°~45°; the diameter of the nozzle outlet is d N , the repeatability error of the nozzle is e, then the layer spacing d is set at 2e-0.8d N within the scope; Step 2.2, select the deformation area and transform all the angles in the object model below the maximum angle θ max The surface and its internal voxels are added to the flattened deformation area. Let the angle between a triangular facet on the model surface and the horizontal plane be θ. When θ<θ max When , the patch and its internal voxels are selected for flattening deformation; Step 2.3, flattening deformation algorithm, appropriately deforming the voxelized grid, deforming the relatively flat surface area into a horizontal plane suitable for planar path planning; Assume that the current iteration number is i, use the iterative algorithm to horizontally deform the patch in the deformation area, define the mapping M as the deformation field of the voxel grid v, and store the new position h(p) of each vertex p in v after deformation in M; Step 2.3.1: The result of the previous iteration h i-1 (p) as input; Step 2.3.2, traverse the voxel grid h i-1 (p) surface triangles, using Denotes the surface area that meets the flattening deformation condition, using represents the remaining surfaces; Step 2.3.3, use the objective function to The patch in is deformed horizontally. The facets in the mesh are then deformed accordingly to ensure the integrity of the model and obtain the new mesh vertex position h i (p); In the function, if all the patches in the flattening are forced, it will usually lead to an unsolvable problem, so only The flattening objective is set as a soft constraint with high weight; Step 2.3.4, Detection and its adjacent triangular faces. If all faces are horizontal, stop the iteration and replace them with h i (p) As a result, if there are still patches that are not horizontal, the next iteration is performed. If any surface of the input model does not meet the flattening requirements, the vertex position of the original voxel grid is the iteration result h0(p); Step 2.4, continuous path generation, select Fermat spiral as the filling method, reduce path segmentation, and make the printing start point and end point close to each other; let the outer contour of the jth area in the i-th layer slice be r i,j,0 , offset the distance d inwards to form a new contour r i,j,1 , repeat this operation to obtain a series of closed paths r i,j,k ; Each layer of closed paths is stored in a tree-like manner. The closed paths are indexed layer by layer from the outside to the inside, and divided into odd circles and even circles; find the position where the feature changes less in each layer of the path, and determine a point at this position as the connection point connecting adjacent contours; disconnect the path of this layer at this point, and connect the adjacent paths inward to establish a continuous path; Step 2.5, use the deformation field to transform the planar continuous path into a multi-degree-of-freedom printing path in space. Suppose the surface contour of the initial model is Ω, and the vertex of the tetrahedron voxel inside it is p i , when the surface contour is deformed, the vertices are also displaced. By comparing the positions of the vertices of the tetrahedral mesh before deformation, the position mapping relationship of the flattened deformation, i.e., the deformation field M, can be obtained; Define the inverse mapping of the flattening deformation operation as M -1 , then perform an inverse mapping operation M on the existing plane continuous path P -1 (P), the desired continuous path in space can be obtained; Step 2.6, mapping the position of the path point in the original model to the position in the deformed model by linear interpolation; After continuous path planning, the printing path can be obtained; since deformation may occur at any position in the printing path, in order to ensure the accuracy of deformation, the sampling rate is set to the nozzle diameter, and the planar printing path after plane slicing is resampled; Each sampling point must be located inside or at the boundary of a tetrahedron in the flattened voxel grid. After determining the tetrahedron where the path point is located, linear interpolation is performed based on the vertex positions in the corresponding tetrahedron in the original model to obtain the spatial position of the transformed path point. Step 2.7, generate support structure based on the STL mesh of the model; Step 2.7.1, find all downward-facing triangular facets based on the STL mesh of the model, and then combine these surfaces to form the upper surface of the support structure. The upper surface of the support structure can be determined by the point coordinates of the triangular facet vertex coordinates; Step 2.7.2, extract the spatial contour of the outermost edge of the upper surface of the support, project it vertically downward, and construct the side surface of the support structure; Step 2.7.3, use the plane where the hot bed is located as the bottom surface of the support structure to form a closed support structure model; In step 2.8, a dense grid structure is used on the top surface of the support to ensure the surface quality of the first layer, but this may make the support difficult to peel off and increase the difficulty of post-processing; a sparse grid structure is used in the middle of the support to increase the printing speed and reduce the use of materials. Its pattern is consistent with the conventional 3D printing filling pattern; Step 2.9, by setting a reasonable lifting distance between the model and the bottom printing plane, the flatness error of the hot bed can be eliminated during the support printing process, improving the printing effect of the support and model; Step 3: Reinforce learning model training, design reward function, and use data for model training; Step 4: System integration and testing: Integrate all parts of the system and evaluate the printing quality and efficiency through printing tests.

2. The method for 3D printing in a multi-degree-of-freedom space according to claim 1, characterized in that: The process of 3D model preparation in step 1 is expressed as: Step 1.1, data conversion, convert the scanned point cloud data into STL format, fit the point cloud data into a surface, and generate an STL format file, which consists of a series of triangular facets. The surface shape of the object is described by the vertex coordinates and normal vectors of the triangular facets. For each triangular facet, three vertex coordinates (x1, y1, z1), (x2, y2, z2), (x3, y3, z3) and the corresponding normal vector (n x ,n y ,n z ) to indicate that n x 、n y 、n z are the components of the normal vector in the x, y, and z directions respectively; Step 1.2, set the size of the voxel in the voxel model. The size formula is as follows: v=k×min(p,f) Among them, v is the voxel side length, p is the printing accuracy, f is the minimum feature size of the model, and k is the adjustment coefficient; Step 1.3, hole repair, use the hole filling algorithm to fill the holes in the STL mesh, by identifying the triangular facets at the hole boundary, generating new triangular facets on the boundary to fill the hole. Assume that the boundary of the hole is composed of n triangular facets, and their vertex coordinates are (x b1 ,y b1 ,z b1 ), (x b2 ,y b2 ,z b2 ),…,(x bn ,y bn ,z bn ), the subscript b indicates the boundary, 1, 2, ..., n is used to distinguish different vertices, calculate the geometric center of these vertices, and generate new triangles based on the center to fill the holes. The geometric center calculation formula is as follows: Among them, (x c ,y c ,z c ) is the geometric center coordinate of the hole, and the subscript i is a variable used to distinguish different vertices; Step 1.4, patch optimization, evaluates the uniformity of patch distribution by calculating the patch area and angle parameters; Step 1.5, optimize the mesh and use the mesh smoothing algorithm to smooth the mesh surface. For each vertex v in the mesh i , the new position after smoothing is: in, is the vertex v i The new position after the mesh smoothing algorithm is processed, w j is the weight of adjacent vertices, and the subscript j is a variable used to distinguish different vertices; Step 1.6, network refinement, refine the mesh, re-divide the mesh into smaller tetrahedrons, increase the mesh density, for a larger tetrahedron, divide it into smaller tetrahedrons by connecting the midpoints of its edges; interpolate between two adjacent tetrahedral mesh units, insert new vertices and tetrahedral units, and increase the mesh density; Step 1.7, voxelization, uses a tetrahedral mesh generation algorithm to convert the STL mesh into a tetrahedral mesh.

3. The method for 3D printing in a multi-degree-of-freedom space according to claim 1, characterized in that: The process of reinforcement learning model training in step 3 is expressed as follows: Step 3.1, design a comprehensive print quality reward function R quality : R quality =k1p+k2s+k3u Among them, p is the printing accuracy, s is the surface finish, u is the support-free printing capability, k1, k2 and k3 are the weight coefficients of printing accuracy, surface finish and support-free printing capability respectively; Step 3.2, design the printing efficiency reward function R efficiency : Where T is the expected printing time, M is the expected material consumption, t is the actual printing time, m is the actual material consumption, k4 and k5 are the weight coefficients of printing time and material consumption respectively; Step 3.3, design the stability reward function R for the robot arm motion and extruder control stability : R stability =k6(1-d arm )+k7(1-v arm )+k8(1-a arm )+k9(1-d extruder )+k 10 (1-in extruder ) Among them, d arm is the deviation of the robot's motion trajectory, v arm is the fluctuation of the robot arm's motion speed, a arm is the fluctuation of the robot arm's motion acceleration, d extruder is the extruder temperature fluctuation, v extruder is the fluctuation of extruder speed, k6, k7, k8, k9 and k 10 They are weight coefficients of robot motion trajectory deviation, robot motion speed fluctuation, robot motion acceleration fluctuation, extruder temperature fluctuation, and extruder speed fluctuation; Step 3.4, define the printing parameters of layer thickness, printing speed, filling density, and temperature as the states of reinforcement learning; Step 3.5, increasing or decreasing the layer thickness and changing the printing speed are defined as reinforcement learning actions; Step 3.6, set the initial printing parameters as the state, initialize the strategy π and the value function Q; Step 3.7, select an action a according to the current state s and strategy π; Step 3.8, the agent performs action a, the environment adjusts the printing parameters according to the action, and performs 3D printing; Step 3.9, calculate the reward based on the printed result; Step 3.10, the environment moves to a new state according to the action performed; Step 3.11, use the Bellman equation to update the value function Q and policy π.

Images