Force-feedback-based Reinforcement Learning Automatic Printing Method, 3D Printing Device, and Computer-readable Medium

Abstract

An automatic printing method based on force feedback for reinforcement learning, the process is as follows: Step 1), obtain environmental information data, including the sliced image information of the printing model; Step 2), encode the environmental information data to obtain a high-dimensional feature vector m; Step 3), input the high-dimensional feature vector m into the first neural network, and output an action parameter set S reflecting the printer action parameters; Step 4), execute the printing action based on the action parameter set S, collect the real-time data of the peeling force, and judge whether the printing is successful based on the real-time data of the peeling force, and collect the printing result information; Step 5), reward or punish the first neural network according to the printing result information, so as to realize iterative training of the first neural network, so as to obtain a reinforcement learning control strategy aiming to ensure printing success and improve printing efficiency.

Description

Technical Field

[0001] The present application relates to the field of additive manufacturing technology, and in particular, to a force-feedback-based reinforcement learning automatic printing method, a 3D printing device, and a computer-readable medium. Background Art

[0002] Stereolithography 3D printing technology utilizes the characteristics of photosensitive resin or photocuring materials. Under the irradiation of ultraviolet light, these materials change from liquid to solid. Among them, in DLP (Digital Light Processing) stereolithography 3D printing, the photosensitive resin cures on the surface of the release element, and then the printing platform moves upward to peel the cured material from the release element and continue printing the next layer, finally completing the entire printed model. The release element is usually a high-transparency thin film material. Each time printing and separation occur, the release film will cause minor surface damage. Therefore, when the printing time or the number of printing times is excessive, the surface of the release film will become rough and the transmittance will decrease, and a new release film must be replaced.

[0003] During the printing process, affected by many factors such as printing speed, material viscosity, exposure time, and the magnitude of the peeling force between the printing platform and the release element, there may be problems with a certain layer (such as delamination, partial detachment), and there will be a high probability that subsequent layers will print unsuccessfully or be missing.

[0004] When using 3D printing technology for large-scale production in industrial applications, if a printing failure of a certain layer cannot be detected in time, it will lead to waste of a large amount of materials and time, disrupting the production arrangement. Currently, the main methods are, one is to increase the frequency of personnel inspections to detect faulty printers in time; the other is to use more conservative printing parameters to give priority to ensuring successful printing, which inevitably sacrifices printing efficiency. Summary of the Invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a reinforcement learning decision-making and system for force-feedback-based automatic printing. After inputting the sliced image of the printed model, automatic printing is performed through the printing decision-making of reinforcement learning, so as to obtain the best printing effect with both printing efficiency and printing quality.

[0006] To achieve the above object, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a force-feedback-based reinforcement learning automatic printing method, and the process is as follows:

[0008] Step 1), obtain environmental information data, and the environmental information data includes sliced image information of the printed model;

[0009] Step 2), encode the environmental information data to obtain a high-dimensional feature vector m;

[0010] Step 3), input the high-dimensional feature vector m into the first neural network, and output an action parameter set S reflecting the printer action parameters;

[0011] Step 4), perform a printing action based on the action parameter set S, collect real-time data on the peeling force between the printing platform and the release element, and determine whether the printing is successful based on the real-time data of the peeling force, and collect printing result information;

[0012] Step 5), reward or punish the first neural network according to the printing result information, so as to perform iterative training on the first neural network, thereby obtaining a reinforcement learning control strategy aiming to ensure printing success and improve printing efficiency, and perform automatic printing according to the reinforcement learning control strategy.

[0013] In a possible implementation manner, the environmental information data further includes printing material information.

[0014] In a possible implementation manner, the printing material information includes at least one of material viscosity, forming strength, light transmittance, crystallinity, volatility, and elongation rate.

[0015] In a possible implementation manner, the action parameters include at least one of: rising speed, lifting height, lifting waiting time, descending speed, descending waiting time, peeling force limit value, slow movement speed, light intensity, and light time.

[0016] In a possible implementation manner, in Step 4), the printing result information includes: the printing time of this layer, the maximum peeling force, and whether the printing is successful.

[0017] In a possible implementation manner, in Step 4), the following method is used to determine whether the printing is successful:

[0018] 41), input the real-time data of the peeling force into the second neural network to generate a predicted printing image;

[0019] 42), compare the predicted printing image with the actual sliced image of the printing model;

[0020] 43) If the comparison difference is greater than the set threshold, it is determined that the printing fails.

[0021] In a possible implementation manner, the second neural network is established by the following method:

[0022] Given a sliced image for printing;

[0023] Build a model to collect the change of the peeling force that the printing platform experiences over time when peeling from the release element, mark the printing results, and establish a mapping between the sliced image, the real-time data of the peeling force, and the printing results.

[0024] Train the model. Use the real-time data of the peeling force as the input and the image as the output to train an encoder-decoder neural network. The output is a low-resolution predicted printing image, and it is required that the predicted printing image be as close as possible to the sliced image.

[0025] In a possible implementation, the printing method further includes: after determining a printing failure, moving the printing platform back to the previous printing position and reprinting.

[0026] In a possible implementation, the penalties include at least one of the penalties for printing failure in this layer, the penalty for increased printing time in this layer, and the penalty for increased peeling force. The rewards include at least one of the rewards for reduced printing time in this layer and the reward for reduced peeling force.

[0027] In a second aspect, the present application provides a 3D printing device, including a printing platform, a sensor, a material tank, a light-emitting device, as well as a processor and a memory. The bottom of the material tank has a release element. The sensor is configured to collect the real-time data of the peeling force of the printing platform and feedback it to the processor. The memory is used to store instructions. When the instructions are executed by the processor, the 3D printer is made to execute the aforementioned automatic printing method based on force feedback reinforcement learning.

[0028] In a third aspect, the present application provides a computer-readable storage medium storing at least one program, which can implement the aforementioned automatic printing method based on force feedback reinforcement learning when called.

[0029] The present application builds a reinforcement learning model aiming to ensure printing success and improve printing efficiency, and continuously adjusts and improves 3D printing parameters through computer neural network reinforcement learning to achieve automatic printing with the best efficiency.

[0030] It can be understood that the 3D printing device described in the second aspect and the computer-readable storage medium described in the third aspect provided above may both include the 3D printing method based on machine learning provided above. Therefore, the beneficial effects they can achieve can refer to the beneficial effects in the aforementioned automatic printing method based on force feedback reinforcement learning, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic flowchart of the automatic printing method based on force feedback reinforcement learning provided by an embodiment of the present application.

[0032] Figure 2Schematic diagram after separation of the printing platform and the material tank provided by an embodiment of the present application.

[0033] Figure 3 Schematic diagram when the printing platform and the material tank of a 3D printing device provided by an embodiment of the present application are peeled off.

[0034] Figure 4 A sliced image provided by an embodiment of the present application;

[0035] Figure 5 For Figure 4 During the printing process of the sliced image shown, the corresponding temporal variation curve of the peeling force;

[0036] Figure 6 For the predicted printing image output by the second neural network based on Figure 5 the temporal variation curve of the peeling force in, with a similarity of 90%.

[0037] Wherein: 1. Frame; 2. Material tank; 3. Vertical guide rail; 4. Lifting platform; 5. Printing platform; 6. Optical machine; 7. Printed model. Detailed implementation manners

[0038] To describe in detail the technical content, structural features, achieved objectives and effects of the invention, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. In the following description, for the purpose of explanation, many specific details are set forth to provide a detailed description of various exemplary embodiments or implementations of the invention. However, various exemplary embodiments can also be implemented without these specific details or in the case of one or more equivalent arrangements. In addition, various exemplary embodiments can be different, but do not have to be exclusive. For example, without departing from the inventive concept, the specific shapes, structures and characteristics of the exemplary embodiments can be used or implemented in another exemplary embodiment.

[0039] For a 3D printing device that cures light in a DLP manner, as the printing area increases, the demolding difficulty increases, and the printed model and the release film are also easily damaged. Therefore, in the present application, a sensor is used to measure the temporal variation curve of the peeling force between the printing platform and the release element, determine whether the printing is successful, and iteratively optimize the printing parameters through deep learning.

[0040] An embodiment of the present application provides a force-feedback-based reinforcement learning automatic 3D printing method. This method establishes a reinforcement learning model aiming to ensure printing success and improve printing efficiency. By continuously adjusting and improving 3D printing parameters through computer neural network reinforcement learning, automatic printing with the best efficiency is achieved. The embodiment of the present application is generally applicable to stereolithography 3D printing devices that require release elements, and no special restrictions are imposed on the specific form of stereolithography. For the convenience of description below, a DLP (Digital Light Processing) stereolithography 3D printer is taken as an example for illustration.

[0041] Please refer to Figure 1 , an automatic printing method based on force-feedback reinforcement learning of the present application, includes the following steps.

[0042] Step 1): Obtain environmental information data.

[0043] The environmental information data includes printing material information and sliced image information of the printing model.

[0044] The printing material information is a key factor that can affect printing failure or success, and includes at least one of material viscosity, forming strength (horizontal direction), light transmittance, crystallinity, volatility, and elongation rate (vertical direction).

[0045] The forming strength is a parameter reflecting the softness and hardness of the printing model. When the forming strength is much lower than the general value, it indicates printing failure and the printing model has not been successfully cured.

[0046] The above-mentioned sliced image information is a set of a series of sliced images formed by slicing the printing model. One sliced image can reflect the forming pattern of the current layer of the stereolithography layer. By stacking layer by layer, the printing model is finally formed.

[0047] Step 2): Input the environmental information data into the second neural network for encoding to obtain a high-dimensional feature vector m.

[0048] The second neural network is an autoencoder, which is an unsupervised learning neural network model. The purpose of the autoencoder is to convert the input signal into a lower-dimensional representation and reconstruct an output from the low-dimensional encoding that is as close as possible to the original input signal. The learning process of the autoencoder is a process of calculating the error between the input and the reconstructed signal, and it learns to automatically extract appropriate features by minimizing the error.

[0049] Step 3): Input the high-dimensional feature vector m into the first neural network to output an action parameter set S reflecting the printer action parameters.

[0050] The first neural network can be trained using a DQN deep reinforcement learning network. It learns how to make optimal actions by interacting with the environment. In reinforcement learning, the algorithm obtains the current environmental state, takes an action according to the policy function, and then receives a reward or punishment. The goal is to find the optimal policy among all possible state and action combinations to maximize the long-term cumulative reward. This learning method is particularly suitable for dealing with continuous state and action spaces, as well as problems that require making decisions in uncertain and dynamic environments. The key to reinforcement learning is to define the form of environmental information, the way of actions, and the method of judging right or wrong.

[0051] The action parameters are the main control parameters of the printer and can affect the working efficiency of the printer and the success of printing. The set S of the action parameters includes at least one of: rising speed, lifting height, lifting waiting time, falling speed, falling waiting time, peeling force limit value, slow movement speed, and light intensity. These control parameters are continuously updated during the iterative training of the first neural network, and each update aims to pursue higher printing efficiency and ensure successful printing.

[0052] For example, the rising speed and the falling speed refer to the moving speed of the printing platform. The faster the rising speed and the falling speed, the higher the printing efficiency. However, the faster the rising speed, the lower the printing success rate. Therefore, during the training iteration of the first neural network, it is necessary to learn the optimal rising speed and falling speed corresponding to each sliced image.

[0053] The lifting height refers to the height by which the printing platform is lifted after each layer of photosensitive material is cured. If the lifting height is too low, it is not conducive to the complete separation of the printing platform from the release component and the entry of the photosensitive material into the photocuring area. If the lifting height is too large, it is not conducive to improving the printing efficiency.

[0054] The lifting waiting time refers to the time required for the printing platform to lift from the current position to the specified position. This time is to ensure that the printing platform moves to the new printing layer stably and accurately.

[0055] The falling waiting time refers to the time required for the printing platform to fall from the current position to the specified position. This is also to ensure that the printing platform can stabilize before moving to the next layer.

[0056] These two waiting times are both for optimizing the printing process and ensuring the printing accuracy and stability. If the lifting or falling is too fast, it may cause defects or distortions in the printed model; if it is too slow, it may prolong the entire printing process. Therefore, it is necessary for the first neural network to adjust these waiting times according to the specific sliced image to achieve the best printing efficiency.

[0057] The peel force limit value is an upper limit value that defines the maximum allowable value of the peel force between the printing platform and the release element. When the real-time collected peel force is less than this limit value, it can ensure that the release element does not break, the printed model and the release element are smoothly separated, and the printing is successful. When the peel force is too large, it indicates that the moving speed of the printing platform is too fast, which may damage the hardware structure of the printer, may also damage the transparency and smoothness of the release element, may cause the release element to break, and may lead to printing failure.

[0058] The jogging speed refers to the acceleration and deceleration of the printing platform during movement. A faster jogging speed can shorten the printing time, but too fast a speed may result in poor bonding between printing layers, leading to printing failure. A slower jogging speed can ensure good bonding between printing layers, but it will prolong the printing time and reduce the printing efficiency. Therefore, an appropriate jogging speed helps to improve the printing efficiency and the printing success rate.

[0059] The light intensity refers to the light intensity used by the stereolithography printer during irradiation. The higher the light intensity, the faster the curing speed, but too high a light intensity may result in poor bonding between printing layers, leading to printing failure. A lower light intensity may result in too slow a curing speed, affecting the printing efficiency.

[0060] During continuous printing, the first neural network learns from the sliced images and the printing results, aiming for the smallest possible peel force, the fastest possible printing speed, and ensuring printing success. It outputs control parameters that conform to the reinforcement learning control strategy with the goal of ensuring printing success and improving printing efficiency for the next printing cycle.

[0061] Step 4): Print and execute the action parameter set S. During the printing process of collecting each sliced image, obtain the time-series change curve of the peel force between the corresponding printing platform and the release element, and acquire the printing result information. The printing result information includes: the printing time of this layer, the maximum peel force, and whether the printing is successful.

[0062] The "printing time of this layer" is a comprehensive parameter that can reflect the printing efficiency.

[0063] "The maximum peeling force" is the maximum resistance value when the printed model is separated from the release element. When the printed model is separated from the release element, the printing platform is subject to suction and adhesion forces. The suction force is caused by atmospheric pressure. The more viscous the material, the more difficult it is to separate, and the greater the suction force. Therefore, there will be a great resistance when the printing platform is lifted and lowered. The peeling force = adhesion force + atmospheric pressure. It is about 12 N per square centimeter, and the force on the area of a palm can reach 1000 N, which is approximately the weight of an adult. Pulling hard can easily damage the release film, glass, and printed model, and even damage the lifting motor. Therefore, the platform should be driven to rise at a slow speed. Once there is a gap between the printed model and the release film, the suction force will disappear rapidly. At this time, it can be pulled up at a faster speed to replenish the resin material. Therefore, during separation, the movement of the printing platform rises slowly first and then quickly.

[0064] "Whether the printing is successful" is directly judged as right or wrong. The first neural network can optimize the motion parameters for the next layer of printing by learning the printing time of this layer and the maximum peeling force.

[0065] The timing change curve of the peeling force between the printing platform and the release element can be measured by installing a sensor on the printing platform. In order to better reduce the damage to the surface of the release element during the printing separation process, in this embodiment, preferably, one of the printing platform and the material tank of the 3D printer can rotate relative to the other, so that the printed model on the printing platform and the release element can be gradually separated from one side to the other. In this way, through the sensor set at the edge of the printing platform, the timing change curve of the peeling force can be monitored more intuitively and conveniently.

[0066] Since the timing change curve of the peeling force changes according to the sliced image, it is possible to judge whether the printing is successful by learning the relationship between the sliced image and the timing change curve of the peeling force. The specific method is as follows:

[0067] 40) Establish a second neural network;

[0068] 41) Input the real-time peeling force data collected by the sensor into the second neural network to generate a predicted printed image, and this predicted printed image can only output a low-resolution image;

[0069] 42) Compare the predicted printed image with the sliced image of the printed model;

[0070] 43) If the comparison difference is greater than the set threshold, it is judged that the printing fails.

[0071] The similarity comparison of pictures can adopt methods such as feature extraction algorithm, direct comparison algorithm, structural similarity index, or perceptual hashing algorithm.

[0072] The second neural network is established by the following method:

[0073] Given environmental information data;

[0074] Build a model, collect the real-time peeling force data received by the printing platform when the printing platform and the release element are peeled off, mark the printing result, and establish a mapping between the environmental information data, the real-time peeling force data, and the printing result;

[0075] Train the model, use the peeling force time-series curve as the input and the sliced image as the output, train the encoder-decoder neural network, and the output is a low-resolution predicted printing image, and it is required that the predicted printing image is as close as possible to the morphology of the sliced image. And use the set of environmental information data to train the encoder-decoder neural network, and the set of environmental information data contains multiple sliced images.

[0076] Through multiple trainings, the second neural network can obtain the relationship between the environmental information data and the time-series change data of the peeling force. Then, by inputting the time-series change curve of the peeling force collected by the sensor into the second neural network, a predicted printing image can be output. By comparing the similarity between the predicted printing image and the actual sliced image, it can be known whether the printing is successful or failed.

[0077] The machine learning model used by the second neural network is an autoencoder, which belongs to an unsupervised learning neural network model and includes an encoder and a decoder. The encoder converts the input signal into a hidden representation and compresses it into a middle vector of a higher dimension; the decoder converts the compressed vector back to the original dimension and reconstructs the output signal. The learning process of the autoencoder is a process of calculating the error between the input and the reconstructed signal, and it learns to automatically extract appropriate features by minimizing the error.

[0078] Figures 4 - 6 This is an embodiment given in this application. In this embodiment, the printing target is Figure 4 the sliced image in Figure 5 It shows the time-series change curves of the peeling force collected by 4 sensors during the printing process. The horizontal axis is the sampling interval, which is 100 milliseconds in this embodiment, and the vertical axis is the magnitude of the peeling force, with the unit of kilogram-force. Figure 6 It is Figure 5 the predicted printing image obtained after inputting the time-series curve of the peeling force in

[0079] Step 5), reward or punish the first neural network based on the printing result information.

[0080] The penalties described above include one of the following: penalty for layer printing failure, penalty for increased layer printing time, and penalty for increased peeling force. The rewards described above include one of the following: reward for decreased layer printing time and reward for decreased peeling force. Rewards or penalties are used to train the first neural network. Any quantifiable parameter or metric can be used to set rewards and penalties, and there is no limit on the quantity.

[0081] To optimize multiple objectives simultaneously, different score coefficients can be set for different rewards and penalties in this application, such as using a weighted method.

[0082] For example, in one embodiment, the total score = 0.2 × time score + 0.2 × peeling force score + 0.6 × printing success. Each type of score is pre-unified to between 0 and 1, and the total score is also between 0 and 1.

[0083] In another embodiment, rewards and punishments are given according to the results

[0084] (1) If the layer printing fails, a severe penalty is imposed, such as deducting 1000 points.

[0085] (2) If the layer printing time is short, for example, less than 2 seconds, a reward of +10 points is given; if the layer printing time is medium, for example, less than 4 seconds, a reward of +2 points is given; if the layer printing time is too long, for example, >4 seconds, a penalty of -2 points is imposed.

[0086] (3) If the peeling force collected by the sensor is too large, for example, greater than 200N, a penalty of -50 points is imposed; if it is greater than 100N, a penalty of -10 points is imposed.

[0087] Among them, rewards and penalties can be represented by discrete values or by a continuous curve, such as a log function curve.

[0088] In summary, the goal of the first neural network in this application is to optimize the printing action, so that the parameters that should be fast are executed quickly, the parameters that should be slow are executed slowly, and the printing platform cannot be rapidly lifted to cause damage to the equipment and the release element. Through multiple printing iteration trainings, the printing action is continuously optimized, and a deep reinforcement learning network based on peeling force feedback is established to achieve continuous optimization of printing.

[0089] The printing method described above further includes, after determining printing failure, moving the printing platform back to the printing position of the previous layer and reprinting. The steps of reprinting can be placed after step 4 or step 5.

[0090] Briefly speaking, the method of this application is to automatically output printing parameters according to environmental information data during the printing process of each slice image, feedback the printing results, iteratively optimize the policy function, and then apply the optimized policy to the continuous printing process of the next slice image.

[0091] The present application also provides a computer-readable storage medium storing at least one program, which when invoked can implement the above-described force-feedback-based reinforcement learning automatic printing method.

[0092] See Figure 2 、 3 As shown in

[0093] See Figure 3 As shown in

[0094] The sensor characterizes the real-time data of the peeling force with a data curve, and the data curve includes at least one of the information such as amplitude, peak time, change rate, and stable time.

[0095] In order to accurately obtain the magnitude and real-time change of the peeling force between the printing platform and the release film, in the preferred embodiment of the present application, a plurality of sensors, such as 2-6 sensors, are used. More sensors can provide more training data for the machine learning model. In this embodiment, 4 sensors are used, and they are respectively distributed in a rectangular shape on two opposite sides of the printing platform. The sensors are located between the lifting platform 4 and the printing platform 5 and are signal-connected to the processor. They are configured to collect the real-time data of the peeling force of the printing platform and feedback it to the processor. The peeling force signals fed back by the two sensors on the same side should be close and have the same change trend. If it is found that the signals of the two sensors on the same side differ greatly, it is very likely that the release film is damaged or malfunctioning.

[0096] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification is only to illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and the scope of protection required by the present invention is defined by the appended claims, the specification and their equivalents.

Claims

1. An automatic printing method for reinforcement learning based on force feedback, characterized in that, The process is as follows: Step 1): Obtain environmental information data, where the environmental information data includes sliced image information of the printing model; Step 2): Encode the environmental information data to obtain a high-dimensional feature vector m; Step 3): Input the high-dimensional feature vector m into the first neural network to output an action parameter set S reflecting the printer action parameters; Step 4): Execute a printing action based on the action parameter set S, collect real-time data of the peeling force between the printing platform and the release element, and judge whether the printing is successful based on the real-time data of the peeling force, and collect printing result information; Step 5): Reward or punish the first neural network according to the printing result information to perform iterative training on the first neural network, so as to obtain a reinforcement learning control strategy aiming to ensure printing success and improve printing efficiency; Among them, in step 4), it is judged whether the printing is successful through the following method: 41): Input the real-time data of the peeling force into the second neural network to generate a predicted printing image; 42): Compare the predicted printing image with the actual sliced image of the printing model; 43) If the comparison difference is greater than the set threshold, it is judged that the printing fails; The second neural network is established through the following method: Perform printing with given environmental information data; Construct a model, collect the change of the peeling force received by the printing platform over time when the printing platform is peeled from the release element, mark the printing result, and establish a mapping between the environmental information data, the real-time data of the peeling force, and the printing result; Train the model, use the real-time data of the peeling force as the input and the image as the output, train the encoding-decoding neural network, and the output is a low-resolution predicted printing image, and require the predicted printing image to be as close as possible to the sliced image.

2. The method according to claim 1, wherein: The environmental information data also includes printing material information.

3. The method according to claim 2, wherein: The printing material information includes at least one of material viscosity, forming strength, light transmittance, crystallinity, volatility, and elongation rate.

4. The method according to claim 1, characterized in that: The action parameters include at least one of rising speed, lifting height, lifting waiting time, descending speed, descending waiting time, peeling force limit value, slow movement speed, light intensity, and light time.

5. The method according to claim 1, wherein: In step 4), the printing result information includes: the printing time of this layer, the maximum peeling force, and whether the printing is successful.

6. The method according to claim 1, characterized in that: The printing method further includes: after judging that the printing fails, retracting the printing platform to the printing position of the previous layer and reprinting.

7. The method according to claim 1, characterized in that: The punishment includes at least one of the punishment for printing failure of this layer, the punishment for increasing the printing time of this layer, and the punishment for increasing the peeling force, and the reward includes at least one of the reward for reducing the printing time of this layer and the reward for reducing the peeling force.

8. A 3D printing device, comprising a printing platform, a sensor, a material tank, a light-emitting device, as well as a processor and a memory. The bottom of the material tank is provided with a release element, and it is characterized in that: The sensor is configured to collect real-time data of the peeling force of the printing platform and feedback it to the processor, and the memory is used to store instructions, and when the instructions are executed by the processor, the 3D printing device executes the force feedback-based reinforcement learning automatic printing method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Store at least one program, which can implement the force-feedback-based reinforcement learning automatic printing method according to any one of claims 1 to 6 when the at least one program is called.

Images