Tool Path Generation by Reinforcement Learning for Computer-Aided Manufacturing

Abstract

Methods, systems, and apparatuses, including media-encoded computer program products, for computer-aided design and manufacturing of physical structures using toolpaths generated by reinforcement learning for use in subtractive manufacturing systems and techniques include obtaining, in a computer-aided design or manufacturing program, a three-dimensional model of a manufacturable object; generating, by the computer-aided design or manufacturing program, a toolpath usable by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to a machine learning algorithm that employs reinforcement learning during training, wherein the machine learning algorithm includes one or more scoring functions that include rewards that correlate with desired toolpath characteristics, including toolpath smoothness, toolpath length, and avoidance of collisions with the three-dimensional model; and providing the toolpath to the computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object.

Description

Technical Field

[0001] Cross - Reference to Related Applications This patent application claims priority to U.S. Provisional Patent Application No. 63 / 042,264, filed on June 22, 2020, and U.S. Utility Patent Application No. 17 / 153,266, filed on January 20, 2021, which are hereby incorporated by reference in their entirety.

Background Art

[0002] This specification relates to computer - aided design and manufacture of physical structures, such as the use of subtractive manufacturing systems and techniques.

[0003] Computer - aided design (CAD) software and computer - aided manufacturing (CAM) software have been developed to generate three - dimensional (3D) representations of objects and to manufacture the physical structures of those objects (e.g., using computer numerical control (CNC) manufacturing techniques). A subtractive manufacturing method refers to any manufacturing process in which a 3D object is formed from a stock material (generally a "blank" or "workpiece" larger than the 3D object) by removing portions of the stock material. Subtractive manufacturing processes often involve using multiple CNC machine cutting tools in a series of operations that follow a toolpath that was previously (at least partially) determined manually.

[0004] Selecting a toolpath in CAM software can be difficult for novice users. A CNC mill can have several axes and functions, and the geometry being machined can have complex shapes that require specific routing paths. Existing methods for selecting a toolpath involve the user knowing which category of toolpath is most appropriate, selecting that category, and manipulating many (often dozens) of parameters to obtain the desired result. However, even when the user is given a clue as to which type of toolpath to use, it is not always clear which category of toolpath to select, and thus the user often spends hours exploring various categories and parameters to find the desired toolpath. Additionally, POWERMILL® software (available from Autodesk, Inc., San Rafael, CA) includes templates that can be used to generate toolpaths. Summary of the Invention Problems to be Solved by the Invention

[0005] This specification describes techniques related to computer-aided design and manufacturing of physical structures using toolpaths generated by reinforcement learning for use in subtractive manufacturing systems and technologies.

[0006] In general, one or more aspects of the subject matter described in this specification can be embodied in one or more ways (and also in one or more non-transitory computer-readable media specifically encoding a computer program operable to cause a data processing apparatus to perform operations), the method comprising, in a computer-aided design or manufacturing program, obtaining a three-dimensional (3D) model of a manufacturable object and providing at least a portion of the 3D model to a machine learning algorithm that employs reinforcement learning during training to manufacture at least a portion of the manufacturable object, generating, by the computer-aided design or manufacturing program, a toolpath usable by a computer-controlled manufacturing system, wherein the machine learning algorithm includes one or more scoring functions including a reward correlated with desired toolpath characteristics including avoiding smoothness of the toolpath, length of the toolpath, and collisions with the 3D model, and providing the toolpath to the computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object.

[0007] Desired tool path characteristics can include tool engagement for the selected cutting tool and the contact track of the selected tool. Machine learning algorithms can employ variable feeds and speeds. Machine learning algorithms can include two or more machine learning algorithms and providing at least a portion of a 3D model, including processing at least a portion of the 3D model using a first algorithm of the two or more machine learning algorithms and processing a portion of the 3D model using a second algorithm of the two or more machine learning algorithms. The first algorithm of the two or more machine learning algorithms can include a convolutional neural network used to generate data from a portion of the 3D model processed by the second algorithm of the two or more machine learning algorithms. The first algorithm of the two or more machine learning algorithms can operate on a low-resolution view of at least a portion of the 3D model, and the second algorithm of the two or more machine learning algorithms can operate on a high-resolution view of a portion of the 3D model.

[0008] Machine learning algorithms can include an advantage-based actor-critical machine learning architecture. The tool path can be for 2.5-axis machining by a computer-controlled manufacturing system. To manufacture at least a portion of a manufacturable object, generating a tool path usable by a computer-controlled manufacturing system can include generating a plurality of two-dimensional (2D) representations of the 3D model in individual 2D layers, providing each 2D representation to a machine learning algorithm to generate a corresponding set of tool paths for manufacturing each individual 2D layer, and combining the corresponding sets of tool paths for the plurality of 2D representations of the 3D model in the individual 2D layers to generate a tool path usable by the computer-controlled manufacturing system.

[0009] The machine learning algorithm can include two or more machine learning algorithms, and providing at least a part of the three-dimensional model to the machine learning algorithms can include generating at least one starting position by processing a global view of at least a part of the three-dimensional model using a first algorithm among the two or more machine learning algorithms, and generating a set of tool paths near each of the at least one starting position by processing a local view of at least a part of the three-dimensional model using a second algorithm among the two or more machine learning algorithms. Generating at least one starting position can include processing the global view with the first algorithm among the two or more machine learning algorithms using a discretized representation of the three-dimensional model of the manufacturable object and a discretized representation of the model of the stock material from which at least a part of the manufacturable object is manufactured. Generating a set of tool paths can include processing the local view with the second algorithm among the two or more machine learning algorithms using a continuous representation of the model of the tool in a computer-controlled manufacturing system used to manufacture at least a part of the manufacturable object. Generating at least one starting position can include processing the global view with the first algorithm among the two or more machine learning algorithms using a discretized representation of the model of the tool, and generating a set of tool paths can include processing the local view with the second algorithm among the two or more machine learning algorithms using a continuous representation of the three-dimensional model of the manufacturable object and a continuous representation of the model of the stock material.

[0010] The desired tool path characteristics can include the Turn direction of the tool set based on the rotational direction of the tool. The TurnThe orientation can be set based on the area of the tool with respect to the 3D model of the manufacturable object, and one or more scoring functions can include one or more rewards that encourage free selection of the Turn orientation for the tool when the area of the tool is greater than a threshold distance from the 3D model of the manufacturable object, and when the area of the tool is within the threshold distance from the 3D model of the manufacturable object, the one or more rewards can encourage exposure of the correct side of the tool in only one direction based on the rotational direction Turn so as to do. The machine learning algorithm can include one or more scoring functions that can include stage-based rewards correlated with the corresponding percentage of completion of the manufacturable object.

[0011] One or more aspects of the subject matter described herein can also be embodied in one or more systems, the one or more systems including a data processing device including at least one hardware processor and a non-transitory computer-readable medium encoding instructions configured to cause the data processing device to perform operations, the operations including obtaining a three-dimensional model of a manufacturable object in a computer-aided design or manufacturing program and generating, by the computer-aided design or manufacturing program, a tool path usable by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to a machine learning algorithm that employs reinforcement learning during training, the generating including the machine learning algorithm including one or more scoring functions including rewards correlated with desired tool path characteristics including smoothness of the tool path, length of the tool path, and avoidance of collisions with the three-dimensional model, and providing the tool path to the computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object.

[0012] Certain embodiments of the subject matter described herein can be implemented to realize one or more of the following advantages. Tool paths that can be used in manufacturing three-dimensional objects can be automatically generated by a machine learning algorithm, which can reduce the time required to create a manufacturing plan and the time required to manufacture a part. The machine learning algorithm uses reinforcement learning and can be trained to generate desired tool path characteristics using rewards for tool path smoothness, tool path length, and collision avoidance with the three-dimensional model of the object. The machine learning algorithm can generate desired tool path characteristics, including tool engagement, smoothness of the contact track, changes in the tool axis, machining time, variable feed, variable speed, etc. The machine learning algorithm can generate tool paths that can be used in 2.5-axis machining from a two-dimensional representation of the three-dimensional model of the object. Furthermore, by making the tool path generation process more automated, more users can design tool paths. For example, the user does not need to investigate and fine-tune various parameters of the tool path template (e.g., the category of tool path type) to find the desired tool path.

[0013] Details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the invention will become apparent from the description of the embodiments, the drawings, and the claims.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 3

Figure 4

Best Mode for Carrying Out the Invention

[0015] Like reference numerals and designations indicate like elements in the various drawings.

[0016] FIG. 1 shows an example of a system 100 that can be used to design and manufacture a physical structure. The computer 110 includes a processor 112 and a memory 114, and the computer 110 can be connected to a network 140 that can be a private network, a public network, a virtual private network, or the like. The processor 112 can be one or more hardware processors, each of which can include a plurality of processor cores. The memory 114 can include both volatile and non-volatile memory such as random access memory (RAM) and flash RAM. The computer 110 can include various types of computer storage media and devices that can include the memory 114 for storing program instructions executed by the processor 112.

[0017] Such programs include one or more 3D modeling, simulation, and manufacturing control programs, such as computer-aided manufacturing (CAM) programs (s), also referred to as computer-aided design (CAD) and / or computer-aided engineering (CAE) programs 116. The program(s) 116 can be executed locally on the computer 110, remotely on a computer of one or more remote computer systems 150 (e.g., one or more server systems of one or more third-party providers accessible by the computer 110 via the network 140), or both locally and remotely. The machine learning algorithm 134 can be stored in the memory 114 (and / or one or more remote computer systems 150) and can be accessed by the CAD / CAM program(s) 116.

[0018] The CAD / CAM program 116 presents a user interface (UI) 122 on a display device 120 of the computer 110 that can be operated using one or more input devices 118 (e.g., keyboard and mouse) of the computer 110. Although shown as separate devices in FIG. 1, it should be noted that the display device 120 and / or the input device 118 can also be integrated with each other and / or with the computer 110 in a tablet computer, or in a virtual reality (VR) or augmented reality (AR) system, etc. For example, the input / output devices 118, 120 can include a VR input glove 118a and a VR headset 120a.

[0019] User 190 can interact with program(s) 116 to create and / or load a 3D model 132 of an object 180 (e.g., from document 130) to be manufactured by a computer-controlled manufacturing system, such as a CNC machine 170, e.g., a multi-axis, multi-tool milling machine. This can be done using known graphical user interface tools, and the 3D model 132 can be defined in the computer using various known 3D modeling formats, such as a solid model (e.g., voxels) or a surface model (e.g., B-Rep (Boundary Representation), surface mesh). Further, user 190 can, if desired, interact with program(s) 116 to modify the 3D model 132 of object 180.

[0020] In some implementations, the 3D model 132 (e.g., from document 130) can include a 3D model of a stock material (i.e., the “workpiece”) that can be removed by the CNC machine 170 in a subtractive manufacturing process. In some implementations, a separate 3D model of the stock material can be obtained by CAD / CAM program(s) 116. The stock material can be removed by the CNC machine 170 following a desired tool path. For ease of explanation, the stock material is shown heart-shaped and the object to be manufactured is shown pentagonal. This illustration does not correspond to typical stock material workpieces or manufactured objects encountered in the field of subtractive manufacturing (e.g., milling).

[0021] Once the 3D model 132 of the object 180 is prepared for manufacturing, a 3D model 132 for manufacturing the physical structure of the object 180 can be prepared by generating a tool path for use in a computer-controlled manufacturing system to manufacture the object 180. For example, the 3D model 132 can be sent to a CNC machine 170 and used to generate a tool path specification document 160 that can be used to control the operation of one or more milling tools. This can be done in response to a request by the user 190 or in view of the user's request for another operation, such as whether the 3D model 132 can be directly connected to the CNC machine 170 or the computer 110, or sent to other manufacturing machines that can be connected via the network 140. This can involve post-processing steps executed on the local computer 110 or in the cloud service, and the 3D model 132 can be exported to an electronic document from which it can be manufactured. Note that the electronic document (simply referred to as a document for simplicity) can be a file, but does not necessarily correspond to a file. The document can be stored within a portion of a file that holds other documents, within a single file dedicated to the document, or within multiple coordinated files.

[0022] In any case, the program(s) 116 can create one or more tool paths within the document 160 and provide the document 160 (in an appropriate format) to the CNC machine 170 to create the physical structure of the object 180. (In some implementations, the computer 110 is integrated with the CNC machine 170, so note that the tool path specification document 160 is created by the same computer that uses the tool path specification document 160 to manufacture the object 180.) The program(s) 116 can generate one or more tool paths by providing the 3D model 132 of the object 180 (e.g., from the document 130) to the machine learning algorithm 134. The machine learning algorithm 134 can automatically generate a tool path 172 (e.g., stored in the document 160) that the CNC machine 170 can use to manufacture the object 180. This automated process can accelerate the tool path generation process, shortening the time required to create a manufacturing plan and, similarly, shortening the time required to manufacture the part, rather than having the user 190 specify the type and parameters of the desired tool path (e.g., via the menu of the UI 122). For example, the CNC machine 170 can be a subtractive manufacturing machine that can manufacture the object 180 by removing stock material 136. The CNC machine 170 can control the cutting tool 174 using the tool path 172 automatically generated by the machine learning algorithm 134 (e.g., stored in the document 160). For example, the cutting tool 174 can include a cutter, which can be programmed to remove excess stock material when manufacturing an object using subtractive manufacturing.

[0023] The program(s) 116 can include a series of menus in the UI 122, which enable the user 190 to accept or reject one or more candidate toolpaths automatically generated by the machine learning algorithm 134. In some implementations, the program(s) 116 can include a series of menus in the UI 122 that enable the user 190 to adjust one or more portions of the candidate toolpath until the user is satisfied with the toolpath. When the user accepts a candidate toolpath, the program(s) 116 can save the candidate toolpath to the toolpath document 160 and provide the document 160 to the CNC machine 170 to manufacture the physical structure of the object 180.

[0024] Figure 2A shows an example of a process for generating a toolpath by a machine learning algorithm for use in manufacturing the physical structure of a modeled object. A three-dimensional model of a manufacturable object is obtained (200), for example, by the program(s) 116. In other words, the geometry of the modeled object to be manufactured by the CNC machine is identified. This can be done automatically by a computer (e.g., by the program 116 on the computer 110) or by receiving user input. For example, the user can select the target surface, contour, or other geometry of the 3D model of interest in machining. In some implementations, the program provides a user interface where the user can directly select the geometry of interest (e.g., a surface or a contour) (e.g., by clicking with a mouse).

[0025] In some implementations, after a 3D model of an object is obtained, one or more preprocessing processes can be performed on the 3D model, for example, by program(s) 116. For example, program(s) 116 can remove portions of the 3D model that are too tight for available tool(s). As another example, program(s) 116 can generate a set of 2D images representing cross-sections of the 3D model. The background region, the region inside the object, and the region outside the object can be represented by different values in the input to the machine learning algorithm, for example, by being represented by different colors in the 2D images.

[0026] The 3D model of the object can be in various representations. Possible representations include image pixels, point clouds, voxels, meshes, contour maps, etc., or combinations of two or more of the above representations. In some implementations, the representation can include 2D images of 2D views of the 3D object, or multiple 2D images from multiple angles of the local geometry of the 3D model. In some implementations, models of available tool(s), models of the object, and models of stock materials can use the same or different representations.

[0027] In some implementations, one or more models of available tools, models of objects, and models of stock materials can use a continuous representation (defined by one or more smooth functions) instead of a discretized representation (e.g., using pixels). For example, instead of using an individual pixel representation, a milling tool can be represented by a continuous circle defined by the center coordinates and radius of the milling tool. The milling tool can be represented as a circle centered at the center coordinates of the tool and having a radius equal to the radius of the tool. Using this circular representation, program(s) 116 can calculate whether pixels of the stock material are within the range of the tool using the radius of the tool. As another example, a model of an object and / or a model of stock material can have a continuous representation (e.g., defined by one or more smooth functions by continuous B-Reps) instead of a discretized representation (e.g., using pixels).

[0028] At least a portion of the three-dimensional model is provided to a machine learning algorithm, e.g., by program(s) 116, to generate one or more tool paths (202). The generated tool paths can be used by a computer-controlled manufacturing system, such as a CNC machine 170, to manufacture at least a portion of a manufacturable object. In some implementations, the tool paths generated by the machine learning algorithm can be used to manufacture the entire object.

[0029] In some implementations, the machine learning model can generate a series of regions through which the tool passes. In some implementations, the machine learning model can generate a series of velocity vectors for the tool (e.g., the direction in which the tool is accelerated). For example, the regions and velocity vectors can be represented in pixels, and the tool can move by the number of pixels at each step. A series of regions can include a series of pixel coordinates in a 2D environment. In a 3D environment, the sequence of regions of the tool can include the coordinates of voxels in 3D and the 3D orientation of the tool. The CAD / CAM program(s) 116 can generate one or more splines in a post-processing step that connects all or a portion of the series of regions. One or more splines can be saved as a toolpath and used to control the tool to move through these points in a smooth route. In some implementations, the machine learning model can generate a control mechanism for the tool, e.g., the angle of the cutter. The CAD / CAM program(s) 116 can generate a toolpath with a control mechanism for the tool generated by a machine learning algorithm.

[0030] Generally, a machine learning algorithm constructs a mathematical model based on training data. The machine learning algorithm receives at least a portion of a 3D model of an object as input. The machine learning algorithm can also receive, as input, a representation of the environment, e.g., a model of the stock material that needs to be removed during manufacturing. In some implementations, the model of the stock material from which the object is cut can be the default model used by the program(s) or can be provided to the program(s) by the user or another process. In some implementations, the representation of the environment can use ray tracing, i.e., describe the current environment using the distances from the stock along a set of rays starting from the tool.

[0031] A machine learning algorithm can be trained to generate a tool path with a set of desired tool path characteristics. FIG. 2B is a flowchart illustrating an example of a process for training a machine learning algorithm to generate a tool path by reinforcement learning. One or more scoring functions are defined (232) that include a reward correlated with the desired tool path characteristics. In a machine learning algorithm, reinforcement learning can be used to include one or more scoring functions that include a reward correlated with the desired tool path characteristics. The machine learning algorithm can include a reward for desired tool path behavior and can include a penalty for other undesired tool path behavior. The main goal of the reward is to encourage good cutting, discourage bad cutting, and discourage bad behavior of the cutting tool. Examples of bad tool behavior can include moving into the CAD model and staying in one spot and changing direction indefinitely, etc.

[0032] Desired tool path characteristics can include maximizing the smoothness of the tool path (e.g., the track of the center of the tool is smooth), minimizing the length of the tool path, minimizing the machining time, etc. For example, a tool path that suddenly turns 90 degrees Turn may not be desirable. A tool path that makes a zigzag movement may also not be desirable. Penalties or negative rewards can be applied to these undesired tool path characteristics.

[0033] Desired tool path characteristics can also include collision avoidance with a 3D model. In some implementations, the machine learning model can include a hard limit to prevent the tool from hitting the CAD model. In some implementations, the machine learning algorithm can include a penalty function that penalizes a machine learning model that attempts to move into the CAD model.

[0034] In some implementations, the desired tool path characteristics can further include selecting and optimizing the side of the cutter within the tool path for a computer-aided manufacturing process. In a given step of the computer-aided manufacturing process, the cutter can include the correct side of the cutter (i.e., the correct region), the incorrect side of the cutter (i.e., the wrong region), and a neutral side (i.e., the neutral region) between the correct side and the incorrect side of the cutter. Using the correct side of the cutter in a given step can enable the tool path to produce good cuts, e.g., pixels of stock material removed on the correct side of the cutter. Using the opposite side of the cutter in a given step can cause the tool path to potentially produce bad cuts, e.g., pixels of stock material removed on the wrong side of the cutter. Using the neutral region of the cutter can enable the tool path to produce neutral cuts, e.g., pixels of stock material removed by the neutral region between the correct and wrong sides of the cutter.

[0035] Selecting and optimizing from the correct, wrong, and neutral sides of the cutter is an important constraint that encourages the machine learning algorithm to create the desired tool path. The desired tool path operation can remove stock material while exposing the correct side of the cutter. The machine learning algorithm can be trained to generate an appropriate tool path that uses the correct side of the tool based on the direction of tool movement. For example, when the tool is in the same position but moving in different directions, the correct and wrong sides of the tool can be different with respect to the direction of tool movement. As another example, the desired tool path characteristics can include removing as much stock material as quickly as possible while using only the correct side of the cutter.

[0036] The correct side, incorrect side, and neutral side of the cutter may each occupy a specific percentage of the cutter. For example, the correct side, incorrect side, and neutral side of the cutter may each occupy 49%, 49%, and 2% of the cutter, respectively. As another example, the correct side, incorrect side, and neutral side of the cutter may each occupy 20%, 70%, and 10% of the cutter, respectively. When the correct side of the cutter occupies a smaller percentage of the cutter, the tool can remove a thinner amount of stock material, and the machine learning algorithm can be trained to generate smaller updates when selecting and changing the side of the cutter at each step.

[0037] In some implementations, the desired tool path characteristics can further include optimizing the tool engagement of the selected cutting tool, maximizing the smoothness of the contact track of the selected cutting tool (e.g., the contact track of the tool is smooth), minimizing the tool axis change, maximizing the smoothness of the tool axis change, avoiding the residue of small chunks of stock material, limiting the tool engagement angle, or other suitable tool path characteristics. For example, the machine learning algorithm can include a reward for using the good part of the cutter (e.g., the cutting edge of the cutter) during down milling or other milling operations to avoid using the bad part of the cutter (e.g., the central part of a ball nose cutter, or the lower part of a bull nose cutter) and / or to avoid using the incorrect side of the cutter (e.g., in down milling or up milling). As another example, the machine learning algorithm can limit the tool engagement angle by imposing a penalty on the score when too many pixels of the image representation of the model are simultaneously engaged (e.g., touched by the tool).

[0038] In some implementations, the machine learning algorithm can include rewards for the area of the tool and / or the rotational characteristics (e.g., whether it rotates in place or advances). For example, the machine learning algorithm can continuously determine to change the direction of the machine learning algorithm in one direction to cause the tool to rotate around a point in the environment, i.e., include a reward that does not encourage rotation in place. In some implementations, the reward for the area of the tool and / or the rotational characteristics can be combined with the reward for the smoothness of the tool path to generate a smooth tool path, e.g., generate a smooth cut around a part. At each step, the machine learning algorithm can include a reward for maintaining or changing the rotational direction of the tool. The machine learning algorithm can include a reward for rotating the tool in the same area before moving the tool to another area. In some implementations, at each step, the machine learning algorithm can include a reward for always moving the tool to another area, preventing the tool from staying in the same area. In some implementations, the machine learning algorithm can include a reward that allows the tool to rotate and advance on the spot but not both at the same time, which can encourage an increase in the completion rate and prevent the tool from colliding with the CAD model. In some implementations, the machine learning algorithm can include a reward for rotating the tool only counterclockwise (i.e., anti-clockwise) or only clockwise or rotating the tool in both clockwise and counterclockwise directions. For example, in some implementations, when the cutter collides with the CAD model, it may be desirable to rotate the cutter counterclockwise rather than clockwise because the correct side of the cutter can be exposed in the default counterclockwise Turn direction. As another example, in some implementations, rotating the tool in both clockwise and counterclockwise directions, e.g., clockwise in one step Turn and then counterclockwise in the subsequent step TurnAnd it may be desirable to repeat these two steps, which facilitates the production of a smoother outer profile of the manufactured object. In some implementations, since the tool does not need to make a plurality of consecutive decisions to face a particular direction, the ability to rotate in both directions allows the tool to more easily create a smooth tool path. Turn In some implementations, the desired tool path characteristics can include setting the direction of the tool based on the area of the tool relative to the model of the manufacturable object. When the area of the tool exceeds a threshold distance from the model of the manufacturable object, one or more scoring functions can include one or more rewards that encourage the tool to freely select its direction. For example, when the tool is far from the CAD model (e.g., more than 1 millimeter away), the system can freely select the direction to create a smoother tool path. When the area of the tool is within the threshold distance range from the model of the manufacturable object, one or more scoring functions can include one or more rewards that encourage the tool to move in only one direction, which causes the correct side of the tool to be exposed based on the direction of rotation. For example, when the tool is close to the CAD model (e.g., less than 1 millimeter), the system can cause the tool to rotate only counterclockwise to prevent the tool from catching on the CAD model.

[0039] In some implementations, the desired tool path characteristics can include setting the direction of the tool based on the area of the tool relative to the model of the manufacturable object. Turn When the area of the tool exceeds a threshold distance from the model of the manufacturable object, one or more scoring functions can include one or more rewards that encourage the tool to freely select its direction. Turn For example, when the tool is far from the CAD model (e.g., more than 1 millimeter away), the system can freely select the direction to create a smoother tool path. Turn When the area of the tool is within the threshold distance range from the model of the manufacturable object, one or more scoring functions can include one or more rewards that encourage the tool to move in only one direction, Turn which causes the correct side of the tool to be exposed based on the direction of rotation. Turn For example, when the tool is close to the CAD model (e.g., less than 1 millimeter), the system can cause the tool to rotate only counterclockwise to prevent the tool from catching on the CAD model.

[0040] In some implementations, the machine learning algorithm can include a reward for removing the representation of the stock material. The machine learning algorithm can increase the amount of reward for removing stock material that is close to the CAD model, and / or can increase the amount of reward for removing stock material as more stock material is removed. In some implementations, the machine learning algorithm can include stage-based rewards. The stage-based rewards can include larger rewards for higher levels of completion (including up to 100%), and can help ensure that the machine learning algorithm removes all of the stock material around the CAD model. For example, various levels of rewards can be set for 50%, 95%, 99% completion, or 80%, 95%, 99%, 100% completion. By obtaining a very high reward at 100% completion, the machine learning algorithm can be prevented from (e.g., being determined to have obtained sufficient reward) removing only the stock material far from the CAD model before actually finishing machining the CAD model. As another example, when only a small amount of stock material remains, the reward for removing the small amount of stock material can be increased. A machine learning algorithm trained with stage-based rewards can generate toolpaths with a high completion rate (including up to 100%) for milling the modeled object.

[0041] In some implementations, the machine learning algorithm can use variable tool feed (e.g., percentage of cutter used), variable tool speed (e.g., feed rate), or variable cutting force, etc. In some implementations, the trochoidal motion can be adopted in the machine learning algorithm. For example, if the tool needs to cut slots in a stock material with CAD models on both sides, the tool can move in a trochoidal motion to avoid excessive tool engagement. As another example, when the tool moves outside the stock material, the tool can move in a helical motion instead of a trochoidal motion. Using variable feed rate allows the tool to increase or decrease its speed during undesirable machining operations (e.g., heavy cuts) that the machine learning algorithm may create, thus reducing the need to achieve optimal engagement 100% of the time. Therefore, the machine learning algorithm can achieve good engagement most of the time (e.g., 99% of the time), and instead of trying to obtain the machine learning algorithm and always achieve 100% tool engagement compliant behavior, the cutter speed can be easily dropped during any undesirable machining operations (e.g., heavy cuts) that may occur occasionally.

[0042] Examples of reward functions can be functions of the following elements, which are The number of good cuts (e.g., the number of pixels of stock material removed by the correct side of the cutter in a given step), The number of bad cuts (e.g., the number of pixels of the CAD model removed by the wrong side of the cutter in a given step), The number of neutral cuts (e.g., the number of pixels in contact with the CAD model removed by the neutral area between the correct and wrong sides of the cutter in a given step).

[0043] Was there a bad cut? Did the CAD model hit? Is the tool in a position it has been to before? Is the tool rotating on the spot or is the tool speed = 0? Has the completion threshold been reached? The reward function can include weight coefficients for each element. Positive weight coefficients can be assigned to desired tool path characteristics such as, for example, the number of good pixel cuts. Negative weight coefficients (e.g., indicating a penalty) can be applied to undesired tool path characteristics such as, for example, the number of bad pixel cuts or the fact that the CAD model was hit. The values of the weight coefficients can be determined in advance or can be learned during the training of a machine learning algorithm.

[0044] In some implementations, the machine learning algorithm can include long-term rewards, short-term rewards, or a combination of both. In some implementations, the machine learning algorithm can employ a reward function for each step of a plurality of steps of the generated tool path. The total reward can be the sum of all rewards corresponding to the plurality of steps. In some implementations, one or more discount rates for the rewards can be applied over time. The discount rate can determine how much a reinforcement learning algorithm values rewards in the distant future relative to rewards in the near future. The discount rate can be a value between 0 and 1. For example, the discount rate can be set to 0.99.

[0045] The input to the machine learning algorithm can be the observation of its environment. In some implementations, at each step, the machine learning algorithm can determine the area of the tool based on the local view of the model without information about the entire model. For example, the machine learning algorithm can use a high-resolution image near the current area of the tool, such as image data for only the stock material, and a CAD model within a defined distance from the tool tip, where the defined distance is one-fourth or half of the tool diameter, or exactly the tool diameter, to efficiently determine how the tool interacts with the stock material. In some implementations, the machine learning algorithm can receive one or more views of the environment as input and make a decision based on one or more views of the environment. For example, in the case of a 3D environment, two or more 2D views of the environment can be provided as input to the machine learning algorithm. In some implementations, the machine learning algorithm can receive as input one or more views of the environment at the current step and one or more views of the environment at one or more previous steps. For example, the observation of the environment can include three images, a current 2D view of the environment and two 2D views of the environment from two previous steps.

[0046] The output of the machine learning algorithm can be a tool path that includes a sequence of tool areas over multiple steps. Each tool area can represent the area where the CAM system moves the representation of the cutting tool. For example, each tool area can be the (x, y, z) coordinates of the tool head. The sequence of tool areas can be adjacent to each other (e.g., those that move only one pixel rotationally or forward within a 2D image representation) or separated (e.g., those that move a long distance in a single step).

[0047] The machine learning algorithm can adopt various reinforcement learning algorithms. Examples of reinforcement learning algorithms include Q-learning, State-Action-Reward-State-Action (SARSA), Deep Q-Learning Network (DQN), Asynchronous Advantage Actor-Critic (A3C) network, Deep Deterministic Policy Gradient (DDPG), hybrid reward architecture (HRA), and the like. The reinforcement learning algorithm can adopt online learning or offline learning, on-policy learning or off-policy learning, hierarchical reinforcement learning, and the like. In some implementation forms, the reinforcement learning algorithm can include a recurrent neural network architecture that uses the previous output state as the input to the next step, such as a gated recurrent unit (GRU) or a long short-term memory (LSTM) neural network. The neural network architecture can include a convolutional neural network (CNN) including one or more convolutional layers of configurable size, one or more fully connected layers, one or more activation layers, or skip connections between layers.

[0048] Figure 2C is a schematic diagram showing an example of a neural network architecture 210 for a machine learning algorithm that generates a tool path for manufacturing the physical structure of a modeled object. This implementation employs an unsupervised machine learning algorithm that does not require a sample solution created by an expert in the training example. This neural network architecture uses a reinforcement learning algorithm, specifically, an advantage actor-critic architecture. The input to the machine learning algorithm can be an environmental observation 212 that describes the model of the object 220, the model of the stock material 222, and the model(s) of the available tool(s) 224. For example, the input to the machine learning algorithm can be a 2D image 214 corresponding to a square region around the tool 224 when the tool moves through the environment.

[0049] The neural network architecture 210 can include a convolutional neural network (i.e., ConvNet 216) that can generate one or more feature vectors from the observed values 212. For example, ConvNet 216 can include a convolutional layer with a stride of 2 that can generate a 32-channel feature vector of size 4. A recurrent neural network, such as GRU 220, can receive as inputs the state vector h i 18 generated by GRU 220 at a previous time step and one or more feature vectors generated from ConvNet 216. A recurrent neural network, such as GRU 220, can generate the state vector h i+1 22 for the current time step. For example, the state vector h i+1 22 can have a predetermined length of 256. The state vector h i+1 22 for the current time step can be processed through one or more linear operators 223. The output of the neural network architecture can include a plurality of actions 224 and one or more values 226. The actions 224 can describe a tool path for manufacturing a modeled object, e.g., a velocity vector for moving a tool or a direction for accelerating a tool. The one or more values 226 can represent a value score for being in a particular state corresponding to the state vector h i+1 22. For example, a state where the tool is very close to the CAD model can have a low value score because there is a possibility that the tool will hit the CAD model, resulting in a large negative reward. In some implementations, a softmax function 228 can be applied to the actions 224, and the output of the softmax function 228 can include a probability distribution of possible actions.

[0050] Machine learning algorithms can determine a sequence of tool regions in multiple steps based on what they have learned from past experiences obtained during the training process. Machine learning algorithms can be trained using training examples that include sample tools and sample environments. Referring again to FIG. 2B, multiple training examples can be received (234), and each training example can include a sample tool and a sample environment. During training, the parameters of the machine learning algorithm (e.g., a set of weights) can be repeatedly updated based on the training examples until a stopping criterion is met. The training examples can include samples from an actual CAM process, or a simulated CAM process, or a combination of both. For example, the training examples can include actual or simulated cutting resistance information for one or more milling tools, and such cutting resistance information can also be added to a scoring mechanism during training. The training examples can include a 2D environment or a 3D environment. The training examples can include representations of 2D tools or 3D tools. In some implementations, one or more preprocessing operations can be performed on the training examples. For example, if a portion of the sample environment is too tight for the available tool(s) to fit, a portion of the sample environment can be removed so that the training sample includes a CAD model that can achieve 100% completion. Thus, the machine learning algorithm can be trained to remove all of the stock material of the training examples.

[0051] Training examples can be used (236) to train a machine learning algorithm to generate tool paths that can maximize the value(s) generated by one or more scoring functions. In some implementations, the machine learning algorithm can employ reinforcement learning algorithm's unsupervised training. During unsupervised training, the machine learning algorithm does not receive the desired output or the expert-labeled sample solutions. The reinforcement learning algorithm can determine the output by maximizing one or more scoring functions that include rewards correlated with the desired tool path characteristics. For example, the reinforcement learning algorithm can be trained to maximize the rewards received from the observed machining environment. By designing the rewards to correlate with the desired tool path characteristics, the reinforcement learning algorithm can be trained to generate the desired tool path.

[0052] In some implementations, the machine learning algorithm can employ a training method appropriate for the selected corresponding reinforcement learning algorithm. For example, a reinforcement learning algorithm based on an actor-critic network can be trained using an asynchronous training method. In the asynchronous training method, for each iteration, a replica of the reinforcement learning network can be created using the current set of weights. Each replica of the network can execute its own simulation by interacting with a part of the environment. The current performance from the replica can be collected from the simulations accumulated over several steps. The update of the set of weights can be calculated based on the performance collected through the optimization algorithm (e.g., Stochastic Gradient Descent (SGD) with or without momentum, Root Mean Square Propagation (RMSProp) with or without shared statistics, etc.). The set of weights can be repeatedly updated based on the performance of the reinforcement learning model until the stopping criterion is met (e.g., a certain number of iterations are completed, the change in weights is less than a threshold, or the accuracy limit is reached, etc.).

[0053] In some implementations, when training a machine learning algorithm, training that does not conform to the policy can be used instead of training that conforms to the policy. Training that does not conform to the policy can evaluate and train the machine learning algorithm using sample tool paths generated from a source other than the machine learning algorithm itself. Sample tool paths generated from different sources can include actual tool path data already used in computer-aided manufacturing, or tool path data designed manually with or without using templates. For example, a reinforcement learning algorithm can have its performance evaluated and learn the algorithm's parameters from tool paths created by experts. In some implementations, experience replay optimization can be used when training a reinforcement learning algorithm. Experience replay can help improve sample efficiency by enabling samples to be reused and by allowing training samples for interesting and challenging scenarios to be used more frequently.

[0054] After training is complete, the machine learning algorithm can generate toolpaths that can be used to manufacture objects not in the training examples or objects for which the machine learning algorithm has not been trained. Additional training examples can be obtained that can represent one or more new objects (e.g., one or more new parts). The machine learning algorithm can be further trained with a combination of the existing training examples and the additional training examples. In some implementations, for the purpose of rapid training, based on a previously trained machine learning model, i.e., instead of the parameters of the machine learning model being calculated from scratch (e.g., random numbers or zero), they are updated from the previously learned parameters, and the machine learning algorithm can be trained by performing fine-tuning. The toolpaths generated by the machine learning algorithm for these new parts can be further improved after adding new training examples to train the machine learning algorithm. In some implementations, the additional training examples can include data corresponding to user changes to toolpaths previously generated by the machine learning algorithm. The data corresponding to user changes can be used to train an improved machine learning algorithm that can generate more desirable toolpaths.

[0055] In some implementations, the machine learning algorithm can include two or more machine learning algorithms. At least a portion of the 3D model can be processed by a first algorithm of the two or more machine learning algorithms. A portion of the 3D model can be further processed by a second algorithm of the two or more machine learning algorithms.

[0056] In some implementations, a first one of two or more machine learning algorithms can include a convolutional neural network (CNN) used to generate data (e.g., image features) from a portion of a 3D model. Examples of CNNs can include AlexNet, InceptionNet, ResNet, DenseNet, etc., or other types of CNNs capable of performing image recognition tasks. In some implementations, the machine learning algorithm can receive as input a 3D model of an object in an environment and a 2D image representing a 2D cross-sectional representation of a stock material. The convolutional neural network can effectively extract useful image features from the 2D image through two or more convolutional layers that perform a series of linear and non-linear operations. The extracted image features can represent the relationship between the remaining stock material, the model of the object, and the area of the tool. For example, the generated data, such as the extracted image features, can be processed by a second one of the two or more machine learning algorithms. For example, the second machine learning algorithm can be a reinforcement learning network (e.g., an Asynchronous Advantage Actor-Critic (A3C) network) capable of generating a tool path usable in computer-controlled manufacturing.

[0057] In some implementations, a first one of two or more machine learning algorithms can operate on a low-resolution view of at least a portion of a 3D model. A second one of the two or more machine learning algorithms can operate on a high-resolution view of a portion of the 3D model. For example, the first algorithm can generate a plurality of starting positions for positioning a tool using a low-resolution view of the model of the object. Based on the high-resolution views around each starting position, the second algorithm can generate a tool path starting from each starting position generated by the first algorithm, and the tool path can be used to manufacture a local portion of the object. Details of the two or more machine learning algorithms are described below in connection with FIG. 3.

[0058] Referring again to FIG. 2A, the toolpath generated by the machine learning algorithm is provided to the user, for example, by program(s) 116 to determine (204) whether the toolpath is the acceptable final toolpath for the object. Program(s) 116 can include in UI122 UI elements that allow user 190 to accept or reject one or more candidate toolpaths automatically generated by the machine learning algorithm. For example, the user can view a video simulating the process of manufacturing an object using one or more candidate toolpaths.

[0059] If the user determines that the generated toolpath is unacceptable for manufacturing at least a portion of the manufacturable object, program(s) 116 can generate an updated toolpath using the machine learning algorithm. In some implementations, program(s) 116 can include in UI122 UI element(s) that allow user 190 to specify updated desired toolpath characteristics. The machine learning algorithm can generate the updated toolpath using one or more scoring functions that include rewards correlated with the updated toolpath characteristics. In some implementations, program(s) 116 can include in UI122 UI element(s) that allow user 190 to manually edit one or more portions of the candidate toolpath until the user is satisfied with the toolpath. Further, if the machine learning algorithm generates a toolpath(s) that cannot remove all of the stock material during the subtractive manufacturing process, user 190 can employ UI element(s) of UI122 to add to the automatically generated toolpath(s) to ensure that all of the stock material is removed during the subtractive manufacturing process, i.e., extending the generated toolpath(s) can additionally be performed in addition to changing the generated toolpath(s).

[0060] When the user determines that the generated tool path is acceptable to manufacture at least a portion of the manufacturable object (204), the tool path is provided to a computer-controlled manufacturing system, for example, by program(s) 116, to manufacture at least a portion of the manufacturable object (206). In some implementations, program(s) 116 can save the candidate tool path in the tool path document 160 of FIG. 1. Program(s) 116 can provide document 160 to the CNC machine 170 to manufacture the physical structure of object 180.

[0061] A computer-controlled manufacturing system uses a tool path generated by a machine learning algorithm to manufacture at least a portion of a manufacturable object (208). The manufacture of the modeled object can include roughing operations, finishing operations, and optionally, semi-finishing operations between these two operations. The roughing operation can include cutting most of the stock material, but a portion of the stock material remains on the modeled object. The finishing operation can include cutting all of the remaining stock material and generating the final manufactured object with a good finish. Each of the roughing, finishing, and semi-finishing operations can have its own tool path. The machine learning algorithm can be used to generate a tool path for roughing, finishing, or semi-finishing.

[0062] FIG. 3 shows an example of a process for generating a tool path by a machine learning algorithm for 2.5-axis machining. 2.5-axis machining is a type of subtractive manufacturing process. 2.5-axis machining can use a three-axis milling tool that can move in all three separate dimensions, but in most cutting operations, the milling tool only moves in two axes relative to the workpiece, resulting in a more efficient manufacturing process. The subtractive process in 2.5-axis machining is a continuous movement within a plane perpendicular to the milling tool, but occurs in discontinuous steps parallel to the milling tool. Compared to three-axis subtractive manufacturing, the 2.5-axis subtractive manufacturing process can quickly remove layers of material in sequence and can create parts with a series of "pockets" at various depths.

[0063] A three-dimensional model 320 of an object for 2.5-axis machining by a computer-controlled manufacturing system is obtained, for example, by a program(s) 116 (302). 2.5-axis generative design can use generative design software to generate a CAD model of a 3D object that includes a plurality of individual layers. For example, the CAD model 320 can have three layers including a bottom layer, an intermediate layer, and a top layer.

[0064] A plurality of two-dimensional representations 322 of the three-dimensional model are generated, for example, by a program(s) 116 (304). The plurality of 2D representations can be generated in individual 2D layers of the 3D model in a preprocessing step. Each 2D representation can be an image representing a cross-section of the 3D model of the object. For example, the 2D representation 322 can be an image representing a cross-section of the 3D model 320 at the height of the intermediate layer. The 2D representation 322 can include a region 326 representing the object (e.g., the part) and a region 328 outside the object (e.g., outside the part) where stock material needs to be removed.

[0065] The 2D representation is provided to the machine learning algorithm, for example, by the program(s) 116. The machine learning algorithm 306 can be trained to generate a toolpath for 2.5-axis machining, that is, the machine learning algorithm 306 creates a toolpath that can be used to manufacture an object in three dimensions but operates only in two dimensions. The toolpath 324 that can be used to manufacture at least a portion of an object using 2.5-axis machining can be generated, for example, by the program(s) 116, based on a plurality of 2D representations (308). In other words, each 2D representation can be provided to the machine learning algorithm to generate a corresponding set of toolpaths for manufacturing each individual 2D layer. In some implementations, the final toolpath can be generated by combining all sets of toolpaths corresponding to the plurality of 2D representations of the 3D model.

[0066] The toolpath can be provided to a computer-controlled manufacturing system to manufacture at least a portion of an object using 2.5-axis machining (310). For example, the toolpath 324 can be provided to remove stock material at the lower right portion 330 of the object using 2.5-axis machining.

[0067] In some implementations, the machine learning algorithm can include two or more machine learning algorithms. At least one starting position of the tool can be generated by processing a global view of the 3D model with a first algorithm of the two or more machine learning algorithms. For each of the at least one starting position, a set of tool paths can be generated by using a second algorithm of the two or more machine learning algorithms to process a local view of the 3D model near each starting position (e.g., an array of values representing pixels around the edge of the cutter's surface, or a set of values representing concentric circles extending from the cutter's surface). The manufacturing process can be operated in a way that teleports and then removes. In each iteration, the tool can quickly move to a desired starting position without performing a cutting operation. Next, the tool can perform cutting in a local area near the at least one starting position. The method of performing long-term planning using two or more machine learning algorithms and then cutting locally can be applied to various types of computer-controlled manufacturing systems and is not limited to 2.5-axis machining.

[0068] For example, the image representation 323 of the object shows four local regions 330, 332, 334, and 336 outside the object. If the tool moves only within a 2D plane perpendicular to the tool, some regions (e.g., region 336) may not be accessible by a tool operating in another region (e.g., region 330). The first machine learning algorithm can generate four starting positions for manufacturing each of the four regions 330, 332, 334, and 336. The second machine learning algorithm can generate tool paths that can be used to remove stock material in each of the four regions 330, 332, 334, and 336.

[0069] In some implementations, generating at least one starting position may include processing a global view of the 3D model of the manufacturable object using a first algorithm of two or more machine learning algorithms, using a discretized representation of the 3D model of the manufacturable object and of the model of the stock material from which at least a portion of the manufacturable object will be manufactured. The discretized representations of the object and the stock material can reduce the computational amount and improve the efficiency of the first machine learning algorithm. In some implementations, generating a set of tool paths may include processing a local view using a second algorithm of two or more machine learning algorithms, using a continuous representation of the model of the tool(s) within a computer-controlled manufacturing system used to manufacture at least a portion of the manufacturable object. For example, the system can use the continuous representation of the tool(s) as input to a second algorithm of two or more machine learning algorithms to generate accurate tool paths for performing local cutting close to the CAD model.

[0070] In some implementations, generating at least one starting position may include processing a global view using a first algorithm of two or more machine learning algorithms, using a discretized representation of the model of the tool(s). In some implementations, generating a set of tool paths may include processing a local view using a second algorithm of two or more machine learning algorithms, using a continuous representation of the 3D model of the manufacturable object and of the model of the stock material. For example, the system can use the continuous representation of the 3D model of the object and of the model of the stock material as input to a second algorithm of two or more machine learning algorithms to generate a portion of the tool path for performing local cutting close to the CAD model. The continuous representations of the object and the stock material can improve the accuracy of local cutting close to the CAD model.

[0071] In some implementations, the system can use a discretized representation of the 3D model of the manufacturable object and the model of the stock material with both the first and second of two or more machine learning algorithms. The system can use a high-resolution discretized representation of the model during the processing of the local view with the second of two or more machine learning algorithms. For example, the system can process the global view with the first of two or more machine learning algorithms using a low-resolution discretized representation (e.g., an image) of the 3D model of the object, and each pixel in the image can have a physical size of 5 mm × 5 mm. The system can process the local view with the second of two or more machine learning algorithms using a high-resolution discretized representation (e.g., an image) of the 3D model of the object, and each pixel in the image can have a physical size of 0.5 mm × 0.5 mm.

[0072] In some implementations, after most of the stock material has been removed, there may still be small pieces of the stock material remaining. These small pieces of material that need to be removed may not be close to each other. The method of teleporting and then removing described above can effectively remove small pieces of stock material that are far apart from each other. Instead of using the local view near the tool to search for the next piece of stock material, the machine learning algorithm can utilize the global view of all the remaining pieces and quickly send the tool to the starting position of the next piece of stock material.

[0073] FIG. 4 is a schematic diagram of a data processing system including a data processing apparatus 400 that can be programmed as a client or a server. The data processing apparatus 400 is connected to one or more computers 490 via a network 480. Although only one computer is shown in FIG. 4, a plurality of computers can be used as the data processing apparatus 400. The data processing apparatus 400 includes various software modules that can be distributed between an application layer and an operating system. These can include executable and / or interpretable software programs or libraries, including tools and services of a 3D modeling / simulation and manufacturing control program 404 that implements the systems and techniques described above. The number of software modules used can vary from implementation to implementation. Further, the software modules can be distributed among one or more data processing apparatuses connected by one or more computer networks or other suitable communication networks.

[0074] The data processing apparatus 400 also includes hardware or firmware devices including one or more processors 412, one or more additional devices 414, a computer-readable medium 416, a communication interface 418, and one or more user interface devices 420. Each processor 412 can process instructions for execution within the data processing apparatus 400. In some implementations, the processor 412 is a single-threaded or multi-threaded processor. Each processor 412 can process instructions stored in the computer-readable medium 416 or in a storage device such as one of the additional devices 414. The data processing apparatus 400 communicates with one or more computers 490 using its communication interface 418, for example, via a network 480. Examples of user interface devices 420 include displays, cameras, speakers, microphones, tactile feedback devices, keyboards, mice, and VR and / or AR devices. The data processing apparatus 400 can store instructions for performing operations associated with the above-described program(s) in, for example, the computer-readable medium 416 or in one or more additional devices 414 such as, for example, one or more of a hard disk device, an optical disk device, a tape device, and a solid state memory device.

[0075] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, hardware, or in combinations of one or more of them that include the structures disclosed in this specification and their structural equivalents. Embodiments of the subject matter described in this specification can be implemented using one or more modules of computer program instructions encoded on a non-transitory computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a manufactured product, such as a hard drive within a computer system, an optical disk sold through a retail channel, or an embedded system. The computer-readable medium can be separately acquired and later encoded with one or more modules of computer program instructions, such as by delivery via a wired network or a wireless network. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.

[0076] The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a runtime environment, or a combination of one or more of them. Further, the apparatus can adopt various different computing model infrastructures, such as web services, distributed computing, grid computing infrastructure, and the like.

[0077] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative languages, or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. The program can be stored in a single file dedicated to the program, or in multiple coordinated files (e.g., files storing one or more modules, subprograms, or portions of code), as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document). A computer program can be deployed to execute on one computer or on computers located at one site, or distributed across multiple sites and executed on multiple computers interconnected by a communication network.

[0078] The processes and logical flows described herein can be executed by one or more programmable processors executing one or more computer programs, by operating on input data to produce output. The processes and logical flows can also be executed by a device, and the device can also be implemented as special purpose logic circuitry, e.g., an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0079] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. In general, a processor will receive instructions and data from, or both, a read only memory or a random access memory. Important elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. In general, a computer will also include, or be operatively coupled to one or more mass storage devices for storing data, such as, by way of example, magnetic disks, magneto-optical disks, or optical disks. However, a computer need not have such devices. Further, a computer may be embedded in another device, by way of example, a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable memory device (such as, a universal serial bus (USB) flash drive). Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices such as, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0080] To provide interaction with a user, embodiments of the subject matter described herein can be implemented on a computer having, for example, an LDC (liquid crystal display) display device, an OLED (organic light emitting diode) display device, or another monitor for displaying information to the user, and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user. For example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and the input from the user can be received in any form, including acoustic, voice, or tactile input.

[0081] A computing system can include a client and a server. The client and the server are generally remote from each other and typically interact via a communication network. The relationship between the client and the server is created by computer programs that are executed on each computer and have a client-server relationship with each other. Embodiments of the subject matter described herein can be implemented in a computing system, which can include, for example, backend components as a data server, or can include, for example, middleware components such as an application server, or can include, for example, a front-end component such as a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein, or any combination of one or more of such backend, middleware, and front-end components. The components of the system can be interconnected by any form or medium of digital data communication, such as, for example, a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), the Internet (such as the Internet), and peer-to-peer networks (such as ad hoc peer-to-peer networks).

[0082] Although this specification contains many implementation details, these should not be construed as limitations on, or as defining the scope of, what is claimed or may be claimed, but rather as descriptions of specific features of particular embodiments of the disclosed subject matter. Specific features described herein in the context of separate embodiments may also be implemented in combination within a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable subcombination. Further, these features may be described above as acting in certain combinations and even initially claimed as such, but one or more features from a claimed combination may in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0083] Similarly, operations are shown in the drawings in a particular order, but this should not be understood as requiring that such operations be performed in the particular order shown, or in a sequential order, or that all of the illustrated operations be performed, to achieve the desired result. In certain circumstances multitasking and parallel processing may be advantageous. Further, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products.

[0084] Thus, particular embodiments of the invention have been described. Other embodiments are within the scope of the following claims. It should be noted that the following aspects are disclosed herein. [Aspect 1] A method comprising: acquiring, using a computer-aided design or manufacturing program, a three-dimensional (3D) model of a manufacturable object; To manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to a machine learning algorithm that employs reinforcement learning during training, a tool path usable by a computer-controlled manufacturing system is generated by a computer-aided design or manufacturing program, the machine learning algorithm including one or more scoring functions that include a reward correlated with a desired tool path characteristic including smoothness of the tool path, length of the tool path, and avoidance of collision with the three-dimensional model, said generating, providing the tool path to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object; comprising said method. [Aspect 2] The method according to aspect 1, wherein the desired tool path characteristics include tool engagement of a selected cutting tool and a contact track of a selected tool. [Aspect 3] The method according to any one of aspects 1 to 2, wherein the machine learning algorithm employs variable feed and / or speed. [Aspect 4] The machine learning algorithm includes two or more machine learning algorithms, and providing at least the portion of the three-dimensional model is processing at least the portion of the three-dimensional model using a first algorithm of the two or more machine learning algorithms; processing the portion of the three-dimensional model using a second algorithm of the two or more machine learning algorithms; comprising the method according to aspect 1. [Aspect 5] The method according to aspect 4, wherein the first algorithm of the two or more machine learning algorithms includes a convolutional neural network used to generate data from the portion of the three-dimensional model processed by the second algorithm of the two or more machine learning algorithms. [Aspect 6] The method according to aspect 4, wherein the first algorithm among the two or more machine learning algorithms operates on a low-resolution view of at least the part of the three-dimensional model, and the second algorithm among the two or more machine learning algorithms operates on a high-resolution view of the part of the three-dimensional model. [Aspect 7] The method according to any one of aspects 1 to 2 and 4, wherein the machine learning algorithm includes an advantage-based actor-critical machine learning architecture. [Aspect 8] The method according to any one of aspects 1 to 2 and 4 to 6, wherein the tool path is for 2.5-axis machining by the computer-controlled manufacturing system. [Aspect 9] To generate the tool path usable by the computer-controlled manufacturing system for manufacturing at least the part of the manufacturable object, Generating a plurality of two-dimensional (2D) representations of the three-dimensional model in individual 2D layers, Providing each 2D representation to a machine learning algorithm to generate a corresponding set of tool paths for manufacturing each individual 2D layer, Generating the tool path usable by the computer-controlled manufacturing system by combining the corresponding sets of the tool paths for the plurality of 2D representations of the three-dimensional model in the individual 2D layers, The method according to aspect 8, comprising: [Aspect 10] The machine learning algorithm includes two or more machine learning algorithms, and providing at least the part of the three-dimensional model to the machine learning algorithm includes: Generating at least one starting position by processing a global view of at least the part of the three-dimensional model using a first algorithm among the two or more machine learning algorithms. Generating a set of tool paths near each of the at least one starting position by processing a local view of at least the portion of the three-dimensional model using a second algorithm of the two or more machine learning algorithms; The method according to aspect 1, comprising. [Aspect 11] Generating the at least one starting position includes processing the global view with a first algorithm of the two or more machine learning algorithms using a discretized representation of the three-dimensional model of the manufacturable object and a discretized representation of a model of a stock material from which at least the portion of the manufacturable object is to be manufactured; The method according to aspect 10, wherein generating the set of tool paths includes processing the local view with a second algorithm of the two or more machine learning algorithms using a continuous representation of a model of a tool in the computer-controlled manufacturing system used to manufacture at least the portion of the manufacturable object. [Aspect 12] Generating the at least one starting position includes The method according to aspect 11, wherein generating the at least one starting position includes processing the global view with a first algorithm of the two or more machine learning algorithms using a discretized representation of the model of the tool, and generating the set of tool paths includes processing the local view with a second algorithm of the two or more machine learning algorithms using a continuous representation of the three-dimensional model of the manufacturable object and a continuous representation of the model of the stock material. [Aspect 13] The method according to aspect 1, wherein the desired tool path characteristic includes a Turn direction of the tool set based on the rotational direction of the tool. [Aspect 14] The TurnThe direction is set based on the area of the tool with respect to the 3D model of the manufacturable object, wherein the one or more scoring functions include one or more rewards that encourage free selection of the direction of the tool when the area of the tool is greater than a threshold distance from the 3D model of the manufacturable object, and when the area of the tool is within the threshold distance from the 3D model of the manufacturable object, the one or more rewards are in only one direction that exposes the correct side of the tool based on the rotational direction, Turn The method according to aspect 13, which encourages the tool to do so. Turn [Aspect 15] The method according to any one of aspects 1 to 2, 4 to 6, and 10 to 14, wherein the machine learning algorithm includes one or more scoring functions including stage-based rewards correlated with the corresponding percentage of completion of the manufacturable object. [Aspect 16] A system, A data processing device including at least one hardware processor, A non-transitory computer-readable medium encoding instructions configured to cause the data processing device to perform operations, The operations include: Obtaining a three-dimensional (3D) model of a manufacturable object with a computer-aided design or manufacturing program; Generating a tool path usable by a computer-controlled manufacturing system by the computer-aided design or manufacturing program to manufacture at least a portion of the manufacturable object by providing at least a portion of the 3D model to a machine learning algorithm that employs reinforcement learning during training, wherein the machine learning algorithm includes one or more scoring functions including rewards correlated with desired tool path characteristics including smoothness of the tool path, length of the tool path, and avoidance of collisions with the 3D model; ​To provide the tool path to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object, The system including. [Aspect 17] The machine learning algorithm includes two or more machine learning algorithms, and providing at least the portion of the three-dimensional model to the machine learning algorithm By processing a global view of at least the portion of the three-dimensional model using a first algorithm of the two or more machine learning algorithms, generating at least one starting position; By processing a local view of at least the portion of the three-dimensional model using a second algorithm of the two or more machine learning algorithms, generating a set of tool paths near each of the at least one starting position; The system according to aspect 16, including. [Aspect 18] The system according to aspect 16, wherein the desired tool path characteristics include tool engagement of a selected cutting tool and a contact track of the selected tool. [Aspect 19] A non-transitory computer-readable medium for causing a data processing device to execute an operation, The operation is In a computer-aided design or manufacturing program, obtaining a three-dimensional (3D) model of a manufacturable object; Generating a tool path usable by a computer-controlled manufacturing system by the computer-aided design or manufacturing program to manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to a machine learning algorithm that employs reinforcement learning during training, the machine learning algorithm including one or more scoring functions including a reward correlated with desired tool path characteristics including smoothness of the tool path, length of the tool path, and avoidance of collision with the three-dimensional model; To manufacture at least the portion of the manufacturable object, providing the tool path to the computer-controlled manufacturing system; The non-transitory computer-readable medium including . [Aspect 20] The machine learning algorithm includes two or more machine learning algorithms, and providing at least the portion of the three-dimensional model to the machine learning algorithm; Generating at least one starting position by processing a global view of at least the portion of the three-dimensional model using a first algorithm of the two or more machine learning algorithms; Generating a set of tool paths near each of the at least one starting position by processing a local view of at least the portion of the three-dimensional model using a second algorithm of the two or more machine learning algorithms; The non-transitory computer-readable medium according to aspect 19, including the above.

Claims

**Claim 1** A method comprising: obtaining, using a computer-aided design or manufacturing program, a three-dimensional (3D) model of a manufacturable object; generating, by the computer-aided design or manufacturing program, a toolpath usable by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the 3D model to two or more machine learning algorithms, wherein at least one of the two or more machine learning algorithms employs reinforcement learning during training; generating the toolpath usable by the computer-controlled manufacturing system, the generating including one or more scoring functions including a reward correlated with desired toolpath characteristics including avoiding toolpath smoothness, toolpath length, and collisions with the 3D model, wherein at least one of the two or more machine learning algorithms includes a reward correlated with desired toolpath characteristics; providing the toolpath to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object; wherein providing at least the portion of the 3D model includes: processing at least the portion of the 3D model using a first algorithm of the two or more machine learning algorithms; processing the portion of the 3D model using a second algorithm of the two or more machine learning algorithms; the method. **Claim 2** The method of claim 1, wherein the desired toolpath characteristics include tool engagement of a selected cutting tool and a contact track of the selected cutting tool, and wherein the tool engagement of the selected cutting tool includes one or more of an engagement angle and an engagement site of the selected tool engagement. **Claim 3** The method of claim 1, wherein at least one of the two or more machine learning algorithms employs variable feed and / or speed. **Claim 4** The method of claim 1, wherein the first algorithm of the two or more machine learning algorithms includes a convolutional neural network used to generate data from at least the portion of the 3D model processed by the second algorithm of the two or more machine learning algorithms. **Claim 5** ​ The method according to claim 1, wherein the first algorithm among the two or more machine learning algorithms operates on a low-resolution view of at least the part of the three-dimensional model, and the second algorithm among the two or more machine learning algorithms operates on a high-resolution view of at least the part of the three-dimensional model.

6. The method according to claim 1, wherein at least one of the two or more machine learning algorithms includes an advantage-based actor-critical machine learning architecture.

7. The method according to claim 1, wherein the tool path is for 2.5-axis machining by the computer-controlled manufacturing system.

8. To manufacture at least the part of the manufacturable object, generating the tool path usable by the computer-controlled manufacturing system includes: Generating a plurality of two-dimensional (2D) representations of the three-dimensional model in separate 2D layers; Providing each 2D representation to the two or more machine learning algorithms to generate a corresponding set of tool paths for manufacturing each individual 2D layer; Combining the corresponding sets of the tool paths for the plurality of 2D representations of the three-dimensional model in the individual 2D layers to generate the tool path usable by the computer-controlled manufacturing system. The method according to claim 7, comprising:

9. Providing at least the part of the three-dimensional model to the two or more machine learning algorithms includes: Generating at least one starting position by processing a global view of at least the part of the three-dimensional model using a first algorithm among the two or more machine learning algorithms; Generating a set of tool paths near each of the at least one starting position by processing a local view near each of the at least one starting position of at least the part of the three-dimensional model using a second algorithm among the two or more machine learning algorithms. The method according to claim 1, comprising:

10. Generating the at least one starting position includes processing the global view with the first algorithm of the two or more machine learning algorithms using a discretized representation of the three-dimensional model of the manufacturable object and a discretized representation of a model of the stock material from which at least the portion of the manufacturable object is manufactured, Generating the set of tool paths includes processing the local view with the second algorithm of the two or more machine learning algorithms using a continuous representation of a model of a tool in the computer-controlled manufacturing system used to manufacture at least the portion of the manufacturable object. The method according to claim 9. **Claim 11** Generating the at least one starting position is Processing the global view with the first algorithm of the two or more machine learning algorithms using a discretized representation of the model of the tool, and generating the set of tool paths includes processing the local view with the second algorithm of the two or more machine learning algorithms using a continuous representation of the three-dimensional model of the manufacturable object and a continuous representation of the model of the stock material. The method according to claim 10. **Claim 12** The method according to claim 1, wherein the desired tool path characteristic includes a turn direction of the tool set based on a rotation direction of the tool. **Claim 13** The turn direction of the tool is set based on an area of the tool relative to the 3D model of the manufacturable object, The one or more scoring functions include one or more rewards that encourage freely selecting the turn direction of the tool when the area of the tool is greater than a threshold distance from the 3D model of the manufacturable object, and when the area of the tool is within the threshold distance from the 3D model of the manufacturable object, the one or more rewards encourage the tool to turn in only one direction that exposes the correct side of the tool based on the rotation direction. The method according to claim 12. **Claim 14** The method of claim 1, wherein at least one of the two or more machine learning algorithms includes one or more scoring functions that include stage-based rewards correlated with the corresponding percentage of completion of the manufacturable object.

15. A system comprising: A data processing device including at least one hardware processor; A non-transitory computer-readable medium encoding instructions configured to cause the data processing device to perform operations, the operations comprising: In a computer-aided design or manufacturing program, obtaining a three-dimensional (3D) model of a manufacturable object; Generating a toolpath usable by a computer-controlled manufacturing system by the computer-aided design or manufacturing program to manufacture at least a portion of the manufacturable object by providing at least a portion of the 3D model to two or more machine learning algorithms, wherein at least one of the two or more machine learning algorithms employs reinforcement learning during training, and at least one of the two or more machine learning algorithms includes one or more scoring functions that include rewards correlated with desired toolpath characteristics including smoothness of the toolpath, length of the toolpath, and avoidance of collisions with the 3D model; Providing the toolpath to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object; Including, Providing at least the portion of the 3D model includes: Processing at least the portion of the 3D model using a first algorithm of the two or more machine learning algorithms; Processing the portion of the 3D model using a second algorithm of the two or more machine learning algorithms; Including, The system.

16. A system comprising: A data processing device including at least one hardware processor; A non-transitory computer-readable medium encoding instructions configured to cause the data processing device to perform operations, the operations comprising: In a computer-aided design or manufacturing program, obtaining a three-dimensional (3D) model of a manufacturable object; ​ ​ To manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to two or more machine learning algorithms, the computer-aided design or manufacturing program generates a tool path usable by a computer-controlled manufacturing system, wherein at least one of the two or more machine learning algorithms employs reinforcement learning during training, and at least one of the two or more machine learning algorithms includes one or more scoring functions including a reward correlated with desired tool path characteristics including avoiding tool path smoothness, tool path length, and collision with the three-dimensional model, generating the tool path usable by the computer-controlled manufacturing system; providing the tool path to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object; comprising; providing at least the portion of the three-dimensional model to the two or more machine learning algorithms is generating at least one starting position by processing a global view of at least the portion of the three-dimensional model using a first algorithm of the two or more machine learning algorithms; generating a set of tool paths near each of the at least one starting position by processing a local view near each of the at least one starting position of at least the portion of the three-dimensional model using a second algorithm of the two or more machine learning algorithms; A system comprising.

17. The system according to claim 15, wherein the desired tool path characteristics include tool engagement of a selected cutting tool and a contact track of the selected cutting tool, and the tool engagement of the selected cutting tool includes one or more of an engagement angle and an engagement site of the selected tool engagement.

18. A computer-aided design or manufacturing program that causes a data processing device to execute operations, wherein the operations are in a computer-aided design or manufacturing program, obtaining a three-dimensional (3D) model of a manufacturable object; To manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to two or more machine learning algorithms, generating a toolpath usable by a computer-controlled manufacturing system by a computer-aided design or manufacturing program, wherein the two or more machine learning algorithms employ reinforcement learning during training, and at least one of the two or more machine learning algorithms includes one or more scoring functions that include a reward correlated with desired toolpath characteristics including avoiding toolpath smoothness, toolpath length, and collisions with the three-dimensional model, generating the toolpath usable by the computer-controlled manufacturing system; Providing the toolpath to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object; Comprising; Providing at least the portion of the three-dimensional model is Processing at least the portion of the three-dimensional model using a first algorithm of the two or more machine learning algorithms; Processing the portion of the three-dimensional model using a second algorithm of the two or more machine learning algorithms; Including; The computer-aided design or manufacturing program. Claim 19 A computer-aided design or manufacturing program that causes a data processing apparatus to execute operations, wherein the operations are The operations are In a computer-aided design or manufacturing program, obtaining a three-dimensional (3D) model of a manufacturable object; Generating, by the computer-aided design or manufacturing program, a tool path usable by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to two or more machine learning algorithms, wherein at least one of the two or more machine learning algorithms employs reinforcement learning during training, and at least one of the two or more machine learning algorithms includes one or more scoring functions that include a reward correlated with desired tool path characteristics including avoiding smoothness of the tool path, length of the tool path, and collisions with the three-dimensional model, and generating the tool path usable by the computer-controlled manufacturing system. Providing the tool path to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object. Including Providing at least the portion of the three-dimensional model to the two or more machine learning algorithms includes Generating at least one starting position by processing a global view of at least the portion of the three-dimensional model using a first algorithm of the two or more machine learning algorithms. Generating a set of tool paths near each of the at least one starting position by processing a local view near each of the at least one starting position of at least the portion of the three-dimensional model using a second algorithm of the two or more machine learning algorithms. Including. The computer-aided design or manufacturing program.

Images