Generating tool paths for computer-aided manufacturing by reinforcement learning

Abstract

Methods, systems, and apparatus, including media encoded computer program products, for computer-aided design and manufacturing of physical structures using tool paths generated through reinforcement learning for use with subtractive manufacturing systems and techniques, including: obtaining a three-dimensional model of a manufacturable object in a computer-aided design or manufacturing program; generating a tool path that can be used by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model to a machine learning algorithm employing reinforcement learning, wherein the machine learning algorithm includes one or more scoring functions that include rewards related to desired tool path characteristics including tool path smoothness, tool path length, and avoidance of collisions with the three-dimensional model; and providing the tool path to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.

Description

[0001] Cross-references to related applications

[0002] This patent application claims priority and benefit to U.S. Provisional Patent Application No. 63 / 042,264, filed June 22, 2020, and U.S. Utility Model Patent Application No. 17 / 153,266, filed January 20, 2021, both of which are incorporated herein by reference in their entirety. Background Technology

[0003] This specification relates to computer-aided design and manufacture of physical structures, such as using subtractive manufacturing systems and techniques.

[0004] Computer-aided design (CAD) software and computer-aided manufacturing (CAM) software have been developed and used to generate three-dimensional (3D) representations of objects, and to manufacture the physical structure of those objects, for example, using computer numerical control (CNC) manufacturing technology. Subtractive manufacturing refers to any manufacturing process that creates a 3D object from a material (typically a "blank" or "workpiece" larger than the 3D object) by removing portions of the material. Subtractive manufacturing processes often involve the use of multiple CNC machine tool cutting tools in a series of operations following a toolpath that has been previously (at least partially) manually determined.

[0005] Choosing toolpaths in CAM software can be difficult for novice users. CNC milling machines can have numerous axes and functions, and the geometry being machined may have complex forms requiring specific routing paths. Existing methods for selecting toolpaths involve the user understanding which categories of toolpaths are most appropriate, selecting that category, and manipulating many (often dozens) parameters to achieve the desired result. However, even when users are given hints about which toolpath to use, the choice of which category is not always obvious, and therefore users often spend hours exploring various categories and parameters trying to find the toolpath they want. Furthermore, The software (available from Autodesk in San Rafael, California) includes templates that can be used to generate tool paths. Summary of the Invention

[0006] This specification describes techniques for use with subtractive manufacturing systems and technologies, related to the computer-aided design and manufacture of physical structures using tool paths generated through reinforcement learning.

[0007] Generally, one or more aspects of the subject matter described in this specification can be embodied in one or more methods (and one or more non-transitory computer-readable media tangibly encoded with computer programs operable to cause data processing equipment to perform operations), the methods comprising: obtaining a three-dimensional model of a manufacturable object in a computer-aided design or manufacturing program; generating, by the computer-aided design or manufacturing program, a tool path capable of being manufactured by a computer-controlled manufacturing system for manufacturing at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model during training to a machine learning algorithm employing reinforcement learning, wherein the machine learning algorithm includes one or more scoring functions, the one or more scoring functions including rewards associated with desired tool path characteristics, the desired tool path characteristics including tool path smoothness, tool path length, and avoidance of collisions with the three-dimensional model; and providing the tool path to the computer-controlled manufacturing system for manufacturing at least said portion of the manufacturable object.

[0008] The desired toolpath characteristics may include tool engagement of the selected cutting tool and the contact trajectory of the selected tool. The machine learning algorithm may employ variable feed and / or speed. The machine learning algorithm may include two or more machine learning algorithms, and providing at least said portions of the 3D model may include: processing at least said portions of the 3D model using a first machine learning algorithm of the two or more machine learning algorithms; and processing said portions of the 3D model using a second machine learning algorithm of the two or more machine learning algorithms. The first machine learning algorithm of the two or more machine learning algorithms may include a convolutional neural network for generating data from said portions of the 3D model to be processed using the second machine learning algorithm of the two or more machine learning algorithms. The first machine learning algorithm of the two or more machine learning algorithms may operate on a low-resolution view of at least said portions of the 3D model, and the second machine learning algorithm of the two or more machine learning algorithms may operate on a high-resolution view of said portions of the 3D model.

[0009] The machine learning algorithm may include a strength-based actor-commenter machine learning architecture. The toolpath can be used for 2.5-axis machining by a computer-controlled manufacturing system. Generating the toolpath capable of being used by the computer-controlled manufacturing system to manufacture at least said portions of the manufacturable object may include: generating multiple two-dimensional (2D) representations of the three-dimensional model at discrete 2D layers; providing each 2D representation to the machine learning algorithm to generate a corresponding set of toolpaths for manufacturing each discrete 2D layer; and combining the corresponding sets of toolpaths for the multiple 2D representations of the three-dimensional model at discrete 2D layers to generate the toolpath capable of being used by the computer-controlled manufacturing system.

[0010] The machine learning algorithm may include two or more machine learning algorithms, and providing at least said portions of the 3D model to the machine learning algorithm may include: generating at least one starting position by processing a global view of at least said portions of the 3D model using a first machine learning algorithm of the two or more machine learning algorithms; and generating a set of tool paths near each of the at least one starting position by processing a local view of at least said portions of the 3D model using a second machine learning algorithm of the two or more machine learning algorithms. Generating the at least one starting position may include processing the global view using a discretized representation of a 3D model of the manufacturable object and a model of a material using the first machine learning algorithm of the two or more machine learning algorithms, from which at least said portions of the manufacturable object will be manufactured, and generating the set of tool paths may include processing the local view using a continuous representation of a tool model in a computer-controlled manufacturing system for manufacturing at least said portions of the manufacturable object using the second machine learning algorithm of the two or more machine learning algorithms. Generating the at least one starting position may include processing the global view using a discretized representation of the tool model using a first machine learning algorithm of two or more machine learning algorithms, and generating the set of tool paths may include processing the local view using a continuous representation of the three-dimensional model of the manufacturable object and the model of the material using a second machine learning algorithm of the two or more machine learning algorithms.

[0011] The desired tool path characteristics may include a tool steering direction set based on the tool's rotation direction. The tool steering direction may be set based on the tool's position relative to a 3D model of the manufacturable object, and one or more scoring functions may include one or more rewards that encourage free choice of the tool's steering direction when the tool's position is greater than a threshold distance from the 3D model of the manufacturable object, and that the one or more rewards may encourage the tool to turn in only one direction when the tool's position is within a threshold distance from the 3D model of the manufacturable object, thereby revealing the correct side of the tool based on the rotation direction. The machine learning algorithm may include one or more scoring functions that may include stage-based rewards related to the corresponding percentage of completion of the manufacturable object.

[0012] One or more aspects of the subject matter described in this specification may also be embodied in one or more systems comprising: a data processing device including at least one hardware processor; and a non-transitory computer-readable medium encoded with instructions configured to cause the data processing device to perform operations including: obtaining a three-dimensional model of a manufacturable object in a computer-aided design or manufacturing program; generating a tool path by the computer-aided design or manufacturing program for a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three-dimensional model during training to a machine learning algorithm employing reinforcement learning, wherein the machine learning algorithm includes one or more scoring functions including rewards associated with desired tool path characteristics, such as tool path smoothness, tool path length, and avoidance of collisions with the three-dimensional model; and providing the tool path to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.

[0013] Specific implementations of the subject matter described in this specification can be implemented to achieve one or more of the following advantages. Toolpaths suitable for manufacturing 3D objects can be automatically generated using machine learning algorithms, which reduces the time required for manufacturing planning and the time required to manufacture parts. The machine learning algorithms use reinforcement learning and can be trained to generate desired toolpath characteristics using rewards for toolpath smoothness, toolpath length, and avoiding collisions with the 3D model of the object. The machine learning algorithms can generate desired toolpath characteristics, including tool engagement, smoothness of the contact trajectory, tool axis variation, machining time, variable feed, variable speed, etc. The machine learning algorithms can generate toolpaths suitable for 2.5-axis machining from a 2D representation of the 3D model of the object. Furthermore, by making the toolpath generation process more automated, more users can design toolpaths. For example, users do not need to explore and adjust various parameters of toolpath templates (e.g., for the category of toolpath type) to find their desired toolpath.

[0014] Details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the invention will become apparent from the description, drawings, and claims. Attached Figure Description

[0015] Figure 1 An example of a system that can be used to design and manufacture physical structures is shown.

[0016] Figure 2A This is a flowchart illustrating an example of a process that uses machine learning algorithms to generate toolpaths for creating the physical structure of a modeled object.

[0017] Figure 2B This is a flowchart illustrating an example of the process of training a machine learning algorithm that generates tool paths through reinforcement learning.

[0018] Figure 2C This is a schematic diagram illustrating an example of a neural network architecture for a machine learning algorithm used to generate tool paths for manufacturing the physical structure of a modeled object.

[0019] Figure 3 An example of the process of generating toolpaths for 2.5-axis machining using a machine learning algorithm is shown.

[0020] Figure 4 It is a schematic diagram of a data processing system that can be used to implement the described system and technology.

[0021] The same reference numerals and titles in various diagrams indicate the same elements. Detailed Implementation

[0022] Figure 1 An example of a system 100 that can be used to design and manufacture physical structures is shown. Computer 110 includes a processor 112 and memory 114, and computer 110 can be connected to a network 140, which can be a private network, a public network, a virtual private network, etc. Processor 112 can be one or more hardware processors, each of which may include multiple processor cores. Memory 114 may include both volatile and non-volatile memory, such as random access memory (RAM) and flash RAM. Computer 110 may include various types of computer storage media and devices, which may include memory 114 to store instructions for a program running on processor 112.

[0023] Such programs include one or more 3D modeling, simulation, and manufacturing control programs, such as computer-aided design (CAD) and / or computer-aided manufacturing (CAM) programs 116, also referred to as computer-aided engineering design (CAE) programs, etc. The CAD program 116 may run locally on computer 110, remotely on one or more remote computer systems 150 (e.g., one or more server systems of one or more third-party providers accessible from computer 110 via network 140), or both locally and remotely. The machine learning algorithm 134 may be stored in memory 114 (and / or in one or more remote computer systems 150) and accessible by the CAD / CAM program 116.

[0024] The CAD / CAM program 116 presents a user interface (UI) 122 on the display device 120 of the computer 110, which can be operated using one or more input devices 118 of the computer 110 (e.g., keyboard and mouse). It should be noted that although in Figure 1 While shown as separate devices, display device 120 and / or input device 118 may also be integrated with each other and / or with computer 110, such as in a tablet computer or a virtual reality (VR) or augmented reality (AR) system. For example, input / output devices 118, 120 may include VR input gloves 118a and VR headset 120a.

[0025] User 190 can interact with program 116 to (e.g., from document 130) create and / or load a 3D model 132 of object 180 to be manufactured by a computer-controlled manufacturing system (e.g., by a CNC machine 170, such as a multi-axis, multi-tool milling machine, etc.). This can be done using known graphical user interface tools, and the 3D model 132 can be defined in the computer using various known 3D modeling formats, such as using solid models (e.g., voxels) or surface models (e.g., B-Rep (boundary representation), surface meshes). Additionally, user 190 can interact with program 116 to modify the 3D model 132 of object 180 as needed.

[0026] In some implementations, 3D model 132 (e.g., from document 130) may include a 3D model of material (i.e., a “workpiece”) that can be removed by a CNC machine 170 during a subtractive manufacturing process. In some implementations, a separate 3D model of the material can be obtained via CAD / CAM program 116. The material can be removed by a CNC machine 170 following a desired toolpath. For ease of illustration, the material is shown as a heart shape, while the object to be manufactured is shown as a pentagon. This illustration does not correspond to typical material workpieces and manufactured objects encountered in the field of subtractive manufacturing (e.g., milling).

[0027] Once the 3D model 132 of object 180 is ready for manufacturing, the 3D model 132, which provides the physical structure for manufacturing object 180, can be prepared by generating toolpaths for use by a computer-controlled manufacturing system. For example, the 3D model 132 can be used to generate a toolpath specification document 160, which can be sent to a CNC machine 170 and used to control the operation of one or more milling tools. This can be done after a request from user 190 or in consideration of a user's request for another action, such as sending the 3D model 132 to a CNC machine 170 or other manufacturing machinery that can be directly connected to computer 110 or connected to said computer via network 140. This can involve post-processing steps performed on local computer 110 or a cloud service to export the 3D model 132 to an electronic document upon which manufacturing is based. It should be noted that an electronic document (which will be simply referred to as a document for simplicity) can be a file, but does not necessarily correspond to a file. A document can be stored as part of a file that holds other documents, as a single file dedicated to the document in question, or as multiple collaborative files.

[0028] In any case, program 116 may create one or more toolpaths in document 160 and provide document 160 (with appropriate format) to CNC machine 170 to create the physical structure of object 180 (note that in some implementations, computer 110 is integrated into CNC machine 170, and therefore toolpath specification document 160 is created by the same computer that will use toolpath specification document 160 to manufacture object 180). Program 116 may generate one or more toolpaths by providing a 3D model 132 of object 180 (e.g., from document 130) to machine learning algorithm 134. Machine learning algorithm 134 may automatically generate toolpath 172 (e.g., saved in document 160) that can be used by CNC machine 170 to manufacture object 180. This automated process, which does not require user 190 (e.g., via menus in UI 122) to specify the type and parameters of the desired toolpath, accelerates the toolpath generation process, reducing the time required for manufacturing planning and, similarly, the time required to manufacture the part. For example, CNC machine 170 may be a subtractive manufacturing machine that manufactures object 180 by removing material 136. CNC machine 170 may use toolpath 172 (e.g., stored in document 160) automatically generated by machine learning algorithm 134 to control cutting tool 174. For example, cutting tool 174 may include a cutting tool that may be programmed to remove excess material when manufacturing the object using subtractive manufacturing.

[0029] Program 116 may include a series of menus in UI 122 that allow user 190 to accept or reject one or more candidate tool paths automatically generated by machine learning algorithm 134. In some implementations, program 116 may include a series of menus in UI 122 that allow user 190 to adjust one or more portions of the candidate tool paths until the user is satisfied with the tool paths. Once the user accepts a candidate tool path, program 116 may save the candidate tool path in tool path document 160 and may provide document 160 to CNC machine 170 to manufacture the physical structure of object 180.

[0030] Figure 2A An example of a process is illustrated where a machine learning algorithm generates toolpaths for manufacturing the physical structure of a modeled object. For example, program 116 obtains 200 3D models of manufacturable objects. In other words, the geometry of the modeled object to be manufactured by a CNC machine is identified. This can be done automatically by a computer (e.g., by program 116 on computer 110) or by receiving user input. For example, a user can select desired faces, contours, or other geometries of the 3D model they want to manufacture. In some implementations, the program provides a user interface in which the user can directly select (e.g., click with a mouse) the geometry of interest (e.g., faces or contours).

[0031] In some implementations, after obtaining a 3D model of the object, one or more preprocessing procedures can be performed on the 3D model, for example, via program 116. For instance, program 116 can remove parts of the 3D model that are too tightly packed to fit into the available tools. As another example, program 116 can generate a set of 2D images representing cross-sections of the 3D model. Background regions, regions inside the object, and regions outside the object can be represented using different values ​​in the input of the machine learning algorithm, for example, by using different colors in the 2D images.

[0032] The three-dimensional model of an object can be represented in various ways. Possible representations may include image pixels, point clouds, voxels, meshes, contour plots, or any combination of two or more of the above representations. In some implementations, the representation may include a 2D image of a 2D view of the 3D object, or multiple 2D images of local geometry of the 3D model viewed from multiple angles. In some implementations, models of available tools, objects, and materials may use the same or different representations.

[0033] In some implementations, one or more of the available tool models, object models, and material models may use a continuous representation (defined by one or more smoothing functions) instead of a discretized representation (e.g., using pixels). For example, a milling tool may be represented by a continuous circle defined by the center coordinates and radius of the milling tool, instead of using a discrete pixel representation. The milling tool may be represented as a circle centered at the tool's center coordinates and with a radius equal to the tool's radius. Using this circular representation, program 116 can use the tool's radius to calculate whether the material's pixels are within the tool's range. As another example, the object model and / or the material model may have a continuous representation (defined by one or more smoothing functions, e.g., defined by continuous B-Rep) instead of a discretized representation (e.g., using pixels).

[0034] For example, program 116 generates one or more toolpaths 202 by providing at least a portion of a 3D model to a machine learning algorithm. A computer-controlled manufacturing system (e.g., CNC machine 170) uses the generated toolpaths to manufacture at least a portion of a manufacturable object. In some implementations, the toolpaths generated by the machine learning algorithm can be used to manufacture the entire object.

[0035] In some implementations, the machine learning model can generate a series of positions for the tool to traverse. In some implementations, the machine learning model can generate a series of velocity vectors for the tool (e.g., directions that allow the tool to accelerate). For example, the positions and velocity vectors can be represented in pixels, and the tool can move a certain number of pixels in each step. The series of positions can include a series of pixel coordinates in a 2D environment. In a 3D environment, the series of positions of the tool can include the coordinates of voxels in 3D and the 3D orientation of the tool. The CAD / CAM program 116 can generate one or more splines in a post-processing step connecting all or part of the series of positions. The one or more splines can be saved as a tool path and can be used to control the tool to travel smoothly through these points. In some implementations, the machine learning model can generate tool control mechanisms, such as tool angles. The CAD / CAM program 116 can utilize the tool control mechanisms generated by the machine learning algorithm to generate the tool path.

[0036] Generally, machine learning algorithms construct mathematical models based on training data. These algorithms take at least a portion of a three-dimensional model of an object as input. They may also take a representation of the environment as input, such as a model of material to be removed during manufacturing. In some implementations, the model of the material from which the object is to be cut can be a default model used by the program, or it can be provided to the program by the user or another process. In some implementations, the environment can be represented using ray tracing, i.e., using a set of rays from the tool to the material to describe the current environment.

[0037] A trainable machine learning algorithm can be used to generate tool paths with a set of desired tool path characteristics. Figure 2B This is a flowchart illustrating an example of training a machine learning algorithm that generates tool paths through reinforcement learning. Definition 232 includes one or more scoring functions that relate a reward to a desired tool path characteristic. The machine learning algorithm may employ reinforcement learning to include one or more scoring functions that include a reward related to the desired tool path characteristic. The machine learning algorithm may include a reward for desired tool path behavior and may include a penalty for other undesired tool path behavior. The primary objective of the reward is to discourage undesirable cutting tool behavior while encouraging good cutting and discouraging bad cutting. Examples of undesirable tool behavior may include moving into a CAD model and remaining in one position without indeterminate direction.

[0038] Desired toolpath characteristics may include maximizing toolpath smoothness (e.g., a smooth trajectory at the tool center), minimizing toolpath length, and minimizing machining time. For example, a toolpath that suddenly makes a 90-degree turn may not be desirable. A toolpath that executes a zigzag path may also be undesirable. Penalties or negative rewards can be applied to these undesirable toolpath characteristics.

[0039] The desired toolpath characteristics may also include avoiding collisions with the 3D model. In some implementations, the machine learning model may include hard constraints to prevent the tool from colliding with the CAD model. In some implementations, the machine learning algorithm may include a penalty function that penalizes the machine learning model for attempting to move into the CAD model.

[0040] In some implementations, the desired toolpath characteristics may also include selecting and optimizing one side of the tool in the toolpath used in the computer-aided manufacturing process. In a given step of the computer-aided manufacturing process, the tool may include a correct side (i.e., the correct region), an incorrect side (i.e., the incorrect region), and a neutral side (i.e., the neutral region) between the correct and incorrect sides of the tool. Using the correct side of the tool in a given step, the toolpath can generate a good cut, for example, pixels of material removed by the correct side of the tool. Using the incorrect side of the tool in a given step, the toolpath can generate a poor cut, for example, pixels of material removed by the incorrect side of the tool. Using the neutral region of the tool, the toolpath can generate a neutral cut, for example, pixels of material removed by the neutral region between the correct and incorrect sides of the tool.

[0041] Selecting and optimizing between the correct, incorrect, and neutral sides of the tool is a crucial constraint that encourages machine learning algorithms to create desired toolpaths. The desired toolpath motion removes material while also exposing the correct side of the tool. Machine learning algorithms can be trained to generate appropriate toolpaths that utilize the correct side of the tool based on its direction of movement. For example, if the tool is in the same position but traveling in different directions, the correct and incorrect sides of the tool will differ relative to its direction of travel. As another example, desired toolpath characteristics could include removing as much material as possible as quickly as possible while using only the correct side of the tool.

[0042] The correct, incorrect, and neutral sides of a cutting tool can each occupy a specific percentage of the tool. For example, the correct, incorrect, and neutral sides could occupy 49%, 49%, and 2% of the tool, respectively. As another example, the correct, incorrect, and neutral sides could occupy 20%, 70%, and 10% of the tool, respectively. When the correct side of the tool occupies a smaller percentage of the tool, the tool can remove a smaller amount of material, and machine learning algorithms can be trained to generate smaller updates as the side of the tool is selected and changed at each step.

[0043] In some implementations, desired toolpath characteristics may also include optimizing tool engagement of the selected cutting tool, maximizing the smoothness of the contact trajectory of the selected cutting tool (e.g., the tool contact trajectory is smooth), minimizing tool axis variation, maximizing the smoothness of tool axis variation, avoiding leaving small pieces of material, limiting tool engagement angles, or any other suitable toolpath characteristics. For example, the machine learning algorithm may include rewards for using good portions of the tool (e.g., the edge of the tool) during climb milling or other milling operations to avoid using bad portions of the tool (e.g., the middle portion of a ball end mill or the bottom portion of a bullnose tool), and / or to avoid using the wrong side of the tool (e.g., in climb milling or conventional milling). As another example, if too many pixels in the model's image representation are engaged simultaneously (e.g., touched by the tool), the machine learning algorithm may limit the tool engagement angle by penalizing the score.

[0044] In some implementations, the machine learning algorithm may include rewards for tool position and / or rotational characteristics (e.g., rotation in place or forward movement). For example, the machine learning algorithm may include rewards that discourage it from continuously deciding to change direction in a way that causes the tool to rotate around a point in the environment (i.e., rotation in place). In some implementations, rewards for tool position and / or rotational characteristics may be combined with tool path smoothness rewards to generate smooth tool paths, such as generating smooth cuts around a part. At each step, the machine learning algorithm may include rewards for maintaining or changing the tool's rotational direction. The machine learning algorithm may include rewards for rotating the tool in the same position before moving it to a different position. In some implementations, at each step, the machine learning algorithm may include rewards for always moving the tool to a different position and preventing the tool from staying in the same position. In some implementations, the machine learning algorithm may include rewards that allow the tool to rotate in place or move forward, but not simultaneously, which can help increase the completion percentage and prevent the tool from hitting the CAD model. In some implementations, the machine learning algorithm may include a reward for allowing the tool to rotate only counterclockwise (i.e., in the opposite direction of clockwise) or only clockwise, or allowing the tool to rotate both clockwise and counterclockwise. For example, in some implementations, it might be preferable to allow the tool to rotate counterclockwise rather than clockwise because the default counterclockwise direction of rotation exposes the correct side of the tool when it strikes the CAD model. As another example, in some implementations, it might be desirable to allow the tool to rotate both clockwise and counterclockwise, for example, turning clockwise in one step and then counterclockwise in the next step and repeating both steps, which can help produce a smoother external profile of the manufactured object. In some implementations, being able to rotate in both directions makes it easier for the tool to produce a smooth toolpath because the tool does not need to make several consecutive decisions about turning in the same direction to face a particular orientation.

[0045] In some implementations, desired toolpath characteristics may include setting the tool's turning direction based on the tool's position relative to the model of the manufacturable object. When the tool's position is greater than a threshold distance from the model of the manufacturable object, one or more scoring functions may include one or more rewards that encourage free choice of the tool's turning direction. For example, when the tool is far from the CAD model (e.g., more than 1 mm), the system may be free to choose a turning direction to create a smoother toolpath. When the tool's position is within a threshold distance from the model of the manufacturable object, one or more scoring functions may include one or more rewards that encourage the tool to turn only in one direction, which may result in the correct side of the tool being exposed based on the rotation direction. For example, when the tool is closer to the CAD model (e.g., less than 1 mm), the system may cause the tool to turn only counterclockwise to prevent the tool from being blocked by the CAD model.

[0046] In some implementations, the machine learning algorithm may include a reward for the representation of removed material. The machine learning algorithm may increase the reward for removing material close to the CAD model and / or increase the reward for removing material as more material is removed. In some implementations, the machine learning algorithm may include stage-based rewards. Stage-based rewards may include larger rewards for higher completion levels (including up to 100%) and can help ensure that the machine learning algorithm completes the removal of all material around the CAD model. For example, different levels of rewards may be set at 50%, 95%, and 99% completion, or at 80%, 95%, 99%, and 100% completion. A significantly high reward at 100% completion prevents the machine learning algorithm from removing only material farther from the CAD model before actually completing the machining of the CAD model (e.g., determining it has earned enough reward). As another example, the reward for removing a small amount of material may be increased when only a small amount remains. Machine learning algorithms trained using stage-based rewards can generate toolpaths with higher percentages of completion (including up to 100%) for milling the modeled object.

[0047] In some implementations, the machine learning algorithm may employ variable tool feed (e.g., percentage of the tool used), variable tool speed (e.g., feed rate), or variable cutting force. In some implementations, the machine learning algorithm may employ subcycloidal motion. For example, if the tool needs to cut through a material channel with CAD models on both sides, the tool can move in a subcycloidal motion to avoid excessive tool engagement. As another example, if the tool is traveling outside the material, the tool can move in a helical motion instead of a subcycloidal motion. The use of a variable feed rate reduces the potential need to achieve optimal engagement 100% of the time by allowing the tool to accelerate or decelerate during otherwise undesirable machining operations (e.g., heavy cutting) that the machine learning algorithm might generate. Therefore, the machine learning algorithm can achieve good engagement most of the time (e.g., 99% of the time) and can easily slow down the tool during any undesirable machining operations (e.g., heavy cutting) that may occasionally occur, rather than trying to make the machine learning algorithm always achieve 100% tool engagement compliance behavior.

[0048] Examples of reward functions could be functions of the following factors:

[0049] The number of good cuts (e.g., the number of pixels of material removed by the correct side of the tool in a given step).

[0050] The number of defective cuts (e.g., the number of pixels in a CAD model removed by the wrong side of the tool in a given step).

[0051] The number of neutral cuts (e.g., the number of pixels in a CAD model that come into contact with the neutral region between the correct and incorrect sides of the tool in a given step).

[0052] Are there any defects in the cutting process?

[0053] Did you bump into the CAD model?

[0054] Is the tool in a location it has been to before?

[0055] Is the tool rotating in place, or is the tool speed zero?

[0056] Has the completion threshold been reached?

[0057] The reward function may include a weighted coefficient for each of the factors. Positive weighted coefficients may be assigned to desired toolpath characteristics, such as the number of good pixel cuts. Negative weighted coefficients (e.g., indicating penalties) may be assigned to undesired toolpath characteristics, such as the number of bad pixel cuts, or the fact that a CAD model has been hit. The values ​​of the weighted coefficients may be predetermined or learned when training a machine learning algorithm.

[0058] In some implementations, the machine learning algorithm may include long-term rewards, short-term rewards, or a combination of both. In some implementations, the machine learning algorithm may apply a reward function to each of the multiple steps in the generated toolpath. The total reward may be the sum of all rewards corresponding to the multiple steps. In some implementations, one or more discount rates may be applied to the rewards over time. The discount rate determines how much the reinforcement learning algorithm evaluates the long-term reward relative to the recent reward. The discount rate may be a value between 0 and 1. For example, the discount rate may be set to 0.99.

[0059] The input to a machine learning algorithm can be observations of its environment. In some implementations, at each step, the machine learning algorithm can determine the tool's position based on a local view of the model, without requiring information about the entire model. For example, the machine learning algorithm can use high-resolution images near the tool's current position (e.g., image data of the material and CAD model only within a defined distance from the tool's edge) to efficiently determine how the tool should interact with the material, said defined distance being a quarter or half of the tool's diameter, or simply the tool's diameter. In some implementations, the machine learning algorithm can take one or more views of the environment as input and can make decisions based on one or more views of the environment. For example, for a 3D environment, two or more 2D views of the environment can be provided as input to the machine learning algorithm. In some implementations, the machine learning algorithm can take one or more views of the environment at the current step and one or more views of the environment at one or more previous steps as input. For example, the observations of the environment can include three images: the current 2D view of the environment and two 2D views of the environment from two previous steps.

[0060] The output of a machine learning algorithm can be a tool path comprising a series of tool positions across multiple steps. Each tool position represents where the CAM system should move the representation of the cutting tool. For example, each tool position can be the (x, y, z) coordinates of the tool head. The series of tool positions can be adjacent to each other (e.g., turning forward or moving one pixel in the 2D image representation) or spaced further apart (e.g., moving a longer distance in a single step).

[0061] Machine learning algorithms can employ various reinforcement learning algorithms. Examples of reinforcement learning algorithms include Q-learning, State-Action-Reward-State-Action (SARSA), Deep Q-Learning Network (DQN), Asynchronous Advantage Performer-Commentator (A3C) Network, Deep Deterministic Policy Gradient (DDPG), Hybrid Reward Architecture (HRA), etc. Reinforcement learning algorithms can employ online or offline learning, same-policy or off-policy learning, hierarchical reinforcement learning, etc. In some implementations, reinforcement learning algorithms may include recurrent neural networks that use previous output states as inputs to the next step, such as gated recurrent units (GRUs) or long short-term memory (LSTM) neural networks. The neural network architecture may include convolutional neural networks (CNNs), including one or more convolutional layers of configurable size, one or more fully connected layers, one or more activation layers, or skip connections between said layers, etc.

[0062] Figure 2CThis is a schematic diagram illustrating an example of a neural network architecture 210 for a machine learning algorithm used to generate tool paths for manufacturing the physical structure of a modeled object. This implementation employs an unsupervised machine learning algorithm that does not require expert-made sample solutions in the training examples. This neural network architecture uses a reinforcement learning algorithm, specifically an advantage performer-commentator architecture. The input to the machine learning algorithm can be observations 212 describing the environment of the object model 220, the material model 222, and the available tool model 224. For example, the input to the machine learning algorithm can be a 2D image 214 corresponding to a square region around the tool as the tool 224 traverses the environment.

[0063] Neural network architecture 210 may include a convolutional neural network (i.e., ConvNet 216) that can generate one or more feature vectors from observations 212. For example, ConvNet 216 may include convolutional layers of size 4 and stride 2 that can generate feature vectors with 32 channels. A recurrent neural network, such as GRU 220, can generate a state vector h generated by GRU 220 in a previous time step. i The recurrent neural network, such as GRU 220, takes one or more feature vectors generated from ConvNet 218 and ConvNet 216 as input. It generates the state vector h for the current time step. i+1 222. For example, the state vector h i+1 222 can have a predetermined length of 256. The state vector h at the current time step. i+1 222 can be processed through one or more linear operations 223. The output of the neural network architecture may include multiple actions 224 and one or more values ​​226. Actions 224 may describe the tool path used to create the modeled object, such as the velocity vector of moving the tool, or the direction in which the tool accelerates. The one or more values ​​226 may represent the state corresponding to the state vector h. i+1 The value score for a specific state of 222. For example, a state where the tool is very close to the CAD model may have a lower value score because the tool may collide with the CAD model and thus result in a larger negative reward. In some implementations, the Softmax function 228 may be applied to action 224, and the output of the Softmax function 228 may include a probability distribution of possible actions.

[0064] Machine learning algorithms can determine a sequence of tool positions across multiple steps based on what they have learned from past experience gained during the training process. Training examples, including exemplary tools and environments, can be used to train machine learning algorithms. See again. Figure 2BThe system can receive over 234 training examples, and each training example may include an exemplary tool and an exemplary environment. During training, the parameters of the machine learning algorithm (e.g., a set of weights) may be iteratively updated based on the training examples until a stopping criterion is met. The training examples may include samples from real CAM processes, simulated CAM processes, or a combination of both. For example, training examples may include real or simulated cutting force information from one or more milling tools, and such cutting force information may also be added to a scoring mechanism during training. The training examples may include 2D or 3D environments. The training examples may include representations of 2D or 3D tools. In some implementations, one or more preprocessing operations may be performed on the training examples. For example, if a portion of the exemplary environment is too tight to fit into a usable tool, that portion may be removed, such that the training example comprises a CAD model that can be 100% completed. Therefore, the machine learning algorithm can be trained to remove all material from the training examples.

[0065] 236 training examples can be used to train a machine learning algorithm to generate tool paths that maximize values ​​produced by one or more scoring functions. In some implementations, the machine learning algorithm may employ unsupervised training of a reinforcement learning algorithm. During unsupervised training, the machine learning algorithm does not receive desired outputs or expert-labeled sample solutions. The reinforcement learning algorithm determines the output by maximizing one or more scoring functions, which include rewards associated with desired tool path characteristics. For example, a reinforcement learning algorithm can be trained to maximize rewards received from observed processing environments. By designing the rewards to be associated with desired tool path characteristics, the reinforcement learning algorithm can be trained to produce desired tool paths.

[0066] In some implementations, the machine learning algorithm may employ an appropriate training method for the chosen reinforcement learning algorithm. For example, an asynchronous training method can be used to train a reinforcement learning algorithm based on an actor-commenter network. In the asynchronous training method, in each iteration, a copy of the reinforcement learning network with the current set of weights can be created. Each copy of the network can run its own simulation by interacting with a part of the environment. The current performance of the copies can be collected from the simulations accumulated over a certain number of steps. An update to the set of weights can be computed based on the collected performance using an optimization algorithm (e.g., stochastic gradient descent (SGD) with or without momentum, root mean square propagation (RMSProp) with or without shared statistical information, etc.). The set of weights can be iteratively updated based on the performance of the reinforcement learning model until a stopping criterion is met, e.g., a fixed number of iterations have been completed, the change in weights is less than a threshold, or the limit of accuracy has been reached.

[0067] In some implementations, off-policy training can be used instead of same-policy training when training machine learning algorithms. Off-policy training can evaluate and train machine learning algorithms using sample toolpaths generated from different sources other than the machine learning algorithm itself. Sample toolpaths generated from different sources can include real-life toolpath data already used in computer-aided manufacturing, or toolpath data designed by humans with or without templates. For example, reinforcement learning algorithms can be evaluated for their performance, and the algorithm's parameters can be learned from expert-made toolpaths. In some implementations, experience replay optimization can be used when training reinforcement learning algorithms. Experience replay can help improve sample efficiency by allowing the reuse of samples and potentially allowing the use of training samples with relevant and challenging scenarios that will be used more frequently.

[0068] After training, the machine learning algorithm can generate toolpaths for creating objects not present in the training examples or objects the algorithm has not yet trained on. Additional training examples representing one or more new objects (e.g., one or more new parts) are available. The machine learning algorithm can be further trained using a combination of existing and additional training examples. In some implementations, for faster training, the machine learning algorithm can be trained by fine-tuning based on a previously trained model; that is, the parameters of the machine learning model are updated from previously learned parameters rather than being computed from scratch (e.g., random numbers or zero). The toolpaths generated by the machine learning algorithm for these new parts can be further improved after training with new training examples. In some implementations, the additional training examples may include data corresponding to user modifications to previously generated toolpaths by the machine learning algorithm. The data corresponding to user modifications can be used to train an improved machine learning algorithm that generates more desirable toolpaths.

[0069] In some implementations, the machine learning algorithm may include two or more machine learning algorithms. A first machine learning algorithm of the two or more machine learning algorithms may be used to process at least a portion of the 3D model. A second machine learning algorithm of the two or more machine learning algorithms may be used to further process the portion of the 3D model.

[0070] In some implementations, the first machine learning algorithm of the two or more machine learning algorithms may include a convolutional neural network (CNN) for generating data (e.g., image features) from a portion of a 3D model. Examples of CNNs may include AlexNet, InceptionNet, ResNet, DenseNet, etc., or other types of CNNs capable of performing image recognition tasks. In some implementations, the machine learning algorithm may take as input a 2D image representing a 2D cross-section of a 3D model of an object and material in the environment. The convolutional neural network may efficiently extract useful image features from the 2D image by using two or more convolutional layers that perform a series of linear and nonlinear operations. The extracted image features may represent the relationship between the remaining material, the object model, and the tool's position. The second machine learning algorithm of the two or more machine learning algorithms may be used to process the generated data, such as the extracted image features. For example, the second machine learning algorithm may be a reinforcement learning network (e.g., an asynchronous advantage enforcer-commenter (A3C) network) capable of generating tool paths in computer-controlled manufacturing.

[0071] In some implementations, the first machine learning algorithm of the two or more machine learning algorithms can operate on a low-resolution view of at least a portion of the 3D model. The second machine learning algorithm of the two or more machine learning algorithms can operate on a high-resolution view of the portion of the 3D model. For example, the first algorithm can generate multiple starting positions to locate a tool using a low-resolution view of the object's model. Based on the high-resolution view surrounding each starting position, the second algorithm can generate a tool path starting from each starting position generated by the first algorithm, and the tool path can be used to create local portions of the object. (The following is in conjunction with...) Figure 3 Describe the details of the two or more machine learning algorithms.

[0072] See again Figure 2A For example, program 116 provides the user with a tool path generated by a machine learning algorithm to determine whether the tool path 204 is an acceptable final tool path for the object. Program 116 may include a UI element in UI 122 that allows the user 190 to accept or reject one or more candidate tool paths automatically generated by the machine learning algorithm. For example, the user may watch a video simulating the process of creating an object using one or more candidate tool paths.

[0073] If the user determines that the generated toolpath is unacceptable for at least a portion of the manufacturable object, program 116 can use a machine learning algorithm to generate an updated toolpath. In some implementations, program 116 may include UI elements in UI 122 that allow the user 190 to specify the desired updated toolpath characteristics. The machine learning algorithm can generate the updated toolpath using one or more scoring functions, which include a reward associated with the updated toolpath characteristics. In some implementations, program 116 may include UI elements in UI 122 that allow the user 190 to manually edit one or more portions of the candidate toolpath until the user is satisfied with the toolpath. Furthermore, in cases where the machine learning algorithm generates a toolpath that cannot remove one or more pieces of material, the user 190 can use UI elements in UI 122 to add to the automatically generated toolpath to ensure that all material is removed during the subtractive manufacturing process; that is, the generated toolpath can be expanded in addition to being modified.

[0074] Once the user determines that the toolpath generated by 204 is acceptable for manufacturing at least a portion of the manufacturable object, the toolpath is provided, for example, by program 116 to 206 to a computer-controlled manufacturing system for manufacturing at least a portion of the manufacturable object. In some implementations, program 116 may store candidate toolpaths in Figure 1 The tool path is in document 160. Program 116 can provide document 160 to CNC machine 170 to manufacture the physical structure of object 180.

[0075] A computer-controlled manufacturing system manufactures at least a portion of a 208 manufacturable object using toolpaths generated by machine learning algorithms. The manufacture of the modeled object may involve roughing operations, finishing operations, and optionally, semi-finishing operations between these two. A roughing operation may involve removing most of the material, but leaving some material on the modeled object. A finishing operation may involve removing all remaining material and producing a final manufactured object with a good surface finish. Each of the roughing, finishing, and semi-finishing operations may have its own toolpath. Machine learning algorithms can be used to generate toolpaths for roughing, finishing, or semi-finishing operations.

[0076] Figure 3An example of a process for generating toolpaths for 2.5-axis machining using a machine learning algorithm is shown. 2.5-axis machining is a subtractive manufacturing process. While 2.5-axis machining utilizes a 3-axis milling machine that can move in all three individual dimensions, during most cutting operations, the milling cutter moves relative to the workpiece in only two axes, resulting in a more efficient manufacturing process. The subtractive process in 2.5-axis machining occurs in discrete steps parallel to the milling cutter, with continuous movement in a plane perpendicular to the milling cutter. Compared to 3-axis subtractive manufacturing, 2.5-axis subtractive manufacturing processes can rapidly and sequentially remove material layers and can produce parts that often have a series of "cavities" of varying depths.

[0077] For example, program 116 obtains a 3D model 320 of an object for 2.5-axis machining by a computer-controlled manufacturing system. 2.5-axis generative design can use generative design software to generate a CAD model of a 3D object that includes multiple discrete layers. For example, CAD model 320 may have three layers, including a bottom layer, a middle layer, and a top layer.

[0078] For example, program 116 generates multiple two-dimensional representations 322 of the three-dimensional model described in 304. Multiple 2D representations can be generated at discrete 2D layers of the 3D model during a preprocessing step. Each 2D representation can be an image representing a cross-section of the 3D model of an object. For example, 2D representation 322 can be an image representing a cross-section of the 3D model 320 at the height of an intermediate layer. 2D representation 322 may include a region 326 representing an object (e.g., a part), and a region 328 outside the object (e.g., outside the part) where material needs to be removed.

[0079] For example, the two-dimensional representation is provided to a machine learning algorithm via procedure 116. The machine learning algorithm 306 can be trained to generate toolpaths for 2.5-axis machining; that is, the machine learning algorithm 306 operates only in two dimensions, even if it creates toolpaths that can be used to manufacture objects in three dimensions. For example, procedure 116 can generate, based on multiple 2D representations, a toolpath 324 308 that can be used to manufacture at least a portion of an object using 2.5-axis machining. In other words, each 2D representation can be provided to the machine learning algorithm to generate a corresponding set of toolpaths for manufacturing each discrete 2D layer. In some implementations, the final toolpath can be generated by combining all sets of toolpaths corresponding to multiple 2D representations of a 3D model.

[0080] The toolpath 310 can be provided to a computer-controlled manufacturing system to manufacture at least a portion of an object using 2.5-axis machining. For example, toolpath 324 can be provided to remove material at the lower right portion 330 of the object using 2.5-axis machining.

[0081] In some implementations, the machine learning algorithm may include two or more machine learning algorithms. At least one starting position for the tool can be generated by processing a global view of the 3D model using a first machine learning algorithm from the two or more machine learning algorithms. For each of the at least one starting position, a set of tool paths (e.g., an array of pixel values ​​representing the edges of the tool surface or multiple sets of values ​​representing concentric circles extending from the tool surface) can be generated by processing a local view of the 3D model near each starting position using a second machine learning algorithm from the two or more machine learning algorithms. The manufacturing process can operate in a transfer-and-remove manner. In each iteration, the tool can be rapidly moved to the desired starting position without performing any cutting operations. The tool can then perform cutting in a local area near at least one starting position. This method of using two or more machine learning algorithms to perform long-term planning followed by local cutting can be applied to various types of computer-controlled manufacturing systems, not just 2.5-axis machining.

[0082] For example, image representation 323 of the object shows four local regions 330, 332, 334, and 336 outside the object. If a tool only moves in a 2D plane perpendicular to the tool, a tool working in another region (e.g., region 330) may be unable to access some regions (e.g., region 336). A first machine learning algorithm generates four starting positions for manufacturing each of the four regions 330, 332, 334, and 336. A second machine learning algorithm generates toolpaths that can be used to remove material from each of the four regions 330, 332, 334, and 336.

[0083] In some implementations, generating at least one starting position may include processing a global view of the 3D model using a first machine learning algorithm of two or more machine learning algorithms, employing a discretized representation of the 3D model of the manufacturable object and a model of the material from which at least a portion of the manufacturable object will be manufactured. The discretized representation of the object and material reduces computation and improves the efficiency of the first machine learning algorithm. In some implementations, generating the set of toolpaths may include processing the local view using a second machine learning algorithm of the two or more machine learning algorithms, employing a continuous representation of the tool's model, in a computer-controlled manufacturing system to be used for manufacturing at least the portion of the manufacturable object. For example, the system may use the continuous representation of the tool as input to the second machine learning algorithm of the two or more machine learning algorithms to generate an accurate toolpath that performs local cutting close to the CAD model.

[0084] In some implementations, generating at least one starting position may include processing a global view using a discretized representation of the tool's model using a first machine learning algorithm from the two or more machine learning algorithms. In some implementations, generating the set of toolpaths may include processing a local view using a continuous representation of the 3D model of the manufacturable object and the model of the material using a second machine learning algorithm from the two or more machine learning algorithms. For example, the system may use the continuous representation of the 3D model of the object and the model of the material as input to the second machine learning algorithm from the two or more machine learning algorithms to generate portions of a toolpath that perform local cuts close to the CAD model. Continuous representations of the object and material can improve the accuracy of local cuts close to the CAD model.

[0085] In some implementations, the system may use a discretized representation of the 3D model of the manufacturable object and the model of the material in a first and second machine learning algorithm of two or more machine learning algorithms, and the system may use a high-resolution discretized representation of the model during the processing of a local view using the second machine learning algorithm of the two or more machine learning algorithms. For example, the system may use a low-resolution discretized representation (e.g., an image) of the 3D model of the object to process a global view using the first machine learning algorithm of the two or more machine learning algorithms, and each pixel in the image may have a physical size of 5mm × 5mm. The system may use a high-resolution discretized representation (e.g., an image) of the 3D model of the object to process a local view using the second machine learning algorithm of the two or more machine learning algorithms, and each pixel in the image may have a physical size of 0.5mm × 0.5mm.

[0086] In some implementations, after most of the material has been removed, some small pieces may still remain. These small pieces that need to be removed may not be close to each other. The aforementioned method of transferring and then removing can effectively remove small pieces that are far apart from each other. Machine learning algorithms can utilize a global view of all remaining pieces of material and can quickly send the tool to the starting position of the next piece of material, rather than using only a local view near the tool and searching for the next piece of material.

[0087] Figure 4 This is a schematic diagram of a data processing system including a data processing device 400, which can be programmed as a client or server. The data processing device 400 is connected to one or more computers 490 via a network 480. Although in Figure 4Only one computer is shown as data processing device 400, but multiple computers may be used. Data processing device 400 includes various software modules that can be distributed between the application layer and the operating system. These may include executable and / or interpretable software programs or libraries, including tools and services for implementing the 3D modeling / simulation and manufacturing control programs 404 of the systems and technologies described above. The number of software modules used may vary depending on the implementation. Furthermore, software modules may be distributed across one or more data processing devices connected via one or more computer networks or other suitable communication networks.

[0088] The data processing device 400 also includes hardware or firmware means including one or more processors 412, one or more additional devices 414, a computer-readable medium 416, a communication interface 418, and one or more user interface devices 420. Each processor 412 is capable of processing instructions for execution within the data processing device 400. In some implementations, the processor 412 is a single-threaded or multi-threaded processor. Each processor 412 is capable of processing instructions stored on the computer-readable medium 416 or on a storage device such as one of the additional devices 414. The data processing device 400 communicates with one or more computers 490, for example, over a network 480, using its communication interface 418. Examples of user interface devices 420 include displays, cameras, speakers, microphones, haptic feedback devices, keyboards, mice, and VR and / or AR devices. The data processing device 400 may, for example, store on a computer-readable medium 416 or one or more additional devices 414 instructions for performing operations associated with the program described above, such as one or more of a hard disk device, an optical disk device, a magnetic tape device, and a solid-state storage device.

[0089] The embodiments of the subject matter and functional operation described in this specification can be implemented in digital electronic circuits or computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations thereof. Embodiments of the subject matter described in this specification can be implemented using one or more modules of computer program instructions encoded on a non-transitory computer-readable medium for execution by a data processing device or for controlling the operation of a data processing device. The computer-readable medium can be an manufactured product, such as a hard disk drive in a computer system, or an optical disc sold through retail channels, or an embedded system. The computer-readable medium can be obtained separately, or it can be encoded later with one or more modules of computer program instructions, for example, via a wired or wireless network. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination thereof.

[0090] The term "data processing device" encompasses all devices, apparatuses, and machines used for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to hardware, the device may also include code that creates the execution environment of the computer program in question, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, runtime environments, or combinations thereof. Furthermore, the device may employ a variety of different computing model infrastructures, such as network services, distributed computing, and grid computing infrastructures.

[0091] Computer programs (also referred to as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs or modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored as a portion of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), as a single file dedicated to the program in question, or as a collection of coordinating files (e.g., files storing portions of one or more modules, subroutines, or code). A computer program can be deployed to execute on a single computer, or on multiple computers located in one location or distributed across multiple locations and interconnected via a communication network.

[0092] The processes and logic flows described in this specification can be executed by one or more programmable processors, which execute one or more computer programs to perform functions by manipulating input data and producing outputs. The processes and logic flows can also be executed by dedicated logic circuitry, and the device can also be implemented as dedicated logic circuitry, such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0093] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, as well as any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to said one or more mass storage devices to receive data from or transfer data to them, or both. However, a computer does not need to have such devices. Additionally, a computer may be embedded in another device, such as (to name only) a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive). Suitable devices for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including (for example): exemplary semiconductor memory devices, such as EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented by dedicated logic circuitry, or the processor and memory may be incorporated into dedicated logic circuitry.

[0094] To enable interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having: a display device, such as an LCD (liquid crystal display), an OLED (organic light-emitting diode) display device, or another monitor for displaying information to a user; and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to enable interaction with the user; for example, feedback provided to the user can be any form of perceptual feedback, such as visual feedback, auditory feedback, or tactile feedback; and the ability to receive input from the user in any form, including auditory, voice, or tactile input.

[0095] The computing system may include clients and servers. Clients and servers are generally geographically isolated and typically interact via a communication network. The client-server relationship arises from computer programs running on respective computers and having a client-server relationship with each other. Embodiments of the subject matter described herein can be implemented in a computing system comprising: back-end components, such as a data server; or middleware components, such as an application server; or front-end components, such as a client computer with a graphical user interface or web browser, through which a user can interact with an implementation of the subject matter described herein; or any combination of one or more such back-end components, middleware components, or front-end components. Components of the system may be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), the internet (e.g., the Internet) and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0096] While this specification contains numerous implementation details, these details should not be construed as limiting the scope of the claimed or claimable content, but rather as descriptions of features specific to particular embodiments of the disclosed subject matter. Certain features described in this specification within the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described within the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations, and even initially claimed to be so, one or more features from a claimed combination may, in some cases, be removed from said combination, and a claimed combination may be for sub-combinations or variations thereof.

[0097] Similarly, although operations are depicted in a specific order in the diagrams, this should not be construed as requiring such operations to be performed in the shown specific order or in a sequential order, or requiring all of the described operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be construed as requiring this separation in all embodiments, and it should be understood that the described program components and systems may be substantially integrated together in a single software product or packaged into multiple software products.

[0098] Therefore, specific embodiments of the invention have been described. Other embodiments are within the scope of the appended claims.

Claims

1. A method, the method comprising: Obtain a three-dimensional (3D) model of the manufacturable object; A toolpath is generated by feeding at least a portion of the 3D model to two or more machine learning algorithms during training, enabling a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object, wherein the generation includes: By processing the portion of the 3D model using a first machine learning algorithm from the two or more machine learning algorithms, data including image feature data or at least one starting position of the tool path is generated, the image feature data or the at least one starting position being input to create the tool path. The tool path is generated by processing data including the image feature data or at least the starting position of the tool path using a second machine learning algorithm among the two or more machine learning algorithms, wherein the at least second machine learning algorithm among the two or more machine learning algorithms employs reinforcement learning including one or more scoring functions, the one or more scoring functions including rewards associated with one or more desired tool path features, and the second machine learning algorithm among the two or more machine learning algorithms is different from the first machine learning algorithm among the two or more machine learning algorithms. as well as The toolpath is provided to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.

2. The method of claim 1, wherein the generation comprises: The first machine learning algorithm of the two or more machine learning algorithms is used to process the global view of the portion of the 3D model to generate the data including at least one starting position of the tool path; as well as The second machine learning algorithm, one of the two or more machine learning algorithms, processes a local view of the portion of the 3D model around at least each of the at least one starting position to generate the tool path near each of the at least one starting position.

3. The method of claim 2, wherein the global view comprises a low-resolution representation of at least a portion of the three-dimensional model, and the local view comprises a high-resolution representation of a portion of the three-dimensional model around each of at least one of the at least one starting positions.

4. The method of claim 1, wherein the first machine learning algorithm of the two or more machine learning algorithms comprises a convolutional neural network, and the data comprises image feature data extracted by the convolutional neural network from one or more images of one or more two-dimensional cross-sections representing at least the portion of the three-dimensional model.

5. The method of claim 1, further comprising: Define one or more scoring functions, wherein the one or more scoring functions include rewards related to the one or more desired tool path features; It accepts multiple training examples, each of which includes example tools and example environments; and The training example is used to train a second machine learning algorithm employing reinforcement learning among the two or more machine learning algorithms to generate a tool path that maximizes one or more values ​​generated by the one or more scoring functions.

6. The method as described in claim 1, characterized in that, The two or more machine learning algorithms generate toolpaths that include the angles of the cutting tools used by the computer-controlled manufacturing system.

7. The method of claim 1, wherein, The second machine learning algorithm of the two or more machine learning algorithms generates a series of positions for a tool used by a computer-controlled manufacturing system, the tool path including one or more splines connecting the series of positions of the tool, and providing the tool path to the computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object includes providing the one or more splines to control the tool to travel along the one or more splines in a smooth path through the series of positions.

8. The method of claim 1, wherein, The second of the two or more machine learning algorithms operates in a three-dimensional environment and uses a three-dimensional representation of the tools used by the computer-controlled manufacturing system.

9. The method of claim 1, wherein the one or more desired tool path features include the desired side of the tool in the tool path.

10. The method of claim 1, wherein, The one or more desired tool path features include the tool's rotation features.

11. The method as described in claim 1, characterized in that, The one or more desired tool path features include tool engagement of the selected cutting tool.

12. The method of claim 1, wherein, The second machine learning algorithm in the two or more machine learning algorithms uses the percentage of tools used.

13. The method of claim 1, wherein the computer-controlled manufacturing system is a multi-axis, multi-tool milling machine.

14. The method of claim 1, wherein, The two or more machine learning algorithms are trained to generate toolpaths for 2.5-axis machining.

15. The method of claim 14, wherein the three-dimensional model is obtained using generative design software that generates a three-dimensional model comprising multiple discrete layers.

16. A system comprising: A data processing device, the data processing device including at least one hardware processor; as well as A non-transitory computer-readable medium encoded with instructions configured to cause the data processing device to perform operations, said operations including: Obtain a three-dimensional (3D) model of the manufacturable object; By feeding at least a portion of the 3D model to two or more machine learning algorithms during training, a toolpath capable of being manufactured by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object is generated, wherein the generation includes: By processing the portion of the 3D model using a first machine learning algorithm from the two or more machine learning algorithms, data including image feature data or at least one starting position of the tool path is generated, the image feature data or the at least one starting position being input to create the tool path. The tool path is generated by processing data including the image feature data or at least the starting position of the tool path using a second machine learning algorithm from the two or more machine learning algorithms, wherein the at least second machine learning algorithm from the two or more machine learning algorithms employs reinforcement learning including one or more scoring functions, the one or more scoring functions including rewards associated with one or more desired tool path features, and the second machine learning algorithm from the two or more machine learning algorithms is different from the first machine learning algorithm from the two or more machine learning algorithms; and The toolpath is provided to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.

17. The system of claim 16, wherein the generation comprises: The data, including at least one starting position of the tool path, is generated by processing a global view of the portion of the 3D model using the first machine learning algorithm of the two or more machine learning algorithms. as well as The second machine learning algorithm, one of the two or more machine learning algorithms, processes a local view of the portion of the 3D model around at least each of the at least one starting position to generate the tool path near each of the at least one starting position.

18. The system of claim 17, wherein the global view comprises a low-resolution representation of at least a portion of the three-dimensional model, and the local view comprises a high-resolution representation of a portion of the three-dimensional model around each of at least one of the at least one starting positions.

19. The system of claim 16, wherein the first machine learning algorithm of the two or more machine learning algorithms comprises a convolutional neural network, and the data comprises image feature data extracted by the convolutional neural network from one or more images of one or more two-dimensional cross-sections representing at least the portion of the three-dimensional model.

20. The system of claim 16, wherein the operation further comprises: Define one or more scoring functions, wherein the one or more scoring functions include rewards related to the one or more desired tool path features; It accepts multiple training examples, each of which includes example tools and example environments; and The training example is used to train a second machine learning algorithm employing reinforcement learning among the two or more machine learning algorithms to generate a tool path that maximizes one or more values ​​generated by the one or more scoring functions.

21. The system of claim 16, wherein, The two or more machine learning algorithms generate toolpaths that include the angles of the cutting tools used by the computer-controlled manufacturing system.

22. The system of claim 16, wherein, The second machine learning algorithm of the two or more machine learning algorithms generates a series of positions for a tool used by a computer-controlled manufacturing system, the tool path including one or more splines connecting the series of positions of the tool, and providing the tool path to the computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object includes providing the one or more splines to control the tool to travel along the one or more splines in a smooth path through the series of positions.

23. The system of claim 16, wherein, The second of the two or more machine learning algorithms operates in a three-dimensional environment and uses a three-dimensional representation of the tools used by the computer-controlled manufacturing system.

24. The system of claim 16, wherein the one or more desired tool path features include the desired side of the tool in the tool path.

25. The system of claim 16, wherein, The one or more desired tool path features include the tool's rotation features.

26. The system as claimed in claim 16, characterized in that, The one or more desired tool path features include tool engagement of the selected cutting tool.

27. The system of claim 16, wherein, The second machine learning algorithm in the two or more machine learning algorithms uses the percentage of tools used.

28. The system of claim 16, wherein the system comprises a computer-controlled manufacturing system.

29. The system of claim 28, wherein the computer-controlled manufacturing system is a multi-axis, multi-tool milling machine.

30. The system of claim 16, wherein, The two or more machine learning algorithms are trained to generate toolpaths for 16.5-axis machining.

31. The system of claim 16, wherein the three-dimensional model is obtained using generative design software that generates a three-dimensional model comprising multiple discrete layers.

32. A non-transitory computer-readable medium, the non-transitory computer-readable medium being encoded with instructions operable to cause a data processing device to perform operations, the operations including: Obtain a 3D model of a manufacturable object; By providing at least a portion of the 3D model during training, a toolpath is generated using two or more machine learning algorithms to enable a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object, wherein the generation includes: By processing the portion of the 3D model using a first machine learning algorithm from the two or more machine learning algorithms, data including image feature data or at least one starting position of the tool path is generated, the image feature data or the at least one starting position being input to create the tool path. The tool path is generated by processing data including the image feature data or at least the starting position of the tool path using a second machine learning algorithm among the two or more machine learning algorithms, wherein the at least second machine learning algorithm among the two or more machine learning algorithms employs reinforcement learning including one or more scoring functions, the one or more scoring functions including rewards associated with one or more desired tool path features, and the second machine learning algorithm among the two or more machine learning algorithms is different from the first machine learning algorithm among the two or more machine learning algorithms. as well as The toolpath is provided to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.

33. The computer-readable medium of claim 32, wherein the generation comprises: The first machine learning algorithm of the two or more machine learning algorithms is used to process the global view of the portion of the 3D model to generate the data including at least one starting position of the tool path; as well as The second machine learning algorithm, one of the two or more machine learning algorithms, processes a local view of the portion of the 3D model around at least each of the at least one starting position to generate the tool path near each of the at least one starting position.

34. The computer-readable medium of claim 32, wherein the first machine learning algorithm of the two or more machine learning algorithms comprises a convolutional neural network, and the data comprises image feature data extracted by the convolutional neural network from one or more images of one or more two-dimensional cross-sections representing at least the portion of the three-dimensional model.

35. The computer-readable medium of claim 33, wherein the global view comprises a low-resolution representation of at least a portion of the three-dimensional model, and the local view comprises a high-resolution representation of a portion of the three-dimensional model around each of at least one of the at least one starting position.

36. The computer-readable medium of claim 32, further comprising: Define one or more scoring functions, wherein the one or more scoring functions include rewards related to the one or more desired tool path features; It accepts multiple training examples, each of which includes example tools and example environments; and The training example is used to train a second machine learning algorithm employing reinforcement learning among the two or more machine learning algorithms to generate a tool path that maximizes one or more values ​​generated by the one or more scoring functions.

37. The computer-readable medium as claimed in claim 32, characterized in that, The two or more machine learning algorithms generate toolpaths that include the angles of the cutting tools used by the computer-controlled manufacturing system.

38. The computer-readable medium of claim 32, wherein, The second machine learning algorithm of the two or more machine learning algorithms generates a series of positions for a tool used by a computer-controlled manufacturing system, the tool path including one or more splines connecting the series of positions of the tool, and providing the tool path to the computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object includes providing the one or more splines to control the tool to travel along the one or more splines in a smooth path through the series of positions.

39. The computer-readable medium of claim 32, wherein, The second of the two or more machine learning algorithms operates in a three-dimensional environment and uses a three-dimensional representation of the tools used by the computer-controlled manufacturing system.

40. The computer-readable medium of claim 32, wherein the one or more desired tool path features include the desired side of the tool in the tool path.

41. The computer-readable medium of claim 32, wherein, The one or more desired tool path features include the tool's rotation features.

42. The computer-readable medium as claimed in claim 32, characterized in that, The one or more desired tool path features include tool engagement of the selected cutting tool.

43. The computer-readable medium of claim 32, wherein, The second machine learning algorithm in the two or more machine learning algorithms uses the percentage of tools used.

44. The computer-readable medium of claim 32, wherein the computer-controlled manufacturing system is a multi-axis, multi-tool milling machine.

45. The computer-readable medium of claim 32, wherein, The two or more machine learning algorithms are trained to generate toolpaths for 2.5-axis machining.

46. ​​The computer-readable medium of claim 32, wherein the three-dimensional model is obtained using generative design software that generates a three-dimensional model comprising multiple discrete layers.

Images