A method and device for micro-EDM based on deep Q network

Abstract

The application provides a micro-EDM method based on a deep Q network, comprising: using the learning ability of a DQN neural network to establish a corresponding relationship between an action performed in EDM and an evaluation score through sample data; obtaining an action set performed on a workpiece to be processed; the action set comprises a plurality of candidate actions; determining an evaluation score set corresponding to the action set through the corresponding relationship; the evaluation scores in the evaluation score set correspond to the candidate actions in the action set one by one; setting the highest evaluation score in the evaluation score set as a predicted score, and setting the candidate action corresponding to the highest evaluation score as a current action. Through the learning ability of the DQN neural network, the processing environment of the current scene can be quickly adapted to rapidly perform micro-EDM operation, improve processing efficiency, and the model is lighter.

Description

Technical Field

[0001] This invention relates to the field of electrical discharge machining, and more particularly to a method and apparatus for micro electrical discharge machining based on a deep Q-network. Background Technology

[0002] With the rapid development of modern technology, micro-electrical discharge machining (EDM) technology, due to its high machining accuracy, is widely used in the manufacture of precision parts and micro-machines. Micro-electrical discharge machining is currently moving towards intelligent and automated processes. However, because the machining process is extremely complex, there is currently no suitable sensor to directly obtain the gap distance between the workpiece and the tool electrode. Since controlling this gap distance is crucial for machining quality, traditional EDM control relies on the operator's observation of the EDM state, as well as the surface roughness and machining accuracy requirements, to continuously adjust machining parameters to ensure stable operation of the process.

[0003] However, relying heavily on human experience is neither intelligent nor efficient, and is prone to subjective changes, which is not conducive to the intelligentization of micro-electrical discharge machining technology.

[0004] Micro-electrical discharge machining (EDM) technology generates high temperatures through electrical discharge, melting and sputtering conductive material in localized areas. This process is repeated multiple times to complete the machining of the entire workpiece. To ensure machining quality, the servo system needs to be able to quickly follow the melting process and then proceed to the next melting cycle. Therefore, the servo motor's speed needs to change dynamically and accurately during machining. Current technologies use convolutional neural networks to infer the electrical discharge state during machining and then control it with a servo system. However, this approach has poor adaptability to the changing industrial production environment, and the image-based convolutional neural network models are excessively large. Summary of the Invention

[0005] In view of the aforementioned problems, this application is proposed to provide a method and apparatus for micro-electrical discharge machining based on a deep Q-network to overcome or at least partially solve the aforementioned problems, comprising:

[0006] A method for micro-electrical discharge machining based on deep Q-networks, comprising:

[0007] By utilizing the learning capability of the DQN neural network, a correspondence between the actions performed in electrical discharge machining and the evaluation scores is established through sample data;

[0008] Obtain the set of actions to be performed on the current workpiece to be processed; the set of actions includes several candidate actions.

[0009] The set of evaluation scores corresponding to the set of execution actions is determined through the correspondence; the evaluation scores in the set of evaluation scores correspond one-to-one with the candidate execution actions in the set of execution actions.

[0010] The highest rating score in the set of rating scores is set as the predicted score, and the candidate action corresponding to the highest rating score is set as the current action.

[0011] Furthermore, the step of utilizing the learning capability of the DQN neural network to establish the correspondence between the actions performed in electrical discharge machining and the evaluation scores through sample data includes:

[0012] Obtain the processing status of the sample workpiece during processing;

[0013] The actions performed during the processing state are scored to obtain an evaluation score;

[0014] The DQN neural network is used to establish the correspondence between the evaluation score and the executed action.

[0015] Furthermore, the steps for obtaining the processing status of the sample workpiece during processing include:

[0016] Obtain the motion space of the sample workpiece, the motion space including all executable actions;

[0017] The state space of the sample workpiece is obtained, and the state space includes potential difference, effective discharge number and arcing number.

[0018] Furthermore, the step of scoring the actions performed in the processing state and obtaining an evaluation score includes:

[0019] Obtain a reward function, wherein the input parameters of the reward function have the same elements as the state space;

[0020] The reward function is used to calculate the element score for each element in the state space;

[0021] The score for the performed action is calculated by summing the scores of all the elements.

[0022] Furthermore, the step of establishing the correspondence between the evaluation score and the executed action through the DQN neural network includes:

[0023] The DQN neural network is used to predict the predicted scores of all candidate actions in the current state.

[0024] Set the candidate action with the highest predicted score as the action to be executed and execute it.

[0025] The evaluation score of the executed action is obtained through the reward function and then evaluated.

[0026] The evaluation results are stored in the experience pool of the predictive model;

[0027] The experience pool is used to establish a correspondence between the evaluation score and the action performed.

[0028] Furthermore, it also includes:

[0029] Randomly select the actions to be executed from the action space;

[0030] The DQN neural network is used to predict the rating score of the performed action and generate a predicted score.

[0031] The execution action is performed, and the evaluation score of the execution action is obtained through the reward function and evaluated.

[0032] The evaluation results are stored in the experience pool of the prediction model.

[0033] The experience pool is used to establish a correspondence between the evaluation score and the action performed.

[0034] Furthermore, it also includes converting the evaluation results into... Storage; among which, Let 'a' represent the state at time t, 'a' represent the action at time t, and 'r' represent the score at time t. This represents the state at time t+1.

[0035] An apparatus for micro-electrical discharge machining based on a deep Q-network, the apparatus implementing the steps of the micro-electrical discharge machining method based on a deep Q-network as described above, includes:

[0036] A module is established to utilize the learning capabilities of the DQN neural network to establish the correspondence between the actions performed in electrical discharge machining and the evaluation scores through sample data;

[0037] The acquisition module is used to acquire the set of execution actions for the current workpiece to be processed; the set of execution actions includes several candidate execution actions;

[0038] The scoring module is used to determine the set of evaluation scores corresponding to the set of execution actions through the correspondence; the evaluation scores in the set of evaluation scores correspond one-to-one with the candidate execution actions in the set of execution actions;

[0039] The determination module is used to set the highest evaluation score in the set of evaluation scores as the predicted score, and set the candidate action corresponding to the highest evaluation score as the current action.

[0040] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for micro-electrical discharge machining based on a deep Q-network as described above.

[0041] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method for micro-electrical discharge machining based on a deep Q-network as described above.

[0042] This application has the following advantages:

[0043] In the embodiments of this application, addressing the shortcomings of existing technologies in intelligent machining of fine electrical discharge machining (EDM) due to its inability to adapt to changing application scenarios, this application provides a method for fine EDM based on a deep Q-network. The method includes: utilizing the learning capability of a DQN neural network to establish a correspondence between executed actions and evaluation scores in EDM through sample data; obtaining a set of executed actions for the current workpiece to be processed; the set of executed actions includes several candidate executed actions; determining a set of evaluation scores corresponding to the set of executed actions through the correspondence; a one-to-one correspondence between the evaluation scores in the set of evaluation scores and the candidate executed actions in the set of executed actions; setting the highest evaluation score in the set of evaluation scores as the predicted score, and setting the candidate executed action corresponding to the highest evaluation score as the current executed action. Through the learning capability of the DQN neural network, it can quickly adapt to the current machining environment to perform fine EDM operations, improving machining efficiency, and the model is also more lightweight. Attached Figure Description

[0044] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the steps of a method for micro-electrical discharge machining based on a deep Q-network, as provided in an embodiment of this application.

[0046] Figure 2 This is a schematic diagram of the module structure of a device for micro-electrical discharge machining based on a deep Q-network according to an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0049] The inventors discovered through analysis of existing technologies that applying the reinforcement learning capabilities of deep Q-networks to fine electrical discharge machining can quickly adapt to the current working environment, complete delicate operations, and the model is also lighter.

[0050] Reference Figure 1 This illustrates a method for micro-electrical discharge machining based on a deep Q-network according to an embodiment of this application;

[0051] The method includes:

[0052] S110. Utilize the learning capability of the DQN neural network to establish the correspondence between the actions performed and the evaluation scores in electrical discharge machining through sample data;

[0053] S120. Obtain the set of actions to be executed for the current workpiece to be processed; the set of actions includes several selectable actions.

[0054] S130. Determine the evaluation score set corresponding to the set of execution actions through the correspondence relationship; the evaluation scores in the evaluation score set correspond one-to-one with the candidate execution actions in the set of execution actions;

[0055] S140. Set the highest rating score in the set of rating scores as the predicted score, and set the candidate action corresponding to the highest rating score as the current action.

[0056] In the embodiments of this application, addressing the shortcomings of existing technologies in intelligent machining of fine electrical discharge machining (EDM) that cannot adapt to changing application scenarios, this application provides a method for fine EDM based on a deep Q-network. The method includes: utilizing the learning capability of a DQN neural network to establish a correspondence between executed actions and evaluation scores in EDM through sample data; obtaining a set of executed actions for the current workpiece to be processed; the set of executed actions includes several candidate executed actions; determining a set of evaluation scores corresponding to the set of executed actions based on the correspondence; a one-to-one correspondence between the evaluation scores in the set of evaluation scores and the candidate executed actions in the set of executed actions; setting the highest evaluation score in the set of evaluation scores as the predicted score, and setting the candidate executed action corresponding to the highest evaluation score as the current executed action. Through the learning capability of the DQN neural network, it can quickly adapt to the current machining environment to perform fine EDM operations, and the model is also more lightweight.

[0057] The rated score is the score obtained after an action is completed. The predicted score is the score predicted after an action is completed in the current state. After training is completed, the difference between the rated score and the predicted score should approach 0. The predicted scores of all executable actions in the current state are obtained through the DQN neural network. The action with the highest predicted score is selected for execution, which is the execution action with the highest rated score, i.e., the optimal execution action.

[0058] The following will further describe a method and apparatus for micro-electrical discharge machining based on a deep Q-network in this exemplary embodiment.

[0059] As described in step S110 above, the learning capability of the DQN neural network is used to establish the correspondence between the actions performed in electrical discharge machining and the evaluation scores through sample data.

[0060] It should be noted that the evaluation score is the score given to each state of the workpiece after the action is executed in the current state, and the total score is the evaluation score. Using the DQN neural network, the relationship between the action executed in the sample data of fine electrical discharge machining and the score obtained after the action is completed is analyzed. Through the self-learning and adaptive characteristics of the DQN neural network, the mapping relationship between the executed action and the evaluation score obtained after the action is completed is found.

[0061] In one embodiment of the present invention, the specific process of "using the learning ability of the DQN neural network to establish the correspondence between the executed actions and the evaluation scores in electrical discharge machining through sample data" described in step S110 can be further explained in conjunction with the following description.

[0062] The following steps are used to obtain the processing status of the sample workpiece during processing.

[0063] It should be noted that the processing parameters can be manually defined or data uploaded by sensors installed on the processing device.

[0064] In one embodiment of the present invention, the specific process of "obtaining the processing state of the sample workpiece during processing" can be further described in conjunction with the following description.

[0065] As described in the following steps, the motion space of the sample workpiece is obtained, the motion space including all executable motions;

[0066] The action space (A) space It includes all possible actions in servo control, such as "forward", "backward", and "no action".

[0067] The state space of the sample workpiece is obtained as described in the following steps. The state space includes potential difference, effective discharge count, and arcing count.

[0068] The state space includes some sensor data configured in the machining system, such as potential difference, effective discharge count, and arcing count (V). t N d N arc ).

[0069] As described in the following steps, the actions performed in the processing state are scored to obtain an evaluation score.

[0070] In one embodiment of the present invention, the specific process of “scoring the execution actions in the processing state and obtaining an evaluation score” can be further explained in conjunction with the following description.

[0071] As described in the following steps, a reward function is obtained, wherein the input parameters contained in the reward function have the same elements as the state space.

[0072] As described in the following steps, the element score for each element in the state space is calculated using the reward function;

[0073] As described in the following steps, the element scores are added together to calculate the evaluation score for the performed action.

[0074] It should be noted that the reward function This is a manually defined value used for posterior evaluation of the score after taking a certain action, where is a vector including the current state. The elements are the same as those in the state space. Each cycle, the state contained in the state space is sampled by the sensor, and then a score for that cycle is calculated using a reward function. The reward function calculation mechanism is as follows: if the potential difference is between the upper and lower thresholds, a higher positive score is obtained; otherwise, a higher absolute value negative score is obtained. Then, a cumulative score is obtained by multiplying the effective discharge count by a smaller positive score, and a cumulative score is obtained by multiplying the number of arc pulls by a smaller absolute value negative score. Finally, the scores of all state evaluations are summed to obtain the total score obtained after the execution of the action, which is the evaluation score.

[0075] As described in the following steps, the correspondence between the score and the executed action is established through the DQN neural network.

[0076] In one embodiment of the present invention, the specific process of "establishing the correspondence between the evaluation score and the execution action through the DQN neural network" can be further explained in conjunction with the following description.

[0077] As described in the following steps, the predicted scores of all candidate actions to be executed in the current state are predicted using the DQN neural network;

[0078] As described in the following steps, the candidate action with the highest predicted score is set as the action to be executed and then executed.

[0079] As described in the following steps, the evaluation score of the performed action is obtained through the reward function and then evaluated;

[0080] As described in the following steps, the evaluation results are stored in the experience pool of the prediction model;

[0081] As described in the following steps, the correspondence between the evaluation score and the performed action is established through the experience pool.

[0082] As described in the following steps, randomly select the actions to be executed from the action space;

[0083] As described in the following steps, the rating score of the action is predicted and a predicted score is generated using the DQN neural network;

[0084] As described in the following steps, the execution action is performed, and the evaluation score of the execution action is obtained through the reward function and evaluated.

[0085] As described in the following steps, the evaluation results are stored in the experience pool of the prediction model;

[0086] As described in the following steps, the correspondence between the evaluation score and the performed action is established through the experience pool.

[0087] As described in the following steps, the evaluation results are... Storage, wherein, the Let 'a' be the state at time t, 'a' be the action at time t, and 'r' be the score at time t. This represents the state at time t+1.

[0088] It should be noted that the DQN neural network is used to assess the possible final score when performing different actions in different states. However, because the DQN neural network has insufficient "experience" in the early stages, it mainly relies on "exploration" to acquire "experience" in the early stages.

[0089] The initial exploration probability is set to 100%. As exploration progresses, the DQN neural network continuously learns and improves, gradually decreasing the exploration rate to near 0, but never equal to 0. When entering the "exploration" state, an action is randomly selected from the action space. The DQN neural network makes a prediction, which is then evaluated by a reward function to determine if the score error is below a preset value. The result is stored in the experience pool to refine the correspondence. Outside the "exploration" state, the DQN neural network calculates and evaluates the possible scores of all actions taken in the action space in the current state. Finally, the action with the highest score is selected and evaluated by the reward function to determine if the score error is below a preset value. The result is also stored in the experience pool to refine the correspondence.

[0090] Whether the action is generated by "exploration" or inferred by the DQN neural network, it will be rewarded by a function after the action is completed. To evaluate the score of this action, and in Each "lesson" is stored in the experience pool to train the DQN network. According to the Bellman equation: State at time t Perform action a t The final score is equal to the score r at time t plus the score at time t+1. Execute a t+1 The maximum score that an action can ultimately obtain is multiplied by a discount factor. Therefore, the DQN network will... As input, the network output is in The final score that can be obtained by performing each action in the action space under a given state. When training the network, an "experience" is divided into two parts, input and output, to fit the Q-function. The input part is... The output section is The network is used to fit the Q-function using the mean squared loss error function. With sufficient training iterations, the Q-function is refined by the network, at which point the Bellman equation's Q-function, even when replaced by a neural network, still maintains a near-equal relationship between the two sides. After training is complete, the correspondence between the evaluation scores and the executed actions is established.

[0091] As described in step S120 above, the set of execution actions for the current workpiece to be processed is obtained; the set of execution actions includes several candidate execution actions.

[0092] It should be noted that the number of candidate actions includes all possible actions that may be performed in the current state of the workpiece to be processed, and the candidate actions can also be set manually.

[0093] As described in step S130 above, the set of evaluation scores corresponding to the set of execution actions is determined through the correspondence; the evaluation scores in the set of evaluation scores correspond one-to-one with the candidate execution actions in the set of execution actions.

[0094] It should be noted that the number of path entries generated by the micro-electrical discharge machining system is enormous. It needs to consider the path of the next movement, and more importantly, it needs to proactively consider the paths for the next few steps and optimize those future paths. By obtaining the evaluation scores corresponding to the executed actions, the high-scoring actions can be accurately and intuitively selected for execution, making micro-electrical discharge machining more precise and efficient.

[0095] As described in step S140 above, the highest rating score in the set of rating scores is set as the predicted score, and the candidate action corresponding to the highest rating score is set as the current action.

[0096] In a specific implementation, the high precision requirements of micro-electrical discharge machining (EDM) place high demands on the real-time performance of servo control. In this embodiment, the micro-EDM system employs an Ubuntu system based on a dual-kernel architecture of Xenomai and Linux. The Qt5-based graphical interface software runs on the native Linux kernel, while the real-time servo control program runs within the real-time Xenomai kernel. This ensures rapid response during servo control, thereby improving machining accuracy.

[0097] To process workpieces made of different materials, a database is set up at the back end to store processing data for each material. In this embodiment, the micro-electrical discharge machining system uses a MySQL database. During processing, the material to be processed is determined, and then a trained neural network is selected from the MySQL database for servo control. If no corresponding trained neural network is found in the MySQL database, a learning mode is entered, where processing is performed while learning. After processing is complete, the network model is stored in the MySQL database.

[0098] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0099] Reference Figure 2 This application illustrates an apparatus for optimizing batch writing of database logs according to an embodiment of the present application.

[0100] Specifically, it includes:

[0101] Module 210 is established to utilize the learning capability of the DQN neural network to establish the correspondence between the actions performed in electrical discharge machining and the evaluation scores through sample data;

[0102] The acquisition module 220 is used to acquire the set of execution actions for the current workpiece to be processed; the set of execution actions includes several candidate execution actions;

[0103] The scoring module 230 is used to determine the set of evaluation scores corresponding to the set of execution actions through the correspondence relationship; the evaluation scores in the set of evaluation scores correspond one-to-one with the candidate execution actions in the set of execution actions;

[0104] The determination module 240 is used to set the highest evaluation score in the set of evaluation scores as the predicted score, and set the candidate action corresponding to the highest evaluation score as the current action.

[0105] In one embodiment of the present invention, the establishment module 210 includes:

[0106] The sample acquisition submodule is used to acquire the processing status of the sample workpiece during processing;

[0107] The sample evaluation score evaluation submodule is used to score the actions performed in the processing state and obtain an evaluation score;

[0108] The correspondence establishment submodule is used to establish the correspondence between the evaluation score and the execution action through the DQN neural network.

[0109] In one embodiment of the present invention, the sample acquisition submodule includes:

[0110] The motion space acquisition submodule is used to acquire the motion space of the sample workpiece, which includes all executable motions.

[0111] The state space acquisition submodule is used to acquire the state space of the sample workpiece, which includes potential difference, effective discharge count, and arcing count.

[0112] In one embodiment of the present invention, the sample evaluation score evaluation submodule includes:

[0113] The reward function acquisition submodule is used to acquire the reward function, which contains the same elements as the state space.

[0114] The element score calculation submodule is used to calculate the element score of each element in the state space through the reward function;

[0115] The score acquisition submodule is used to add the scores of the elements to calculate the score of the executed action.

[0116] In one embodiment of the present invention, the correspondence establishment sub-module includes:

[0117] The All Action Score Prediction Submodule is used to predict the predicted scores of all candidate actions in the current state using the DQN neural network.

[0118] The prediction action execution submodule is used to set the candidate action with the highest prediction score as the execution action and execute it.

[0119] The evaluation submodule is used to obtain the evaluation score of the executed action through the reward function and perform the evaluation;

[0120] The experience pool refinement submodule is used to store the evaluation results in the experience pool of the prediction model.

[0121] The correspondence improvement submodule is used to establish the correspondence between the evaluation score and the executed action through the experience pool.

[0122] In one embodiment of the present invention, the correspondence establishment submodule further includes:

[0123] The random submodule is used to randomly select the actions to be executed from the action space;

[0124] The action score prediction submodule is used to predict the evaluation score of the performed action and generate a predicted score through the DQN neural network;

[0125] The evaluation submodule is used to execute the execution action, obtain the evaluation score of the execution action through the reward function, and perform evaluation.

[0126] The evaluation submodule is used to store the evaluation results in the experience pool of the prediction model;

[0127] The correspondence improvement submodule is used to establish the correspondence between the evaluation score and the executed action through the experience pool.

[0128] In one embodiment of the present invention, it further includes: a storage submodule, used to store the evaluation results as... Storage, wherein Let 'a' be the state at time t, 'a' be the action at time t, and 'r' be the score at time t. This represents the state at time t+1.

[0129] Reference Figure 3 The computer device illustrating a method for micro-electrical discharge machining based on a deep Q-network according to the present invention may specifically include the following:

[0130] The computer device 12 described above is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0131] Bus 18 refers to one or more of several types of bus 18 architectures, including memory bus 18 or memory controller, peripheral bus 18, graphics acceleration port, processor, or local bus 18 using any of the various bus 18 architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus 18, Micro Channel Architecture (MAC) bus 18, Enhanced ISA bus 18, Audio / Video Electronics Standards Association (VESA) local bus 18, and Peripheral Component Interconnect (PCI) bus 18.

[0132] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0133] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42 configured to perform the functions of the embodiments of the present invention.

[0134] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory. Such program modules 42 include—but are not limited to—an operating system, one or more application programs, other program modules 42, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0135] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, camera, etc.), and with one or more devices that enable medical personnel to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN)), wide area network (WAN), and / or public networks (e.g., the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although... Figure 3 Not shown, it can be combined with computer device 12 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing unit 16, external disk drive array, RAID system, tape drive and data backup storage system 34, etc.

[0136] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a method for optimizing batch writing of database logs provided in an embodiment of the present invention.

[0137] That is, when the processing unit 16 executes the above program, it achieves the following: by utilizing the learning capability of the DQN neural network, it establishes a correspondence between the actions performed in electrical discharge machining and the evaluation scores through sample data;

[0138] Obtain the set of actions to be performed on the current workpiece to be processed; the set of actions includes several candidate actions.

[0139] The set of evaluation scores corresponding to the set of execution actions is determined through the correspondence; the evaluation scores in the set of evaluation scores correspond one-to-one with the candidate execution actions in the set of execution actions.

[0140] The highest rating score in the set of rating scores is set as the predicted score, and the candidate action corresponding to the highest rating score is set as the current action.

[0141] In this embodiment of the invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements a method for micro-electrical discharge machining based on a deep Q-network as provided in all embodiments of this application:

[0142] That is, when the program is executed by the processor, it implements the following: by utilizing the learning ability of the DQN neural network, a correspondence between the actions performed in electrical discharge machining and the evaluation scores is established through sample data;

[0143] Obtain the set of actions to be performed on the current workpiece to be processed; the set of actions includes several candidate actions.

[0144] The set of evaluation scores corresponding to the set of execution actions is determined through the correspondence; the evaluation scores in the set of evaluation scores correspond one-to-one with the candidate execution actions in the set of execution actions.

[0145] The highest rating score in the set of rating scores is set as the predicted score, and the candidate action corresponding to the highest rating score is set as the current action.

[0146] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-to-signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPOM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0147] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0148] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the healthcare worker's computer, partially on the healthcare worker's computer, as a standalone software package, partially on the healthcare worker's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the healthcare worker's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0149] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0150] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0151] The above provides a detailed description of a micro-electrical discharge machining method based on a deep Q-network provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for micro-EDM based on deep Q network, characterized in that, The method comprises the following steps: The learning ability of the DQN neural network is used to establish the corresponding relationship between the execution action and the evaluation score in the electric spark machining through sample data; the evaluation score is the score of each state of the workpiece after the execution of the action in the current state, and the total score is the evaluation score; the machining state of the sample workpiece during machining is obtained; the execution action in the machining state is scored to obtain the evaluation score; the corresponding relationship between the evaluation score and the execution action is established through the DQN neural network; the prediction score of all candidate execution actions in the current state is predicted through the DQN neural network; the candidate execution action with the highest prediction score is set as the execution action and is executed; the evaluation score of the execution action is obtained through a preset reward function and is evaluated; the evaluation result is stored in the experience pool of the prediction model; the corresponding relationship between the evaluation score and the execution action is established through the experience pool; wherein the input parameters contained in the reward function and the state space have the same elements; An execution action set of a current workpiece to be machined is obtained; the execution action set comprises a plurality of candidate execution actions; The corresponding evaluation score set of the execution action set is determined through the corresponding relationship; the evaluation scores in the evaluation score set correspond to the candidate execution actions in the execution action set one by one; The highest evaluation score in the evaluation score set is set as a prediction score, and the candidate execution action corresponding to the highest evaluation score is set as a current execution action.

2. The method of claim 1, wherein, The step of obtaining the machining state of the sample workpiece during machining comprises the following steps: An action space of the sample workpiece is obtained, wherein the action space comprises all executable actions; A state space of the sample workpiece is obtained, wherein the state space comprises an electric potential difference, an effective discharge number and an arc drawing frequency.

3. The method of claim 2, wherein, The step of scoring the execution action in the machining state to obtain the evaluation score comprises the following steps: A reward function is obtained; The element score of each element in the state space is calculated through the reward function; The evaluation score of the execution action is calculated by adding the element scores.

4. The method of claim 3, wherein, Further comprising: An execution action in the action space is randomly extracted; The evaluation score of the execution action is predicted through the DQN neural network to generate a prediction score; The execution action is executed, the evaluation score of the execution action is obtained through the reward function, and the evaluation is performed; The evaluation result is stored in the experience pool of the prediction model; The corresponding relationship between the evaluation score and the execution action is established through the experience pool.

5. The method of claim 4, wherein, Also included is, the evaluation result is sent to storage; wherein, the state at t moment, action at t moment, score at t moment, state at t+1 moment.

6. A device for micro-EDM based on deep Q network, characterized in that, The device for micro electric spark machining based on a deep Q network realizes the steps of the method for micro electric spark machining based on a deep Q network according to any one of claims 1 to 5, and comprises: An establishment module is configured to use the learning ability of the DQN neural network to establish the corresponding relationship between the execution action and the evaluation score in the electric spark machining through sample data; An acquisition module is configured to obtain an execution action set of a current workpiece to be machined; the execution action set comprises a plurality of candidate execution actions; The scoring module is configured to determine a set of evaluation scores corresponding to the set of execution actions according to the correspondence; each evaluation score in the set of evaluation scores corresponds to a candidate execution action in the set of execution actions; The determining module is configured to set a highest evaluation score in the set of evaluation scores as a prediction score, and set a candidate execution action corresponding to the highest evaluation score as a current execution action.

7. An electronic device, comprising: A computer program product comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for micro-EDM based on deep Q network according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, A computer program product comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for micro-EDM based on deep Q network according to any one of claims 1-5.

Images