Knowledge graph and reinforcement learning based processing parameter dynamic adjustment method and system

Abstract

The application belongs to numerical control machining control technology, and provides a machining parameter dynamic adjustment method and system based on a knowledge graph and reinforcement learning, the method comprising: constructing a knowledge graph of four types of entities of equipment, machining parameters, products and faults; evaluating the use state level based on the basic information and historical data of the equipment; dividing the machining groups according to the state level and establishing the adaptation relationship with the target product; identifying abnormal / suspicious equipment through error threshold and trend analysis; training a neural network adjustment model, combining reinforcement learning and the knowledge graph to dynamically adjust the parameters; and feeding back the adjustment results to the knowledge graph to realize closed-loop optimization. Through the service time and use state of the equipment, the machining groups are divided in combination with different production orders, the machining parameters of the equipment in different use state levels in the order are dynamically adjusted, the self-sensing, self-decision and self-evolution of the manufacturing process are realized, and the machining quality stability and equipment reliability are significantly improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent manufacturing and industrial artificial intelligence, and relates to CNC machining control technology, specifically to a method and system for dynamic adjustment of machining parameters based on knowledge graphs and reinforcement learning. Background Technology

[0002] In high-end discrete manufacturing fields (such as aerospace structural components, precision molds, and medical devices), CNC machining processes demand extremely high product quality stability. In actual production, machining parameters (such as feed rate, spindle speed, and depth of cut) need to be dynamically adjusted based on equipment condition, tool wear, batch material variations, and environmental disturbances. However, most mainstream manufacturing enterprises still primarily use static process specification cards or rely on experienced technicians to set parameters, lacking the ability to perceive individual equipment performance degradation and group behavior differences, leading to the following prominent problems:

[0003] 1. High product deviation rate: Under the same order, the key dimensions (such as hole diameter and flatness) of the same model of parts processed by different equipment fluctuate significantly, and the sampling pass rate is unstable;

[0004] 2. Delayed response to anomalies: Traditional statistical process control (SPC) only alarms when parameters exceed the ±3σ control limit, and cannot identify the "slow degradation" trend. It is often not discovered until a batch of parts are scrapped.

[0005] 3. Uneven equipment utilization: New and old equipment are mixed in production. Old equipment frequently exceeds tolerances due to performance degradation and is forced to be shut down for debugging, while the capacity of new equipment is not fully utilized.

[0006] 4. Unreliable decision-making: Some companies have tried to introduce AI models to recommend parameters, but the models are "black boxes," and engineers cannot understand why the feed rate should be adjusted from 800 to 750, making it difficult to adopt the changes on-site.

[0007] Existing technical solutions have the following significant limitations: Supervised learning-based parameter prediction models rely on isolated training data stored in MES or databases, lacking semantic relationships between equipment, parameters, products, and faults, resulting in weak model generalization ability. For example, if a machine experiences increased vibration due to aging spindle bearings, the model cannot associate this with a "size oversized" fault, treating it merely as ordinary noise. While using rule engines or expert systems to adjust machining parameters offers some interpretability, the rules are rigid, difficult to adapt to new operating conditions, and fail to consider equipment state grouping, treating 5-year-old equipment the same as 48-month-old equipment, leading to misadjustments.

[0008] Furthermore, most existing intelligent control systems are open-loop structures: once an adjustment suggestion is generated, it is immediately executed, lacking a mechanism to verify the rationality of the adjustment results. For example, the model might suggest drastically reducing the spindle speed from 10,000 rpm to 6,000 rpm, which, while theoretically reducing vibration, could actually cause chatter or a sharp drop in efficiency. Simultaneously, the system lacks the ability to continuously learn based on real production feedback, failing to accumulate experience from whether the product has improved after the adjustment, causing the model to gradually become ineffective over time.

[0009] Although knowledge graph technology has been used to build manufacturing knowledge bases and reinforcement learning has shown potential in robot control, the two have not yet been deeply integrated in scenarios involving dynamic adjustment of processing parameters. Especially in typical discrete manufacturing environments characterized by high value, small batches, and multiple varieties, there is an urgent need for an intelligent control method that can both perceive the behavior of equipment groups and integrate multi-source knowledge for interpretable decision-making, while also possessing safety verification and closed-loop evolution capabilities. Summary of the Invention

[0010] To address the technical problems of fragmented knowledge, weak model generalization, and delayed response in traditional processing parameter adjustment, and to achieve equipment status awareness, early anomaly identification, interpretable decision generation, and closed-loop self-optimization, this invention discloses a dynamic processing parameter adjustment method based on knowledge graphs and reinforcement learning. The method includes the following steps:

[0011] S1. Collect basic information and historical processing data of production equipment, and construct a knowledge graph that associates four types of entities: equipment, processing parameters, products and faults.

[0012] S2. Based on the basic information and the historical processing data, perform a status assessment on the production equipment to obtain the usage status level;

[0013] S3. Divide the production equipment into different processing groups according to the usage status level, and establish the relationship points between each processing group and the target product corresponding to the current production order in the knowledge graph;

[0014] S4. Based on the aforementioned relationship points, collect the current product processing parameters of each processing group during the production process, analyze the changing trend of the current product processing parameters, and identify abnormal and suspicious processing equipment within each processing group.

[0015] S5. Based on the historical processing data corresponding to each processing group, train a neural network adjustment model for different processing groups, and dynamically adjust the processing parameters of the abnormal processing equipment and / or suspicious processing equipment based on the reinforcement learning strategy combined with the associated knowledge in the knowledge graph.

[0016] Further, in step S2, a status assessment is performed to obtain the usage status level, including:

[0017] S21. Classify the devices by type based on the basic information;

[0018] S22. For the same type of equipment, based on the historical processing data, the failure frequency is statistically analyzed, and a threshold is set according to the failure frequency to divide the equipment into three categories: low failure rate, medium failure rate, and high failure rate, which are used as the usage status level.

[0019] S23. Based on the historical processing data, the service time intervals of the three types of equipment are statistically analyzed, and a correspondence between service time and usage status level is established.

[0020] Furthermore, in step S3, the establishment of the relationship points includes:

[0021] S31. For the target product involved in the new production order, select equipment that has processed the target product or similar products from the historical processing data, and group the equipment according to their corresponding usage status level, so that each processing group contains equipment with the same usage status level.

[0022] S32. In the knowledge graph, create a group node for each processing group and a product node for the target product. Establish a relationship edge representing the processing adaptability between the group node and the product node as the relationship point.

[0023] Further, in step S4, abnormal and suspicious processing equipment within each processing group is identified, including:

[0024] S41. For new production orders, collect the current product processing parameters of each processing group and determine whether they exceed the error range based on the preset error.

[0025] S42. Mark the equipment whose current product processing parameters exceed the error range as abnormal processing equipment;

[0026] S43. For equipment whose current product processing parameters do not exceed the error range, calculate the average value of the product processing parameters in the current sampling round and compare it with the corresponding average value in the previous sampling round to determine the trend of the processing parameters of the processing group to which the equipment belongs. The trend of change includes stabilization, improvement or deterioration.

[0027] S44. Among the equipment that has not been marked as abnormal processing equipment, the equipment whose trend of change is deteriorating is marked as suspicious processing equipment.

[0028] Furthermore, in step S5, the training process of the neural network regulation model for different processing groups includes:

[0029] S51. Collect historical processing data of the processing group involved in the current order. The historical processing data includes the equipment error sampling interval value, the corresponding processing parameters and their numerical adjustment range, and the corresponding processing parameters and their numerical adjustment range when the product processing parameters exceed the error standard.

[0030] S52. Semantically associate the historical processing data with the knowledge graph, divide the associated data into training set, validation set and test set, train the neural network adjustment model of different processing groups, the neural network adjustment model adopts the ReLU activation function, and calculates the loss between the model prediction output and the real label through the mean squared error loss function.

[0031] Further, in step S5, the processing parameters of the abnormal processing equipment and / or suspicious processing equipment are dynamically adjusted, including:

[0032] S53. The reinforcement learning strategy network is used to control the multi-source knowledge retrieval process, and processing parameter adjustment decisions are generated based on the retrieval results. The processing parameter adjustment values ​​are verified and output, and the reinforcement learning strategy network is updated according to actual production feedback.

[0033] Furthermore, in step S53, a reinforcement learning policy network is used to control the multi-source knowledge retrieval process, and a processing parameter adjustment decision is generated based on the retrieval results. The adjusted processing parameter values ​​are verified and then output. The reinforcement learning policy network is updated based on actual production feedback, including:

[0034] S531. Decompose the parameter adjustment task into sub-tasks: determine the type of equipment abnormality, determine the direction of parameter adjustment, and calculate the specific adjustment value. Use a hierarchical reinforcement learning reward mechanism to allocate rewards to each sub-task.

[0035] S532. Using a reinforcement learning strategy network, perform graph traversal retrieval on the knowledge graph and keyword matching and semantic similarity retrieval on the text knowledge, and perform weighted fusion of the retrieval results;

[0036] S533. Generate multiple reasoning paths based on fused knowledge, and select the reasoning path with the highest reward value as the optimal path;

[0037] S534. Using the ProofTrace mechanism, the generated processing parameter adjustment values ​​are verified based on the mean ± 2 standard deviations of the historical adjustment values ​​of the same parameters to exclude statistical outliers and ensure that the adjustment values ​​are within the reasonable fluctuation range of history.

[0038] S535. After the verification is passed, the final processing parameter adjustment value is output, and the actual production feedback, including equipment operation data and product quality data, is used to update the reinforcement learning policy network through the Q-learning algorithm.

[0039] In an improved embodiment of the above-described method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning, the method further includes:

[0040] S6. After completing the dynamic adjustment of the processing parameters and executing the processing, collect the actual product processing parameters and their quality judgment results obtained from production, write the processing parameter adjustment action, actual product processing parameters and quality judgment results as new knowledge into the knowledge graph, and dynamically update and optimize the knowledge graph.

[0041] This invention also provides a system for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning. The system includes a data acquisition and knowledge graph construction module, an equipment grouping management module, a relationship establishment module, a production status monitoring and anomaly identification module, and a parameter dynamic adjustment module.

[0042] Among them, the data acquisition and knowledge graph construction module is used to collect equipment types and historical processing data of production equipment, and construct a knowledge graph that associates four types of entities: equipment, processing parameters, products and faults.

[0043] The equipment grouping management module is used to perform status assessment on the production equipment based on the equipment type and the historical processing data to obtain the usage status level;

[0044] The relationship establishment module is used to divide the production equipment into different processing groups according to the usage status level, and to establish relationship points between each processing group and the target product corresponding to the current production order in the knowledge graph;

[0045] The production status monitoring and anomaly identification module is used to collect the current product processing parameters of each processing group during the production process based on the relationship points, and analyze the changing trend of the current product processing parameters in order to identify abnormal and suspicious processing equipment in each processing group.

[0046] The parameter dynamic adjustment module is used to train adjustment models for different processing groups based on the historical processing data corresponding to each processing group, and dynamically adjust the processing parameters of the abnormal processing equipment and / or suspicious processing equipment based on reinforcement learning strategies combined with the associated knowledge in the knowledge graph.

[0047] Furthermore, the aforementioned system also includes:

[0048] The knowledge graph update and optimization module is used to collect the actual product processing parameters and their quality judgment results after the processing parameters are dynamically adjusted and the processing is executed. The processing parameter adjustment action, the actual product processing parameters and the quality judgment results are written into the knowledge graph as new knowledge, and the knowledge graph is dynamically updated and optimized in a closed loop.

[0049] Compared with the prior art, the beneficial effects that at least one technical solution adopted in the embodiments of this specification can achieve include at least:

[0050] 1. Equipment grouping mechanism based on usage status level improves the pertinence of anomaly identification and parameter adjustment: By integrating equipment type, failure frequency and service time, the usage status level of the equipment is quantitatively evaluated, and the equipment is divided into homogeneous processing groups accordingly. This ensures that subsequent parameter modeling and trend analysis are based on equipment groups with consistent reliability characteristics, effectively avoiding misjudgments or ineffective adjustments caused by differences in equipment status.

[0051] 2. Construct a knowledge graph for multidimensional manufacturing to realize the structured expression and dynamic retrieval of process knowledge: Incorporate four types of entities, namely equipment, products, processing parameters and faults and their relationships, into a unified knowledge graph. This not only supports efficient retrieval of historical experience, but also provides semantic context for reinforcement learning decisions, significantly enhancing the interpretability and engineering rationality of parameter adjustment suggestions.

[0052] 3. Employing a dual-dimensional criterion of "absolute error + trend change" to accurately identify potentially risky equipment: Based on traditional out-of-tolerance judgment, the introduction of trend analysis of the time-series average value of processing parameters can identify "suspicious equipment" that, although not out of tolerance, is continuously deteriorating in performance in advance, realizing the transformation from "post-event alarm" to "pre-event warning" and reducing the risk of quality fluctuations.

[0053] 4. Each group's dedicated neural network model is combined with knowledge-guided reinforcement learning to balance personalization and intelligence: The adjustment model is trained separately for processing groups with different usage levels to ensure that the model is adapted to the actual working conditions of the equipment; at the same time, the adjustment decision is generated by integrating multi-source information from the knowledge graph using reinforcement learning strategies, and statistical verification is performed through the ProofTrace mechanism to balance adjustment accuracy, security and generalization ability.

[0054] 5. Form a closed loop of "execution-feedback-update" to achieve continuous self-evolution of the system: Each parameter adjustment action and its corresponding actual processing result are written into the knowledge graph as new knowledge, enabling the system to have the ability to learn and accumulate experience over a long period of time. As data grows, the decision quality is continuously optimized, and the closed-loop self-optimization of the intelligent manufacturing system is truly realized. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of the method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to the present invention;

[0057] Figure 2 This is an architecture diagram of the system for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning, as described in this invention.

[0058] Figure 3 This is a schematic diagram of a computer device disclosed in an embodiment of the present invention;

[0059] The module includes: 201. Data acquisition and knowledge graph construction module; 202. Equipment group management module; 203. Relationship establishment module; 204. Production status monitoring and anomaly identification module; 205. Parameter dynamic adjustment module; 206. Knowledge graph update and optimization module; 301. Memory; 302. Processor. Detailed Implementation

[0060] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0061] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features of the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] This invention discloses a method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning. This method constructs a multi-dimensional knowledge graph linking equipment, products, parameters, and faults, and intelligently groups equipment based on its usage status level to accurately identify abnormal and suspicious equipment. It then utilizes a grouped neural network model and reinforcement learning strategies for knowledge-driven dynamic parameter optimization, ultimately forming a closed loop of "perception—analysis—decision—execution—feedback," thereby improving the adaptability and intelligence of the manufacturing system. Specifically, see [link to relevant documentation]. Figure 1 As shown, the method includes:

[0063] S1. Collect basic information and historical processing data of production equipment, and construct a knowledge graph that associates four types of entities: equipment, processing parameters, products and faults.

[0064] S2. Based on the basic information and the historical processing data, perform a status assessment on the production equipment to obtain the usage status level;

[0065] S3. Divide the production equipment into different processing groups according to the usage status level, and establish the relationship points between each processing group and the target product corresponding to the current production order in the knowledge graph;

[0066] S4. Based on the aforementioned relationship points, collect the current product processing parameters of each processing group during the production process, analyze the changing trend of the current product processing parameters, and identify abnormal and suspicious processing equipment within each processing group.

[0067] S5. Based on the historical processing data corresponding to each processing group, train a neural network adjustment model for different processing groups, and dynamically adjust the processing parameters of the abnormal processing equipment and / or suspicious processing equipment based on the reinforcement learning strategy combined with the associated knowledge in the knowledge graph.

[0068] In one embodiment of step S1 above, the establishment of the knowledge graph includes the following steps:

[0069] S11. Collect basic information on the current production equipment under the enterprise and historical production data of the enterprise's historical orders. Basic information includes equipment name, model, service time, serial number, etc. Historical production data includes product name, specifications, production quantity, processing technology, processing parameters, product quality indicators, and fault records.

[0070] In addition, it should be noted that the processing parameters are related to the processed product and environment, including temperature, pressure, speed, time, etc. Product quality indicators include dimensional accuracy, surface roughness, physical performance requirements, etc., and fault records include fault type, fault parameters, etc.

[0071] S12. Using natural language processing technology, extract specific entities from historical processing data, including identifying equipment entities from equipment information, processing parameters and product entities from processing data, and fault type entities from fault records. Combine this with order definition to establish entity relationships. The identified entities and relationships are stored in a graph database in the form of a graph, thus obtaining a basic knowledge graph of enterprise production.

[0072] It should be noted that defining the relationships between entities needs to be based on information in the order, such as the "applicability" relationship between equipment and processing parameters in the order, i.e., the equipment in a certain order is suitable for specific processing parameters; the "influence" relationship between processing parameters and products, i.e., the impact of different processing parameters of equipment in the order on the quality and performance of the products; and the "potential occurrence" relationship between equipment and fault types, i.e., the types of faults that may occur in a certain order. By collecting relevant data from historical orders, a relationship classification model is built based on a convolutional neural network. Local features of the data are extracted through convolutional layers, pooling layers perform dimensionality reduction and feature selection on the features, and finally, classification is performed through fully connected layers.

[0073] In one embodiment of step S2 above, performing a status assessment to obtain a usage status level includes:

[0074] S21. Classify the devices by type based on the basic information;

[0075] S22. For the same type of equipment, based on the historical processing data, the failure frequency is statistically analyzed, and a threshold is set according to the failure frequency to divide the equipment into three categories: low failure rate, medium failure rate, and high failure rate, which are used as the usage status level.

[0076] S23. Based on the historical processing data, the service time intervals of the three types of equipment are statistically analyzed, and a correspondence between service time and usage status level is established.

[0077] In one feasible embodiment of usage status level assessment, the process of classifying usage status levels based on the basic information of the equipment under the enterprise, different equipment types, and equipment service time includes:

[0078] First, based on the processing data of historical orders, the service time and failure frequency of processing equipment are statistically analyzed. Based on the failure frequency, the usage status levels of different equipment under different service times are divided into excellent (low failure rate), normal (medium failure rate), and poor (high failure rate).

[0079] Secondly, based on the failure frequency of equipment during historical order processing, the relationship between the service time and usage status level of processing equipment in the current order is established. The specific steps are as follows:

[0080] (1) Statistical analysis of the failure frequency of historical processing equipment in the current order. Based on the first fault frequency threshold Second fault frequency threshold ,in, < Determine the equipment's usage status level:

[0081] (2) When < When the time is right, it means that the equipment has experienced few processing failures in the current order, and its usage status is excellent;

[0082] (3) When ≤ ≤ When this occurs, it indicates that a processing failure has occurred in the current equipment within the current order, and the usage status level is normal.

[0083] (4) When > This indicates that the equipment has experienced a high number of processing failures in the current order, and its usage status is poor.

[0084] Finally, the service time intervals of the equipment under different current usage status levels are statistically analyzed to obtain the status intervals of the current order's service time and usage status level, including the excellent interval, the normal interval, and the poor interval.

[0085] It should be noted that the fault frequency threshold and It is necessary to combine the historical processing data of the current orders, sort the failure frequency data of the historical orders from smallest to largest, select appropriate percentiles as thresholds, use the 25th percentile as the threshold for dividing the "excellent" and "normal" states, and the 75th percentile as the threshold for dividing the "normal" and "poor" states. By classifying the relationship between equipment service time and usage status level in different orders, it is possible to achieve targeted classification of equipment usage status levels in the enterprise according to different orders, so as to ensure that the relationship network in the knowledge graph is more consistent with the relationship between orders, products and equipment in the actual production process of the enterprise.

[0086] In one embodiment of step S3 above, the establishment of the relationship points includes:

[0087] S31. For the target product involved in the new production order, select equipment that has processed the target product or similar products from the historical processing data, and group the equipment according to their corresponding usage status level, so that each processing group contains equipment with the same usage status level.

[0088] S32. In the knowledge graph, create a group node for each processing group and a product node for the target product. Establish a relationship edge representing the processing adaptability between the group node and the product node as the relationship point.

[0089] In one feasible embodiment of the relationship point, based on the equipment status intervals under different usage status levels obtained from the historical data of the current order, the processing equipment involved in the current order is grouped according to the status interval, and then... Establish a database to record the service time of each processing device for the current order, and update it every [period]. The service time of the processing equipment in the database is updated periodically in real time. Equipment in the same status range is grouped into the same group and recorded in the database. At the same time, the equipment in the group is updated in real time based on the service time of the processing equipment in the database.

[0090] It should be noted that, The cycle is usually set to 24 hours, but it can also be adjusted according to the actual production situation of the enterprise and the characteristics of the processing equipment. By updating the service time in the database and updating the equipment in the group, it is convenient to dynamically adjust the processing parameters when processing new orders.

[0091] In one embodiment of step S4 above, identifying abnormal and suspicious processing equipment within each processing group includes:

[0092] S41. For new production orders, collect the current product processing parameters of each processing group and determine whether they exceed the error range based on the preset error.

[0093] S42. Mark the equipment whose current product processing parameters exceed the error range as abnormal processing equipment;

[0094] S43. For equipment whose current product processing parameters do not exceed the error range, calculate the average value of the product processing parameters in the current sampling round and compare it with the corresponding average value in the previous sampling round to determine the trend of the processing parameters of the processing group to which the equipment belongs. The trend of change includes stabilization, improvement or deterioration.

[0095] S44. Among the equipment that has not been marked as abnormal processing equipment, the equipment whose trend of change is deteriorating is marked as suspicious processing equipment.

[0096] In one feasible embodiment for identifying abnormal / suspicious processing equipment, two processes are included: product status change trend identification and processing equipment identification. For a new production order, product status change trend identification is achieved by statistically analyzing the parameters of the products produced by different processing groups within the order's equipment, and identifying the product status change trends of different processing groups based on error standards. Processing equipment identification is based on the product status change trends of different processing groups during the current order's production process to identify whether there are any abnormal or suspicious processing equipment within those groups.

[0097] Specifically, step 1, identifying product status change trends, includes the following steps:

[0098] Step 1.1: For the processing equipment in different equipment processing groups within the current order, based on a fixed cycle... Sampling inspections were conducted on the products produced by each processing equipment, and the product parameters of the sampled products were statistically analyzed and timestamps were recorded.

[0099] Step 1.2: Remove products that do not meet the error criteria, statistically analyze the product parameters within the error criteria, compare the product parameters in this round of sampling with those in the previous round, and determine the current trend of product status changes in the equipment processing group, including whether it is stable, improving, or deteriorating. This specifically includes the following steps:

[0100] Step 1.2.1: For each product parameter in the current order, calculate the mean of the sampled products. Standard deviation of the sample ,in , ;

[0101] in, As parameter number, Representative sample The One parameter, Calculate the change in mean for each sample size. With fluctuation change value :

[0102] ;

[0103] ;

[0104] in, These represent the mean and standard deviation of the samples from the previous round of sampling, respectively.

[0105] Step 1.2.2, when and At this time, it indicates that the current production status of the equipment processing group is stable;

[0106] Steps 1, 2, and 3, when and When this time, it indicates that the current production status of the equipment processing group is trending towards improvement;

[0107] Step 1.2.4, when or When this time, it indicates that the current production status of the equipment processing group is trending towards deterioration.

[0108] It should be noted that the cycle The sampling inspection rules for the current order products need to be determined, which can be either a time period or a product quantity period. These represent the allowable drift threshold of the mean, i.e., the safety margin of the tolerance band, and the volatility change threshold, i.e., the random fluctuation range of the production process. The initial values ​​are set based on historical data, and subsequent optimization is performed iteratively using Xbar-R charts to determine the mean change value. Reflecting systematic deviations in equipment processing, when This indicates that the machining center position remained stable during two random checks in the actual production process. This indicates that parameter fluctuations were under control during two random checks in the actual production process. This indicates an improvement in parameter fluctuations after two random checks in the actual production process. This increases the risk of parameters getting out of control.

[0109] Specifically, step 2, identification of abnormal / suspicious processing equipment, includes the following steps:

[0110] Step 2.1: For processing equipment whose sampled products fall outside the error standard, directly mark it as abnormal processing equipment;

[0111] Step 2.2: For the equipment processing groups where the trend is deteriorating, conduct statistical analysis within each equipment processing group. The parameters and degree of variation of products produced by different equipment under random sampling are compared with the degree of variation of other equipment in the group to identify potentially abnormal processing equipment, which is then marked as suspicious processing equipment. The specific steps are as follows:

[0112] For equipment processing groups with a deteriorating trend, based on cycle Continuous processing of products in the equipment processing group For each sampling inspection, the sampling interval value for the error of the production product parameters of different processing equipment was calculated. The parameter error sampling interval is the difference between the parameter error of the current round and the parameter error of the previous round. The mean of all parameter error sampling intervals within the statistical group is calculated. with standard deviation The interval for random sampling of parameter errors for each device within the group. Make a judgment:

[0113] When the equipment in the group exists < - or > + When this happens, the current processing equipment in the current group will be marked as a suspicious processing equipment.

[0114] It should be noted that, The sampling inspection rules for the current order products need to be determined, which can be either a time period or a product quantity period. It is usually set to 20, that is, every cycle. Twenty consecutive samples are taken from the equipment in the current equipment processing group. The parameter error sampling interval value between the 20 samples is calculated to identify potential abnormal equipment. The value of K is generally 2, but it can be adjusted according to the actual situation. By collecting the error values ​​and calculating the parameter error sampling interval value in the same equipment processing group, the aging of equipment under different service times can be combined to identify potential abnormal equipment.

[0115] Step 2.3: For equipment processing groups whose production status trend is stable or improving, based on the cycle... Continuous processing of products in the equipment processing group For the second sampling inspection, repeat step 2.2 to identify suspicious processing equipment.

[0116] In one embodiment of step S5 above, a neural network regulation model is constructed for different processing groups. The training process of the neural network regulation model for different processing groups includes:

[0117] S51. Collect historical processing data of the processing group involved in the current order. The historical processing data includes the equipment error sampling interval value, the corresponding processing parameters and their numerical adjustment range, and the processing parameters and their numerical adjustment range when the product processing parameters exceed the error standard.

[0118] In practice, historical processing data of different equipment processing groups involved in the current order can be collected from the company's production records. This includes the current equipment error sampling interval value, the corresponding processing parameters and corresponding value adjustment ranges, and the processing parameters and corresponding value adjustment ranges when the current equipment sampled products are outside the error standard. The processing parameters include the feed, speed, depth of cut, etc. of CNC machine tool tools.

[0119] S52. Semantically associate the historical processing data with the knowledge graph, divide the associated data into training set, validation set and test set, train the neural network adjustment model of different processing groups, the neural network adjustment model adopts the ReLU activation function, and calculates the loss between the model prediction output and the real label through the mean squared error loss function.

[0120] In practice, the collected data can be associated with the knowledge graph through the KoPA prefix adapter. This prefix adapter can convert the vectors in the historical processed data dataset into vectors that are semantically consistent with the knowledge graph, thereby realizing the association between the historical processed data and the knowledge graph.

[0121] The data was divided into training, validation, and test sets to train targeted neural network adjustment models for different equipment processing groups. Input, hidden, and output layers were set, ReLU was selected as the activation function, and the loss between the model's predicted output and the true label was calculated using the mean squared error loss function.

[0122] In another embodiment of step S5 above, the processing parameters of the abnormal processing equipment and / or suspicious processing equipment are dynamically adjusted, including:

[0123] S53. The reinforcement learning strategy network is used to control the multi-source knowledge retrieval process, and processing parameter adjustment decisions are generated based on the retrieval results. The processing parameter adjustment values ​​are verified and output, and the reinforcement learning strategy network is updated according to actual production feedback.

[0124] In an optional embodiment of step S53 above, a reinforcement learning policy network is used to control the multi-source knowledge retrieval process, and a processing parameter adjustment decision is generated based on the retrieval results. The adjusted processing parameter values ​​are verified and output, and the reinforcement learning policy network is updated based on actual production feedback. This includes:

[0125] S531. The parameter adjustment task is decomposed into sub-tasks: judging the type of equipment abnormality, determining the direction of parameter adjustment, and calculating the specific adjustment value. A hierarchical reinforcement learning reward mechanism is used to allocate rewards to each sub-task.

[0126] In practical implementation, the task is decomposed into multiple sub-tasks. Specifically, the device anomaly type is first determined, then the parameter adjustment direction is determined, and finally the specific adjustment value is calculated. ConciseR's L-RPO is used as a hierarchical reward mechanism, and a corresponding reward function is designed for each sub-task. The knowledge graph is represented by a graph structure, and text knowledge is converted into word vector representation.

[0127] By designing corresponding reward functions for each subtask, the current model understands the goal of each subtask. In the task of generating processing parameter adjustment values ​​based on the parameters of abnormal and suspicious processing equipment, it is decomposed into subtasks of judging the type of equipment abnormality, determining the direction of parameter adjustment, and calculating the specific adjustment value. Rewards are set for each subtask so that the model will strive to complete each subtask during the learning process, thereby strengthening the purposefulness of learning.

[0128] S532. Using a reinforcement learning strategy network, perform graph traversal retrieval on the knowledge graph and keyword matching and semantic similarity retrieval on the text knowledge, and perform weighted fusion of the retrieval results.

[0129] In practice, the RL strategy network is used to determine the order and method of retrieval in different knowledge sources. Knowledge graphs are retrieved using graph traversal algorithms, while text knowledge is retrieved using keyword matching and semantic similarity calculation methods. A weighted average method is used to fuse the retrieved knowledge.

[0130] The RL policy network is used to determine whether to invoke knowledge graphs or textual knowledge. The input of the policy network is the current model state B, and the output is the action A of selecting the knowledge source. The policy function is π = (A|B), which represents the probability of selecting action A under the current model state B. Random initial values ​​are assigned to the model's weights and biases. Samples from the training set are input into the model. After calculation by the input layer, hidden layer and output layer, the model's predicted output is obtained. The model parameters are updated using the validation set. After training, the model is finally tested using the test set.

[0131] S533. Generate multiple reasoning paths based on fused knowledge, and select the reasoning path with the highest reward value as the optimal path.

[0132] In practice, the anomaly is initially inferred based on the currently retrieved knowledge, and the process is repeated iteratively. Multiple inference paths are generated, and the inference path with the highest reward value is selected as the optimal path.

[0133] S534. Using the ProofTrace mechanism, the generated processing parameter adjustment values ​​are verified based on the mean ± 2 standard deviations of the historical adjustment values ​​of the same parameters to exclude statistical outliers and ensure that the adjustment values ​​are within the reasonable fluctuation range of history.

[0134] In practice, the ProofTrace mechanism is used to verify the generated processing equipment parameter adjustment values. The verification criteria include the mean of historical parameter adjustment values ​​within a range of plus or minus two standard deviations. A positive reward is given if the verification passes. The reward is evaluated based on the inference results and the actual situation to determine the accuracy of the inference. If the reward is low, it indicates that there is a problem with the current model's inference, and the retrieval strategy needs to be adjusted, knowledge sources need to be reselected, and the inference process needs to be corrected. The above process is repeated to ensure that the inference results meet the requirements of the current application environment.

[0135] S535. After the verification is passed, the final processing parameter adjustment value is output, and the actual production feedback, including equipment operation data and product quality data, is used to update the reinforcement learning policy network through the Q-learning algorithm.

[0136] In practice, based on the optimal inference path and verification results, the final processing parameter adjustment values ​​of the processing equipment are compiled. At the same time, the application effect of the output processing parameter adjustment values ​​in actual production is fed back to the system. The application effect includes collecting equipment production data and product quality data, and updating the policy network through Q-learning.

[0137] In an improved embodiment of the above-described method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning, the method further includes:

[0138] S6. After completing the dynamic adjustment of the processing parameters and executing the processing, collect the actual product processing parameters and their quality judgment results obtained from production, write the processing parameter adjustment action, actual product processing parameters and quality judgment results as new knowledge into the knowledge graph, and dynamically update and optimize the knowledge graph.

[0139] The method for dynamically adjusting processing parameters of the present invention constructs a basic knowledge graph of enterprise production based on enterprise production orders, and associates equipment, parameters, products, and fault types based on different production orders. At the same time, based on the service life of different equipment within the enterprise and the actual usage status of equipment of different service life in different orders, the orders in the knowledge graph are grouped into equipment processing teams to monitor the equipment production and processing status under different conditions and identify abnormal equipment. This helps to formulate reasonable equipment maintenance plans to extend the actual service life of equipment.

[0140] By grouping the production and processing equipment of enterprises involved in different orders according to their actual production standards, potential equipment anomalies can be identified during actual production, taking into account equipment aging. Processing parameters can be flexibly adjusted according to the service life of different equipment to ensure product processing accuracy and avoid a one-size-fits-all approach to equipment adjustment. By analyzing the actual usage status of equipment in different orders and classifying the relationship between equipment service time and actual usage status, the actual usage status of equipment in enterprises can be classified according to different orders. This ensures that the relationship network in the knowledge graph is more consistent with the relationship between orders, products, and equipment in the actual production process of enterprises. By collecting error values ​​and calculating parameter error sampling intervals within the same equipment processing group, potential abnormal equipment can be identified by combining the aging status of equipment under different service times.

[0141] Based on the same inventive concept, this invention also provides a dynamic adjustment system for processing parameters based on knowledge graphs and reinforcement learning, as described in the following embodiments. Since the principle of the dynamic adjustment system for processing parameters based on knowledge graphs and reinforcement learning is similar to the dynamic adjustment method for processing parameters based on knowledge graphs and reinforcement learning disclosed in the above embodiments, the implementation of the dynamic adjustment system for processing parameters based on knowledge graphs and reinforcement learning can refer to the implementation of the dynamic adjustment method for processing parameters based on knowledge graphs and reinforcement learning; repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0142] Figure 2 This is a structural block diagram of a dynamic adjustment system for processing parameters based on knowledge graphs and reinforcement learning disclosed in an embodiment of the present invention, such as... Figure 2 As shown, the system includes a data acquisition and knowledge graph construction module 201, an equipment group management module 202, a relationship establishment module 203, a production status monitoring and anomaly identification module 204, and a parameter dynamic adjustment module 205. The structure is described below.

[0143] Among them, the data acquisition and knowledge graph construction module 201 is used to collect equipment type and historical processing data of production equipment, and construct a knowledge graph that associates four types of entities: equipment, processing parameters, products and faults.

[0144] The equipment grouping management module 202 is used to perform a status assessment on the production equipment based on the equipment type and the historical processing data to obtain the usage status level;

[0145] The relationship establishment module 203 is used to divide the production equipment into different processing groups according to the usage status level, and to establish relationship points between each processing group and the target product corresponding to the current production order in the knowledge graph;

[0146] The production status monitoring and anomaly identification module 204 is used to collect the current product processing parameters of each processing group during the production process based on the relationship points, and analyze the changing trend of the current product processing parameters in order to identify abnormal processing equipment and suspicious processing equipment in each processing group.

[0147] The parameter dynamic adjustment module 205 is used to train an adjustment model for different processing groups based on the historical processing data corresponding to each processing group, and dynamically adjust the processing parameters of the abnormal processing equipment and / or suspicious processing equipment based on the reinforcement learning strategy combined with the associated knowledge in the knowledge graph.

[0148] Further, see Figure 2 As shown, the system also includes a knowledge graph update and optimization module 206. The knowledge graph update and optimization module 206 is used to collect the actual product processing parameters and their quality judgment results after completing the dynamic adjustment of the processing parameters and executing the processing, and to write the processing parameter adjustment action, the actual product processing parameters, and the quality judgment results as new knowledge into the knowledge graph, thereby dynamically updating and optimizing the knowledge graph in a closed loop.

[0149] The embodiments of the present invention achieve the following technical effects:

[0150] 1. This invention constructs a basic knowledge graph of enterprise production based on enterprise production orders, and associates equipment, parameters, products, and fault types based on different production orders. At the same time, based on the service life of different equipment within the enterprise and the actual usage status of equipment of different service life in different orders, the orders in the knowledge graph are grouped into equipment processing teams to monitor the equipment production and processing status under different conditions and identify abnormal equipment. This helps to formulate reasonable equipment maintenance plans to extend the actual service life of equipment.

[0151] 2. This invention groups the production and processing equipment of relevant enterprises according to the actual production standards of different orders, so as to identify potential equipment abnormalities in the actual production process by combining the aging of the equipment, and flexibly adjust the processing parameters according to the service time of different equipment to ensure the processing accuracy of the products and avoid a one-size-fits-all equipment adjustment strategy.

[0152] 3. This invention, by classifying the relationship between the service time and actual usage status of equipment in different orders, can achieve targeted classification of the actual usage status of equipment in an enterprise based on different orders. This ensures that the relationship network in the knowledge graph is more consistent with the relationship between orders, products, and equipment in the actual production process of the enterprise. By collecting error values ​​and calculating parameter error sampling intervals in the same equipment processing group, the aging status of equipment under different service times can be combined to identify potential abnormal equipment.

[0153] 4. This invention collects historical processing data from different equipment processing groups to construct a targeted parameter adjustment model, which can predict and adjust processing parameters to ensure that the equipment operates in the best condition, reduce production errors caused by improper parameter settings, and improve product quality and production efficiency. At the same time, by decomposing the task into multiple sub-tasks, it can quickly identify the type of equipment anomaly, determine the direction of parameter adjustment, and calculate the specific adjustment value, thereby improving the overall response speed. By using a KoPA prefix adapter to associate historical processing data with a knowledge graph, and using an RL policy network to determine the invocation of knowledge graph and text knowledge, it achieves effective integration of different knowledge sources and realizes data-driven decision-making.

[0154] In this embodiment, a computer device is provided, such as... Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning.

[0155] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.

[0156] In this embodiment, a computer-readable storage medium is provided, which stores a computer program that executes any of the above-described methods for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning.

[0157] Specifically, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media does not include transient media, such as modulated data signals and carrier waves.

[0158] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0159] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning, characterized in that, include: Collect basic information and historical processing data of production equipment, and construct a knowledge graph that associates four types of entities: equipment, processing parameters, products, and faults. Based on the basic information and the historical processing data, the production equipment is assessed to obtain its usage status level. The production equipment is divided into different processing groups according to the usage status level, and the relationship points between each processing group and the target product corresponding to the current production order are established in the knowledge graph. Based on the aforementioned relationship points, the current product processing parameters of each processing group during the production process are collected, and the changing trends of the current product processing parameters are analyzed in order to identify abnormal and suspicious processing equipment within each processing group. Based on the historical processing data of each processing group, a neural network adjustment model is trained for different processing groups. Based on the reinforcement learning strategy and combined with the associated knowledge in the knowledge graph, the processing parameters of the abnormal processing equipment and / or suspicious processing equipment are dynamically adjusted.

2. The method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to claim 1, characterized in that, A status assessment is conducted to obtain the usage status level, including: The devices are classified according to the basic information provided. For the same type of equipment, based on the historical processing data, the failure frequency is statistically analyzed, and a threshold is set according to the failure frequency to classify the equipment into three categories: low failure rate, medium failure rate, and high failure rate, which are used as the usage status level. Based on the historical processing data, the service time intervals of the three types of equipment were statistically analyzed, and a correspondence between service time and usage status level was established.

3. The method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to claim 1, characterized in that, The establishment of relationship points includes: For the target product involved in the new production order, the equipment that has processed the target product or similar products is selected from the historical processing data, and the equipment is grouped according to its corresponding usage status level, so that each processing group contains equipment with the same usage status level. In the knowledge graph, a group node is created for each processing group and a product node is created for the target product. Relationship edges representing processing adaptability are established between the group nodes and the product nodes, serving as the relationship points.

4. The method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to claim 1, characterized in that, Identify abnormal and suspicious processing equipment within each processing group, including: For new production orders, the current product processing parameters of each processing group are collected, and it is determined whether they exceed the error range based on the preset error. Equipment whose current product processing parameters exceed the error range is marked as abnormal processing equipment; For equipment whose current product processing parameters do not exceed the error range, calculate the average product processing parameters of the current sampling round and compare them with the corresponding average of the previous sampling round to determine the trend of the processing parameters of the processing group to which the equipment belongs. The trend of the trend includes stabilization, improvement or deterioration. Among the equipment that is not marked as abnormal processing equipment, the equipment whose trend of change is deteriorating is marked as suspicious processing equipment.

5. The method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to claim 1, characterized in that, The training process of the neural network regulation model for different processing groups includes: Collect historical processing data of the processing group involved in the current order. The historical processing data includes the equipment error sampling interval, the corresponding processing parameters and their adjustment range, and the processing parameters and their adjustment range when the product processing parameters exceed the error standard. The historical processing data is semantically associated with the knowledge graph. The associated data is then divided into training, validation, and test sets. Neural network adjustment models for different processing groups are trained. The neural network adjustment models use the ReLU activation function, and the loss between the model's predicted output and the true label is calculated using the mean squared error loss function.

6. The method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to claim 5, characterized in that, Dynamically adjusting the processing parameters of the abnormal and / or suspicious processing equipment includes: The reinforcement learning strategy network is used to control the multi-source knowledge retrieval process, and processing parameter adjustment decisions are generated based on the retrieval results. The processing parameter adjustment values ​​are verified and output, and the reinforcement learning strategy network is updated according to actual production feedback.

7. The method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to claim 6, characterized in that, The reinforcement learning policy network is used to control the multi-source knowledge retrieval process, and processing parameter adjustment decisions are generated based on the retrieval results. The adjusted processing parameter values ​​are verified and then output. The reinforcement learning policy network is updated based on actual production feedback. This includes: The parameter adjustment task is broken down into sub-tasks: determining the type of equipment malfunction, determining the direction of parameter adjustment, and calculating the specific adjustment value. A hierarchical reinforcement learning reward mechanism is used to allocate rewards to each sub-task. A reinforcement learning strategy network is used to perform graph traversal retrieval on the knowledge graph and keyword matching and semantic similarity retrieval on the text knowledge, and the retrieval results are weighted and fused. Multiple reasoning paths are generated based on fused knowledge, and the reasoning path with the highest reward value is selected as the optimal path. The ProofTrace mechanism is used to verify the generated processing parameter adjustment values ​​based on the mean ± 2 standard deviations of the historical adjustment values ​​of the same parameters, in order to exclude statistical outliers and ensure that the adjustment values ​​are within the reasonable fluctuation range of history. After the verification is passed, the final processing parameter adjustment value is output, and the actual production feedback, including equipment operation data and product quality data, is used to update the reinforcement learning policy network through the Q-learning algorithm.

8. The method for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning according to claim 1, characterized in that, Also includes: After the processing parameters are dynamically adjusted and the processing is executed, the actual product processing parameters and their quality judgment results are collected. The processing parameter adjustment action, the actual product processing parameters and the quality judgment results are written into the knowledge graph as new knowledge, and the knowledge graph is dynamically updated and optimized in a closed loop.

9. A system for dynamically adjusting processing parameters based on knowledge graphs and reinforcement learning, characterized in that, include: The data acquisition and knowledge graph construction module is used to collect equipment types and historical processing data of production equipment, and to construct a knowledge graph that associates four types of entities: equipment, processing parameters, products, and faults. The equipment grouping management module is used to perform status assessment on the production equipment based on the equipment type and the historical processing data to obtain the usage status level; The relationship establishment module is used to divide the production equipment into different processing groups according to the usage status level, and to establish relationship points between each processing group and the target product corresponding to the current production order in the knowledge graph. The production status monitoring and anomaly identification module is used to collect the current product processing parameters of each processing group during the production process based on the relationship points, analyze the changing trend of the current product processing parameters, and identify abnormal and suspicious processing equipment in each processing group. The parameter dynamic adjustment module is used to train an adjustment model for different processing groups based on the historical processing data of each processing group, and dynamically adjust the processing parameters of the abnormal processing equipment and / or suspicious processing equipment based on the reinforcement learning strategy combined with the associated knowledge in the knowledge graph.

10. The dynamic adjustment system for processing parameters based on knowledge graphs and reinforcement learning according to claim 9, characterized in that, Also includes: The knowledge graph update and optimization module is used to collect the actual product processing parameters and their quality judgment results after the processing parameters are dynamically adjusted and the processing is executed. The processing parameter adjustment action, the actual product processing parameters and the quality judgment results are written into the knowledge graph as new knowledge, and the knowledge graph is dynamically updated and optimized in a closed loop.

Images