Numerical control machine tool code early warning method based on fine-tuning large model

By constructing a professional training corpus for full-stage code semantic information of CNC machine tools and a pre-trained fine-tuning large model with differentiated masks and low-rank mapping enhancement, combined with self-attention convolution classifiers, it solves the difficulty of traditional CNC machine tools' code inspection methods to cope with The problems of complex workpieces and variable situations have achieved higher early warning accuracy and reliability of CNC machine tools.

CN119045830APending Publication Date: 2024-11-29SHANDONG DESA BIG DATA CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411180076.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Traditional CNC machine tool code inspection methods are difficult to cope with complex surface machining workpieces and variable actual industrial situations, resulting in CNC code logic errors or omissions, which may lead to processing failures, equipment damage or even safety accidents.

Method used

Using the CNC machine tool code early warning method based on the fine-tuning large model, a professional training corpus covering the semantic information of the entire stage of CNC machining is constructed, and a pre-trained fine-tuning large model with differentiated masks and low-rank mapping enhancement is designed, combining self-attention convolution scores. Classifier, realizes continuous evaluation and accurate prediction of code warning levels.

Benefits of technology

By integrating the semantic information of the full-stage code and multi-step preprocessing operations, the professionalism and accuracy of CNC machine tool code warning is improved, the false alarm rate of early warning evaluation in traditional methods is reduced, and the reliability and processing efficiency of CNC machine tools are enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045830A_ABST
    Figure CN119045830A_ABST
Patent Text Reader

Abstract

The invention discloses a numerical control machine tool code early warning method based on a fine-tuning large model, and belongs to the technical field of new-generation information technology and high-end manufacturing equipment, and the method comprises the steps: constructing code semantic information covering the whole stage of numerical control machining, and forming a corpus special for numerical control machine tool code early warning; and constructing a pre-training fine-tuning large model enhanced by a differentiation mask and low-rank mapping, and comprehensively designing a self-attention convolution classifier connected with a hidden layer of the fine-tuning large model for result output, thereby realizing continuous evaluation and accurate prediction of numerical control machine tool machining code early warning classification based on domain knowledge fine tuning. According to the method, full-stage code semantic information and multi-step preprocessing operation are integrated, so that a specialized training corpus in the field of numerical control machine tools is formed, and efficient parameter fine adjustment and early warning task migration of a pre-training large model are driven; and the challenge that the professionality and accuracy of early warning evaluation are limited due to the fact that only traditional numerical control code recording or a general language model is relied on is overcome.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of new generation information technology and high-end manufacturing equipment technology, and specifically to a CNC machine tool code early warning method based on a fine-tuning large model. Background Art

[0002] Traditional CNC machine tool code checking methods rely on rules and templates, which are difficult to cope with complex surface machining workpieces and changeable actual industrial situations, resulting in CNC code logic errors or instruction omissions, leading to risks such as machining failure, equipment damage and even safety accidents. Therefore, how to effectively detect and warn potential risks in CNC machine tool codes has become the key to improving the reliability and machining efficiency of CNC machine tools. Summary of the invention

[0003] The present invention is directed to CNC machine tools and CNC systems thereof, and discloses a CNC machine tool code warning method based on a fine-tuning large model, constructs code semantic information covering all stages of CNC processing, forms a special corpus for CNC machine tool code warning, constructs a pre-trained fine-tuning large model enhanced with differentiated masks and low-rank mappings, and comprehensively designs a self-attention convolution classifier connected to the hidden layer of the fine-tuning large model for result output, thereby achieving continuous evaluation and accurate prediction of CNC machine tool machining code warning classification based on domain knowledge fine-tuning; by integrating all-stage code semantic information and multi-step preprocessing operations, a specialized training corpus in the field of CNC machine tools is formed, driving efficient parameter fine-tuning of the pre-training large model and migration of warning tasks, thereby overcoming the challenges of limited professionalism and accuracy of warning evaluation caused by relying solely on traditional CNC code records or general language models.

[0004] The present invention is implemented through the following technical scheme: first, the specific processing tasks performed by multiple types of CNC machine tools are clarified and training corpus instances are defined and constructed; then, preprocessing operations are performed based on a single training corpus instance; then, a differentiated mask method training instance is designed; a low-rank mapping fine-tuning module is further designed to implement model fine-tuning; a self-attention convolution classifier is designed to implement classification judgment of warning levels; finally, the fine-tuned CNC machine tool code warning model is selected and deployed in a production environment.

[0005] The method of clarifying the specific machining tasks performed by multiple types of CNC machine tools and defining and constructing training corpus instances specifically includes: comprehensively collecting CNC system code information and CNC system feedback information before, during and after CNC machining, and taking the operation code and feedback code corresponding to a single machining task as a single training corpus instance; the information processing software for the CNC system code information and CNC system feedback information of specific machining tasks performed by multiple types of CNC machine tools is PYTHON; the general CNC machine simulation software for pre-importing CNC code files, simulating and setting machine tool workpiece parameters, starting simulation operation, executing code generation, and analyzing and verifying a series of operations is Siemens NX; the CNC machine communication interface for obtaining the CNC system code information and CNC system feedback information during and after CNC machining is OPCUA.

[0006] The preprocessing operation based on a single training corpus instance refers to: cleaning the original text of the corpus and deduplicating non-processing instructions, standardizing the definition and expanding the internal elements of each training corpus instance, so that each instance contains four types of elements: unique code id, code snippet content, code feedback summary, and code warning level. By integrating and improving each corpus instance, an expanded representation of the specialized training corpus of CNC machine tool codes is constructed; the code feedback summary specifically includes: for the code warning level, from low to high, it includes four categories: information alarm, warning alarm, error alarm, and fatal alarm.

[0007] The design of the differentiated masking method training instance refers to: customizing the fine-tuning training task for the pre-trained large language model, designing the differentiated masking method for each element in the training corpus instance, distinguishing and executing the Masked Language Modeling and Next Sentence Prediction tasks, recorded as MLM tasks and NSP tasks; improving the ability of the fine-tuned pre-trained large language model to understand the relationship between the code snippet content, code feedback summary and code warning level; designing the differentiated masking method is to randomly mask the token corresponding to the code feedback summary, and for the token corresponding to the code feedback summary, the code snippet content, and the code warning level, whether the context corresponds to the NSP task of a sentence; the differentiated masking method refers to: only setting the NSP task for the code snippet content and the code warning level, and executing the MLM and NSP tasks for the code feedback summary at the same time; the pre-trained large language model is specifically: a Bidirectional EncoderRepresentations from Transformers model, recorded as a BERT model; the random masking refers to: randomly replacing the original token with "[MASK]".

[0008] The described design of the low-rank mapping fine-tuning module specifically includes: freezing the main parameters of the large language model during fine-tuning, fine-tuning by adding an adapter module to each layer of the pre-trained large language model, mapping each hidden layer output of the pre-trained large language model to a newly added projection layer with a lower rank, and then connecting the newly added feedforward sublayer through a nonlinear layer, mapping the low-dimensional features after the low-rank mapping back to the high-dimensional features, and realizing the parameter fine-tuning modification of the pre-trained large language model.

[0009] The designed self-attention convolution classifier refers to: connecting the hidden layer output of the pre-trained large language model that integrates the low-rank mapping fine-tuning module to the convolutional neural network layer, performing a deep evaluation of the hidden layer semantic feature vector, and further outputting it to the multi-head self-attention network layer to obtain the mutual correlation information between the deep features and the deep features, and finally connecting it to the fully connected feedforward network, and cooperating with the Softmax layer to complete the classification and judgment of the code warning level; the classification judgment refers to: a four-classification task to realize the recognition of four types of code warning levels including "information alarm", "warning alarm", "error alarm" and "fatal alarm".

[0010] The selecting of the fine-tuned CNC machine tool code warning model refers to: for the pre-trained large language model after the fine-tuning design is completed, the training parameters are set to perform large model training supported by domain knowledge, and the pre-trained large language model with the highest classification accuracy and the lowest perplexity on the specialized training corpus of CNC machine tool codes is selected as the CNC machine tool code warning model after completing domain knowledge fine-tuning; the setting of training parameters includes but is not limited to learning rate, training steps, batch size, and weight decay rate; the pre-trained large language model with the highest classification accuracy and the lowest perplexity refers to: when selecting a model, first select the pre-trained large language model with the highest classification accuracy, and if the classification accuracy rates are the same, select the pre-trained large language model with the lowest perplexity.

[0011] The deployment to the production environment refers to: deploying the CNC machine tool code warning model that has been fine-tuned by integrating machine tool domain knowledge into the production environment, automatically inputting the CNC machine tool code to be detected into the warning model for self-analysis and warning, forming a CNC machine tool code warning device based on the fine-tuned large model, continuously monitoring the performance of the fine-tuned large model in actual applications, recording the model's prediction accuracy and false alarm rate, updating the CNC machine tool code specialized training corpus according to the latest annotated data and feedback, and regularly fine-tuning and updating the designed pre-trained large model to ensure that the model always maintains optimal performance; the machine tool domain knowledge specifically refers to the CNC machine tool code specialized training corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of the CNC machine tool code warning method based on fine-tuning the large model;

[0013] Figure 2 It is a schematic diagram of the principle of the CNC machine tool code early warning method based on fine-tuning the large model;

[0014] Figure 3 It is the effect diagram of the fine-tuning parameter quantity of the CNC machine tool code warning method based on the fine-tuning large model;

[0015] Figure 4 This is the actual prediction effect diagram of the CNC machine tool code early warning method based on the fine-tuning large model. DETAILED DESCRIPTION

[0016] This embodiment is implemented on the premise of the technical solution of the present invention, and provides a detailed implementation method and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0017] Step 1: For specific machining tasks performed by multiple types of CNC machine tools, comprehensively collect CNC system code information and CNC system feedback information before, during and after CNC machining. CNC system code information includes but is not limited to G code, M code, T code, S code, F code, D code, H code, P code, and N code. CNC system feedback information includes but is not limited to mechanical fault alarm, electrical fault alarm, system fault alarm, sensor fault alarm, communication fault alarm, operation fault alarm, environmental fault alarm, and maintenance prompt alarm. The operation code and feedback code corresponding to a single machining task are used as a single training corpus instance.

[0018] Step 2: Perform preprocessing operations based on a single training corpus instance, clean the original text and remove duplicate non-processing instructions, standardize the definition and expand the internal elements of each training corpus instance, so that each instance contains four types of elements: unique code id, code snippet content, code feedback summary, and code warning level. Define a single instance as ,in Is the unique code id, For the code snippet content, Summary for code feedback, For code warning levels, by integrating and improving various corpus examples , construct a specialized training corpus of CNC machine tool codes after expansion representation ,in is the total number of corpus instances collected.

[0019] 2-1 Further Each element in is mapped to the word embedding space, expressed as: , where the word embedding space operation is adopted Layer implementation.

[0020] 2-2 Further The position of each element in is encoded. Word embedding vectors added to the specialized training corpus of CNC machine tool codes In this paper, we obtain an initial representation that can be input into the pre-trained large model. .

[0021] Step 3: Customize fine-tuning training tasks for the pre-trained large language model, and use the BERT pre-trained large language model to perform customized fine-tuning training tasks. Use the design of differentiated masking methods to distinguish the elements in the training corpus instances and perform MLM and NSP tasks to improve the ability of the fine-tuned pre-trained large language model to understand the relationship between code snippet content, code feedback summary, and code warning level.

[0022] 3-1 Input sequence after initial representation With the input sequence For example, , represents the first In the corpus examples . Depend on tokens, Depend on tokens, namely , , concatenate the two sequences into one input sequence in the form of: x i comb =[ CLS , hx i,1 0,2 , hx i,2 0,2 ,…, hx i, n x i 2 0,2 , SEP , hx i,1 0,3 , hx i,2 0,3 ,…, hx i, n x i 3 0,3 , SEP ] .

[0023] 3-2 BERT model receives the concatenated sequence As input, the context representation of each token is generated as: ,in is the hidden layer output of the pre-trained large model, specifically expressed as: h i =[ h CLS , h hx i,1 0,2 , h hx i,2 0,2 ,…, h hx i, n x i 2 0,2 , h [ SEP ] , h hx i,1 0,3 , h hx i,2 0,3 ,…, h hx i, n x i 3 0,3 , h [ SEP ] ] , from using Hidden representation of markup The next sentence prediction begins by calculating the probabilities of two categories through a fully connected layer and a Softmax activation function to complete the NSP task for all tokens; the two categories refer to the next sentence and non-next sentence, respectively.

[0024] 3-3 Input sequence For example, suppose Depend on tokens, namely . With a certain probability facing the original sequence All tags of an instance are randomly selected from non-fixed tags , and replace it with a special marker [MASK]. The masked sequence is x i 3,m =[ x i, 1 3 , x i, 2 3 ,…, MASK , …, x i, n x i 3 3 ] ,in x i,q 3 = [ MASK] , further Implement the initial representation to obtain a sequence that can be input to the BERT model , for each masked , the BERT model can be represented by its corresponding hidden Predictions represent the original markup , in order to complete the MLM task of all tokens.

[0025] Step 4: Design a low-rank mapping fine-tuning module to freeze the main parameters of the large language model during fine-tuning, and perform fine-tuning by adding an adapter module to each layer of the pre-trained large language model. The adapter module can map the 768-dimensional features of the BERT hidden layer to 256-dimensional low-dimensional features, that is, map the output of each hidden layer of the pre-trained large language model to a newly added projection layer with a lower rank, and then connect the newly added feedforward sublayer through a nonlinear layer to map the low-dimensional features after the low-rank 256-dimensional mapping back to the high-dimensional 768-dimensional features, thereby realizing the parameter fine-tuning modification of the pre-trained large language model.

[0026] Step 5: Design a self-attention convolution classifier, connect the hidden layer output of the pre-trained large language model that integrates the low-rank mapping fine-tuning module to the convolutional neural network layer, perform a deep evaluation of the hidden layer semantic feature vector, and further output it to the multi-head self-attention network layer to obtain the mutual correlation information between deep features and deep features. Finally, connect it to the fully connected feedforward network and cooperate with the Softmax layer to complete the probabilistic classification judgment of the code warning level.

[0027] 5-1 Assume that the input sequence is fine-tuned through the pre-trained BERT model, and the output of the last hidden layer is represented as ,in is the length of the sequence represented by the corpus instance, The dimension of the hidden layer is 768.

[0028] 5-2 Further use the self-attention mechanism to capture the relationship between different positions in the sequence. They are query, key and value matrices respectively, and the specific calculation method is: ,in, , , For The corresponding trainable weights have dimensions , is the dimension in the attention mechanism.

[0029] 5-3 Further calculate the attention score of the self-attention mechanism on the hidden layer output, that is ,in is the attention weight matrix, through the attention weight Sum Matrix , calculate the attention output , .

[0030] 5-4 Further output of attention , passed to the one-dimensional convolution layer, assuming the convolution kernel size is , the convolution kernel weight is ,in is the number of channel outputs, and the specific convolution operation can be expressed as: ,in , is the output of the convolutional layer.

[0031] 5-5 Convolutional layer Flatten the output of to get the vector , and then passed to the fully connected feedforward network, the weight of the fully connected layer is set to , the bias is , and further get the fully connected layer output ), output the fully connected layer Pass it to the Softmax layer to get the probability distribution of the four code warning categories. Suppose the weight of the Softmax layer is , the bias is , get the final output of the warning for: .

[0032] Step 6: For the pre-trained large language model after fine-tuning, set the training parameters to train the large model supported by domain knowledge, and select the pre-trained large language model with the highest classification accuracy and lowest perplexity on the specialized training corpus of CNC machine tool codes as the CNC machine tool code warning model after completing domain knowledge fine-tuning. Figure 2 The principle diagram of the CNC machine tool code warning method based on fine-tuning the large model is provided. The training parameters are set on the training parameters of BERT, including the learning rate is set to 10 -7 , the number of training steps is 70000, the batch size is 8, and the weight decay rate is 0.01.

[0033] Step 7: Deploy the CNC machine tool code warning model that has been fine-tuned by integrating machine tool domain knowledge into the production environment, automatically input the CNC machine tool code to be detected into the warning model for self-analysis and warning, and form a CNC machine tool code warning device based on the fine-tuned large model. Continuously monitor the performance of the fine-tuned large model in actual applications, record the model's prediction accuracy and false alarm rate, update the CNC machine tool code specialized training corpus according to the latest annotated data and feedback, and regularly fine-tune and update the designed pre-trained large model to ensure that the model always maintains optimal performance; among them, machine tool domain knowledge specifically refers to the CNC machine tool code specialized training corpus.

[0034] Figure 3 The parameter details of the CNC machine tool code warning method based on fine-tuning the large model are provided to illustrate the parameter effectiveness of the invention; Figure 4 An actual CNC machining code early warning method for a Siemens 840D CNC system is provided to illustrate the specific implementation and effect of the invention.

[0035] The present invention proposes a CNC machine tool code warning method based on a fine-tuning large model, constructs code semantic information covering all stages before, during and after CNC machining, designs a pre-trained fine-tuning large model enhanced by differential masking and low-rank mapping, and outputs the result through a self-attention convolution classifier connected to the hidden layer of the fine-tuning large model, driving the efficient fine-tuning of the parameters of the pre-trained large model and the migration of warning tasks, and realizing the continuous evaluation and accurate prediction of the CNC machine tool machining code warning classification based on domain knowledge fine-tuning. Compared with the prior art, the single training corpus instance and the specialized training corpus of the present invention guarantee the training basis of the fine-tuning large model, and the designed differential masking method and low-rank mapping fine-tuning module improve the logical understanding ability of the relationship between the code snippet content, code feedback summary and code warning level, overcome the application challenges of insufficient CNC machine tool machining code warning classification ability and poor prediction accuracy due to lack of specific domain knowledge, and have important engineering practical value.

[0036] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A CNC machine tool code early warning method based on fine-tuning large model, characterized in that: Firstly, the specific processing tasks performed by multiple types of CNC machine tools are clarified, and training corpus instances are defined and constructed. Then, preprocessing operations are performed based on a single training corpus instance. Finally, a differentiated mask method training instance is designed. Further, a low-rank mapping fine-tuning module is designed to implement model fine-tuning. A self-attention convolution classifier is designed to realize the classification and judgment of warning levels. Finally, the fine-tuned CNC machine tool code warning model is selected and deployed in the production environment.

2. According to claim 1, a CNC machine tool code early warning method based on fine-tuning large model is characterized in that: The specific machining tasks performed by various types of CNC machine tools are clarified, and training corpus instances are defined and constructed, including: comprehensive collection of CNC system code information and CNC system feedback information before, during and after CNC machining, with the operation code and feedback code corresponding to a single machining task as a single training corpus instance; the information processing software for the CNC system code information and CNC system feedback information of specific machining tasks performed by various types of CNC machine tools is PYTHON; the general CNC machine simulation software for a series of operations such as pre-import of CNC code files, simulation setting of machine tool workpiece parameters, start of simulation operation, execution code generation, and feedback report analysis and verification is Siemens NX; the CNC machine tool communication interface for obtaining CNC system code information and CNC system feedback information during and after CNC machining is OPC UA.

3. According to claim 1, a CNC machine tool code early warning method based on fine-tuning large model is characterized in that: The preprocessing operation based on a single training corpus instance refers to: cleaning the original text of the corpus and deduplicating non-processing instructions, standardizing the definition and expanding the internal elements of each training corpus instance, so that each instance contains four types of elements: unique code id, code snippet content, code feedback summary, and code warning level. By integrating and improving each corpus instance, an expanded representation of the specialized training corpus of CNC machine tool codes is constructed; the code feedback summary specifically includes: for the code warning level, from low to high, it includes four categories: information alarm, warning alarm, error alarm, and fatal alarm.

4. The CNC machine tool code early warning method based on fine-tuning large model according to claim 1 is characterized in that: The design of the differentiated masking method training instance refers to: customizing the fine-tuning training task for the pre-trained large language model, designing the differentiated masking method for each element in the training corpus instance, distinguishing and executing the Masked Language Modeling and NextSentence Prediction tasks, recorded as MLM tasks and NSP tasks; improving the ability of the fine-tuned pre-trained large language model to understand the relationship between the code snippet content, code feedback summary and code warning level; designing the differentiated masking method is to randomly mask the Token corresponding to the code feedback summary, and for the code feedback summary, the code snippet content, and the code warning level The corresponding Token execution context corresponds to the NSP task of a sentence; the differentiated masking method refers to: only setting the NSP task for the code snippet content and the code warning level, and performing the MLM and NSP tasks for the code feedback summary at the same time; the pre-trained large language model is specifically: a Bidirectional Encoder Representations from Transformers model, recorded as a BERT model; the random masking refers to: randomly replacing the original Token with "[MASK]".

5. The CNC machine tool code early warning method based on fine-tuning large model according to claim 1 is characterized in that: The described design of the low-rank mapping fine-tuning module specifically includes: freezing the main parameters of the large language model during fine-tuning, fine-tuning by adding an adapter module to each layer of the pre-trained large language model, mapping each hidden layer output of the pre-trained large language model to a newly added projection layer with a lower rank, and then connecting the newly added feedforward sublayer through a nonlinear layer, mapping the low-dimensional features after the low-rank mapping back to the high-dimensional features, and realizing the parameter fine-tuning modification of the pre-trained large language model.

6. The CNC machine tool code early warning method based on fine-tuning large model according to claim 1 is characterized in that: The designed self-attention convolution classifier refers to: connecting the hidden layer output of the pre-trained large language model that integrates the low-rank mapping fine-tuning module to the convolutional neural network layer, performing a deep evaluation of the hidden layer semantic feature vector, and further outputting it to the multi-head self-attention network layer to obtain the mutual correlation information between deep features and deep features, and finally connecting it to the fully connected feedforward network, and cooperating with the Softmax layer to complete the classification and judgment of the code warning level; the classification judgment refers to: a four-classification task to realize the recognition of four types of code warning levels including "information alarm", "warning alarm", "error alarm" and "fatal alarm".

7. The CNC machine tool code early warning method based on fine-tuning large model according to claim 1 is characterized in that: The selecting of the fine-tuned CNC machine tool code warning model refers to: for the pre-trained large language model after the fine-tuning design is completed, the training parameters are set to perform large model training supported by domain knowledge, and the pre-trained large language model with the highest classification accuracy and the lowest perplexity on the specialized training corpus of CNC machine tool codes is selected as the CNC machine tool code warning model after completing domain knowledge fine-tuning; the setting of training parameters includes but is not limited to learning rate, training steps, batch size, and weight decay rate; the pre-trained large language model with the highest classification accuracy and the lowest perplexity refers to: when selecting a model, first select the pre-trained large language model with the highest classification accuracy, and if the classification accuracy rates are the same, select the pre-trained large language model with the lowest perplexity.

8. The CNC machine tool code early warning method based on fine-tuning large model according to claim 1 is characterized in that: The deployment to the production environment refers to: deploying the CNC machine tool code warning model that has been fine-tuned by integrating machine tool domain knowledge into the production environment, automatically inputting the CNC machine tool code to be detected into the warning model for self-analysis and warning, forming a CNC machine tool code warning device based on the fine-tuned large model, continuously monitoring the performance of the fine-tuned large model in actual applications, recording the model's prediction accuracy and false alarm rate, updating the CNC machine tool code specialized training corpus according to the latest annotated data and feedback, and regularly fine-tuning and updating the designed pre-trained large model to ensure that the model always maintains optimal performance; the machine tool domain knowledge specifically refers to the CNC machine tool code specialized training corpus.

Citation Information

Cited By

  • Code defect detection method and system based on large model

    CN119690512A