Smart factory production line parameter optimization system and method based on large model

Abstract

The invention discloses a smart factory production line parameter optimization system and method based on a large model. The system comprises a corpus and the large model. Data acquisition, data analysis and knowledge base construction are performed on factory production line data to form a corpus for large model training. And introducing a multi-layer time sequence, a mixed attention mechanism and a parameter optimization decision into the large model to realize process optimization. A corpus generation scheme designed for a factory production line comprises data discretization grouping processing, a knowledge base and the like. A large model network structure designed for a factory production line comprises a multi-layer time sequence structure, covariance is introduced to carry out mixed attention mechanism calculation, loss is independently calculated for each parameter, and knowledge base retrieval is introduced into a parameter optimization decision module. According to the optimization method, parameter process optimization is achieved through production line process parameter collection, data processing and large model training, the percent of pass is increased, and cost reduction and efficiency improvement are achieved.

Description

Technical Field

[0001] The present invention relates to a factory production line optimization method, and in particular to a smart factory production line parameter optimization system and method based on a large model. Background Art

[0002] Traditional factory production line optimization methods usually rely on trial and error and experience, and engineers adjust production parameters based on experience and experimental results. There are the following problems: over-reliance on engineers' experience and technical knowledge, requiring a lot of experiments and time costs for parameter adjustment, and difficulty in fully considering the complex interaction of parameters in extreme scenarios, and it is easy to fall into local optimal solutions. In today's industrial production, the concept of smart factories is increasingly valued, and companies tend to apply digital technology and big data to production processes to improve efficiency, reduce costs and optimize product quality. Summary of the invention

[0003] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a smart factory production line parameter optimization system and method based on a large model.

[0004] To achieve the purpose of the present invention, the technical solution adopted by the present invention is: A large-model-based intelligent factory production line parameter optimization system, including a corpus and a large model, the corpus includes a data acquisition module, a data analysis module, and a knowledge base construction module, and the large model includes a multi-layer time series module, a hybrid attention mechanism module, and a parameter optimization decision module; The data collection module is used to collect the temperature, humidity, air pressure, raw material model parameters, and machine operating parameters of the working environment; the data analysis module is used to summarize the relevance of multi-dimensional data after discretizing and grouping the collected data; the knowledge base construction module is used to convert relevant text knowledge into data tables and search for data consistent with the corresponding knowledge in the collected data; The multi-layer time series module vectorizes the input parameters, extracts parameter features through the feature extraction network, and then accumulates the parameter features extracted from multiple frames according to the weights; The hybrid attention mechanism module uses statistical covariance in the traditional transform structure, combines deep learning with traditional mathematical statistics, and infers the related properties of different parameters through sample data training. A layer perceptron is designed for each parameter. The dimension of the output weight of each perceptron is the same as the dimension of vectorization. The corresponding parameter is multiplied by the transpose of the corresponding weight, and each parameter obtains an independent evaluation result. The parameter optimization decision module adjusts parameters according to the weight judgment of the model output. At the same time, it retrieves similar data in the knowledge base and makes secondary adjustments based on it.

[0005] Furthermore, the data analysis module also includes separately counting the range of each parameter when the factory production line is normal / abnormal, comparing each group of data step by step, eliminating abnormal extreme value data, and obtaining the effective and reasonable range of each group of parameters; at the same time, calculating the covariance of the sample data and introducing it into the attention mechanism of the large model.

[0006] Furthermore, in the hybrid attention mechanism module, the hybrid attention mechanism is implemented by multiplying the q and k sequences after the linear transformation of the parameters to the attention weight matrix, and then adding it to the covariance weight matrix after the linear transformation of the statistical covariance, and then normalizing it through softmax to obtain the hybrid attention weight matrix.

[0007] A method for optimizing production line parameters of a smart factory based on a large model, comprising the steps of: (1) During the data collection phase, the temperature, humidity, air pressure, raw material model parameters, and machine operating parameters of the working environment are collected; (2) In the data analysis stage, the collected data is discretized and grouped, and the correlation of multidimensional data is summarized. At the same time, the interval range of each parameter of the factory production line when it is normal / abnormal is counted separately, and each group of data is compared level by level, and abnormal extreme value data is eliminated to obtain the effective and reasonable interval of each group of parameters. At the same time, the covariance of the sample data is calculated and introduced into the attention mechanism of the large model. (3) In the knowledge base construction stage, the relevant text knowledge is converted into data tables, and data consistent with the corresponding knowledge is searched in the collected data; (4) To train a large model network, firstly vectorize the input parameters and introduce multi-layer time series. Firstly, extract parameter features through feature extraction network, and then accumulate the parameter features extracted from multiple frames according to the weights. (5) In the hybrid attention mechanism, statistical covariance is referenced in the traditional transform structure, deep learning and traditional mathematical statistics are combined, and the related properties of different parameters are inferred through sample data training; in the weight layer, a layer perceptron is designed for each parameter. The dimension of the output weight of each perceptron is the same as the dimension of vectorization. The corresponding parameter is multiplied by the transpose of the corresponding weight, and each parameter obtains an independent evaluation result; (6) In the parameter optimization decision, the parameters are adjusted according to the weight judgment of the model output. At the same time, similar data will be retrieved from the knowledge base and used as a reference for secondary adjustment.

[0008] Furthermore, in step (1), the collected data includes positive / negative samples, that is, data on normal / abnormal operation of the factory production line.

[0009] Furthermore, in step (5), the hybrid attention mechanism is implemented by multiplying the q and k sequences after the linear transformation of the parameters to the attention weight matrix, and then adding it to the covariance weight matrix after the linear transformation of the statistical covariance, and then normalizing it through softmax to obtain the hybrid attention weight matrix.

[0010] Furthermore, in step (6), in the parameter optimization decision, the parameter adjustment is performed according to the weight judgment of the model output. If the output ratio is greater than 1, the parameter is adjusted down, otherwise the parameter is adjusted up.

[0011] Furthermore, the loss is calculated independently for each parameter, and the formula is as follows:

[0012] in, and represents weight, N represents parameter type, represents the predicted value of the i-th parameter, represents the true value of the i-th parameter. CELoss is a multi-classification loss function, corresponding to the parameter grouping in the corpus generation module.

[0013] The beneficial effect of the present invention is that, compared with the prior art, the optimization method of the present invention realizes parameter process optimization, improves the pass rate, and achieves cost reduction and efficiency improvement by collecting production line process parameters, data processing, and large model training; it can realize automation and save costs; and compared with traditional machine learning solutions, this solution is highly robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the intelligent factory production line parameter optimization system based on a large model according to the present invention; Figure 2 It is a schematic diagram of the large model network; Figure 3 It is a schematic diagram of the hybrid attention mechanism. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of this application.

[0016] like Figure 1 As shown, the intelligent factory production line parameter optimization system based on the big model of the present invention includes a corpus and a big model. The corpus includes a data acquisition module, a data analysis module, and a knowledge base construction module, and the big model includes a multi-layer time series module, a hybrid attention mechanism module, and a parameter optimization decision module.

[0017] The data acquisition module is used to collect the temperature, humidity, air pressure, raw material model parameters, and machine operating parameters of the working environment; the data analysis module is used to summarize the correlation of multidimensional data after discretizing and grouping the collected data; the knowledge base construction module is used to convert relevant text knowledge into data tables and search for data consistent with the corresponding knowledge in the collected data.

[0018] The multi-layer time series module vectorizes the input parameters, extracts parameter features through the feature extraction network, and then accumulates the parameter features extracted from multiple frames according to the weights; The hybrid attention mechanism module uses statistical covariance in the traditional transform structure, combines deep learning with traditional mathematical statistics, and infers the related properties of different parameters through sample data training. A layer perceptron is designed for each parameter. The dimension of the output weight of each perceptron is the same as the dimension of vectorization. The corresponding parameter is multiplied by the transpose of the corresponding weight, and each parameter obtains an independent evaluation result. The parameter optimization decision module adjusts parameters according to the weight judgment of the model output. At the same time, it retrieves similar data in the knowledge base and makes secondary adjustments based on it.

[0019] Data collection, data analysis, and knowledge base construction are performed on factory production line data to form a corpus for large model training. Multi-layer time series, mixed attention mechanism, and parameter optimization decision are introduced into the large model to achieve process optimization.

[0020] Corpus generation scheme for factory production line design, including data discretization and grouping processing, knowledge base, etc. Large model network structure for factory production line design, including multi-layer time series structure, introduction of covariance for mixed attention mechanism calculation, independent calculation of loss for each parameter, and introduction of knowledge base retrieval in parameter optimization decision module.

[0021] The present invention also provides a method for optimizing production line parameters of a smart factory based on a large model, comprising the steps of: During the data collection stage, the temperature, humidity, air pressure, raw material model parameters, machine operating parameters, etc. of the working environment are first collected, which needs to include positive / negative samples, that is, data on normal / abnormal operation of the factory production line.

[0022] In the data analysis stage, a common solution is to directly train the model on the collected continuous data. Considering that the collected data has a wide dimension and a wide range, for the application of large models of smart factories, the present invention discretizes and groups the data. For example, the temperature data is divided into several groups according to the statistical maximum and minimum values. If the collected data is in a certain group, the group number is used to represent its temperature value.

[0023] Compared with continuous data, this solution allows the model to converge quickly and summarize the correlation of multi-dimensional data. In addition, the interval range of each parameter when the factory production line is normal / abnormal is counted separately, and each group of data is compared level by level, and abnormal extreme value data is eliminated to obtain the effective and reasonable interval of each group of parameters.

[0024] In order to deeply explore the relevant properties of each parameter, the covariance of the sample data is calculated and introduced into the attention mechanism of the large model.

[0025] In the knowledge base construction stage, a dedicated knowledge base is built based on the operating experience of process engineers and industrial knowledge, which is different from the common large text model. The relevant text knowledge is converted into a data table. In addition, to ensure the rationality of the knowledge, that is, to avoid the introduction of erroneous knowledge due to unclear expressions by process engineers, data consistent with the corresponding knowledge will be searched in the collected data. For example: in summer, for raw material No. 2, there is a sudden rainstorm scene, and the pressure value of pressure valve No. 29 needs to be increased; therefore, the sensitive parameters in the corresponding table data are: temperature, material model, humidity, and pressure value No. 29.

[0026] like Figure 2 As shown in the figure, considering the particularity of the process parameters of the factory production line, the large model network training process includes the following steps: First, the input parameters are vectorized. Compared with text, parameter values ​​are integers, which are easier to understand. However, considering the complexity of finding internal associations, the parameters are still vectorized. Considering the complexity of factory production line data, the anomaly at the current moment, and the coupling of parameters at previous times, the present invention introduces a multi-layer time series. This part first extracts parameter features through a feature extraction network, and then accumulates the parameter features extracted from multiple frames according to the weights (the closer the time, the greater the weight).

[0027] In the hybrid attention mechanism, statistical covariance is referenced in the traditional transform structure, combining deep learning and traditional mathematical statistics, which can infer the related properties of different parameters through sample data training. The implementation method is to multiply the q and k sequences after the linear transformation of the parameters to the attention weight matrix, and then add it to the covariance weight matrix after the linear transformation of the statistical covariance, and then perform normalization through softmax to obtain the hybrid attention weight matrix. Figure 3 shown.

[0028] In the weight layer, the present invention designs a layer perceptron for each parameter. The dimension of the output weight of each perceptron is the same as the dimension of vectorization. The corresponding parameters are multiplied by the transpose of the corresponding weights, and each parameter obtains an independent evaluation result.

[0029] The process parameters are similar to the barrel theory. Any abnormal parameter will cause problems in the system. Therefore, the present invention calculates the loss of each parameter independently. The formula is as follows:

[0030] in, and represents the weight, which is adjusted according to the training effect. N represents the parameter type. represents the predicted value of the i-th parameter, Represents the true value of the i-th parameter. CELoss is a common multi-classification loss function, which corresponds to the parameter grouping in the corpus generation module.

[0031] In the parameter optimization decision, the parameter can be adjusted according to the weight of the model output. If the output ratio is greater than 1, the parameter is adjusted down, otherwise the parameter is adjusted up. At the same time, similar data will be retrieved from the knowledge base and used as a reference for secondary adjustments to prevent model phantoms.

[0032] The beneficial effect of the present invention is that, compared with the prior art, the optimization method of the present invention realizes parameter process optimization, improves the pass rate, and achieves cost reduction and efficiency improvement by collecting production line process parameters, data processing, and large model training; it can realize automation and save costs; and compared with traditional machine learning solutions, this solution is highly robust.

[0033] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A smart factory production line parameter optimization system based on a large model, characterized in that: It includes a corpus and a large model. The corpus includes a data collection module, a data analysis module, and a knowledge base construction module. The large model includes a multi-layer time series module, a hybrid attention mechanism module, and a parameter optimization decision module. The data collection module is used to collect the temperature, humidity, air pressure, raw material model parameters, and machine operating parameters of the working environment; the data analysis module is used to summarize the relevance of multi-dimensional data after discretizing and grouping the collected data; the knowledge base construction module is used to convert relevant text knowledge into data tables and search for data consistent with the corresponding knowledge in the collected data; The multi-layer time series module vectorizes the input parameters, extracts parameter features through the feature extraction network, and then accumulates the parameter features extracted from multiple frames according to the weights; The hybrid attention mechanism module uses statistical covariance in the traditional transform structure, combines deep learning with traditional mathematical statistics, and infers the related properties of different parameters through sample data training. A layer perceptron is designed for each parameter. The dimension of the output weight of each perceptron is the same as the dimension of vectorization. The corresponding parameter is multiplied by the transpose of the corresponding weight, and each parameter obtains an independent evaluation result. The parameter optimization decision module adjusts parameters according to the weight judgment of the model output. At the same time, it retrieves similar data in the knowledge base and makes secondary adjustments based on it.

2. According to the large model-based smart factory production line parameter optimization system of claim 1, it is characterized in that: The data analysis module also includes statistically analyzing the range of each parameter when the factory production line is normal / abnormal, comparing each group of data step by step, eliminating abnormal extreme value data, and obtaining the effective and reasonable range of each group of parameters; at the same time, the covariance of the sample data is calculated and introduced into the attention mechanism of the large model.

3. The large model-based intelligent factory production line parameter optimization system according to claim 2 is characterized in that: In the hybrid attention mechanism module, the hybrid attention mechanism is implemented by multiplying the q and k sequences after the linear transformation of the parameters to the attention weight matrix, and then adding it to the covariance weight matrix after the linear transformation of the statistical covariance, and then normalizing it through softmax to obtain the hybrid attention weight matrix.

4. A method for optimizing production line parameters of a smart factory based on a large model, characterized in that: Includes steps: (1) During the data collection phase, the temperature, humidity, air pressure, raw material model parameters, and machine operating parameters of the working environment are collected; (2) In the data analysis stage, the collected data is discretized and grouped, and the correlation of multidimensional data is summarized. At the same time, the interval range of each parameter of the factory production line when it is normal / abnormal is counted separately, and each group of data is compared level by level, and abnormal extreme value data is eliminated to obtain the effective and reasonable interval of each group of parameters. At the same time, the covariance of the sample data is calculated and introduced into the attention mechanism of the large model. (3) In the knowledge base construction stage, the relevant text knowledge is converted into data tables, and data consistent with the corresponding knowledge is searched in the collected data; (4) To train a large model network, firstly vectorize the input parameters and introduce multi-layer time series. Firstly, extract parameter features through feature extraction network, and then accumulate the parameter features extracted from multiple frames according to the weights. (5) In the hybrid attention mechanism, statistical covariance is referenced in the traditional transform structure, deep learning and traditional mathematical statistics are combined, and the related properties of different parameters are inferred through sample data training; in the weight layer, a layer perceptron is designed for each parameter. The dimension of the output weight of each perceptron is the same as the dimension of vectorization. The corresponding parameter is multiplied by the transpose of the corresponding weight, and each parameter obtains an independent evaluation result; (6) In the parameter optimization decision, the parameters are adjusted according to the weight judgment of the model output. At the same time, similar data will be retrieved from the knowledge base and used as a reference for secondary adjustment.

5. The method for optimizing production line parameters of a smart factory based on a large model according to claim 4 is characterized in that: In step (1), the collected data contains positive / negative samples, that is, data on normal / abnormal operation of the factory production line.

6. The method for optimizing production line parameters of a smart factory based on a large model according to claim 4 is characterized in that: In step (5), the hybrid attention mechanism is implemented by multiplying the q and k sequences after the linear transformation of the parameters to the attention weight matrix, and then adding it to the covariance weight matrix after the linear transformation of the statistical covariance, and then normalizing it through softmax to obtain the hybrid attention weight matrix.

7. The method for optimizing production line parameters of a smart factory based on a large model according to claim 4 is characterized in that: In step (6), in the parameter optimization decision, the parameter adjustment is performed according to the weight judgment of the model output. If the output ratio is greater than 1, the parameter is adjusted down, otherwise the parameter is adjusted up.

8. The method for optimizing production line parameters of a smart factory based on a large model according to claim 4 is characterized in that: The loss is calculated independently for each parameter, and the formula is as follows: ； in, and represents weight, N represents parameter type, represents the predicted value of the i-th parameter, represents the true value of the i-th parameter. CELoss is a multi-classification loss function, corresponding to the parameter grouping in the corpus generation module.

Images