A scrap foreign matter weight deduction learning method based on multi-task gain regression

Abstract

The application discloses a scrap steel foreign matter weight deduction learning method based on multi-task gain regression, adopts an MTSN prediction model, adopts an Embedding module to realize data continuous vector representation, adopts a multi-layer perception network under a hierarchical condition to realize data learning, respectively generates cognitive data loss metrics and loss metrics of discrete scrap steel grades, finally adopts an MTL multi-task automatic learning weight mechanism, pays attention to variables with large correlation and weakens variables with small correlation, and automatically adjusts weight coefficients; the multi-level regression network in the application trains respectively according to different data types, separately calculates feature influence factors for data with uncertain influence, improves the data representation capability, and can solve the problem of poor data representation capability; the data gain network in the application initializes weights, calculates data prediction parameters, iteratively trains, and converts weight coefficients into gain coefficients.

Description

Technical Field

[0001] This invention belongs to the field of scrap steel foreign object depreciation technology, specifically involving a scrap steel foreign object depreciation learning method based on multi-task gain regression. Background Technology

[0002] Scrap steel is a crucial raw material for steel recycling. Based on thickness and integrity, scrap steel is graded into different levels. After being smelted in a furnace, it produces new usable steel. Scrap steel recycling is one of the main methods of steel production. During scrap steel recycling, different grades of steel are mixed in scrap steel trucks, containing foreign objects, including various types of metallic impurities and natural impurities. When settling accounts, manufacturers need to deduct the weight of these impurities from the scrap steel to determine the price. In reality, these foreign objects are not separated for separate weighing. Furthermore, the high coverage of the truck bed makes it impossible to see the interior of the truck. Traditional foreign object deduction requires on-site quality inspectors to judge the scrap steel grade based on experience and estimate the weight of foreign objects based on the coverage of the truck bed. This requires a high level of experience from the staff, meaning that traditional foreign object deduction is greatly affected by human factors. Moreover, the complex on-site environment, influenced by weather and lighting, makes it impossible for the human eye to accurately assess the situation of foreign objects in the scrap steel inside the truck bed.

[0003] Deep regression network prediction combines the big data analysis characteristics of neural networks and regression. Deep regression networks can make accurate predictions on data. Applying deep regression networks to scrap steel foreign object deduction can automatically calculate scrap steel foreign object deduction information after obtaining the loading information of scrap steel freight trucks. This can not only provide a basis for settlement, but also reduce the workload of accounting personnel and ensure the safety of workers.

[0004] There are currently two challenges:

[0005] (1) How to enhance the interpretability of scrap steel grade data for treasury prediction. After the scrap steel grade is coded, it is a fuzzy ordered data with unclear hierarchy between sequences. After discretization, the feature information is singular and cannot reflect the data distribution of different categories and different intervals.

[0006] (2) For the same grade of scrap steel, the foreign object deduction values ​​of different manufacturers' transport vehicles vary greatly, the deduction data has poor regularity, there is a lot of interference, and the correlation between the data and the test values ​​is poor. Summary of the Invention

[0007] The purpose of this invention is to provide a learning method for foreign object deduction in scrap steel based on multi-task gain regression, in order to solve the problems mentioned in the background art, such as scrap steel grade coding being a fuzzy ordered data, unclear hierarchical relationships between sequences, single feature information after discretization, inability to reflect the data distribution of different categories and intervals, large differences in foreign object deduction values, poor regularity of deduction data, many interferences, and poor correlation between data and test values.

[0008] To achieve the above objectives, this invention provides the following technical solution: a learning method for foreign object depreciation in scrap steel based on multi-task gain regression, employing the MTSN prediction model and using an embedding module to achieve continuous vector representation of data. A multilayer perceptron under hierarchical conditions is used to learn the data. Training is performed separately according to different data types. For data with uncertain influence, feature influence factors are calculated separately to improve the data's representation ability. Cognitive data loss metrics and loss metrics for discrete scrap steel grade data are generated separately. Finally, an MTL multi-task automatic learning weight mechanism is used to focus on highly correlated variables and weaken less correlated variables, achieving automatic adjustment of weight coefficients. Specifically, the method includes the following steps:

[0009] Step 1: Based on data analysis and understanding, perform data preprocessing, discretize the data, and reduce the impact of noise;

[0010] Step 2: Based on the multi-level MLP regression algorithm, generate loss functions for different features to achieve multi-task learning, improve practical results, and the information gain module focuses on feature influencing factors, strengthens favorable factors, weakens noise interference, and improves learning ability and subsequent prediction accuracy.

[0011] Step 3: Based on the MTL multi-task loss optimization learning algorithm, automatically adjust the weights to achieve multi-task learning, output the model, and realize weight deduction prediction;

[0012] The main steps of MTSN are as follows:

[0013] S1: Based on data analysis, data smoothing processing is performed to filter out extreme data, facilitating subsequent data analysis;

[0014] S2: Construct a multi-level gain regression network, differentiate data processing, and use a weight gain module to enhance data representation capabilities, strengthen network learning, and improve prediction accuracy;

[0015] S3: Construct a prediction network for uncertain data, with multiple task inputs and balanced outputs to improve prediction accuracy.

[0016] As a preferred technical solution of the present invention, the method further includes feeding the raw data into a data preprocessing module, the specific steps of which are as follows:

[0017] (1): Based on the correlation analysis theory, data pre-selection reduces noise and extreme value data;

[0018] (2): Process scrap steel grade data based on one-hot encoding;

[0019] (3): Based on the Embedding structure, sparse one-hot vectors are transformed into dense Embedding vectors.

[0020] As a preferred technical solution of the present invention, in step two, the specific steps for completing the multi-task MLP regression algorithm include:

[0021] Ⅰ: Utilize multilayer perceptrons to mine high-order and nonlinear features, and perform representation learning on the aforementioned target data;

[0022] II: Utilize feedforward networks to weight the data and adaptively adjust the "attention level" of the data;

[0023] III: Utilize the SoftMax mechanism to implement logistic regression, and use the SGD optimizer to adjust the gradient, enabling the model to continuously learn;

[0024] IV: Calculate the network weights and obtain the gain coefficients using the Squeezed gain mechanism;

[0025] V: Use gain coefficients to weight the data and continuously adjust the model's focus on difficult-to-separate data.

[0026] As a preferred technical solution of the present invention, the specific steps for implementing multi-task loss optimization in step three include:

[0027] ①: Using a Bayesian model, determine the cognitive uncertainty and accidental uncertainty of task types;

[0028] ②: Based on maximum likelihood estimation, maximize the log-likelihood of the model, and maximize the likelihood of the model parameters and noise;

[0029] ③: Based on the prediction task, starting with homoscedasticity, the variance of the prediction is used to represent the weights, and the model is trained until the loss reaches the expected value.

[0030] As a preferred technical solution of the present invention, S1 mainly includes:

[0031] S11: Data analysis to derive characteristic data that may affect tare weight;

[0032] S12: One-hot data encoding, which discretizes scrap steel grade data;

[0033] S13: The Embedding layer transforms each feature vector, and different features do not interfere with each other.

[0034] As a preferred technical solution of the present invention, S2 mainly includes:

[0035] S21: Based on the multi-task soft parameter sharing method, sub-tasks are pre-divided, and different task data are divided into different network layers so that each task has its own model and parameters.

[0036] S22: Network output. The bottom layer of the multilayer perceptron is the input layer, the middle layer is the hidden layer, and the last layer is the output layer.

[0037] S23: Treat scrap steel grade information as uncertain data The input is fed into a gain MLP network, and the weight distribution {1,1,1…1} is initialized. The dropout structure adds... Subsequently, the network structure changed;

[0038] S24: Utilize the weight update rules of ensemble learning to optimize the learning ability and data representation ability of MLP networks;

[0039] S25: Data training, output loss1 and loss2, iteratively update loss1 and loss2.

[0040] As a preferred technical solution of the present invention, step S3 mainly includes:

[0041] S31: Input the multi-task output parameters and structure into the MTL multi-task learning network;

[0042] S32: Bayesian modeling can identify two types of uncertainty in the data: cognitive uncertainty and accidental uncertainty, reflecting the inherent uncertainty of the regression task;

[0043] S33: In regression tasks, the log-likelihood can be expressed as:

[0044] ;

[0045] S34: Based on maximum likelihood estimation, for multi-task learning, it is assumed that the model output consists of two vectors. and The system consists of two regression tasks and follows a Gaussian distribution.

[0046] S35: To optimize the Loss function Likelihood estimation is transformed into the minimum negative logarithm.

[0047] S36: Use the method of alternating model training to train the final loss. When the loss converges to a certain value, stop training, save the model and predict the model effect. If the requirements are not met, continue training.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] 1. The multi-level regression network in this invention is trained separately according to different data types. For data with uncertain influence, the feature influence factor is calculated separately to improve the data representation ability and solve the problem of poor data representation ability.

[0050] 2. The data gain network in this invention initializes weights, calculates data prediction parameters, iteratively trains, and transforms weight coefficients into gain coefficients, so that the input data receives different model attention. Using data with weight distribution for training can improve prediction accuracy.

[0051] 3. The scrap steel foreign object depreciation prediction network based on uncertainty learning in this invention uses data with cognitive uncertainty as a separate task for training, unifies the multi-task learning output, and the multi-task joint training is more effective than single-task training. Attached Figure Description

[0052] Figure 1 This is an example diagram of the data types of the present invention;

[0053] Figure 2 This is an illustration of the multi-level gain network (MTSN) of the present invention;

[0054] Figure 3 This is a schematic diagram of the industrial processing flow for implementing the present invention.

[0055] Figure 4 This is a diagram illustrating the regression model of the present invention;

[0056] Figure 5 This is an introductory diagram of the gain network model of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figures 1 to 5 This invention provides a technical solution: a learning method for foreign object depreciation in scrap steel based on multi-task gain regression. It employs the MTSN prediction model and uses an embedding module to achieve continuous vector representation of data. A hierarchical multilayer perceptron is used to learn from the data, generating loss metrics for cognitive data and discrete scrap steel grade data respectively. Finally, an MTL multi-task automatic weight learning mechanism is used to focus on highly correlated variables and weaken less correlated variables, thereby automatically adjusting the weight coefficients. Specifically, the method includes the following steps:

[0059] Step 1: Based on data analysis and understanding, perform data preprocessing, discretize the data, and reduce the impact of noise;

[0060] Step 2: Based on the hierarchical MLP regression algorithm, generate loss functions for different features to achieve multi-task learning. The information gain module focuses on feature influencing factors, strengthens favorable factors, and weakens noise interference.

[0061] Step 3: Based on the MTL multi-task loss optimization learning algorithm, automatically adjust the weights to achieve multi-task learning, output the model, and realize weight deduction prediction;

[0062] The main steps of MTSN are as follows:

[0063] S1: Based on data analysis, data smoothing processing, and filtering of extreme value data;

[0064] S1 mainly includes:

[0065] S11: Data analysis yields characteristic data that may affect tare weight, with a total of approximately 60,000 data points;

[0066] S12: One-hot data encoding, which discretizes scrap steel grade data;

[0067] S13: The Embedding layer transforms each feature vector without interfering with each other;

[0068] Using matrix factorization to generate embeddings decomposes the relationship matrix between two entities into two embedding matrices, obtaining the embedding for each entity. This method can directly and effectively obtain the co-occurrence matrix of features. However, this method can only be used to solve the relationship between two feature entities and cannot be used to process the category attributes of features. That is, after encoding the scrap steel grade, the feature matrix cannot be used to solve for co-occurrence. To address this, a gain network and a data gain network were designed. Weights were initialized, data prediction parameters were calculated, and iterative training was performed. The weight coefficients were converted into gain coefficients, allowing the input data to receive different model attention. Training with data that has a weight distribution can improve prediction accuracy. The inverse gain prior data is shown in S2 below.

[0069] S2: Construct a multi-level gain regression network, differentiate data processing, and use a weight gain module to enhance data representation capabilities, strengthen network learning, and improve prediction accuracy;

[0070] Step S2 mainly includes:

[0071] S21: Based on the multi-task soft parameter sharing method, sub-tasks are pre-defined, and different task data are assigned to different network layers, so that each task has its own model and parameters. Continuous data, such as gross weight, is assigned to the MLP task, while discrete data, such as scrap steel grade data, is assigned to the MLP gain task. The multilayer perceptron layer consists of multiple fully connected layers, which can not only explore the nonlinear mapping relationship between feature vectors, but also map finite features to a high-dimensional space through automatic cross-validation. When the input is gross weight and tare weight, the multilayer perceptron outputs the loss after different cross-combinations. When the gross weight is 50 and the tare weight parameter value is high, the actual foreign matter deduction value will increase.

[0072] S22: Network output. In a multilayer perceptron, the bottom layer is the input layer, the middle layers are hidden layers, and the final layer is the output layer. The output of the output layer... ,y represents the value of the output layer, which is the final result. This represents the weighted sum of the inputs to the output neuron. For activation functions;

[0073] S23: Treat scrap steel grade information as uncertain data The input is fed into a gain MLP network, and the weight distribution {1,1,1…1} is initialized. The dropout structure adds... Subsequently, the network structure changed.

[0074] ;

[0075] S24: Utilize ensemble learning's weight update rules to optimize the learning and data representation capabilities of the MLP network. First, the predicted values... Normalize to the [-1, 1] interval and calculate the gain coefficient. , To predict the probability of error, ,in It is the actual value. For the expected value, The coefficients can be obtained. Update data distribution Among them, the weight update coefficient Under this gain condition, discrete scrap steel grade data has a higher mapping ability and can truly reflect the positive correlation between scrap steel grade data and the prediction of foreign object deduction in scrap steel.

[0076] S25: Data training, output loss1 and loss2, iteratively update loss1 and loss2;

[0077] S3: Construct a prediction network for uncertain data, with multiple task inputs, balanced outputs, and improved prediction accuracy;

[0078] Step S3 mainly includes:

[0079] S31: Input the multi-task output parameters and structure into the MTL multi-task learning network;

[0080] S32: Bayesian modeling can identify two types of uncertainty in the data: cognitive uncertainty and accidental uncertainty, reflecting the inherent uncertainty of the regression task;

[0081] S33: In regression tasks, the log-likelihood can be expressed as:

[0082]

[0083] S34: Based on maximum likelihood estimation, for multi-task learning, it is assumed that the model output consists of two vectors. and Composed of two regression tasks, following a Gaussian distribution:

[0084]

[0085] S35: To optimize the Loss function Likelihood estimation is transformed into the minimum negative logarithm for regression tasks.

[0086]

[0087] here, and These are the loss functions for the two tasks, respectively. As can be seen from the formula above, the parameters... Equivalent to Adaptive weighting based on data Increase weight reduction Decrease The weights increase, and the negative logarithm is taken. As a regularization term, to prevent Excessive noise can cause training imbalance and suppress excessive noise increase.

[0088] In this way, this function can freely combine discrete and continuous loss functions. The loss is smooth, and the weights of the task will not converge to 0, unlike the method of directly adding the loss, which also does not converge to 0 quickly. This is an effective method for loss weighting.

[0089] S36: Use the method of alternating model training to train the final loss. When the loss converges to a certain value, stop training, save the model and predict the model effect. If the requirements are not met, continue training.

[0090] As a preferred technical solution of the present invention, the method further includes feeding the raw data into a data preprocessing module, the specific steps of which are as follows:

[0091] (1): Based on the correlation analysis theory, data pre-selection reduces noise and extreme value data;

[0092] (2): Process scrap steel grade data based on one-hot encoding;

[0093] (3): Based on the Embedding structure, sparse one-hot vectors are transformed into dense Embedding vectors.

[0094] As a preferred technical solution of the present invention, the specific steps for completing the multi-task MLP regression algorithm in step two include:

[0095] Ⅰ: Utilize multilayer perceptrons to mine high-order and nonlinear features, and perform representation learning on the aforementioned target data;

[0096] II: Utilize feedforward networks to weight the data and adaptively adjust the "attention level" of the data;

[0097] III: Utilize the SoftMax mechanism to implement logistic regression, and use the SGD optimizer to adjust the gradient, enabling the model to continuously learn;

[0098] IV: Calculate the network weights and obtain the gain coefficients using the Squeezed gain mechanism;

[0099] V: Use gain coefficients to weight the data and continuously adjust the model's focus on difficult-to-separate data.

[0100] As a preferred technical solution of the present invention, the specific steps for implementing multi-task loss optimization in step three include:

[0101] ①: Using a Bayesian model, determine the cognitive uncertainty and accidental uncertainty of task types;

[0102] ②: Based on maximum likelihood estimation, maximize the log-likelihood of the model, and maximize the likelihood of the model parameters and noise;

[0103] ③: Based on the prediction task, starting with homoscedasticity, the variance of the predictions is used to represent the weights to train the model until the loss reaches the expected value.

[0104] This invention proposes an MTSN (Multi-Lask Squeezed Network) prediction model for automatic scrap foreign object deduction based on a regression network. First, it proposes a novel multi-level task embedding and a multi-layer perceptron (MLP) regression network to locate information affecting deduction from collected scrap foreign object quality inspection information. Starting from the data itself, it performs correlation analysis, classifies tasks, and applies targeted processing to different data. Second, it proposes an information-weighted Squeezed module, using the branch loss function as prior knowledge for the gain network. Utilizing the ensemble weighting concept, it continuously enhances the model's focus on the data, deeply analyzes data correlation, and obtains interval information, paying attention to contextual information. Finally, this invention proposes an MTL (Multi-Task Learning) module to learn the weights of multiple tasks, weakening long-distance information, strengthening key information, and improving the model's prediction accuracy.

[0105] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A learning method for foreign object depreciation in scrap steel based on multi-task gain regression, characterized in that: The MTSN prediction model is adopted, and the Embedding module is used to realize the continuous vector representation of data. A multi-level gain regression network is used to learn from the data and generate loss measures for cognitive data and discrete scrap steel grade data, respectively. Finally, the MTL multi-task automatic learning weight mechanism is used to focus on variables with high correlation and weaken variables with low correlation, so as to realize automatic adjustment of weight coefficients. The specific steps include the following: Step 1: Based on data analysis and understanding, perform data preprocessing, discretize the data, and reduce the impact of noise; Step 2: Based on the hierarchical MLP regression algorithm, generate loss functions for different features to achieve multi-task learning. The weight gain module focuses on feature influencing factors, strengthens favorable factors, and weakens noise interference. Step 3: Based on the MTL multi-task loss optimization learning algorithm, automatically adjust the weights to achieve multi-task learning, output the model, and realize weight deduction prediction; The main steps of MTSN are as follows: S1: Based on data analysis, data smoothing is performed, and extreme value data is removed; S2: Construct a multi-level gain regression network, differentiate data processing, and enhance data representation capabilities with a weight gain module; Step S2 mainly includes: S21: Based on the multi-task soft parameter sharing method, sub-tasks are pre-divided, and different task data are assigned to different network layers, so that each task has its own model and parameters. Among them, continuous data is assigned to MLP tasks, while discrete data is assigned to MLP gain tasks. The multilayer perceptron layer is composed of multiple fully connected layers, which can not only explore the nonlinear mapping relationship between feature vectors, but also map finite features to a high-dimensional space through automatic cross-validation. S22: Network output. In a multilayer perceptron, the bottom layer is the input layer, the middle layers are hidden layers, and the final layer is the output layer; the output of the output layer... ,y represents the value of the output layer, which is the final result. This represents the weighted sum of the inputs to the output neuron. For activation functions; S23: Treat scrap steel grade information as uncertain data The input is fed into a gain MLP network, and the weight distribution {1,1,1…1} is initialized. The dropout structure adds... Subsequently, the network structure changed. ； S24: Utilize ensemble learning's weight update rules to optimize the learning and data representation capabilities of the MLP network. First, the predicted values... Normalize to the [-1, 1] interval and calculate the gain coefficient. , To predict the probability of error, ,in It is the actual value. For the expected value, The coefficients can be obtained. Update data distribution Among them, the weight update coefficient Under this gain condition, discrete scrap steel grade data has a higher mapping ability and can truly reflect the positive correlation between scrap steel grade data and the prediction of foreign object deduction in scrap steel. S25: Data training, output loss1 and loss2, iteratively update loss1 and loss2; S3: Construct an MTL multi-task learning network with multi-task inputs and balanced outputs to improve prediction accuracy.

2. The scrap steel foreign object depreciation learning method based on multi-task gain regression according to claim 1, characterized in that: In step two, the specific steps for completing the multi-task MLP regression algorithm include: Ⅰ: Utilize multilayer perceptrons to mine high-order and nonlinear features, and perform representation learning on target data; II: Utilize feedforward networks to weight the data and adaptively adjust the "attention level" of the data; III: Utilize the SoftMax mechanism to implement logistic regression, and use the SGD optimizer to adjust the gradient, enabling the model to continuously learn; IV: Calculate the network weights and obtain the gain coefficients using the Squeezed weight gain module; V: Use gain coefficients to weight the data and continuously adjust the model's focus on difficult-to-separate data.

3. The scrap steel foreign object depreciation learning method based on multi-task gain regression according to claim 1, characterized in that: In step three, the specific steps for implementing multi-task loss optimization include: ①: Using a Bayesian model, determine the cognitive uncertainty and accidental uncertainty of task types; ②: Based on maximum likelihood estimation, maximize the log-likelihood of the model, and maximize the likelihood of the model parameters and noise; ③: Based on the prediction task, starting with homoscedasticity, the variance of the prediction is used to represent the weights, and the model is trained until the loss reaches the expected value.

4. The scrap steel foreign object depreciation learning method based on multi-task gain regression according to claim 1, characterized in that: Step S3 mainly includes: S31: Input the multi-task output parameters and structure into the MTL multi-task learning network; S32: Bayesian modeling can identify two types of uncertainty in the data: cognitive uncertainty and accidental uncertainty, reflecting the inherent uncertainty of the regression task; S33: In regression tasks, the log-likelihood can be expressed as: ； S34: Based on maximum likelihood estimation, for multi-task learning, it is assumed that the model output consists of two vectors. and Composed of elements that follow a Gaussian distribution; S35: To optimize the Loss function Likelihood estimation is transformed into the minimum negative logarithm. S36: Use the method of alternating model training to train the final loss. When the loss converges to a certain value, stop training, save the model, and predict the model performance.

Images