Mine nonferrous metal feature extraction method based on multi-source heterogeneous data fusion

Abstract

The invention discloses a mine nonferrous metal feature extraction method based on multi-source heterogeneous data fusion. The method comprises the following steps: carrying out heterogeneous data processing by using DNN and CNN networks; constructing a feature fusion network through a stacked auto-encoder; training the feature fusion network to complete initialization of the feature fusion network; training and optimizing the initialized feature fusion network by constructing a loss function to obtain an optimized network model; and non-ferrous metal identification is carried out by using the optimized network model. According to the method, the complementary advantages of heterogeneous data are fully exerted, data confusion is reduced, common features of data of different dimensions are fully extracted, and data redundancy is avoided; important information sources which are more beneficial to non-ferrous metal detection in a complex beneficiation and metallurgy environment can be distinguished, interference of the complex environment is reduced, and the non-ferrous metal detection robustness and precision of a follow-up algorithm are effectively guaranteed.

Description

Technical Field

[0001] The invention relates to a mine nonferrous metal feature extraction method based on multi-source heterogeneous data fusion, belonging to the technical field of nonferrous metal detection. Background Art

[0002] In the process of coal mining and production, how to detect and identify non-ferrous metals is crucial to the entire beneficiation process. Thanks to the rapid development of computer and network communication technology, the application of various sensors and information processing technology, the efficiency of intelligent identification of non-ferrous metals in the beneficiation process has also been greatly improved. However, in the process of coal mining and beneficiation, various equipment not only obtains a large amount of heterogeneous data, but also increases the redundancy of storage. At the same time, the processing, fusion and decision-making between different data also limit the further improvement of the intelligent detection effect of non-ferrous metals. In recent years, the use of deep learning technology modeling, extraction and fusion of similar features under various heterogeneous data, strengthening core features, and building a unified framework can give full play to the complementary advantages of multi-source heterogeneous data, enhance the scene understanding ability and robustness of the algorithm, and greatly reduce the redundancy of the algorithm, and improve the overall accuracy of the subsequent detection algorithm.

[0003] The heterogeneous data obtained during the mineral processing process mainly include one-dimensional voice, text and vibration data, two-dimensional image data, and three-dimensional video stream data. The features contained in the heterogeneous data obtained by these different sensors are highly similar, which is prone to feature confusion. In this case, when deep learning technology is directly used for data fusion, it is difficult for the model to accurately extract and fuse the common features between heterogeneous data, which will cause data redundancy and waste of computing resources. In addition, due to the noise interference of complex mineral processing scenes, it is more difficult for the model to identify the important information sources that are more conducive to mine non-ferrous metal detection in the current scene, resulting in a significant decrease in algorithm accuracy.

[0004] In summary, the current detection and identification of non-ferrous metals in mines have the following main defects: (1) The existing deep learning model is difficult to accurately extract the common features between heterogeneous data, resulting in data redundancy; (2) The existing model is easily disturbed by complex mineral processing scenarios, and it is difficult to identify important information sources that are more conducive to non-ferrous metal detection in complex mineral processing environments, resulting in a decrease in algorithm accuracy. Summary of the invention

[0005] The purpose of the present invention is to provide a method for extracting mine nonferrous metal features based on multi-source heterogeneous data fusion, which can reduce data confusion and fully extract common features of data of different dimensions to avoid data redundancy; it can identify important information sources that are more conducive to nonferrous metal detection in complex smelting environments, and improve the robustness and accuracy of subsequent nonferrous metal detection algorithms.

[0006] In order to achieve the above object, the present invention provides a method for extracting features of nonferrous metals in mines based on multi-source heterogeneous data fusion, comprising the following steps:

[0007] S1. Use DNN and CNN networks to process heterogeneous data;

[0008] S2, construct a feature fusion network by stacking autoencoders;

[0009] S3, training the feature fusion network to complete the initialization of the feature fusion network;

[0010] S4, training and optimizing the initialized feature fusion network by constructing a loss function to obtain an optimized network model;

[0011] S5. Use the optimized network model to identify nonferrous metals.

[0012] Furthermore, the specific process of S1 is as follows:

[0013] The DNN network and CNN network are used to extract features from the collected speech text data and image data respectively, and the number of neurons in the output layer of the DNN network and the CNN network are set to be consistent, and the network output feature structure is kept the same for feature fusion and feature similarity extraction of heterogeneous data. The formula is as follows:

[0014] F dnn =N dnn (X 1D |θ dnn )=W dnn X 1D ++b dnn

[0015] F cnn =N cnn (X 2D |θ cnn )=W cnn X 2D ++b cnn ;

[0016] Where, X 1D and X 2D They are one-dimensional speech and text data and two-dimensional image data; N dnn (·) and N cnn (·) represent the DNN and CNN network models used; θ dnn is the mapping relationship learned by the DNN network, represented by the weight W dnn and bias b dnn Composition, θ cnn The mapping relationship learned by the CNN network is represented by the weight W cnn and bias bcnn Composition; F dnn and F cnn They are the output features of the DNN and CNN network models respectively.

[0017] Furthermore, the specific process of S2 is:

[0018] S2.1. Build the autoencoder layer by layer, use the expression obtained by each autoencoder as a hidden layer, then add the data label, and use the back propagation algorithm to fine-tune the parameters in the network again to obtain a deep neural network, that is, a feature fusion network, specifically:

[0019] Let the input data be i, the hidden representation be h, and the autoencoder function be f enc ; The process of the autoencoder converting the input data i into the hidden representation h is expressed as:

[0020] h=f enc (i);

[0021] Combining multiple autoencoders forms a stacked autoencoder structure, which is expressed by the following formula:

[0022] Y=σ(H2·W3)

[0023] H2=σ(H1·W2)

[0024] H1=σ(X·W1);

[0025] Where X and Y are the input layer and output layer respectively; σ represents the activation function; H1 and H2 are the hidden layers of the stacked autoencoder, corresponding to h = f enc (i); W k ,k=1,2,3 is the mapping relationship from the previous layer to the next layer;

[0026] S2.2. The output features of the DNN and CNN network models are respectively passed through the feature fusion network to obtain the following output:

[0027]

[0028] Where N f (·) is the constructed feature fusion network; θ f Represents the randomly initialized network mapping relationship, represented by the weight W f and bias b f constitute; and F cnn and F dnn Output after a feature fusion network;

[0029] S2.3, calculate F separately cnn and Fdnn The reconstruction error is as follows:

[0030]

[0031] In the formula, n is the number of samples, and Indicates F cnn and F dnn Reconstruction loss after a feature fusion.

[0032] Furthermore, the specific process of S3 is as follows:

[0033] S3.1. and Compare and calculate, if Then it is judged as the mapping relationship θ learned by the CNN network cnn Compared with the mapping relationship θ learned by the DNN network dnn More importantly in the current scenario, the mapping relationship θ learned by the CNN network cnn As the initial weight of the next feature fusion network, avoid the problem of increased training time and computational complexity caused by re-randomly initializing the weights, and output the DNN network As the input of the feature network, it is optimized again to obtain the optimized feature

[0034] S3.2. Features Perform another reconstruction loss calculation and get Will and Compare and calculate, if It is judged that the features output by the CNN network are better than the mapping relationship of the previous feature fusion network, and the mapping relationship of the CNN network is θ cnn As the initialization weight of the next feature fusion network, and the output of the previous feature fusion network As the input of this fusion network, it is optimized and the output is obtained

[0035] S3.3, the feature fusion network is reconstructed twice, and the results of the two reconstruction losses are compared, including

[0036] and and and and The four situations, referred to as NN, NY, YN and YY, are expressed as follows using mathematical relationships:

[0037] (1) In the case of NN:

[0038]

[0039] θ dnn =(W dnn |b dnn )

[0040]

[0041] In the formula, Net i (·) is the feature fusion network i; θ dnn is the mapping relationship of the DNN network, represented by the weight W dnn and bias b dnn composition; As the input of feature fusion network 1, At the same time, it serves as the output of feature fusion network 1 and the input of feature fusion network 3. is the output of feature fusion network 3;

[0042] (2) In the case of NY:

[0043]

[0044] θ dnn =(W dnn |b dnn )

[0045]

[0046] θ1=(W1|b1);

[0047] In the formula, θ1 is the mapping relationship of feature fusion network 1, and the feature fusion network 4 is initialized, which consists of weight W1 and bias b1; and They are used as the input and output of feature fusion network 4 respectively;

[0048] (3) In the case of YN:

[0049]

[0050] θ cnn =(W cnn |b cnn )

[0051]

[0052] θ2=(W2|b2);

[0053] In the formula, and As the input and output of feature fusion network 2 respectively; θ cnn is the mapping relationship of the CNN network, and is the initialization of the feature fusion network 2, with weight W cnn and bias b cnn composition; and They are respectively used as the input and output of the feature fusion network 5; θ2 is the mapping relationship of the feature fusion network 2, and the feature fusion network 5 is initialized, which is composed of weight W2 and bias b2;

[0054] (4) In the case of YY:

[0055]

[0056] θ cnn =(W cnn |b cnn )

[0057]

[0058] θ cnn =(W cnn |b cnn );

[0059] In the formula, and are the input and output of feature fusion network 6 respectively; θ cnn is the mapping relationship of the CNN network, represented by the weight W cnn and bias b cnn Composition, complete the initialization of feature fusion network 6.

[0060] Furthermore, in order to effectively extract relevant features between heterogeneous data and improve the recognition accuracy of subsequent algorithms in S4, the following constraints are imposed on the network model in the cases of NN, NY, YN and YY:

[0061] (1) The network loss function in the NN case is expressed as:

[0062]

[0063] L NN =L dnn ++L cnn ;

[0064] Where, L dnn is the reconstruction loss of speech-text data; L cnn is the reconstruction loss of image data; L NN is the reconstruction loss of the network in the NN case;

[0065] (2) The network loss function in the NY case is expressed as:

[0066]

[0067]

[0068] L NY =L dnn ++L cnn ;

[0069] Where, L NY is the reconstruction loss of the network in the NY case;

[0070] (3) The network loss function in the YN case is expressed as:

[0071]

[0072] L YN =L dnn ++L cnn ;

[0073] Where, L YN is the network reconstruction loss in the YN case. The difference from the NY case is that the initialization weights of the feature fusion network are different;

[0074] (4) The network loss function in the YY case is expressed as:

[0075]

[0076] L YY =L dnn ++L cnn ;

[0077] Where, L YY is the network reconstruction loss in case YY.

[0078] Furthermore, the process of using the optimized network model to identify nonferrous metals in S5 is:

[0079] S5.1. Use the fused features as the input of the Softmax classifier and train it:

[0080] result = Softmax(F f |θ s );

[0081] In the formula, θ s is the mapping relationship learned by the classifier; F f is the input of the classifier; result is the output of the classifier;

[0082] S5.2. In order to prevent the network from overfitting during training, L2 regularization is added, and its mathematical expression is as follows:

[0083]

[0084] In the formula, x and are the input and output of the classifier respectively; y is the original true label; α is the L2 regularization parameter; w is the column vector of the weight matrix; n is the number of samples.

[0085] The present invention is guided by prior knowledge, and by constructing a dual-branch multi-source heterogeneous data fusion network, different network branches are set for multi-source heterogeneous data of different dimensions, which effectively reduces data confusion while improving the model's feature extraction capability for heterogeneous data of different dimensions, and ensures the efficiency of subsequent feature fusion operations to obtain common features; the iterative feature fusion method based on loss function comparison judgment, on the basis of the dual-branch multi-source heterogeneous data fusion network, effectively screens and refines the key data sources that are more conducive to non-ferrous metal detection in complex beneficiation scenarios by setting weight factors and iterative fusion operations, and effectively integrates the features of other data sources on the basis of ensuring the effective extraction of the core features of the data source. The present invention gives full play to the complementary advantages between heterogeneous data, reduces the interference of complex environments, and effectively ensures the robustness and accuracy of subsequent algorithms for non-ferrous metal detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a flowchart of the present invention;

[0087] Figure 2 It is a schematic diagram of the structure of the autoencoder in the present invention;

[0088] Figure 3 It is a schematic diagram of the stacked autoencoder structure in the present invention. DETAILED DESCRIPTION

[0089] The present invention will be further described below in conjunction with the accompanying drawings.

[0090] like Figure 1 As shown, a mine nonferrous metal feature extraction method based on multi-source heterogeneous data fusion includes the following steps:

[0091] S1. Use DNN and CNN networks to process heterogeneous data;

[0092] S2, construct a feature fusion network by stacking autoencoders;

[0093] S3, training the feature fusion network to complete the initialization of the feature fusion network;

[0094] S4, training and optimizing the initialized feature fusion network by constructing a loss function to obtain an optimized network model;

[0095] S5. Use the optimized network model to identify nonferrous metals.

[0096] As a preferred implementation, the specific process of S1 is:

[0097] The DNN network and CNN network are used to extract features from the collected speech text data and image data respectively, and the number of neurons in the output layer of the DNN network and the CNN network are set to be consistent, and the network output feature structure is kept the same for feature fusion and feature similarity extraction of heterogeneous data. The formula is as follows:

[0098] F dnn =N dnn (X 1D |θ dnn )=W dnn X 1D +b dnn

[0099] F cnn =N cnn (X 2D |θ cnn )=W cnn X 2D +b cnn ;

[0100] Where, X 1D and X 2D They are one-dimensional speech and text data and two-dimensional image data; N dnn (·) and N cnn (·) represent the DNN and CNN network models used; θ dnn is the mapping relationship learned by the DNN network, represented by the weight W dnn and bias b dnn Composition, θ cnn The mapping relationship learned by the CNN network is represented by the weight W cnn and bias b cnn Composition; F dnn and F cnn They are the output features of the DNN and CNN network models respectively.

[0101] As a preferred implementation, the specific process of S2 is:

[0102] S2.1, such as Figure 2As shown in the figure, the autoencoder is constructed layer by layer, and the expression obtained by each autoencoder is used as a hidden layer. Then the label of the data is added, and the back propagation algorithm is used again to fine-tune the parameters in the network to obtain a deep neural network, that is, a feature fusion network, which is specifically:

[0103] Let the input data be i, the hidden representation be h, and the autoencoder function be f enc ; The process of the autoencoder converting the input data i into the hidden representation h is expressed as:

[0104] h=f enc (i);

[0105] like Figure 3 As shown, multiple autoencoders are combined to form a stacked autoencoder structure, which is expressed by the following formula:

[0106] Y=σ(H2·W3)

[0107] H2=σ(H1·W2)

[0108] H1=σ(X·W1);

[0109] Where X and Y are the input layer and output layer respectively; σ represents the activation function; H1 and H2 are the hidden layers of the stacked autoencoder, corresponding to h = f enc (i); W k ,k=1,2,3 is the mapping relationship from the previous layer to the next layer;

[0110] S2.2. The output features of the DNN and CNN network models are respectively passed through the feature fusion network to obtain the following output:

[0111]

[0112] θ f =(W f |b f )

[0113] Where N f (·) is the constructed feature fusion network; θ f Represents the randomly initialized network mapping relationship, represented by the weight W f and bias b f constitute; and F cnn and F dnn Output after a feature fusion network;

[0114] S2.3, calculate F separately cnn and F dnn The reconstruction error is as follows:

[0115]

[0116] In the formula, n is the number of samples, and Indicates F cnn and F dnn Reconstruction loss after a feature fusion.

[0117] As a preferred implementation, the specific process of S3 is:

[0118] S3.1. and Compare and calculate, if Then it is judged as the mapping relationship θ learned by the CNN network cnn Compared with the mapping relationship θ learned by the DNN network dnn More importantly in the current scenario, the mapping relationship θ learned by the CNN network cnn As the initial weight of the next feature fusion network, avoid the problem of increased training time and computational complexity caused by re-randomly initializing the weights, and output the DNN network As the input of the feature network, it is optimized again to obtain the optimized feature

[0119] S3.2. Features Perform another reconstruction loss calculation and get Will and Compare and calculate, if It is judged that the features output by the CNN network are better than the mapping relationship of the previous feature fusion network, and the mapping relationship of the CNN network is θ cnn As the initialization weight of the next feature fusion network, and the output of the previous feature fusion network As the input of this fusion network, it is optimized and the output is obtained

[0120] S3.3, the feature fusion network is reconstructed twice, and the results of the two reconstruction losses are compared, including

[0121] and and and and The four situations, referred to as NN, NY, YN and YY, are expressed as follows using mathematical relationships:

[0122] (1) In the case of NN:

[0123]

[0124] θ dnn =(W dnn |b dnn )

[0125]

[0126] In the formula, Net i (·) is the feature fusion network i; θ dnn is the mapping relationship of the DNN network, represented by the weight W dnn and bias b dnn composition; As the input of feature fusion network 1, At the same time, it serves as the output of feature fusion network 1 and the input of feature fusion network 3. is the output of feature fusion network 3;

[0127] (2) In the case of NY:

[0128]

[0129] θ dnn =(W dnn |b dnn )

[0130]

[0131] θ1=(W1|b1);

[0132] In the formula, θ1 is the mapping relationship of feature fusion network 1, and the feature fusion network 4 is initialized, which consists of weight W1 and bias b1; and They are used as the input and output of feature fusion network 4 respectively;

[0133] (3) In the case of YN:

[0134]

[0135] θ cnn =(W cnn |b cnn )

[0136]

[0137] θ2=(W2|b2);

[0138] In the formula, and As the input and output of feature fusion network 2 respectively; θ cnn is the mapping relationship of the CNN network, and is the initialization of the feature fusion network 2, with weight Wcnn and bias b cnn composition; and They are respectively used as the input and output of the feature fusion network 5; θ2 is the mapping relationship of the feature fusion network 2, and the feature fusion network 5 is initialized, which is composed of weight W2 and bias b2;

[0139] (4) In the case of YY:

[0140]

[0141] θ cnn =(W cnn |b cnn )

[0142]

[0143] θ cnn =(W cnn |b cnn );

[0144] In the formula, and are the input and output of feature fusion network 6 respectively; θ cnn is the mapping relationship of the CNN network, represented by the weight W cnn and bias b cnn Composition, complete the initialization of feature fusion network 6.

[0145] As a preferred implementation, in order to effectively extract relevant features between heterogeneous data and improve the recognition accuracy of subsequent algorithms, the following constraints are imposed on the network model in the cases of NN, NY, YN and YY:

[0146] (1) The network loss function in the NN case is expressed as:

[0147]

[0148] L NN =L dnn ++L cnn ;

[0149] Where, L dnn is the reconstruction loss of speech-text data; L cnn is the reconstruction loss of image data; L NN is the reconstruction loss of the network in the NN case;

[0150] (2) The network loss function in the NY case is expressed as:

[0151]

[0152] L NY =L dnn ++L cnn ;

[0153] Where, L NY is the reconstruction loss of the network in the NY case;

[0154] (3) The network loss function in the YN case is expressed as:

[0155]

[0156] L YN =L dnn ++L cnn ;

[0157] Where, L YN is the network reconstruction loss in the YN case. The difference from the NY case is that the initialization weights of the feature fusion network are different;

[0158] (4) The network loss function in the YY case is expressed as:

[0159]

[0160] L YY =L dnn ++L cnn ;

[0161] Where, L YY is the network reconstruction loss in case YY.

[0162] As a preferred implementation, the process of using the optimized network model to identify nonferrous metals in S5 is:

[0163] S5.1. Use the fused features as the input of the Softmax classifier and train it:

[0164] result = Softmax(F f |θ s );

[0165] In the formula, θ s is the mapping relationship learned by the classifier; F f is the input of the classifier; result is the output of the classifier;

[0166] S5.2. In order to prevent the network from overfitting during training, L2 regularization is added, and its mathematical expression is as follows:

[0167]

[0168] In the formula, x and are the input and output of the classifier respectively; y is the original true label; α is the L2 regularization parameter; w is the column vector of the weight matrix; n is the number of samples.

Claims

1. A mine nonferrous metal feature extraction method based on multi-source heterogeneous data fusion, characterized in that: The steps include: S1. Use DNN and CNN networks to process heterogeneous data; S2, construct a feature fusion network by stacking autoencoders; S3, training the feature fusion network to complete the initialization of the feature fusion network; S4, training and optimizing the initialized feature fusion network by constructing a loss function to obtain an optimized network model; S5. Use the optimized network model to identify nonferrous metals.

2. The method for extracting features of nonferrous metals in mines based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The specific process of S1 is as follows: The DNN network and CNN network are used to extract features from the collected speech text data and image data respectively, and the number of neurons in the output layer of the DNN network and the CNN network are set to be the same. The formula is as follows: F dnn =N dnn (X 1D |θ dnn )=W dnn X 1D ++b dnn F cnn =N cnn (X 2D |θ cnn )=W cnn X 2D ++b cnn ; Where, X 1D and X 2D They are one-dimensional speech and text data and two-dimensional image data; N dnn (·) and N cnn (·) represent the DNN and CNN network models used; θ dnn is the mapping relationship learned by the DNN network, represented by the weight W dnn and bias b dnn Composition, θ cnn is the mapping relationship learned by the CNN network, represented by the weight W cnn and bias b cnn Composition; F dnn and F cnn They are the output features of the DNN and CNN network models respectively.

3. The method for extracting features of nonferrous metals in mines based on multi-source heterogeneous data fusion according to claim 2 is characterized in that: The specific process of S2 is: S2.

1. Build the autoencoder layer by layer, use the expression obtained by each autoencoder as a hidden layer, then add the data label, and use the back propagation algorithm to fine-tune the parameters in the network again to obtain a deep neural network, that is, a feature fusion network, specifically: Let the input data be i, the hidden representation be h, and the autoencoder function be f enc ; The process of the autoencoder converting the input data i into the hidden representation h is expressed as: h=f enc (i); Combining multiple autoencoders forms a stacked autoencoder structure, which is expressed by the following formula: Y=σ(H2·W3) H2=σ(H1·W2) H1=σ(X·W1); Where X and Y are the input layer and output layer respectively; σ represents the activation function; H1 and H2 are the hidden layers of the stacked autoencoder, corresponding to h = f enc (i); W k ,k=1,2,3 is the mapping relationship from the previous layer to the next layer; S2.

2. The output features of the DNN and CNN network models are respectively passed through the feature fusion network to obtain the following output: i f =(W f |b f ) Where N f (·) is the constructed feature fusion network; θ f Represents the randomly initialized network mapping relationship, represented by the weight W f and bias b f constitute; and F cnn and F dnn Output after a feature fusion network; S2.3, calculate F separately cnn and F dnn The reconstruction error is as follows: In the formula, n is the number of samples, and Indicates F cnn and F dnn Reconstruction loss after a feature fusion.

4. The method for extracting features of nonferrous metals in mines based on multi-source heterogeneous data fusion according to claim 3 is characterized in that: The specific process of S3 is as follows: S3.

1. and Compare and calculate, if Then it is judged as the mapping relationship θ learned by the CNN network cnn Compared with the mapping relationship θ learned by the DNN network dnn More importantly in the current scenario, the mapping relationship θ learned by the CNN network cnn As the initial weight of the next feature fusion network, and output the DNN network As the input of the feature network, it is optimized again to obtain the optimized feature S3.

2. Features Perform another reconstruction loss calculation and get Will and Compare and calculate, if It is judged that the features output by the CNN network are better than the mapping relationship of the previous feature fusion network, and the mapping relationship of the CNN network is θ cnn As the initialization weight of the next feature fusion network, and the output of the previous feature fusion network As the input of this fusion network, it is optimized and the output is obtained S3.3, the feature fusion network is reconstructed twice, and the results of the two reconstruction losses are compared, including and and and and The four situations, referred to as NN, NY, YN and YY, are expressed as follows using mathematical relationships: (1) In the case of NN: i dnn =(W dnn |b dnn ) In the formula, Net i (·) is the feature fusion network i; θ dnn is the mapping relationship of the DNN network, represented by the weight W dnn and bias b dnn composition; As the input of feature fusion network 1, At the same time, it serves as the output of feature fusion network 1 and the input of feature fusion network 3. is the output of feature fusion network 3; (2) In the case of NY: i dnn =(W dnn |b dnn ) θ1=(W1|b1); In the formula, θ1 is the mapping relationship of feature fusion network 1, and the feature fusion network 4 is initialized, which consists of weight W1 and bias b1; and They are used as the input and output of feature fusion network 4 respectively; (3) In the case of YN: i cnn =(W cnn |b cnn ) θ2=(W2|b2); In the formula, and As the input and output of feature fusion network 2 respectively; θ cnn is the mapping relationship of the CNN network, and is the initialization of the feature fusion network 2, with weight W cnn and bias b cnn composition; and They are respectively used as the input and output of the feature fusion network 5; θ2 is the mapping relationship of the feature fusion network 2, and the feature fusion network 5 is initialized, which is composed of weight W2 and bias b2; (4) In the case of YY: i cnn =(W cnn |b cnn ) i cnn =(W cnn |b cnn ); In the formula, and are the input and output of feature fusion network 6 respectively; θ cnn is the mapping relationship of the CNN network, represented by the weight W cnn and bias b cnn Composition, complete the initialization of feature fusion network 6.

5. The method for extracting features of nonferrous metals in mines based on multi-source heterogeneous data fusion according to claim 4 is characterized in that: In S4, the following constraints are imposed on the network model for NN, NY, YN and YY respectively: (1) The network loss function in the NN case is expressed as: L NN =L dnn ++L cnn ; Where, L dnn is the reconstruction loss of speech-text data; L cnn is the reconstruction loss of image data; L NN is the reconstruction loss of the network in the NN case; (2) The network loss function in the NY case is expressed as: L NY =L dnn ++L cnn ; Where, L NY is the reconstruction loss of the network in the NY case; (3) The network loss function in the YN case is expressed as: L YN =L dnn ++L cnn ; Where, L YN is the network reconstruction loss in the YN case. The difference from the NY case is that the initialization weights of the feature fusion network are different; (4) The network loss function in the YY case is expressed as: L YY =L dnn ++L cnn ; Where, L YY is the network reconstruction loss in case YY.

6. The method for extracting features of nonferrous metals in mines based on multi-source heterogeneous data fusion according to claim 5 is characterized in that: The process of using the optimized network model to identify nonferrous metals in S5 is as follows: S5.

1. Use the fused features as the input of the Softmax classifier and train it: result=Softmax(F f |θ s ); In the formula, θ s is the mapping relationship learned by the classifier; F f is the input of the classifier; result is the output of the classifier; S5.2, add L2 regularization, its mathematical expression is as follows: In the formula, x and are the input and output of the classifier respectively; y is the original true label; α is the L2 regularization parameter; w is the column vector of the weight matrix; n is the number of samples.

Images