A method for identifying models of discarded mobile phones based on convolutional neural networks

Abstract

In view of the problem that the model is difficult to accurately identify during the recycling of used mobile phones, the present invention proposes a method for identifying the model of used mobile phones based on a convolutional neural network. The present invention uses an edge detection algorithm to analyze the regional features in the mobile phone inspection photo, constructs a feature extraction convolutional network with shared weights, evaluates the similarity between the regional features of the used mobile phone image and the standard sample, and realizes the rapid identification of the mobile phone model. The present invention maintains good rapidity and accuracy in mobile phone model identification in different scenarios, which can improve the efficiency of used mobile phone recycling and the economic benefits of recycling enterprises.

Description

Technical Field

[0001] The present invention utilizes a method for identifying the model of a used mobile phone based on a low-rank convolutional neural network to achieve accurate identification of the model of the mobile phone in the process of recycling used mobile phones. In the process of recycling used mobile phones, classifying and processing mobile phones by model can obtain greater economic benefits. The identification of mobile phone models has become an important factor affecting the efficiency of recycling used mobile phones. There are many brands and models of mobile phones with high similarity, so it is necessary to have a certain amount of experience accumulation to skillfully distinguish the mobile phone models. Applying the method for identifying the model of a used mobile phone based on a convolutional neural network to the process of recycling used mobile phones can avoid problems such as classification errors and low classification efficiency caused by insufficient experience of personnel, improve the accuracy and speed of recycling used mobile phones, and is an important branch of the field of image recognition, and also belongs to the field of solid waste treatment. Background Art

[0002] The rapid and accurate identification of the models of used mobile phones can improve the efficiency of mobile phone recycling, save manpower, and improve the economic benefits of used mobile phone recycling enterprises. It is an important measure to improve the recycling of urban solid waste resources; it not only has good economic benefits, but also has significant environmental and social benefits. Therefore, the research results of the present invention have broad application prospects.

[0003] Identifying the models of used mobile phones is a process of image recognition and classification. Since the inspection photos taken by inspectors when recycling mobile phones have different shooting conditions such as angles, equipment, light sources, and resolutions, the accuracy of identifying the models of used mobile phones is seriously affected.

[0004] Since some mobile phone models are too similar, and in actual mobile phone recycling scenarios, the categories of mobile phone models are dynamically updated as new models are launched, and since there are fewer training samples for new models, it is difficult for the model to learn and extract effective feature information in a timely manner, which increases the difficulty of building a model. Measuring the differences between mobile phone models based on the similarity of mobile phones can reduce the amount of calculation required for model learning, increase the calculation speed, and meet the needs of recycling used mobile phones. This will improve recycling efficiency, speed up the circulation process of used mobile phones, and also reduce labor costs and improve the benefits of recycling companies.

[0005] The present invention designs a mobile phone model recognition method based on a low-rank bilinear convolutional neural network. The method mainly extracts the identifiable mobile phone area in the inspection photo through a low-rank bilinear convolution algorithm, and uses the convolutional neural network to realize fast and accurate recognition of the models of used mobile phones. Summary of the invention

[0006] The present invention obtains a mobile phone model recognition method using a low-rank bilinear convolutional neural network. The recognition method extracts the identifiable mobile phone area in the inspection photo through a low-rank convolution algorithm, and uses a convolutional neural network to realize rapid and accurate recognition of the models of used mobile phones. The method solves the problem of model recognition in the process of recycling used mobile phones and improves the recycling efficiency of mobile phones.

[0007] The present invention adopts the following technical solutions and implementation steps:

[0008] 1. A method for mobile phone model recognition based on a low-rank bilinear convolutional neural network, which realizes accurate recognition of mobile phone models by designing a low-rank bilinear convolutional network structure, characterized in that it includes the following steps:

[0009] (1) The input variables of the mobile phone model recognition model are: the pixel matrix I1 of the first mobile phone image to be identified; the pixel matrix I2 of the second mobile phone image to be identified; the pixel matrix I3 of the red channel in the first mobile phone image I1 to be identified R1 , the green channel pixel matrix I in the first mobile phone image I1 to be identified G1 , the blue channel pixel matrix I in the first mobile phone image I1 to be identified B1 , the red channel pixel matrix I in the second mobile phone image I2 to be identified R2 , the green channel pixel matrix I in the second mobile phone image I2 to be identified G2 , the blue channel pixel matrix I in the second mobile phone image I2 to be identified B2 , the model label z1 of the first mobile phone image I1 to be identified; the model label z2 of the second mobile phone image I2 to be identified;

[0010] (2) Establish a bilinear convolutional network mobile phone feature extraction model

[0011] The input variable I of the two mobile phone model image samples R ,I G ,I B After graying, bilinear convolution feature extraction is performed. The specific calculation formula is as follows:

[0012] r1(t)=0.299×I R1 (t)+0.588×I G1 (t)+0.114×I B1 (t) (1)

[0013] r2(t)=0.299×I R2 (t)+0.588×I G2 (t)+0.114×I B2 (t) (2)

[0014] r1(t+1)=f(w(t)×r1(t)+λ1) (3)

[0015] r2(t+1)=f(w(t)×r2(t)+λ2) (4)

[0016]

[0017]

[0018] Where: r1(t) is the pixel matrix of the mobile phone image I1 to be identified after grayscale; r2(t) is the pixel matrix of the mobile phone image I2 to be identified after grayscale; r1(t+1) is the activated feature pixel matrix of the mobile phone image to be identified at the t+1th iteration of the first bilinear convolution structure; r2(t+1) is the activated feature pixel matrix of the mobile phone image to be identified at the t+1th iteration of the second bilinear convolution structure; f(·) is the activation function; w(t) represents the parameter weight of the bilinear convolution structure; λ1 is the output bias parameter of the first bilinear convolution structure, and λ1 takes a random value in the interval (0,1); λ2 is the output bias parameter of the second bilinear convolution structure, and λ2 takes a random value in the interval (0,1); t is the number of iterations; P1( r(t)) is the output vector of r1(t) after pooling; r1 is the element in the feature vector r1(t); P2(r(t)) is the output vector of r2(t) after pooling; r2 is the element in the feature vector r2(t); S1 is the horizontal pooling step size; S2 is the vertical pooling step size; a is the horizontal dimension of the convolution feature map after average pooling; b is the vertical dimension of the convolution feature map after average pooling; i represents the number of rows in the feature matrix; j represents the number of columns in the feature matrix; B1(t) is the feature matrix of P1(t) after dimensionality reduction by singular value decomposition; B2(t) is the feature matrix of P2(t) after dimensionality reduction by singular value decomposition; δ1(t) is the singular value matrix of P1(t); U1(t) is the left singular matrix of P1(t); V1 T (t) is the transpose of the right singular matrix of P1(t); δ2(t) is the singular value matrix of P2(t); U2(t) is the left singular matrix of P2(t); V2 T (t) is the transpose of the right singular matrix of P2(t); Z1(t) is the output vector after the regularization of each characteristic element in the matrix B1(t); Z2(t) is the output vector after the regularization of each characteristic element in the matrix B2(t); ||||2 represents the two-norm normalization operation; sign is the sign function;

[0019] (2) Designing a joint supervised classification model

[0020] The mobile phone image joint supervised classification model adopts a regression classification algorithm that combines Softmax and contrast loss. The center loss is used to calculate the similarity between samples of the same type, while the Softmax loss is directly used for calculation when the samples are different. The specific calculation method is shown in formula (11):

[0021]

[0022] Where: L(t) is the output of the joint loss function; μ is the loss trade-off coefficient, when z1=z2, that is, the samples are of the same model of mobile phones, μ=1; when z1≠z2, that is, the samples are of different models of mobile phones, μ=0; w T (t) represents the transpose of the parameter weights of the bilinear convolution structure;

[0023] (3) Mobile phone model identification process

[0024] The specific process of using the bilinear convolutional network structure to identify mobile phone models is as follows:

[0025] ①Use the real images collected during the recycling and inspection of used mobile phones as training data, select any two images I1 and I2 from them and input them into the bilinear convolution feature extraction model to obtain the fusion features B1(t) and B2(t) of each training sample;

[0026] ② The bilinear feature matrix is ​​reduced in rank by using the low-rank matrix parameter dimension reduction method shown in formulas (5) to (6) to reduce the computational complexity of the outer product aggregation operation and improve the operation speed, and finally the low-rank bilinear feature matrices Z1(t) and Z2(t) are obtained;

[0027] ③ Input the reduced feature matrix into the joint loss function as shown in formula (11) to obtain the joint supervision value L(t) of the sample, back-propagate and repeatedly adjust the bilinear convolution model parameter weight w(t) so that the joint supervision value L(t) reaches the global minimum.

[0028] ④ Randomly input the samples into the bilinear convolution recognition model in pairs, set the weight parameter of the model to w(t), and input the low-rank bilinear feature matrix output by the model into the joint loss function for classification and recognition, and then obtain the target model label value of the sample after classification, which is the model of the mobile phone.

[0029] The creativity of the present invention is mainly reflected in:

[0030] (1) The present invention is aimed at the mobile phone model recognition process in the current waste mobile phone recycling process. It is necessary to accurately and quickly identify the mobile phone model to improve the efficiency of waste mobile phone recycling. However, the shooting conditions such as the angle, equipment, and light source of the mobile phone photos are different, and the resolution is also different, which seriously affects the accuracy of the waste mobile phone model recognition. In addition, the similarity of some mobile phone models is too high. In the actual mobile phone recycling scenario, the category of mobile phone models is dynamically updated as new models are launched. In addition, due to the small number of training samples for new models, it is difficult for the model to learn and extract effective feature information in time. A model recognition algorithm based on a convolutional neural network is adopted, which has the characteristics of high accuracy and short detection time.

[0031] (2) The present invention proposes a method for identifying the models of used mobile phones based on a convolutional neural network. The method extracts the identifiable mobile phone area in the inspection photo through an edge detection algorithm, and uses a convolutional neural network to achieve rapid and accurate identification of the models of used mobile phones. It solves the problem of model identification in the process of recycling used mobile phones and improves the recycling efficiency of mobile phones.

[0032] It should be noted that the present invention is only for the convenience of description. Image recognition methods based on the same principles, such as using convolutional neural networks and edge detection algorithms to process mobile phone images and combining other feature extraction algorithms with recognition area extraction algorithms, should all fall within the scope of the present invention. DETAILED DESCRIPTION

[0033] 1. A method for mobile phone model recognition based on a low-rank bilinear convolutional neural network, which realizes accurate recognition of mobile phone models by designing a low-rank bilinear convolutional network structure, characterized in that it includes the following steps:

[0034] (1) The input variables of the mobile phone model recognition model are: the pixel matrix I1 of the first mobile phone image to be identified; the pixel matrix I2 of the second mobile phone image to be identified; the pixel matrix I3 of the red channel in the first mobile phone image I1 to be identified R1 , the green channel pixel matrix I in the first mobile phone image I1 to be identified G1 , the blue channel pixel matrix I in the first mobile phone image I1 to be identified B1 , the red channel pixel matrix I in the second mobile phone image I2 to be identified R2 , the green channel pixel matrix I in the second mobile phone image I2 to be identified G2 , the blue channel pixel matrix I in the second mobile phone image I2 to be identified B2 , the model label z1 of the first mobile phone image I1 to be identified; the model label z2 of the second mobile phone image I2 to be identified;

[0035] (2) Establish a bilinear convolutional network mobile phone feature extraction model

[0036] The input variable I of the two mobile phone model image samples R ,I G ,I B After graying, bilinear convolution feature extraction is performed. The specific calculation formula is as follows:

[0037] r1(t)=0.299×I R1 (t)+0.588×I G1 (t)+0.114×I B1 (t) (1)

[0038] r2(t)=0.299×I R2 (t)+0.588×I G2 (t)+0.114×I B2 (t) (2)

[0039] r1(t+1)=f(w(t)×r1(t)+λ1) (3)

[0040] r2(t+1)=f(w(t)×r2(t)+λ2) (4)

[0041]

[0042]

[0043] Where: r1(t) is the pixel matrix of the mobile phone image I1 to be identified after grayscale; r2(t) is the pixel matrix of the mobile phone image I2 to be identified after grayscale; r1(t+1) is the activated feature pixel matrix of the mobile phone image to be identified at the t+1th iteration of the first bilinear convolution structure; r2(t+1) is the activated feature pixel matrix of the mobile phone image to be identified at the t+1th iteration of the second bilinear convolution structure; f(·) is the activation function; w(t) represents the parameter weight of the bilinear convolution structure; λ1 is the output bias parameter of the first bilinear convolution structure, and λ1 takes a random value in the interval (0,1); λ2 is the output bias parameter of the second bilinear convolution structure, and λ2 takes a random value in the interval (0,1); t is the number of iterations; P1(r 1(t)) is the output vector of r1(t) after pooling; r1 is the element in the feature vector r1(t); P2(r2(t)) is the output vector of r2(t) after pooling; r2 is the element in the feature vector r2(t); S1 is the horizontal pooling step size; S2 is the vertical pooling step size; a is the horizontal dimension of the convolution feature map after average pooling; b is the vertical dimension of the convolution feature map after average pooling; i represents the number of rows in the feature matrix; j represents the number of columns in the feature matrix; B1(t) is the feature matrix of P1(t) after dimensionality reduction by singular value decomposition; B2(t) is the feature matrix of P2(t) after dimensionality reduction by singular value decomposition; δ1(t) is the singular value matrix of P1(t); U1(t) is the left singular matrix of P1(t); V1 T (t) is the transpose of the right singular matrix of P1(t); δ2(t) is the singular value matrix of P2(t); U2(t) is the left singular matrix of P2(t); V2 T (t) is the transpose of the right singular matrix of P2(t); Z1(t) is the output vector after the regularization of each characteristic element in the matrix B1(t); Z2(t) is the output vector after the regularization of each characteristic element in the matrix B2(t); ||||2 represents the two-norm normalization operation; sign is the sign function;

[0044] (2) Designing a joint supervised classification model

[0045] The mobile phone image joint supervised classification model adopts a regression classification algorithm that combines Softmax and contrast loss. The center loss is used to calculate the similarity between samples of the same type, while the Softmax loss is directly used for calculation when the samples are different. The specific calculation method is shown in formula (11):

[0046]

[0047] Where: L(t) is the output of the joint loss function; μ is the loss trade-off coefficient, when z1=z2, that is, the samples are of the same model of mobile phones, μ=1; when z1≠z2, that is, the samples are of different models of mobile phones, μ=0; wT (t) represents the transpose of the parameter weights of the bilinear convolution structure;

[0048] (3) Mobile phone model identification process

[0049] The specific process of using the bilinear convolutional network structure to identify mobile phone models is as follows:

[0050] ⑤Use the real images collected during the recycling and inspection of used mobile phones as training data, select any two images I1 and I2 from them and input them into the bilinear convolution feature extraction model to obtain the fusion features B1(t) and B2(t) of each training sample;

[0051] ⑥ The bilinear feature matrix is ​​reduced in rank by using the low-rank matrix parameter dimensionality reduction method shown in formulas (5) to (6) to reduce the computational complexity of the outer product aggregation operation and improve the operation speed, and finally the low-rank bilinear feature matrices Z1(t) and Z2(t) are obtained;

[0052] ⑦ Input the reduced feature matrix into the joint loss function as shown in formula (11) to obtain the joint supervision value L(t) of the sample, back-propagate and repeatedly adjust the bilinear convolution model parameter weight w(t) so that the joint supervision value L(t) reaches the global minimum.

[0053] ⑧Randomly input the samples into the bilinear convolution recognition model in pairs, set the weight parameter of the model to w(t), and input the low-rank bilinear feature matrix output by the model into the joint loss function for classification and recognition, and then obtain the target model label value of the sample after classification, which is the model of the mobile phone.

Claims

1. A method for identifying the model of used mobile phones based on convolutional neural networks, characterized in that: The following steps are involved: (1) The input variables of the mobile phone model recognition model are: the pixel matrix I1 of the first mobile phone image to be identified; the pixel matrix I2 of the second mobile phone image to be identified; the pixel matrix I3 of the red channel in the first mobile phone image I1 to be identified R1 , the green channel pixel matrix I in the first mobile phone image I1 to be identified G1 , the blue channel pixel matrix I in the first mobile phone image I1 to be identified B1 , the red channel pixel matrix I in the second mobile phone image I2 to be identified R2 , the green channel pixel matrix I in the second mobile phone image I2 to be identified G2 , the blue channel pixel matrix I in the second mobile phone image I2 to be identified B2 , the model label z1 of the first mobile phone image I1 to be identified; the model label z2 of the second mobile phone image I2 to be identified; (2) Establish a bilinear convolutional network mobile phone feature extraction model The input variable I of the two mobile phone model image samples R ,I G ,I B After graying, bilinear convolution feature extraction is performed. The specific calculation formula is as follows: r1(t)=0.299×I R1 (t)+0.588×I G1 (t)+0.114×I B1 (t) (1) r2(t)=0.299×I R2 (t)+0.588×I G2 (t)+0.114×I B2 (t) (2) r1(t+1)=f(w(t)×r1(t)+λ1) (3) r2(t+1)=f(w(t)×r2(t)+λ2) (4) Where: r1(t) is the pixel matrix of the mobile phone image I1 to be identified after grayscale; r2(t) is the pixel matrix of the mobile phone image I2 to be identified after grayscale; r1(t+1) is the activated feature pixel matrix of the mobile phone image to be identified at the t+1th iteration of the first bilinear convolution structure; r2(t+1) is the activated feature pixel matrix of the mobile phone image to be identified at the t+1th iteration of the second bilinear convolution structure; f(·) is the activation function; w(t) represents the parameter weight of the bilinear convolution structure; λ1 is the output bias parameter of the first bilinear convolution structure, and λ1 takes a random value in the interval (0,1); λ2 is the output bias parameter of the second bilinear convolution structure, and λ2 takes a random value in the interval (0,1); t is the number of iterations; P1(r1(t)) is the output vector of r1(t) after pooling; r1 is the element in the feature vector r1(t); P2(r2(t)) is the output vector of r2(t) after pooling; r2 is the element in the feature vector r2(t); S1 is the horizontal pooling step; S2 is the vertical pooling step; a is the horizontal dimension of the convolution feature map after average pooling; b is the vertical dimension of the convolution feature map after average pooling; i represents the number of rows in the feature matrix; j represents the number of columns in the feature matrix; B1(t) is the feature matrix of P1(t) after dimensionality reduction by singular value decomposition; B2(t) is the feature matrix of P2(t) after dimensionality reduction by singular value decomposition; δ1(t) is the singular value matrix of P1(t); U1(t) is the left singular matrix of P1(t); V1 T (t) is the transpose of the right singular matrix of P1(t); δ2(t) is the singular value matrix of P2(t); U2(t) is the left singular matrix of P2(t); V2 T (t) is the transpose of the right singular matrix of P2(t); Z1(t) is the output vector after the regularization of each characteristic element in the matrix B1(t); Z2(t) is the output vector after the regularization of each characteristic element in the matrix B2(t); ||·||2 represents the two-norm normalization operation; sign is the sign function; (2) Designing a joint supervised classification model The mobile phone image joint supervised classification model adopts a regression classification algorithm that combines Softmax and contrast loss. The center loss is used to calculate the similarity between samples of the same type, while the Softmax loss is directly used for calculation when the samples are different. The specific calculation method is shown in formula (11): Where: L(t) is the output of the joint loss function; μ is the loss trade-off coefficient, when z1=z2, that is, the samples are of the same model of mobile phones, μ=1; when z1≠z2, that is, the samples are of different models of mobile phones, μ=0; w T (t) represents the transpose of the parameter weights of the bilinear convolution structure; (3) Mobile phone model identification process The specific process of using the bilinear convolutional network structure to identify mobile phone models is as follows: ①Use the real images collected during the recycling and inspection of used mobile phones as training data, select any two images I1 and I2 from them and input them into the bilinear convolution feature extraction model to obtain the fusion features B1(t) and B2(t) of each training sample; ② The bilinear feature matrix is ​​reduced in rank by using the low-rank matrix parameter dimension reduction method shown in formulas (5) to (6) to reduce the computational complexity of the outer product aggregation operation and improve the operation speed, and finally the low-rank bilinear feature matrices Z1(t) and Z2(t) are obtained; ③ Input the reduced feature matrix into the joint loss function as shown in formula (11) to obtain the joint supervision value L(t) of the sample, back-propagate and repeatedly adjust the bilinear convolution model parameter weight w(t) so that the joint supervision value L(t) reaches the global minimum; ④ Randomly input the samples into the bilinear convolution recognition model in pairs, set the weight parameter of the model to w(t), and input the low-rank bilinear feature matrix output by the model into the joint loss function for classification and recognition, and then obtain the target model label value of the sample after classification, which is the model of the mobile phone.

Images