A lightweight tire pattern classification method and system

Abstract

The present application belongs to the technical field of image recognition, and provides a lightweight tire pattern classification method and system and a tire pattern classification method training strategy based on knowledge distillation. The local feature representation of the student network model is constrained to be consistent with the local feature representation of the teacher network, so that the classification accuracy of the student network model is improved without affecting the classification efficiency of the student network model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field related to image recognition, and particularly relates to a lightweight tire pattern classification method and system. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] With the rapid increase in the number of motor vehicles, the traffic accident rate in China is rising year by year, such as hit-and-run. The tire pattern at the scene of the accident often provides very important clues for the handling of these traffic accidents. For example, according to the tire indentation pattern, the brand of the tire is predicted, and based on the tire brand information, the types and models of motor vehicles searched are further narrowed down, and the case handling efficiency is improved. As can be seen, it is necessary to develop an efficient and automated tire pattern image classification and recognition system to further improve the work efficiency of law enforcement departments. According to the inventors' previous research, there is very little research related to tire pattern image classification and recognition. Their processing procedures can be summarized as follows: 1) using manual feature extraction or feature enhancement methods to extract different features of tire pictures; 2) inputting the extracted features into a SVM classifier or a pre-trained deep image classification model for classification prediction, such as AlexNet and VGGNet. In order to further improve the classification accuracy, some methods propose multi-level feature fusion and the addition of attention mechanisms. Although these methods have achieved preliminary results, they have large memory requirements and large computational loads, and cannot be run on mobile devices or embedded devices. Designing a lightweight tire image classification model that is not only small in size and fast in classification has become a challenging research problem that needs to be solved. SUMMARY

[0004] To overcome the shortcomings of the prior art, the present application provides a lightweight tire pattern classification method and system, which constrains the local feature representation of the student network to be consistent with the local feature representation of the teacher network, improves the classification accuracy of the student network without affecting the classification efficiency of the student network.

[0005] To achieve the above-mentioned purpose, one or more embodiments of the present application provide the following technical solutions:

[0006] The first aspect of the present application provides a lightweight tire pattern classification method, comprising:

[0007] The tire pattern image and the corresponding tire category label information are input into the student network model to obtain the local feature representation of the tire pattern image; the student network model takes ShuffleNet-V2 network as the backbone network;

[0008] The local feature representation is input into a teacher network model to obtain a comprehensive discriminative enhanced local feature representation, the local feature representation and the comprehensive discriminative enhanced local feature representation are constrained to be consistent, the student network model is trained, and a trained student network model is obtained; the teacher network model comprises a Gaussian dictionary learning module and a category memory module.

[0009] The trained student network model is used for identifying a tire pattern image to be identified to determine a tire category of the tire pattern image to be identified.

[0010] The second aspect of the present application provides a lightweight tire pattern classification system, comprising:

[0011] The feature extraction module inputs the tire pattern image and the corresponding tire category label information into the student network model to obtain the local feature representation of the tire pattern image; the student network model takes a ShuffleNet-V2 network as a backbone network;

[0012] The feature extraction and model training module inputs the local feature representation into the teacher network model to obtain the comprehensive discriminative enhanced local feature representation, the local feature representation and the comprehensive discriminative enhanced local feature representation are constrained to be consistent, the student network model is trained, and the trained student network model is obtained; the teacher network model comprises a Gaussian dictionary learning module and a category memory module;

[0013] The identification output module uses the trained student network model to identify the tire pattern image to be identified to determine the tire category of the tire pattern image to be identified.

[0014] The above one or more technical solutions have the following beneficial effects:

[0015] 1. In view of the problem that the existing tire pattern image classification method occupies too much computing resources, the present application uses a ShuffleNet-V2 classification model as a student network, greatly reduces the consumption of computing resources, and improves the classification efficiency;

[0016] 2. The present application proposes a tire pattern classification method training strategy based on knowledge distillation, which constrains the local feature representation of the student network to be consistent with the local feature representation of the teacher network, improves the classification accuracy of the student network without affecting the classification efficiency of the student network.

[0017] 3. The present application introduces a teacher network integrating a category memory module and a Gaussian dictionary learning module to capture comprehensive discriminative enhanced local feature representation, improve the robustness and discriminability of the backbone network feature representation, and then improve the generalization of the tire pattern image classification method.

[0018] Advantages of the additional aspects of the application will become apparent in the following description, which is given for the purpose of illustration and not of limitation. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which form a part of this specification, are included to provide a further understanding of the application and are incorporated by reference herein. The embodiments illustrated in the drawings are presented by way of example in

[0020] Figure 1 Figure 1 is a schematic diagram of a tire pattern image recognition process according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] It should be noted that the following detailed description is merely exemplary and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0022] Embodiment One

[0023] As shown in the following table, the present embodiment discloses a lightweight tire pattern classification method, which comprises the following steps: Figure 1

[0024] The tire pattern image and the corresponding tire category label information are input into the student network model to obtain a local feature representation of the tire pattern image. The student network model takes a ShuffleNet-V2 network as a backbone network.

[0025] The local feature representation is input into a teacher network model to obtain a comprehensive discriminative enhanced local feature representation. The local feature representation and the comprehensive discriminative enhanced local feature representation are constrained to be consistent. The student network model is trained to obtain a trained student network model. The teacher network model includes a Gaussian dictionary learning module and a category memory module.

[0026] The trained student network model is used to identify the tire pattern image to be identified to determine the tire category of the tire pattern image to be identified.

[0027] The present embodiment proposes a lightweight tire pattern classification method, which specifically comprises:

[0028] Step 1: receiving tire pattern image data and tire category label information, inputting the tire pattern image data into a student network model to obtain a local feature representation and a cross-entropy loss function. The student network model takes a ShuffleNet-V2 as a backbone network. Specifically, it includes:

[0029] Step 1-1: receiving tire pattern image data, denoted as ​; receiving tire pattern class label information , denoted as ; wherein, represents the number of tire pattern categories.

[0030] Step 1-2: input the tire pattern image data into the ShuffleNet-V2 network, and the local feature representation output by the Conv5 component in the ShuffleNet-V2 network is , denoted as , wherein R in this embodiment is used to represent the dimension.

[0031] Step 1-3: perform global pooling on the local feature representation to obtain the global representation of the image , denoted as .

[0032] Step 1-4: input the global representation of the image into a classification network composed of a fully connected layer and a SoftMax activation function to obtain the classification probability of the tire pattern image , denoted as , wherein represents the number of tire pattern categories; at the same time, the cross-entropy loss is calculated according to the classification probability of the tire pattern image and the true label information .

[0033] , wherein the ShuffleNet-V2 network includes Conv1, Maxpool, Stage2, Stage3, Stage4, Conv5 and Globalpool connected in sequence.

[0034] Step 2: obtain the discriminative local feature representation of the image by using the Gaussian dictionary learning module and the local feature representation, specifically including:

[0035] Step 2-1: specify the number of required Gaussian functions, which is related to the number of representative representations in the local feature representation of the tire pattern image, denoted as ; and the maximum number of iterations A of the Gaussian dictionary learning module; wherein, according to step 1, the number of local regions in the local feature representation of the tire pattern image is 49= , therefore, the specified number of Gaussian functions n needs to be less than the number of local regions 49 and greater than 1;

[0036] The form of the Gaussian function used in this embodiment is:

[0037]

[0038] wherein is the input feature information, i.e. the local feature representation of the tire pattern image is the dimension transformation result is the input feature information is the dimension number of, i.e. 1024, is the mean vector of the Gaussian function to be learned in the subsequent step, is the unit matrix, denotes the transpose.

[0039] Step 2-2: Randomly initialize the mean vector of all Gaussian functions and splice them together, denoted as ; at the same time, fix the variance of all Gaussian functions as the unit matrix , denoted as .

[0040] Step 2-3: Perform dimension transformation on the local feature representation matrix to obtain ; at the same time, assign the iteration index variable to 1, i.e. .

[0041] Step 2-4: Calculate the relevance score between the local feature representation at different positions and the Gaussian mean vector matrix of the tire pattern image learned in the first (i.e. the ) iteration to obtain the relevance score matrix : , is the specified required number of Gaussian functions, denotes the transpose. Step 2-5: Update the Gaussian mean vector matrix

[0042] according to the relevance score matrix: ; at the same time, increment the iteration index variable by 1, i.e. denotes the transpose. Step 2-6: Repeat steps 2-4 and 2-5 until the value of

[0043] reaches the maximum number of iterations, and the image discriminative local feature representation matrix is obtained. is the specified required number of Gaussian functions. Step 3: Obtain the class discriminative local feature representation using the class memory module and the image discriminative local feature representation, which specifically includes:

[0044]

[0045] ​Step 3-1: randomly initialize a category memory feature representation matrix , denoted as ; and zero-initialize a category memory feature score matrix , each element value in the category memory feature score matrix representing a confidence score of the corresponding position vector in the category memory feature representation matrix; wherein represents the number of each tire pattern category feature representation stored in the category memory feature representation matrix, is required to be less than the number n of Gaussian functions in step 2 and greater than 1, represents the number of categories of tire patterns.

[0046] Step 3-2: input the image discriminative local feature representation matrix into a classification network to obtain classification probability values of different local regions, denoted as , wherein represents the classification probability of the th image discriminative feature representation, is the specified required number of Gaussian functions, represents the number of categories of tire patterns; 1≤ ≤n.

[0047] Step 3-3: take the maximum value of each classification probability , denoted as , and return the index value of the position where the maximum value is located as the predicted tire category , denoted as . If and is greater than a certain value in the th row of the category memory feature score matrix, change the position score to , and store the data in the th row of the image discriminative local feature representation matrix , denoted as , in the th row and the th column of the category memory feature representation matrix . Otherwise, do not modify the information in the category memory feature representation matrix and the category memory feature score matrix .

[0048] The embodiment introduces a teacher network model, wherein the teacher network model includes a category memory module and a Gaussian dictionary learning module, to capture comprehensive discriminative enhanced local feature representations, improve the robustness and discriminativeness of student network feature representations, and then improve the generalization of the tire pattern image classification method.

[0049] Step 4: Strengthen the local feature representation of the student network respectively by using the image discriminative local feature representation and the category discriminative local feature representation, and add the strengthened results to obtain the comprehensive discriminative enhanced local feature representation, which specifically includes:

[0050] Step 4-1: Calculate the correlation score matrix between the local feature representation and the image discriminative local feature representation matrix Step 4-2: According to the label information, extract the memory representation of the corresponding category from the category memory feature representation matrix , denoted as ; calculate the correlation score matrix between the local feature representation and , denoted as

[0051] , (1)

[0052] wherein is a pre-defined balance parameter, which determines the proportion of the image discriminative local feature representation when the image discriminative local feature representation and the local feature representation are fused, and the value is between [0, 1], is the image discriminative local feature representation matrix, is the local feature representation, and T represents transposition.

[0053] Step 4-2: According to the label information , extract the memory representation of the corresponding category from the category memory feature representation matrix , denoted as ; calculate the correlation score matrix between the local feature representation and , denoted as , , , wherein represents the number of each tire pattern category feature representation stored in the category memory feature representation matrix, and the value needs to be less than the number n of Gaussian functions in step 2 and greater than 1; then calculate the memory feature strengthened local representation :

[0054] (2)

[0055] wherein is a pre-defined balance parameter, which determines the proportion of the category discriminative local feature representation when the category discriminative local feature representation and the local feature representation are fused, and the value is between [0, 1], is the local feature representation, and T represents transposition.

[0056] Step 4-3: Add the memory feature strengthened local representation and the image discriminative strengthened local representation adding, a comprehensive discriminative enhanced local feature representation matrix is obtained , denoted as .

[0057] Step 5: inputting the comprehensive discriminative enhanced local feature representation and the local feature representation into the classification network at the same time , obtaining their category probability results, inputting into a consistency loss function, adding the cross-entropy loss function obtained in step 1 and the consistency loss function to obtain a final optimization function, and training the classification network by using the final optimization function. Specifically, it includes:

[0058] Step 5-1: inputting the comprehensive discriminative enhanced local feature representation matrix and the local feature representation into the classification network respectively, obtaining classification probability prediction scores denoted as and .

[0059] Step 5-2: calculating the consistency loss by using the following formula ;

[0060] (3)

[0061] wherein, the i-th row of is denoted as , the i-th row of is denoted as , and i takes values [1, 49].

[0062] Step 5-3: adding the consistency loss and the cross-entropy loss to obtain a final optimization function for network training, thereby obtaining a trained student network model.

[0063] In the embodiment, during the training of the student network, a total of B sets of tire pattern image data and tire pattern category label information are received, denoted as , wherein and respectively represent the i-th set of tire pattern image data and tire pattern category label information. Any set of tire pattern image data and tire pattern category label information will undergo the processes of steps 1-2 to obtain a corresponding image discriminative local feature representation matrix. Herein, the discriminative local feature representation matrix of the i-th set of tire pattern image data is denoted as ​​Then, the image discriminative local feature representation matrix will undergo step 3-2 and step 3-3 operations in turn to update the category memory feature representation matrix and the category memory feature score matrix in turn. As can be seen, the category memory module memorizes all representative discriminative feature representation information in the historical training data, i.e. the category memory feature representation matrix and the category memory feature score matrix will be updated continuously with the training of the student network, and the discriminative feature representation of each category will become more diverse and more representative, thereby improving the generalization and robustness of the student network.

[0064] Comparison of the proposed model of the present application with experimental data of other models:

[0065] MobileNetV2 model: million floating point operations per second: 43; GPU batch image processing number per second: 333; ARM image processing number per second: 33.2;

[0066] DenseNet model: million floating point operations per second: 42; GPU batch image processing number per second: 366; ARM image processing number per second: 39.7;

[0067] Xception model: million floating point operations per second: 40; GPU batch image processing number per second: 384; ARM image processing number per second: 52.9;

[0068] The proposed model of the present embodiment: million floating point operations per second: 41; GPU batch image processing number per second: 417; ARM image processing number per second: 57.0;

[0069] Wherein, the GPU card configuration is NVIDIA GeForce GTX 1080Ti, the ARM configuration is Qualcomm Snapdragon 810, and the batch processing size is set to 8. As can be seen from the above, the proposed model of the present embodiment has the most image data processing per second in both GPU and ARM environments under the condition of fewer floating point operation times, and the proposed classification method of the present embodiment is small in size and fast in classification speed.

[0070] Embodiment two

[0071] The purpose of the present embodiment is to provide a lightweight tire pattern classification system, which comprises:

[0072] A feature extraction module: input the tire pattern image and the corresponding tire category label information into the student network model to obtain the local feature representation of the tire pattern image; the student network model takes ShuffleNet-V2 network as the backbone network;

[0073] The feature extraction and model training module: the local feature representation is input into a teacher network model to obtain a comprehensive discriminative enhanced local feature representation, the local feature representation and the comprehensive discriminative enhanced local feature representation are constrained to be consistent, the student network model is trained, and a trained student network model is obtained; the teacher network model comprises a Gaussian dictionary learning module and a category memory module;

[0074] The recognition output module: the trained student network model is used to recognize a tire pattern image to be recognized to determine a tire category of the tire pattern image to be recognized.

[0075] The above describes the specific embodiments of the application in combination with the drawings, but is not a limitation on the protection scope of the application, and those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the application without creative labor are still within the protection scope of the application.

Claims

1. A lightweight tire tread pattern classification method, characterized in that, include: The tire tread image and the corresponding tire category label information are obtained and input into the student network model to obtain the local feature representation of the tire tread image; the student network model uses the ShuffleNet-V2 network as the backbone network; The local feature representation is input into the teacher network model to obtain a local feature representation with enhanced discriminative power. The local feature representation and the local feature representation with enhanced discriminative power are constrained to be consistent. The student network model is then trained to obtain a trained student network model. The teacher network model includes a Gaussian dictionary learning module and a category memory module. By inputting local feature representations into the teacher network model, a comprehensive discriminative enhanced local feature representation is obtained, specifically: By utilizing the Gaussian dictionary learning module and local feature representation, a discriminative local feature representation of the image is obtained; By utilizing the category memory module and image discriminative local feature representation, a category discriminative local feature representation is obtained; The local feature representations are enhanced by strengthening the image discriminative local feature representation and the category discriminative local feature representation respectively, resulting in a comprehensive discriminatively enhanced local feature representation, specifically: Calculate the first correlation score matrix between the local feature representation and the image discriminative local feature representation, and use the first correlation score matrix to obtain the local feature representation for image discriminative enhancement; Calculate the second relevance score matrix of the corresponding category memory representation in the local feature representation and category memory feature representation matrix, and use the second relevance score matrix to obtain the local feature representation with enhanced category discriminativeness; The local feature representation of image discriminative enhancement is added to the local feature representation of category discriminative enhancement to obtain the local feature representation of comprehensive discriminative enhancement; The trained student network model is used to identify the tire tread images to determine the tire category of the images.

2. The lightweight tire tread pattern classification method as described in claim 1, characterized in that, The output of the Conv5 component in the ShuffleNet-V2 network is used as a local feature representation of the tire tread image.

3. The lightweight tire tread pattern classification method as described in claim 1, characterized in that, The discriminative local feature representation of the image is obtained using a local feature representation and a Gaussian dictionary learning module, specifically as follows: The number of Gaussian functions and the number of iterations are predefined. Based on the determined number of Gaussian functions, the mean vector of the Gaussian functions is randomly initialized. Based on the initial mean vector of the Gaussian function, calculate the correlation score between the local feature representation and the mean vector of different Gaussian functions; The mean vector of the Gaussian function is iteratively updated based on the correlation score and local feature representation. After reaching the required number of iterations, the final Gaussian function mean vector is used as the discriminative local feature representation of the image.

4. The lightweight tire tread pattern classification method as described in claim 3, characterized in that, The discriminative local feature representation of the image is input into the category memory module, which outputs a category-discriminative local feature representation. Specifically: Randomly initialize the category memory feature representation matrix and zero-initialize the category memory feature score matrix. Each element value in the category memory feature score matrix represents the confidence score of the corresponding position in the category memory feature representation matrix. The discriminative local feature representation of the image is input into the classification network to obtain the corresponding classification probability value; If the maximum classification probability corresponding to the discriminative local feature representation of the j-th image is greater than the corresponding value in the category memory feature score matrix, and the tire prediction category corresponding to the maximum classification probability is the same as the tire category label, then the maximum classification probability is updated in the category memory feature score matrix, and the data in the j-th row of the discriminative local feature representation of the image is updated in the category memory feature representation matrix.

5. The lightweight tire tread pattern classification method as described in claim 1, characterized in that, The classification network is trained using local feature representation and local feature representation with enhanced discriminative power. Specifically, the local feature representation with enhanced discriminative power and the local feature representation are input into the classification network to obtain two classification prediction probabilities. By constraining the two classification probabilities to be consistent, a consistency loss function is obtained, and the classification network is trained.

6. The lightweight tire tread pattern classification method as described in claim 5, characterized in that, The classification network consists of fully connected layers and a SoftMax activation function.

7. The lightweight tire tread pattern classification method as described in claim 5, characterized in that, The classification loss function of the classification network is the cross-entropy classification loss function. The consistency loss function is added to the cross-entropy loss function to obtain the final optimization function, which is then used to train the classification network.

8. A lightweight tire tread pattern classification system, employing the lightweight tire tread pattern classification method as described in any one of claims 1-7, characterized in that, include: Feature extraction module: Obtains tire tread image and corresponding tire category label information and inputs them into student network model to obtain local feature representation of tire tread image; the student network model uses ShuffleNet-V2 network as backbone network; Feature extraction and model training module: Local feature representations are input into the teacher network model to obtain a comprehensively discriminative enhanced local feature representation. The local feature representation and the comprehensively discriminative enhanced local feature representation are constrained to be consistent. The student network model is then trained to obtain a trained student network model. The teacher network model includes a Gaussian dictionary learning module and a category memory module. The Gaussian dictionary learning module and local feature representations are used to obtain image discriminative local feature representations. The category memory module and image discriminative local feature representations are used to obtain category-discriminative local feature representations. Recognition Output Module: Uses a trained student network model to recognize the tire tread image to determine the tire category of the image.

Images