An image recognition uncertainty knowledge distillation method and system

Abstract

This invention discloses a knowledge distillation method and system for image recognition uncertainty. The method involves collecting labeled training image samples; selecting a first neural network model and training it using the training image samples to obtain a trained first neural network model; inputting training sample images to obtain the intermediate layer sample feature representations and output soft label information of the first neural network model; selecting a second neural network model and processing the training image samples to obtain intermediate layer sample feature representations; performing uncertainty modeling between the intermediate layer sample feature representations of the first neural network model and the first neural network model to obtain a first loss function; using the soft label information output by the first neural network model and the training image samples, combined with the first loss function, updating the parameters of the second neural network model to obtain a trained second neural network model; and simultaneously using the second neural network model of this invention to perform image recognition processing on the image to be processed, thereby improving the accuracy of image recognition.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a method and system for knowledge distillation of uncertainty in image recognition. Background Technology

[0002] Knowledge distillation is a technique for compressing and accelerating neural network models. It effectively reduces the resource requirements of neural network-based image classification or object detection models on resource-constrained devices while maintaining high recognition accuracy. Its basic principle is to use the feature representations and predicted image label information of a highly accurate neural network model (the first neural network model) to guide the training of a smaller neural network model (the second neural network model) with fewer parameters and lower computational cost. This method significantly improves the image recognition accuracy of the second neural network model while requiring minimal resources. However, current knowledge distillation techniques assume that the knowledge representation extracted and distilled from the first neural network model is accurate and discriminative, ignoring noise and other misleading information contained in the knowledge representation of the first neural network model. This results in a weaker knowledge representation acquired by the second neural network model, hindering further improvements in image recognition accuracy. Summary of the Invention

[0003] To address the shortcomings of existing technologies and extract more deterministic and discriminative knowledge representations during knowledge distillation, thereby improving the image recognition accuracy of the second neural network model, this invention adopts the following technical solution:

[0004] On one hand, the present invention provides a method for knowledge distillation of uncertainty in image recognition, the method comprising:

[0005] S100: Collect sample images, process and label them to obtain the first number of labeled training image samples;

[0006] S200: Select the first neural network model, train it using the first number of training image samples, update the parameter values ​​of the network model, obtain the trained first neural network model, input the training sample images into the first neural network model to obtain the sample feature representation of the intermediate layer of the model and the soft label information of the image processing output;

[0007] S300: Select a second neural network model, process a first number of training image samples to obtain intermediate layer sample feature representations, perform uncertainty modeling on the intermediate layer sample feature representations obtained from the first neural network model, and obtain a first loss function;

[0008] S400: Using the image processing soft label information and training image samples output by the first neural network model, the first loss function is combined to obtain the overall loss function and update the parameters of the second neural network model to obtain the trained second neural network model;

[0009] Optionally, a second neural network model is selected, and a first number of training image samples are processed to obtain intermediate layer sample feature representations. Uncertainty modeling is then performed between these intermediate layer sample feature representations and those obtained from the first neural network model to obtain a first loss function, including:

[0010] Select the intermediate convolutional layer of the second neural network model and input training image samples to obtain the feature representation of the intermediate layer samples of the second neural network model;

[0011] Calculate the semantic similarity between the channels of the intermediate layer sample feature representations of the second neural network model and the intermediate layer sample feature representations of the first neural network model;

[0012] Calculate the semantic similarity between the spatial feature representations of the intermediate layer samples of the second neural network model and the spatial feature representations of the intermediate layer samples of the first neural network model;

[0013] Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, the mean value of the uncertainty of each sample feature expression in the first neural network model is obtained;

[0014] Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, a fully connected layer is used to further process and obtain the variance value of the uncertainty of each sample feature expression in the first neural network model.

[0015] Based on the mean and variance of the uncertainty expressed by the sample features obtained from the calculation, the uncertainty value is obtained by using the reparameterization technique, and thus the first loss function of knowledge distillation is obtained;

[0016] Optionally, the image processing soft label information output by the first neural network model and the training image samples are used together with the first loss function to obtain the overall loss function and update the parameters of the second neural network model, including:

[0017] The training sample images are processed using a second neural network model, and the processed predicted label information is output.

[0018] The second loss function is calculated using the image processing soft label information output by the first neural network model and the predicted label information output by the second neural network model.

[0019] The third loss function is calculated using the predicted label information output by the second neural network model and the label information of the training image samples.

[0020] The overall loss function is obtained by adding the first loss function, the second loss function, and the third loss function.

[0021] In a second aspect, the present invention provides an image recognition uncertainty knowledge distillation system, comprising:

[0022] The acquisition module is used to collect sample images, process and label them to obtain a first number of labeled training image samples;

[0023] First update module: connected to the acquisition module, used to select a first neural network model, train it using a first number of training image samples, update the parameter values ​​of the network model, obtain a trained first neural network model, and input training sample images into the first neural network model to obtain the model's intermediate layer sample feature expression and soft label information output by image processing.

[0024] Uncertainty modeling module: connected to the acquisition module and the first update module, used to select a second neural network model, process a first number of training image samples to obtain intermediate layer sample feature expressions, and perform uncertainty modeling with the intermediate layer sample feature expressions obtained from the first neural network model to obtain a first loss function;

[0025] The second update module, connected to the acquisition module, the first update module, and the uncertainty modeling module, uses the image processing soft-label information output by the first neural network model and training image samples. It combines these with the first loss function to obtain the overall loss function and update the parameters of the second neural network model, resulting in a trained second neural network model.

[0026] Optionally, the uncertainty modeling module specifically includes:

[0027] Select the intermediate convolutional layer of the second neural network model and input training image samples to obtain the feature representation of the intermediate layer samples of the second neural network model;

[0028] Calculate the semantic similarity between the channels of the intermediate layer sample feature representations of the second neural network model and the intermediate layer sample feature representations of the first neural network model;

[0029] Calculate the semantic similarity between the spatial feature representations of the intermediate layer samples of the second neural network model and the spatial feature representations of the intermediate layer samples of the first neural network model;

[0030] Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, the mean value of the uncertainty of each sample feature expression in the first neural network model is obtained;

[0031] Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, a fully connected layer is used to further process and obtain the variance value of the uncertainty of each sample feature expression in the first neural network model.

[0032] Based on the mean and variance of the uncertainty expressed by the sample features obtained from the calculation, the uncertainty value is obtained by using the reparameterization technique, and thus the first loss function of knowledge distillation is obtained;

[0033] Optionally, the second update module specifically includes:

[0034] The training sample images are processed using a second neural network model, and the processed predicted label information is output.

[0035] The second loss function is calculated using the image processing soft label information output by the first neural network model and the predicted label information output by the second neural network model.

[0036] The third loss function is calculated using the predicted label information output by the second neural network model and the label information of the training image samples.

[0037] The overall loss function is obtained by adding the first loss function, the second loss function, and the third loss function.

[0038] Thirdly, the present invention provides an image processing method for image recognition uncertainty knowledge distillation, comprising:

[0039] Obtain a second number of test image samples to be processed;

[0040] The second neural network model trained using the aforementioned image recognition uncertainty knowledge distillation method is used to perform image recognition processing on the second number of test image samples to be processed.

[0041] Obtain the recognition and processing results and output them;

[0042] Fourthly, the present invention provides an image processing system, comprising:

[0043] The acquisition module is used to acquire a second number of test image samples to be processed;

[0044] The recognition processing module, connected to the acquisition module, is used to perform image recognition processing on the second number of test image samples to be processed using the second neural network model trained by the aforementioned image recognition uncertainty knowledge distillation system.

[0045] The recognition output module is connected to the recognition processing module and is used to acquire the image to be processed. The second neural network model trained by the aforementioned image recognition uncertainty knowledge distillation system is used to perform image recognition processing on the second number of test image samples to be processed, and the recognition processing result is obtained and output.

[0046] The present invention also relates to an image recognition processing device, comprising an image acquisition unit, a memory, one or more processors, and an external output unit. The image acquisition unit is used to acquire a first and a second number of image samples. The memory stores executable code. When the one or more processors execute the executable code, they are used to implement an image recognition uncertainty knowledge distillation method and an image processing method of the present invention. The external output unit is used to output and display the image recognition processing result obtained by the image processing method.

[0047] The present invention also relates to a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute an image recognition uncertainty knowledge distillation method and an image processing method of the present invention.

[0048] This invention also relates to a computer program product, including a computer program / instruction, characterized in that, when the computer program / instruction is executed by a processor, it implements an image recognition uncertainty knowledge distillation method and an image processing method of this invention.

[0049] As can be seen from the above technical solution, compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0050] This invention can be flexibly applied to any teacher-student model combination and can be easily deployed into existing neural network models for image recognition. Compared with existing knowledge distillation image recognition techniques, it can significantly improve the image recognition accuracy of the second neural network model, and the computational load of the trained model is greatly reduced. Using ResNet32x4 as the first neural network model and VGG8 as the second neural network model, this invention can improve the image classification accuracy of the second neural network model from 70.36% to 75.88% on the CIFAR100 dataset. Attached Figure Description

[0051] Figure 1 This is a flowchart of an image recognition uncertainty knowledge distillation method according to an embodiment of the present invention.

[0052] Figure 2 This is a flowchart illustrating the process of obtaining the first loss function in an image recognition uncertainty knowledge distillation method according to an embodiment of the present invention.

[0053] Figure 3This is a flowchart illustrating the parameter update of the second neural network model in an image recognition uncertainty knowledge distillation method according to an embodiment of the present invention.

[0054] Figure 4 This is a schematic diagram of the structure of an image recognition uncertainty knowledge distillation system according to an embodiment of the present invention.

[0055] Figure 5 This is a flowchart of an image processing method according to an embodiment of the present invention.

[0056] Figure 6 This is a schematic diagram of the structure of an image processing system according to an embodiment of the present invention.

[0057] Figure 7 This is a schematic diagram of the structure of an image recognition device according to an embodiment of the present invention, which describes an image recognition uncertainty knowledge distillation method. Detailed Implementation

[0058] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0059] Example 1

[0060] like Figure 1 As shown, an image recognition uncertainty knowledge distillation method includes the following steps:

[0061] S100: Collect sample images, process and label them to obtain the first number of labeled training image samples;

[0062] In embodiments of the present invention, sample images are collected, processed, and labeled, divided into a first number of labeled training image samples, and saved to a data storage device. Labeling refers to manually identifying the categories of images or pixels in images according to task requirements. For example, in image classification tasks, this involves labeling the main object categories in the image; in object detection tasks, it involves labeling the number of pixels and categories of objects in the image. The collected images include, but are not limited to, face images and natural scene images.

[0063] S200: Select the first neural network model, train it using the first number of training image samples, update the parameter values ​​of the network model, obtain the trained first neural network model, input the training sample images into the first neural network model to obtain the sample feature representation of the intermediate layer of the model and the soft label information of the image processing output;

[0064] In an embodiment of the present invention, a first neural network model is selected and trained using a first number of labeled training image samples in step S100. The trained neural network model structure file and parameter values ​​are saved to a data storage device. The first neural network model can be any existing neural network model, including but not limited to ResNet, VGGnet, etc., or a new convolutional neural network model can be redesigned.

[0065] In this embodiment of the invention, a first neural network model is used to process labeled training image samples to obtain intermediate layer sample feature representations and soft label information. Specifically, the obtained intermediate layer sample feature representations of the first neural network model are formalized as follows: (c t h represents the number of channels representing the sample features. t w represents the height of the sample feature representation. t The soft label information obtained (representing the width of the sample feature representation) is formalized as follows: (C represents the number of target categories in the image).

[0066] S300: Select a second neural network model, process a first number of training image samples to obtain intermediate layer sample feature representations, perform uncertainty modeling on the intermediate layer sample feature representations obtained from the first neural network model, and obtain a first loss function;

[0067] In embodiments of the present invention, a second neural network model is selected. The second neural network model may be any existing neural network model, including but not limited to MobileNet, ShuffleNet, etc., or a new convolutional neural network model may be redesigned.

[0068] In this embodiment of the invention, optionally, as shown... Figure 2 As shown, step S300 involves selecting a second neural network model, processing a first number of training image samples to obtain intermediate layer sample feature representations, and performing uncertainty modeling on the intermediate layer sample feature representations obtained from the first neural network model to obtain a first loss function. Specific steps include:

[0069] S301: Select the intermediate convolutional layer of the second neural network model and input training image samples to obtain the feature representation of the intermediate layer samples of the second neural network model;

[0070] In an embodiment of the present invention, optionally, an intermediate convolutional layer of the second neural network model is selected, and training image samples are input to obtain the feature representation of the intermediate layer samples of the second neural network model. Specifically, the obtained feature representation of the intermediate layer samples of the second neural network model is formalized as follows: (c s h represents the number of channels representing the sample features.s w represents the height of the sample feature representation. s (Indicates the width of the sample feature representation)

[0071] S302: Calculate the semantic similarity between the channels of the intermediate layer sample feature representation of the second neural network model and the intermediate layer sample feature representation of the first neural network model;

[0072] In this embodiment of the invention, optionally, the semantic similarity between channels is calculated using cosine distance, and formally represented as:

[0073]

[0074]

[0075] In the formula, Let φ represent the feature dimension transformation function, and let φ represent the transformation function, such that the number of channels in the feature representation of the intermediate layer samples of the second neural network model is the same as the number of channels in the feature representation of the intermediate layer samples of the first neural network model. Let ||||2 represent the binary norm.

[0076] S303: Calculate the semantic similarity between the spatial feature representations of the intermediate layer samples of the second neural network model and the spatial feature representations of the intermediate layer samples of the first neural network model;

[0077] In this embodiment of the invention, optionally, the semantic similarity between spatial feature representations of sample feature representations is calculated using cosine distance, and formally expressed as:

[0078]

[0079] S304: Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, the mean value of the uncertainty of each sample feature expression in the first neural network model is obtained;

[0080] In this embodiment of the invention, optionally, the mean value of the uncertainty in the feature representation of each sample in the first neural network model is obtained, which is formally expressed as:

[0081] μ=diag(α ch )⊙diag(α sp (4)

[0082] In the formula, diag represents the operation of taking the diagonal value of the matrix, and ⊙ represents scalable element-wise multiplication.

[0083] S305: Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, a fully connected layer is used to further process and obtain the variance value of the uncertainty of each sample feature expression in the first neural network model;

[0084] In this embodiment of the invention, optionally, based on the calculated channel semantic similarity and spatial feature expression semantic similarity, a fully connected layer is used to further process and obtain the variance value of the uncertainty of the feature expression of each sample in the first neural network model, which is formally expressed as:

[0085]

[0086]

[0087]

[0088] In the formula, Indicates a fully connected layer. This indicates a convolution operation.

[0089] S306: Based on the mean and variance of the uncertainty expressed by the sample features obtained from the calculation, the uncertainty value is obtained by using the reparameterization technique, thereby obtaining the first loss function of knowledge distillation;

[0090] In this embodiment of the invention, optionally, the uncertainty value is obtained by using a reparameterization technique based on the mean and variance of the calculated sample feature expression uncertainty, and is formally expressed as:

[0091] ω=μ+σ×∈ (8)

[0092] In the formula, ∈ represents the standard Gaussian distribution.

[0093] In this embodiment of the invention, optionally, the first loss function of knowledge distillation is obtained, which is formally expressed as:

[0094]

[0095] In the formula, M represents the total number of feature locations in the intermediate layer feature representation of the first and second neural network models, and λ represents the loss weight.

[0096] S400: Using the image processing soft label information and training image samples output by the first neural network model, the first loss function is combined to obtain the overall loss function and update the parameters of the second neural network model to obtain the trained second neural network model;

[0097] In embodiments of the present invention, optionally, as shown below, Figure 3 As shown, step S400, which uses the image processing soft label information output by the first neural network model and training image samples, combined with the first loss function, to obtain the overall loss function and update the parameters of the second neural network model, specifically includes the following steps:

[0098] S401: Use the second neural network model to process the training sample images and output the processed predicted label information;

[0099] In this embodiment of the invention, optionally, a second neural network model is used to process the training sample images, and the processed predicted label information is output as follows: (C represents the number of target categories in the image).

[0100] S402: The second loss function is calculated using the image processing soft label information output by the first neural network model and the predicted label information output by the second neural network model;

[0101] In this embodiment of the invention, optionally, the image processing soft label information output by the first neural network model and the predicted label information output by the second neural network model are used to calculate the second loss function, which is formally expressed as:

[0102]

[0103] In the formula, This represents the softmax function.

[0104] S403: The third loss function is calculated using the predicted label information output by the second neural network model and the label information of the training image samples;

[0105] In this embodiment of the invention, optionally, a third loss function is calculated using the predicted label information output by the second neural network model and the label information of the training image samples, which is formally expressed as:

[0106]

[0107] S404: The total loss function is obtained by adding the first loss function, the second loss function, and the third loss function;

[0108] In this embodiment of the invention, optionally, the first loss function, the second loss function, and the third loss function are added together to obtain the overall loss function, which is formally expressed as:

[0109]

[0110] In the formula, T and γ represent the weight values ​​of the loss function.

[0111] Example 2

[0112] like Figure 4 As shown, this embodiment provides an image recognition uncertainty knowledge distillation system for implementing the method described in Embodiment 1, comprising:

[0113] The acquisition module 501 is used to collect sample images, process and label them to obtain a first number of labeled training image samples;

[0114] First update module 502: connected to the acquisition module 501, used to select a first neural network model, train it using a first number of training image samples, update the parameter values ​​of the network model, obtain a trained first neural network model, and input training sample images to the first neural network model to obtain the model's intermediate layer sample feature expression and soft label information output by image processing.

[0115] Uncertainty modeling module 503: connected to the acquisition module 501 and the first update module 502, used to select a second neural network model, process a first number of training image samples to obtain intermediate layer sample feature expressions, perform uncertainty modeling with the intermediate layer sample feature expressions obtained by the first neural network model, and obtain a first loss function;

[0116] The second update module 504 is connected to the acquisition module 501, the first update module 502, and the uncertainty modeling module 503. It uses the image processing soft-label information output by the first neural network model and training image samples, combined with the first loss function, to obtain the overall loss function and update the parameters of the second neural network model, thus obtaining the trained second neural network model.

[0117] In this embodiment of the invention, optionally, the uncertainty modeling module 503 is specifically used for:

[0118] Select the intermediate convolutional layer of the second neural network model and input training image samples to obtain the feature representation of the intermediate layer samples of the second neural network model;

[0119] Calculate the semantic similarity between the channels of the intermediate layer sample feature representations of the second neural network model and the intermediate layer sample feature representations of the first neural network model;

[0120] Calculate the semantic similarity between the spatial feature representations of the intermediate layer samples of the second neural network model and the spatial feature representations of the intermediate layer samples of the first neural network model;

[0121] Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, the mean value of the uncertainty of each sample feature expression in the first neural network model is obtained;

[0122] Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, a fully connected layer is used to further process and obtain the variance value of the uncertainty of each sample feature expression in the first neural network model.

[0123] Based on the mean and variance of the uncertainty expressed by the sample features obtained from the calculation, the uncertainty value is obtained by using the reparameterization technique, and thus the first loss function of knowledge distillation is obtained;

[0124] In this embodiment of the invention, optionally, the second update module 504 is specifically used for:

[0125] The training sample images are processed using a second neural network model, and the processed predicted label information is output.

[0126] The second loss function is calculated using the image processing soft label information output by the first neural network model and the predicted label information output by the second neural network model.

[0127] The third loss function is calculated using the predicted label information output by the second neural network model and the label information of the training image samples.

[0128] The overall loss function is obtained by adding the first loss function, the second loss function, and the third loss function.

[0129] Example 3

[0130] like Figure 5 As shown, this embodiment of the invention provides an image processing method for applying the image recognition uncertainty knowledge distillation described in Embodiment 1. The specific steps of the image processing method include:

[0131] S601: Obtain the second number of test image samples to be processed;

[0132] In this embodiment of the invention, the test image sample to be processed can be a natural landscape image, a human face image, etc.

[0133] S602: The second neural network model trained using the aforementioned image recognition uncertainty knowledge distillation method performs image recognition processing on the second number of test image samples to be processed;

[0134] S603: Obtain the recognition processing result and output it.

[0135] Example 4

[0136] like Figure 6 As shown, this embodiment provides an image processing system that implements the image processing method of Embodiment 3. The image processing system includes:

[0137] The acquisition module 701 is used to acquire a second number of test image samples to be processed.

[0138] The recognition processing module 702, connected to the acquisition module 701, is used to perform image recognition processing on the second number of test image samples to be processed using the second neural network model trained by the aforementioned image recognition uncertainty knowledge distillation system.

[0139] The recognition output module 703 is connected to the recognition processing module 702 and is used to acquire the image to be processed, and to perform image recognition processing on the second number of test image samples to be processed using the second neural network model trained by the aforementioned image recognition uncertainty knowledge distillation system, so as to obtain the recognition processing result and output it.

[0140] Example 5

[0141] Corresponding to the embodiment of the image recognition uncertainty knowledge distillation method described in Example 1, the present invention also provides an embodiment of an image recognition device for implementing the image recognition uncertainty knowledge distillation method.

[0142] See Figure 7 The image recognition device provided in this embodiment of the invention, which implements an image recognition uncertainty knowledge distillation method, includes an image acquisition unit, a memory, one or more processors, and an external output unit. The image acquisition unit is used to acquire a first and a second number of image samples. The memory stores executable code. When the one or more processors execute the executable code, they are used to implement the image recognition uncertainty knowledge distillation method in Embodiment 1 and the image processing method in Embodiment 3. The external output unit is used to output and display the image recognition processing result obtained by the image processing method in Embodiment 3.

[0143] The embodiments of the image recognition uncertainty knowledge distillation method and image recognition device of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The device embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 7 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including the image recognition uncertainty knowledge distillation method and image recognition device of the present invention. (Except for...) Figure 7 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0144] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0145] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0146] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the image recognition uncertainty knowledge distillation method and the image recognition method described above.

[0147] The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0148] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0149] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A knowledge distillation method for image recognition uncertainty, comprising the following steps: S100: Collect sample images, process and label them to obtain the first number of labeled training image samples; S200: Select the first neural network model, train it using the first number of training image samples, update the parameter values ​​of the network model, obtain the trained first neural network model, input the training sample images into the first neural network model to obtain the sample feature representation of the intermediate layer of the model and the soft label information of the image processing output; S300: Select a second neural network model, process a first number of training image samples to obtain intermediate layer sample feature representations, and perform uncertainty modeling on the intermediate layer sample feature representations obtained from the first neural network model to obtain a first loss function; specific steps include: S301: Select the intermediate convolutional layer of the second neural network model and input training image samples to obtain the feature representation of the intermediate layer samples of the second neural network model; S302: Calculate the semantic similarity between the channels of the intermediate layer sample feature representation of the second neural network model and the intermediate layer sample feature representation of the first neural network model; S303: Calculate the semantic similarity between the spatial feature representations of the intermediate layer samples of the second neural network model and the spatial feature representations of the intermediate layer samples of the first neural network model; S304: Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, the mean value of the uncertainty of each sample feature expression in the first neural network model is obtained; S305: Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, a fully connected layer is used to further process and obtain the variance value of the uncertainty of each sample feature expression in the first neural network model; S306: Based on the mean and variance of the uncertainty expressed by the sample features obtained from the calculation, the uncertainty value is obtained by using the reparameterization technique, thereby obtaining the first loss function of knowledge distillation; S400: Using the image processing soft label information and training image samples output by the first neural network model, the first loss function is combined to obtain the overall loss function and update the parameters of the second neural network model, thus obtaining the trained second neural network model.

2. The image recognition uncertainty knowledge distillation method according to claim 1, characterized in that, The steps of using the image processing soft label information output by the first neural network model and training image samples, combined with the first loss function, to obtain the overall loss function and update the parameters of the second neural network model include: S401: Use the second neural network model to process the training sample images and output the processed predicted label information; S402: The second loss function is calculated using the image processing soft label information output by the first neural network model and the predicted label information output by the second neural network model; S403: The third loss function is calculated using the predicted label information output by the second neural network model and the label information of the training image samples; S404: The overall loss function is obtained by adding the first loss function, the second loss function, and the third loss function.

3. An image recognition uncertainty knowledge distillation system, used to implement the image recognition uncertainty knowledge distillation method of claim 1, specifically comprising: The acquisition module is used to collect sample images, process and label them to obtain a first number of labeled training image samples; First update module: connected to the acquisition module, used to select a first neural network model, train it using a first number of training image samples, update the parameter values ​​of the network model, obtain a trained first neural network model, and input training sample images into the first neural network model to obtain the model's intermediate layer sample feature expression and soft label information output by image processing. Uncertainty modeling module: connected to the acquisition module and the first update module, used to select a second neural network model, process a first number of training image samples to obtain intermediate layer sample feature expressions, and perform uncertainty modeling with the intermediate layer sample feature expressions obtained from the first neural network model to obtain a first loss function; The second update module is connected to the acquisition module, the first update module, and the uncertainty modeling module. It is used to process the soft label information and training image samples output by the first neural network model, combine them with the first loss function to obtain the overall loss function, and update the parameters of the second neural network model to obtain the trained second neural network model.

4. The image recognition uncertainty knowledge distillation system according to claim 3, characterized in that, The uncertainty modeling module specifically includes: Select the intermediate convolutional layer of the second neural network model and input training image samples to obtain the feature representation of the intermediate layer samples of the second neural network model; Calculate the semantic similarity between the channels of the intermediate layer sample feature representations of the second neural network model and the intermediate layer sample feature representations of the first neural network model; Calculate the semantic similarity between the spatial feature representations of the intermediate layer samples of the second neural network model and the spatial feature representations of the intermediate layer samples of the first neural network model; Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, the mean value of the uncertainty of each sample feature expression in the first neural network model is obtained; Based on the calculated channel semantic similarity and spatial feature expression semantic similarity, a fully connected layer is used to further process and obtain the variance value of the uncertainty of each sample feature expression in the first neural network model. Based on the mean and variance of the uncertainty expressed by the sample features obtained through calculation, the uncertainty value is obtained by using the reparameterization technique, thereby obtaining the first loss function of knowledge distillation.

5. The image recognition uncertainty knowledge distillation system according to claim 3, characterized in that, The second update module specifically includes: The training sample images are processed using a second neural network model, and the processed predicted label information is output. The second loss function is calculated using the image processing soft label information output by the first neural network model and the predicted label information output by the second neural network model. The third loss function is calculated using the predicted label information output by the second neural network model and the label information of the training image samples. The overall loss function is obtained by adding the first loss function, the second loss function, and the third loss function.

6. An image processing method applying the image recognition uncertainty knowledge distillation method of claim 1, comprising: Obtain a second number of test image samples to be processed; The second neural network model trained using the aforementioned image recognition uncertainty knowledge distillation method is used to perform image recognition processing on the second number of test image samples to be processed. Obtain the recognition and processing results and output them.

7. An image processing system implementing the method of claim 6, characterized in that, include: The acquisition module is used to acquire a second number of test image samples to be processed; The recognition processing module, connected to the acquisition module, is used to perform image recognition processing on the second number of test image samples to be processed using the second neural network model trained by the aforementioned image recognition uncertainty knowledge distillation system. The recognition output module, connected to the recognition processing module, is used to acquire the image to be processed, and to perform image recognition processing on the second number of test image samples to be processed using the second neural network model trained by the aforementioned image recognition uncertainty knowledge distillation system, so as to obtain the recognition processing result and output it.

8. An image recognition processing device, comprising an image acquisition unit, a memory, one or more processors, and an external output unit, wherein the image acquisition unit is used to acquire a first and a second number of image samples, the memory stores executable code, and the one or more processors, when executing the executable code, are used to implement the image recognition uncertainty knowledge distillation method of claim 1 and the image processing method of claim 6, and the external output unit is used to output and display the image recognition processing result obtained by the image processing method of claim 7.

9. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the image recognition uncertainty knowledge distillation method of claim 1 and the image processing method of claim 6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the image recognition uncertainty knowledge distillation method of claim 1 and the image processing method of claim 6.

Images