Open set identification method and system based on image reconstruction

Abstract

The invention relates to the technical field of design picture open set recognition, in particular to an open set recognition method and system based on image reconstruction, and the method comprises the following steps: collecting classified images with the image type being RGB through an image collection device, and constructing an image classification data set; the image classification number is divided into a known class and an unknown class according to the proportion of 9: 1, the data set is divided into a closed data set and an open data set according to the image class classification result, and the closed data set is used for training; the method has the beneficial effects that the CNN encoder is arranged to perform down-sampling on an input image, the image is encoded, the depth semantic features of the image are extracted, down-sampling information is fused in the reconstruction process of the decoder to output information, the reconstruction error of the model is reduced, and the generalization ability of the model is improved; meanwhile, on the premise that the reconstruction error of the model on the known class is small and the reconstruction error of the model on the unknown class which is not trained by the model is large, the reconstruction error is converted into a boundary condition for judging the known class and the unknown class, and the unknown class is judged.

Description

Technical Field

[0001] The present invention relates to the technical field of open set recognition of design pictures, and in particular to an open set recognition method and system based on image reconstruction. Background Art

[0002] Open Set Recognition (OSR) is an important branch of machine learning. It aims to solve a key problem in the real world: how to deal with new categories that have not been seen in the training phase. This challenge is critical in many applications, such as security monitoring, natural language processing, and medical diagnosis. At present, there are mainly traditional methods based on support vector machines, distance metrics, and deep learning methods in the field of image open recognition. The open recognition model based on support vector machines improves generalization ability by maximizing the interval of the decision boundary, and can effectively distinguish known categories from unknown categories. However, when processing large-scale data sets, it may encounter problems of low computational efficiency and sensitivity to parameter selection. Improper parameter adjustment may lead to performance degradation. The distance metric-based method identifies unknown categories by calculating the distance between samples. This method is simple and intuitive, and has high computational efficiency. However, the distance metric method may be affected by the data distribution. If the data distribution of different categories overlaps, the recognition effect will be greatly reduced. The open recognition model based on deep learning can learn complex data representations and has good recognition effects on high-dimensional image data. They can automatically extract features through end-to-end training, reducing manual intervention. A large amount of labeled data is required for training, and the model has poor interpretability, making it difficult to track the cause when errors occur.

[0003] The proposed method in the field of general open recognition has made some progress, but it still faces many challenges in the field of image open recognition: (1) Distribution change: The model needs to be able to handle test data that is different from the distribution of training data, which involves complex statistical and probability problems; (2) Rejection options: The open recognition model needs to be able to reject categories that it is uncertain about, rather than mistakenly classifying them into known categories. Therefore, how to propose a method with high generalization ability, processing different image data, and a method that can clearly define the boundary between known and unknown categories has become a technical problem in the field of image open recognition. Summary of the invention

[0004] The purpose of the present invention is to provide an open set recognition method and system based on image reconstruction to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: an open set recognition method based on image reconstruction, the method comprising the following steps:

[0006] Using an image acquisition device to collect classified images, the image type is RGB, and an image classification data set is constructed;

[0007] The number of image classifications is divided into known classes and unknown classes in a ratio of 9:1. According to the image category classification results, the data set is divided into a closed data set and an open data set. The closed data set is used for training, and the open data set is used for testing.

[0008] Modify the irregularly sized images to the same size. The modified image (X) resolution H×W×C should not be less than 512×512×3;

[0009] The image X is input into the convolutional neural network CNN encoder to extract the image depth feature information for encoding. The encoding process will perform four downsamplings and output vector Avoid loss of information loss, where N is an arbitrary number;

[0010] X 4 Input to the CNN decoder for decoding. During the reconstruction process, CNN will perform four upsampling and reconstruct the output to the same dimension as X. Fusion X 1 , X 2 , X 3 , to prevent large reconstruction errors caused by information loss during the encoding process, the calculation process of the fusion process is as follows:

[0011] X′ 3 =F 3 (X 4 )+X 3

[0012] X′ 2 =F 2 (X′ 3 )+X 2

[0013] X′ 1 =F 1 (X′ 2 )+X 1

[0014]

[0015] Where F(x) is the convolutional network calculation process, the output after X′ is upsampled;

[0016] Calculate X and The Euclidean distance between them is used as the image reconstruction error and input into the extreme value distribution probability model to obtain the probability p to determine whether the current input is a known class;

[0017] X 4 Convert to a one-dimensional vector Xl Input into the classifier to obtain the classification result of the input image in the known class;

[0018] The comprehensive classification result judgment method is: if p is greater than 0.5, it is considered to be an unknown class, and if p is less than 0.5, it is considered to be a known class. The image category is the classification result obtained by the classifier.

[0019] Preferably, the method further comprises constructing a CNN encoder, using the GoogleNet deep learning network launched by Google, the GoogleNet model encodes the model through the CNN layer, and performs four downsamplings of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 in the encoding process, and outputs X 1 512×512×32 , X 2 256×256×288 , X 3 128×128×768 , X 4 64×64×2038 ;

[0020] Construct a decoder, which consists of a convolutional layer and a Relu layer. 4 64×64×2038 After convolution calculation and four times upsampling output Each upsampling combines the feature information output by the fusion decoder;

[0021] Construct a classifier, which consists of a linear layer, an activation layer, and a softmax layer. 4 64×64×2038 Convert to a one-dimensional vector X l As input, it passes through multiple linear layers and outputs N is the number of known classes.

[0022] Preferably, the method also includes constructing an extreme value distribution probability model, performing reconstruction training on images in a training set of known classes, and obtaining image reconstruction decoder and encoder weights; using the trained reconstruction model to reconstruct images of known classes in the training set, and obtaining reconstruction errors as a set for constructing an extreme value probability distribution model; and using the training set and extreme value theory to construct a reconstruction error extreme value probability model.

[0023] Preferably, the optimal extreme value distribution model theory selects the basic generalized extreme value theory, and the construction process is as follows:

[0024] Input all the images in the training data set into the trained model to obtain the image reconstruction error of the data set, and build the image reconstruction error distribution probability model based on the generalized extreme value distribution theory;

[0025] The extreme value theory model is introduced as follows:

[0026] For a given sufficiently large threshold ω and a class of distributions denoted by D, the class has n samples, {D 1 ,D 2 ,…,D n}, GEV can be described as the following equation, denoted as p evt :

[0027]

[0028] Where μ is the location parameter, σ is the scale parameter, ξ is the shape parameter, μ,ξ∈R,σ>0; p evt is the generalized extreme value distribution function CFD, so p evt It represents the probability of the D-type extreme value distribution.

[0029] Preferably, the model training process includes: the encoder is used as a common part of the image classification and reconstruction process, the encoder, decoder, and classifier weights are trained together, and the training loss function is as follows:

[0030] The classification loss function calculates the cross entropy loss function by comparing the result y of the classifier data with the target value.

[0031] The calculation formula is as follows:

[0032]

[0033] N is the batch size, Y is the number of categories;

[0034] The image reconstruction loss function calculates the Euclidean distance between the decoder output and the input image as the reconstruction

[0035]

[0036] The total training loss function calculation formula is as follows:

[0037]

[0038] λ c ,λ r is the ratio of classification loss to reconstruction loss, and the values ​​in this paper are 0.4 and 0.6.

[0039] An open set recognition system based on image reconstruction, the system comprising:

[0040] The image acquisition module uses an image acquisition device to collect classified images, the image type is RGB, and constructs an image classification data set;

[0041] Image classification module: divides the number of image classifications into known classes and unknown classes in a ratio of 9:1. According to the image category classification results, the data set is divided into a closed data set and an open data set. The closed data set is used for training, and the open data set is used for testing.

[0042] Image resizing module, resizes irregular images to the same size. The resolution of the resized image (X) H×W×C should not be less than 512×512×3;

[0043] The encoding module inputs the image X into the convolutional neural network CNN encoder to extract the image depth feature information for encoding. The encoding process will perform four downsampling and output vector Avoid loss of information loss, where N is an arbitrary number;

[0044] Decoding module, X 4 Input to the CNN decoder for decoding. During the reconstruction process, CNN will perform four upsampling and reconstruct the output to the same dimension as X. Fusion X 1 , X 2 , X 3 , to prevent large reconstruction errors caused by information loss during the encoding process, the calculation process of the fusion process is as follows:

[0045] X′ 3 =F 3 (X 4 )+X 3

[0046] X′ 2 =F 2 (X′ 3 )+X 2

[0047] X′ 1 =F 1 (X′ 2 )+X 1

[0048]

[0049] Where F(x) is the convolutional network calculation process, the output after X′ is upsampled;

[0050] Calculation module, calculates X and The Euclidean distance between them is used as the image reconstruction error and input into the extreme value distribution probability model to obtain the probability p to determine whether the current input is a known class;

[0051] Conversion module, X 4 Convert to a one-dimensional vector X lInput into the classifier to obtain the classification result of the input image in the known class;

[0052] The discrimination module comprehensively classifies the results: if p is greater than 0.5, it is considered to be an unknown class; if p is less than 0.5, it is considered to be a known class. The image category is the classification result obtained by the classifier.

[0053] Preferably, it also includes a CNN encoder construction module, using the GoogleNet deep learning network launched by Google. The GoogleNet model encodes the model through the CNN layer. During the encoding process, four downsamplings of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 are performed in sequence to output X 1 512×512×32 , X 2 256×256×288 , X 3 128×128×768 , X 4 64×64×2038 ;

[0054] Decoder building block, the decoder consists of convolutional layers, ReLU layers, and the input vector X 4 64×64×2038 After convolution calculation and four times upsampling output Each upsampling combines the feature information output by the fusion decoder;

[0055] Classifier building block, the classifier consists of a linear layer, an activation layer, and a softmax layer. 4 64×64×2038 Convert to a one-dimensional vector X l As input, it passes through multiple linear layers and outputs N is the number of known classes.

[0056] Preferably, it also includes constructing an extreme value distribution probability model, performing reconstruction training on images in a training set of known classes, and obtaining image reconstruction decoder and encoder weights; using the trained reconstruction model to reconstruct images of known classes in the training set, and obtaining reconstruction errors as a set for constructing an extreme value probability distribution model; and using the training set and extreme value theory to construct a reconstruction error extreme value probability model.

[0057] Preferably, the optimal extreme value distribution model theory selects the basic generalized extreme value theory, and the construction process is as follows:

[0058] Input all the images in the training data set into the trained model to obtain the image reconstruction error of the data set, and build the image reconstruction error distribution probability model based on the generalized extreme value distribution theory;

[0059] The extreme value theory model is introduced as follows:

[0060] For a given sufficiently large threshold ω and a class of distributions denoted by D, the class has n samples, {D 1 ,D 2 ,…,D n}, GEV can be described as the following equation, denoted as p evt :

[0061]

[0062] Where μ is the location parameter, σ is the scale parameter, ξ is the shape parameter, μ,ξ∈R,σ>0; p evt is the generalized extreme value distribution function CFD, so p evt It represents the probability of the D-type extreme value distribution.

[0063] Preferably, the model training process includes: the encoder is used as a common part of the image classification and reconstruction process, the encoder, decoder, and classifier weights are trained together, and the training loss function is as follows:

[0064] The classification loss function calculates the cross entropy loss function by comparing the result y of the classifier data with the target value.

[0065] The calculation formula is as follows:

[0066]

[0067] N is the batch size, Y is the number of categories;

[0068] The image reconstruction loss function calculates the Euclidean distance between the decoder output and the input image as the reconstruction

[0069]

[0070] The total training loss function calculation formula is as follows:

[0071]

[0072] λ c ,λ r is the ratio of classification loss to reconstruction loss, and the values ​​in this paper are 0.4 and 0.6.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] The open set recognition method and system based on image reconstruction proposed in the present invention downsample the input image to encode the image by setting a CNN encoder to extract the deep semantic features of the image. The decoder reconstruction process fuses the downsampled information to output information, thereby reducing the reconstruction error of the model and improving the generalization ability of the model. At the same time, under the assumption that the reconstruction error of the model for known classes is small and the reconstruction error for unknown classes that have not been trained by the model is large, the reconstruction error is converted into a boundary condition for judging known classes and unknown classes, thereby realizing the judgment of unknown classes. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a flow chart of the steps of the method of the present invention;

[0076] Figure 2 This is an example of the decoder structure of the present invention;

[0077] Figure 3 This is an example of the classifier structure of the present invention. DETAILED DESCRIPTION

[0078] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] For example, see Figures 1 to 3 The present invention provides a technical solution: an open set recognition method based on image reconstruction, the method comprising the following steps:

[0080] Using an image acquisition device to collect classified images, the image type is RGB, and an image classification data set is constructed;

[0081] The number of image classifications is divided into known classes and unknown classes in a ratio of 9:1. According to the image category classification results, the data set is divided into a closed data set and an open data set. The closed data set is used for training, and the open data set is used for testing.

[0082] Modify the irregularly sized images to the same size. The modified image (X) resolution (dimension) H×W×C should not be less than 512×512×3;

[0083] The image X is input into the convolutional neural network (CNN) encoder to extract the image depth feature information for encoding. The encoding process will be downsampled four times and the output vector Avoid loss of information loss, where N is an arbitrary number;

[0084] X 4 Input to the CNN decoder for decoding. During the reconstruction process, CNN will perform four upsampling and reconstruct the output to the same dimension as X. To merge X 1 , X 2 , X 3 , to prevent large reconstruction errors caused by information loss during the encoding process, the calculation process of the fusion process is as follows:

[0085] X′ 3 =F 3 (X 4 )+X 3

[0086] X′ 2 =F 2 (X′ 3 )+X 2

[0087] X′ 1 =F 1 (X′ 2 )+X 1

[0088]

[0089] Where F(x) is the convolutional network calculation process, and X′ is the output after upsampling

[0090] Calculate X and The Euclidean distance between them is used as the image reconstruction error and input into the extreme value distribution probability model to obtain the probability p to determine whether the current input is a known class;

[0091] X 4 Convert to a one-dimensional vector X l Input into the classifier to obtain the classification result of the input image in the known class;

[0092] The comprehensive classification result judgment method is: if p is greater than 0.5, it is considered to be an unknown class, and if p is less than 0.5, it is considered to be a known class. The image category is the classification result obtained by the classifier.

[0093] Construct a CNN encoder and use the GoogleNet deep learning network launched by Google. The GoogleNet model encodes the model through the CNN layer. During the encoding process, it performs four downsamplings of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 in sequence, and outputs X 1 512×512×32 , X 2 256×256×288 , X 3128×128×768 , X 4 64×64×2038 .

[0094] The decoder consists of a convolutional layer, a ReLU layer, etc., and processes the input vector X 4 64×64×2038 After convolution calculation and four times upsampling output Each upsampling combines the output feature information of the fusion decoder. The structure can be referred to Figure 2 , other decoder structures are also possible;

[0095] The classifier consists of a linear layer, an activation layer, and a softmax layer. 4 64×64×2038 Convert to a one-dimensional vector X l As input, it passes through multiple linear layers and outputs N is the number of known classes. Optional Figure 3 structure.

[0096] The construction process of the extreme value distribution probability model: reconstruct the images in the known class training set to obtain the image reconstruction decoder and encoder weights; use the trained reconstruction model to reconstruct the images of the known classes in the training set to obtain the reconstruction error as a set for constructing the extreme value probability distribution model; use the training set and extreme value theory to construct the reconstruction error extreme value probability model.

[0097] The optimal extreme value distribution model theory selects the basic generalized extreme value theory, and the construction process is as follows:

[0098] Input all the images in the training data set into the trained model to obtain the image reconstruction error of the data set, and build an image reconstruction error distribution probability model based on the generalized extreme value distribution theory. The generalized extreme value theory model is introduced as follows:

[0099] For a given sufficiently large threshold ω and a class of distributions denoted by D, the class has n samples, {D 1 ,D 2 ,…,D n}, GEV can be described as the following equation, denoted as p evt :

[0100]

[0101] Where μ is the location parameter, σ is the scale parameter, ξ is the shape parameter, μ,ξ∈R,σ>0. evt is the generalized extreme value distribution function (CFD), so p evtIt represents the probability of the extreme value distribution of class D. The obtained probability score can be used to determine whether a given sample is a known class or an unknown class. In this example, the known class is the plant species and the unknown class is other plants. The parameters ξ and σ are estimated by the maximum value in the known class sample data, and the estimation method is maximum likelihood estimation. The selection process of the maximum value estimation data of plant image reconstruction error data: First, sort the image reconstruction error samples from small to large; second, select the five largest values ​​after sorting to construct the extreme value distribution model.

[0102] Description of the model training process: The encoder is a common part of the image classification and reconstruction process. The encoder, decoder, and classifier weights are trained together. The training loss function is as follows:

[0103] The classification loss function calculates the cross entropy loss function by comparing the result y of the classifier data with the target value.

[0104] The calculation formula is as follows:

[0105]

[0106] N is the batch size and Y is the number of categories.

[0107] The image reconstruction loss function calculates the Euclidean distance between the decoder output and the input image as the reconstruction

[0108]

[0109] The total training loss function calculation formula is as follows:

[0110]

[0111] λ c ,λ r This paper uses 0.4 and 0.6 as the ratio of classification loss and reconstruction loss.

[0112] Loss function during model training It consists of classification loss and leaf reconstruction loss. The classification loss adopts the cross entropy loss function.

[0113] Embodiment 2, based on Embodiment 1, proposes an open set recognition system based on image reconstruction according to the open set recognition method based on image reconstruction according to any one of claims 1 to 5, characterized in that: the system comprises:

[0114] The image acquisition module uses an image acquisition device to collect classified images, the image type is RGB, and constructs an image classification data set;

[0115] Image classification module: divides the number of image classifications into known classes and unknown classes in a ratio of 9:1. According to the image category classification results, the data set is divided into a closed data set and an open data set. The closed data set is used for training, and the open data set is used for testing.

[0116] Image resizing module, resizes irregular images to the same size. The resolution of the resized image (X) H×W×C should not be less than 512×512×3;

[0117] The encoding module inputs the image X into the convolutional neural network CNN encoder to extract the image depth feature information for encoding. The encoding process will perform four downsampling and output vector Avoid loss of information loss, where N is an arbitrary number;

[0118] Decoding module, X 4 Input to the CNN decoder for decoding. During the reconstruction process, CNN will perform four upsampling and reconstruct the output to the same dimension as X. Fusion X 1 , X 2 , X 3 , to prevent large reconstruction errors caused by information loss during the encoding process, the calculation process of the fusion process is as follows:

[0119] X′ 3 =F 3 (X 4 )+X 3

[0120] X′ 2 =F 2 (X′ 3 )+X 2

[0121] X′ 1 =F 1 (X′ 2 )+X 1

[0122]

[0123] Where F(x) is the convolutional network calculation process, the output after X′ is upsampled;

[0124] Calculation module, calculates X and The Euclidean distance between them is used as the image reconstruction error and input into the extreme value distribution probability model to obtain the probability p to determine whether the current input is a known class;

[0125] Conversion module, X 4 Convert to a one-dimensional vector X lInput into the classifier to obtain the classification result of the input image in the known class;

[0126] The discrimination module comprehensively classifies the results: if p is greater than 0.5, it is considered to be an unknown class; if p is less than 0.5, it is considered to be a known class. The image category is the classification result obtained by the classifier.

[0127] It also includes a CNN encoder building module, which uses the GoogleNet deep learning network launched by Google. The GoogleNet model encodes the model through the CNN layer. During the encoding process, it performs four downsamplings of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 in sequence, and outputs X 1 512×512×32 , X 2 256×256×288 , X 3 128×128×768 , X 4 64×64×2038 ;

[0128] Decoder building block, the decoder consists of convolutional layers, ReLU layers, and the input vector X 4 64×64×2038 After convolution calculation and four times upsampling output Each upsampling combines the feature information output by the fusion decoder;

[0129] Classifier building block, the classifier consists of a linear layer, an activation layer, and a softmax layer. 4 64×64×2038 Convert to a one-dimensional vector X l As input, it passes through multiple linear layers and outputs N is the number of known classes.

[0130] It also includes constructing an extreme value distribution probability model, reconstructing images in a training set of known classes, and obtaining image reconstruction decoder and encoder weights; using the trained reconstruction model to reconstruct images of known classes in the training set, and obtaining the reconstruction error as a set for constructing an extreme value probability distribution model; and using the training set and extreme value theory to construct a reconstruction error extreme value probability model.

[0131] The optimal extreme value distribution model theory selects the basic generalized extreme value theory, and the construction process is as follows:

[0132] Input all the images in the training data set into the trained model to obtain the image reconstruction error of the data set, and build the image reconstruction error distribution probability model based on the generalized extreme value distribution theory;

[0133] The extreme value theory model is introduced as follows:

[0134] For a given sufficiently large threshold ω and a class of distributions denoted by D, the class has n samples, {D 1 ,D 2 ,…,D n}, GEV is described by the following equation, denoted as p evt :

[0135]

[0136] Where μ is the location parameter, σ is the scale parameter, ξ is the shape parameter, μ,ξ∈R,σ>0; p evt is the generalized extreme value distribution function CFD, so p evt It represents the probability of the D-type extreme value distribution.

[0137] The model training process includes: the encoder is the common part of the image classification and reconstruction process, the encoder, decoder, and classifier weights are trained together, and the training loss function is as follows:

[0138] The classification loss function calculates the cross entropy loss function by comparing the result y of the classifier data with the target value. The calculation formula is as follows:

[0139]

[0140] N is the batch size, Y is the number of categories;

[0141] The image reconstruction loss function calculates the Euclidean distance between the decoder output and the input image as the reconstruction

[0142]

[0143] The total training loss function calculation formula is as follows:

[0144]

[0145] λ c ,λ r is the ratio of classification loss to reconstruction loss, and the values ​​in this paper are 0.4 and 0.6.

[0146] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An open set recognition method based on image reconstruction, characterized in that: The method comprises the following steps: Using an image acquisition device to collect classified images, the image type is RGB, and an image classification data set is constructed; The number of image classifications is divided into known classes and unknown classes in a ratio of 9:

1. According to the image category classification results, the data set is divided into a closed data set and an open data set. The closed data set is used for training, and the open data set is used for testing. Modify the irregularly sized images to the same size. The modified image (X) resolution H×W×C should not be less than 512×512×3; The image X is input into the convolutional neural network CNN encoder to extract the image depth feature information for encoding. The encoding process will perform four downsampling and output vector Avoid loss of information loss, where N is an arbitrary number; Input X4 into the CNN decoder for decoding. During the reconstruction process, CNN will perform four upsampling and reconstruct the output to have the same dimension as X. Fusion of X1, X2, and X3 prevents large reconstruction errors caused by information loss during encoding. The calculation process of the fusion process is as follows: X′3=F3(X4)+X3 X′2=F2(X′3)+X2 X1′=F1(X2′)+X1 Where F(x) is the convolutional network calculation process, the output after X′ is upsampled; Calculate X and The Euclidean distance between them is used as the image reconstruction error and input into the extreme value distribution probability model to obtain the probability p to determine whether the current input is a known class; Convert X4 to a one-dimensional vector X l Input into the classifier to obtain the classification result of the input image in the known class; The comprehensive classification result judgment method is: if p is greater than 0.5, it is considered to be an unknown class, and if p is less than 0.5, it is considered to be a known class. The image category is the classification result obtained by the classifier.

2. The open set recognition method based on image reconstruction according to claim 1, characterized in that: The method also includes constructing a CNN encoder, using the GoogleNet deep learning network launched by Google. The GoogleNet model encodes the model through the CNN layer. During the encoding process, four downsamplings are performed in sequence: 1 / 2, 1 / 4, 1 / 8, and 1 / 16, and the outputs are X1 and X2, respectively. 512 ×512×32 , X2 256×256×288 、X3 128×128×768 、X4 64×64×2038 ; Construct a decoder, which consists of a convolutional layer and a Relu layer. 64×64×2038 After convolution calculation and four times upsampling output Each upsampling combines the feature information output by the fusion decoder; Construct a classifier, which consists of a linear layer, an activation layer, and a softmax layer. 64×64×2038 Convert to a one-dimensional vector X l As input, it passes through multiple linear layers and outputs N is the number of known classes.

3. The open set recognition method based on image reconstruction according to claim 1, characterized in that: The method also includes constructing an extreme value distribution probability model, performing reconstruction training on images in a training set of known classes, and obtaining image reconstruction decoder and encoder weights; using the trained reconstruction model to reconstruct images of known classes in the training set, and obtaining reconstruction errors as a set for constructing an extreme value probability distribution model; and using the training set and extreme value theory to construct a reconstruction error extreme value probability model.

4. The open set recognition method based on image reconstruction according to claim 3, characterized in that: The optimal extreme value distribution model theory selects the basic generalized extreme value theory, and the construction process is as follows: Input all the images in the training data set into the trained model to obtain the image reconstruction error of the data set, and build an image reconstruction error distribution probability model based on the generalized extreme value distribution theory; The extreme value theory model is introduced as follows: For a given sufficiently large threshold ω and a class of distributions denoted by D, the class has n samples, {D1,D2,…,D n }, GEV is described by the following equation, denoted as p evt : Where μ is the location parameter, σ is the scale parameter, ξ is the shape parameter, μ,ξ∈R,σ>0; p evt is the generalized extreme value distribution function CFD, so p evt It represents the probability of the D-type extreme value distribution.

5. The open set recognition method based on image reconstruction according to claim 4, characterized in that: The model training process includes: the encoder is the common part of the image classification and reconstruction process, the encoder, decoder, and classifier weights are trained together, and the training loss function is as follows: The classification loss function calculates the cross entropy loss function by comparing the result y of the classifier data with the target value. The calculation formula is as follows: N is the batch size, Y is the number of categories; The image reconstruction loss function calculates the Euclidean distance between the decoder output and the input image as the reconstruction The total training loss function calculation formula is as follows: λ c ,λ r is the ratio of classification loss to reconstruction loss, and the values ​​in this paper are 0.4 and 0.

6.

6. An open set recognition system based on image reconstruction according to the open set recognition method based on image reconstruction according to any one of claims 1 to 5, characterized in that: The system comprises: The image acquisition module uses an image acquisition device to collect classified images, the image type is RGB, and constructs an image classification data set; Image classification module: divides the number of image classifications into known classes and unknown classes in a ratio of 9:

1. According to the image category classification results, the data set is divided into a closed data set and an open data set. The closed data set is used for training, and the open data set is used for testing. Image resizing module, resizes irregular images to the same size. The resolution of the resized image (X) H×W×C should not be less than 512×512×3; The encoding module inputs the image X into the convolutional neural network CNN encoder to extract the image depth feature information for encoding. The encoding process will perform four downsampling and output vector Avoid loss of information loss, where N is an arbitrary number; The decoding module inputs X4 into the CNN decoder for decoding. During the reconstruction process, CNN will perform four upsampling and reconstruct the output to have the same dimension as X. Fusion of X1, X2, and X3 prevents large reconstruction errors caused by information loss during encoding. The calculation process of the fusion process is as follows: X3′3=F3(X4)+X3 X′2=F2(X′3)+X2 X1′=F1(X2′)+X1 Where F(x) is the convolutional network calculation process, the output after X′ is upsampled; Calculation module, calculates X and The Euclidean distance between them is used as the image reconstruction error and input into the extreme value distribution probability model to obtain the probability p to determine whether the current input is a known class; Conversion module, converts X4 into a one-dimensional vector X l Input into the classifier to obtain the classification result of the input image in the known class; The discrimination module comprehensively classifies the results: if p is greater than 0.5, it is considered to be an unknown class; if p is less than 0.5, it is considered to be a known class. The image category is the classification result obtained by the classifier.

7. The open set recognition system based on image reconstruction according to claim 6, characterized in that: It also includes a CNN encoder building module, which uses the GoogleNet deep learning network launched by Google. The GoogleNet model encodes the model through the CNN layer. During the encoding process, it performs four downsamplings of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 in sequence, and outputs X1 512 ×512×32 , X2 256×256×288 、X3 128×128×768 、X4 64×64×2038 ; Decoder building module, the decoder consists of convolutional layer, Relu layer, for the input vector X4 64×64×2038 After convolution calculation and four times upsampling output Each upsampling combines the feature information output by the fusion decoder; Classifier building block, the classifier consists of a linear layer, an activation layer, and a softmax layer, X4 64×64×2038 Convert to a one-dimensional vector X l As input, it passes through multiple linear layers and outputs N is the number of known classes.

8. The open set recognition system based on image reconstruction according to claim 6, characterized in that: It also includes constructing an extreme value distribution probability model, reconstructing images in a training set of known classes, and obtaining image reconstruction decoder and encoder weights; using the trained reconstruction model to reconstruct images of known classes in the training set, and obtaining the reconstruction error as a set for constructing an extreme value probability distribution model; and using the training set and extreme value theory to construct a reconstruction error extreme value probability model.

9. The open set recognition system based on image reconstruction according to claim 8, characterized in that: The optimal extreme value distribution model theory selects the basic generalized extreme value theory, and the construction process is as follows: Input all the images in the training data set into the trained model to obtain the image reconstruction error of the data set, and build an image reconstruction error distribution probability model based on the generalized extreme value distribution theory; The extreme value theory model is introduced as follows: For a given sufficiently large threshold ω and a class of distributions denoted by D, the class has n samples, {D1,D2,…,D n }, GEV is described by the following equation, denoted as p evt : Where μ is the location parameter, σ is the scale parameter, ξ is the shape parameter, μ,ξ∈R,σ>0; p evt is the generalized extreme value distribution function CFD, so p evt It represents the probability of the D-type extreme value distribution.

10. The open set recognition system based on image reconstruction according to claim 9, characterized in that: The model training process includes: the encoder is the common part of the image classification and reconstruction process, the encoder, decoder, and classifier weights are trained together, and the training loss function is as follows: The classification loss function calculates the cross entropy loss function by comparing the result y of the classifier data with the target value. The calculation formula is as follows: N is the batch size, Y is the number of categories; The image reconstruction loss function calculates the Euclidean distance between the decoder output and the input image as the reconstruction The total training loss function calculation formula is as follows: λ c ,λ r is the ratio of classification loss to reconstruction loss, and the values ​​in this paper are 0.4 and 0.6.

Images