A method for extracting appearance features of small visual targets

Abstract

The present invention discloses a method for extracting appearance features of a visual small target, and the method is as follows: a training data set image is composed of four target images, namely, large, medium, small and standard; the large and medium target images are scaled to the same size as the standard target image, which is the first image; the four sides of the small target image are filled with gray to obtain a second image, which is placed in a first neural network model for deconvolution to obtain a third image; the third image is placed in a second neural network model for appearance feature extraction to obtain a feature vector of the third image; the feature vector of the first image is extracted using the second neural network model; the cosine distance between the extracted two-dimensional feature vectors is used to judge the similarity between the first and third images, and the extraction of appearance features of the visual small target is completed. The present invention uses a deconvolution method to enlarge the small target image, and then uses a convolutional network to extract the target appearance information, and trains parameters in an adversarial manner to achieve the extraction of appearance information of the small target image.

Description

Technical Field

[0001] The invention belongs to the technical field of visual target tracking, and in particular is a method for extracting appearance features of a small visual target. Background Art

[0002] Appearance features mainly include the skeleton, decoration, and appearance of objects. They are one of the main features of objects and are widely used in many fields. In the field of visual target tracking, appearance features are an important feature of the target and an important basis for matching the target with the trajectory. In the field of face recognition, appearance features are the key to face matching.

[0003] Small target images are often at a disadvantage in the use of appearance features. The use of appearance features often starts with the target recognition algorithm. First, the target location is screened out from the image through the target recognition algorithm. Then, the image at the target location is segmented from the image. Next, the captured image is scaled to the pixel size required by the neural network entry. Finally, the scaled image is placed in the neural network to extract appearance features. Although the captured image removes unnecessary background information, it also makes the pixel sizes of the captured images different, and the image needs to be scaled before it can be uniformly placed in the neural network model. The image with a size close to the pixel requirement of the neural network entry has a smaller scaling ratio, and the image information loss is less, but the small target image is too different from the pixel requirement of the neural network entry. If it is scaled directly, it will cause large-scale image distortion and most of the image appearance information will be lost.

[0004] In summary, the existing appearance feature extraction technology has the following shortcomings: Since the pixel size of small target images is too different from the pixel requirements of the neural network entrance, directly changing the pixels to the required size of the neural network entrance through scaling transformation will lose most of the information, which will affect the appearance matching of small target images. Summary of the invention

[0005] The purpose of the present invention is to provide a method for extracting the appearance features of visual small targets, which uses a deconvolution method to enlarge the small target image, then uses a convolutional network to extract the target appearance information, and trains parameters in an adversarial manner, thereby achieving the protection and extraction of the appearance information of the small target image.

[0006] The technical solution to achieve the purpose of the present invention is: a method for extracting appearance features of a small visual target, comprising the following steps:

[0007] Step 1: Collect all training dataset images, which consist of large target images, medium target images, small target images, and target images;

[0008] Step 2: Scale the large target image and the medium target image to the same size as the target image using bilinear interpolation, which is called the first image.

[0009] Step 3: Fill the four sides of the small target image with gray to make the target pixel become 64*32, and obtain the second image;

[0010] Step 4: Place the second image into the first neural network model for deconvolution to obtain a third image, where the pixel size of the third image is the same as that of the first image;

[0011] Step 5: Place the third image into the second neural network model to extract appearance features, and obtain a feature vector of the third image, where the feature vector length is 128 dimensions;

[0012] Step 6: Also use the second neural network model to extract a 128-dimensional feature vector of the first image;

[0013] Step 7: Use the 128-dimensional feature vectors extracted from steps 5 and 6

[0014] Compared with the prior art, the present invention has the following significant advantages: (1) by using deconvolution technology and adversarial networks, the small target image is enlarged, and the image appearance information is effectively enhanced and protected; (2) by using a convolutional neural network to extract the appearance features of the enlarged image, and using the cosine distance to describe the degree of similarity between the appearances of the two images, it better reflects the degree of similarity between the image appearances; (3) by using the network adversarial method to train parameters, the generator network is trained as the enlargement function of the small target image, and the discriminator network is trained as the extractor of the target feature vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the method for extracting the appearance features of a small visual target of the present invention.

[0016] Figure 2 This is a schematic diagram of the file format after the data set is classified.

[0017] Figure 3 It is a schematic diagram of the deconvolution model.

[0018] Figure 4 It is a schematic diagram of the appearance feature extraction network.

[0019] Figure 5 It is a schematic diagram of the adversarial network generator model.

[0020] Figure 6 It is a schematic diagram of the adversarial network discriminator model. DETAILED DESCRIPTION

[0021] Combination Figure 1 The present invention provides a method for extracting appearance features of a small visual target, comprising the following steps:

[0022] Step 1: Collect all training dataset images, which consist of large target images, medium target images, small target images, and target images;

[0023] Step 2: Scale the large target image and the medium target image to the same size as the target image using bilinear interpolation, which is called the first image.

[0024] Step 3: Fill the four sides of the small target image with gray to make the target pixel become 64*32, and obtain the second image;

[0025] Step 4: Place the second image into the first neural network model for deconvolution to obtain a third image, where the pixel size of the third image is the same as that of the first image;

[0026] Step 5: Place the third image into the second neural network model to extract appearance features, and obtain a feature vector of the third image, where the feature vector length is 128 dimensions;

[0027] Step 6: Also use the second neural network model to extract a 128-dimensional feature vector of the first image;

[0028] Step 7: Use the cosine distance between the 128-dimensional feature vectors extracted in steps 5 and 6 to judge the similarity between the first image and the third image. That is, use the cosine distance between the 128-dimensional feature vectors to judge the similarity between all data sets in step 1, and complete the extraction of the appearance features of the small visual target.

[0029] As a specific example, in step 1, images of the same car model are placed in the same folder, and the car model images in the same folder have different sizes.

[0030] As a specific example, in step 1, the target image is defined as an image with pixels equal to 128*64, a large target image is defined as an image with length and width pixels greater than 128 and 64 respectively, a small target image is defined as an image with length and width pixels less than 64 and 32 respectively, and the rest of the images are medium target images.

[0031] As a specific example, the first neural network model in step 4 and the second neural network model in step 5 constitute an adversarial model.

[0032] As a specific example, the first neural network model is a generator model in an adversarial network, which consists of the following parts:

[0033] 1. A feature extraction convolutional neural network, for extracting features of the second image;

[0034] 2. A two-layer deconvolution network is used to visualize the features into an image of the target image size.

[0035] As a specific example, the second neural network model is a discriminator model in an adversarial network, which is composed of a three-layer convolutional neural network model and is used to obtain a feature vector of an image.

[0036] As a specific example, an adversarial model consisting of a first neural network model and a second neural network model is trained in an adversarial manner, and the specific steps are as follows:

[0037] (1) Take any image in the first image and put it into the second neural network to obtain a 128-dimensional feature vector, which is called the first feature vector;

[0038] (2) Take the small target image in the same folder as the image in (1), and obtain a 128-dimensional feature vector according to steps 3, 4, and 5, which is called the second feature vector;

[0039] (3) Fix the discriminator network parameters and update the generator network parameters using the cosine distance between the first eigenvector and the second eigenvector as the loss;

[0040] (4) The small target image in (2) is obtained again according to the method of step 3, step 4, and step 5 to obtain a 128-dimensional feature vector, which is called the third feature vector;

[0041] (5) Fix the generator network parameters and update the discriminator network parameters using the cosine distance between the negative vector of the first eigenvector and the fourth eigenvector as the loss;

[0042] (6) Repeat the above steps (1) to (5) until the loss function in (3) is less than 0.2 and the loss function in (5) is also less than 0.2.

[0043] As a specific example, the formula for the cosine distance in step 7 is:

[0044]

[0045] Among them, x, y are 128-dimensional feature vectors, and i is the value of the feature vector of the i-th dimension.

[0046] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Example

[0048] In order to solve the problem that in the extraction of target appearance features, the small target image and the large target image are often scaled uniformly using the image scaling function, resulting in the distortion of the small target image and the loss of appearance information, the present invention proposes to use the deconvolution method to enlarge the small target image and use the adversarial network to train the parameters. The specific steps are as follows:

[0049] S1: Take photos of the smart car from different angles and capture images without background;

[0050] S2: Smart cars of the same type are grouped into one category and put into one folder to obtain a smart car dataset. The images in the same file have different sizes and can be divided into small target images, medium target images, large target images and target images according to pixel size.

[0051] S3: Fill the four sides of the small target image with gray to make its pixels become 64*32, which is called image 1; use bilinear interpolation to enlarge the large target image and the medium target image to make their pixels become 128*64; which is called image 2.

[0052] S4: put image 1 into Figure 3 The deconvolution neural network model shown above obtains image 3, where the pixel size of image 3 is 128*64;

[0053] S5: Place image 3 into Figure 4 The convolutional neural network model shown above obtains the feature vector of image 3, and the feature vector length is 128 dimensions;

[0054] S6: Similarly, put image 2 into Figure 4 The neural network shown obtains the 128-dimensional feature vector of image 2;

[0055] S7: The cosine distance between 128-dimensional vectors is used to judge the similarity between images in all datasets.

[0056] In step S2, the classified file format is as follows Figure 2 shown.

[0057] In step S2, the target image is defined as a pixel whose length and width are equal to 128*64 pixels, a large target image is defined as an image whose length and width are greater than 128 and 64 respectively, a small target image is defined as an image whose length and width are less than 64 and 32 respectively, and the rest of the images are medium target images.

[0058] In steps S4 and S5, Figure 3 Deconvolutional neural network model and Figure 4 The parameters of the convolutional neural network model are trained in an adversarial manner. Figure 3 It is a deconvolution model. The input is a small target image after padding, the input size is 3*64*32, and the output is an enlarged target image, the output size is 3*128*64. Figure 4 It is an appearance feature extraction network. The input is the network to be extracted. The input size is 3*128*64, and the output is a 128-dimensional feature vector.

[0059] Figure 3 Deconvolutional neural network model and Figure 4The convolutional neural network model in constitutes an adversarial model, and the deconvolutional neural network model is the generator model in the adversarial network, such as Figure 5 As shown; the convolutional neural network model is the discriminator model in the adversarial network, such as Figure 6 shown.

[0060] The specific training steps are:

[0061] (1) Randomly select a large target image or a medium target image, called image s1;

[0062] (2) Use bilinear interpolation to scale it to an image with a pixel size of 128*64, called image s2;

[0063] (3) Put image s2 into Figure 4 In the convolutional neural network model shown, a 128-dimensional feature vector of image s2 is obtained, which is called feature vector s1;

[0064] (4) Select any small target image in the same folder as image s1, called image s3;

[0065] (5) Fill the four sides of image s3 with gray to expand its pixels to 64*32, which is called image s4;

[0066] (6) Place image s4 into Figure 3 In the deconvolution neural network shown, image s5 is obtained, and the pixel size of image s5 is 128*64 at this time;

[0067] (7) Place image s5 in the same Figure 4 In the convolutional neural network shown, a 128-dimensional feature vector of image s5 is obtained, which is called feature vector s2;

[0068] (8) Calculate the cosine distance between eigenvector s1 and eigenvector s2, and use this distance as the error update Figure 3 Deconvolutional neural network parameters shown;

[0069] (9) Recalculate steps (6) and (7) using the new network parameters to obtain a new 128-dimensional feature vector, called feature vector s3;

[0070] (10) Calculate the cosine distance between eigenvector s1 and eigenvector s3, and use this distance as the error update Figure 4 Convolutional neural network parameters shown;

[0071] (11) Repeat the above steps (1) to (10) until the cosine distance in (8) is less than 0.2 and the cosine distance in (10) is less than 0.2.

[0072] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting appearance features of a small visual target, characterized in that: The following steps are involved: Step 1: Collect all training dataset images, which consist of large target images, medium target images, small target images, and target images; Step 2: Scale the large target image and the medium target image to the same size as the target image using bilinear interpolation, which is called the first image. Step 3: Fill the four sides of the small target image with gray to make the target pixel become 64*32, and obtain the second image; Step 4: Place the second image into the first neural network model for deconvolution to obtain a third image, where the pixel size of the third image is the same as that of the first image; Step 5: Place the third image into the second neural network model to extract appearance features, and obtain a feature vector of the third image, where the feature vector length is 128 dimensions; Step 6: Also use the second neural network model to extract a 128-dimensional feature vector of the first image; Step 7: Use the cosine distance between the 128-dimensional feature vectors extracted in Step 5 and Step 6 to judge the similarity between the first image and the third image, that is, use the cosine distance between the 128-dimensional feature vectors to judge the similarity between all data sets in Step 1, and complete the extraction of the appearance features of the visual small target; The first neural network model in step 4 and the second neural network model in step 5 constitute an adversarial model; The first neural network model is the generator model in the adversarial network, which consists of the following parts:

1. A feature extraction convolutional neural network, for extracting features of the second image; 2. A two-layer deconvolution network to visualize the features into an image of the target image size; The second neural network model is a discriminator model in the adversarial network, which consists of a three-layer convolutional neural network model to obtain the feature vector of the image; Use an adversarial approach to train the adversarial model consisting of the first neural network model and the second neural network model. The specific steps are: (1) Take any image in the first image and put it into the second neural network to obtain a 128-dimensional feature vector, which is called the first feature vector; (2) Take the small target image in the same folder as the image in (1), and obtain a 128-dimensional feature vector according to steps 3, 4, and 5, which is called the second feature vector; (3) Fix the discriminator network parameters and update the generator network parameters using the cosine distance between the first eigenvector and the second eigenvector as the loss; (4) The small target image in (2) is obtained again according to the method of step 3, step 4, and step 5 to obtain a 128-dimensional feature vector, which is called the third feature vector; (5) Fix the generator network parameters and update the discriminator network parameters using the cosine distance between the negative vector of the first eigenvector and the fourth eigenvector as the loss; (6) Repeat the above steps (1) to (5) until the loss function in (3) is less than 0.2 and the loss function in (5) is also less than 0.

2.

2. The method for extracting appearance features of a small visual target according to claim 1, characterized in that: In step 1, images of the same car model are placed in the same folder, and the sizes of the car model images in the same folder are different.

3. The method for extracting appearance features of a small visual target according to claim 1, characterized in that: In step 1, the target image is defined as an image with pixels equal to 128*64, the large target image is defined as an image with length and width pixels greater than 128 and 64 respectively, the small target image is defined as an image with length and width pixels less than 64 and 32 respectively, and the rest of the images are medium target images.

4. The method for extracting appearance features of a small visual target according to claim 1, characterized in that: The formula for the cosine distance in step 7 is: Among them, x, y are 128-dimensional feature vectors, and i is the value of the feature vector of the i-th dimension.

Images