A method for vehicle recognition in infrared remote sensing images based on semi-supervised learning

Through a semi-supervised learning method, combined with deep learning and SVM model, the samples are reselected from unlabeled infrared remote sensing images, solving the problem of low recognition accuracy of infrared remote sensing image vehicle recognition model when the sample size is small, and improving vehicle recognition accuracy.

CN115019212BActive Publication Date: 2024-11-26THE 54TH RESEARCH INSTITUTE OF CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210755231.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-30
Publication Date
2024-11-26
Estimated Expiration
2042-06-30

AI Technical Summary

Technical Problem

Infrared remote sensing image vehicle recognition model is difficult to build a deep learning model when the sample size is small, resulting in low recognition accuracy and traditional methods are difficult to obtain target deep features and deal with noise interference.

Method used

Using a semi-supervised learning method, samples are reselected from unlabeled infrared remote sensing images. By combining deep learning models and SVM models, an infrared remote sensing image vehicle sample library is constructed to increase sample size and improve recognition accuracy.

Benefits of technology

The infrared remote sensing image vehicle samples are effectively increased, the recognition accuracy of the deep learning model is improved, and the problem of low reliability of the deep learning model when the sample size is small is solved, achieving higher vehicle recognition accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115019212B_ABST
    Figure CN115019212B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for infrared remote sensing image vehicle recognition based on semi-supervised learning. The method comprehensively utilizes a deep learning vehicle recognition model and an SVM vehicle recognition model. When the deep learning vehicle recognition probability is greater than a certain threshold and the SVM model recognizes a vehicle information entropy value less than a certain threshold, the target is added to the infrared remote sensing image vehicle sample library for retraining the deep learning vehicle recognition model. The method effectively increases the infrared remote sensing image vehicle samples, solves the problem of low reliability of the deep learning model when the sample size is small by comprehensively utilizing the deep learning model and the SVM model, and improves the recognition accuracy of the deep learning recognition model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent interpretation of remote sensing images, and in particular relates to a vehicle recognition method on infrared remote sensing images. Background Art

[0002] Since the thermal infrared radiation of vehicle targets and backgrounds will be affected by atmospheric absorption and random scattering during atmospheric transmission, the infrared images of vehicle targets and backgrounds lack good contrast and resolution compared to visible light images. As a result, the pixels of infrared images have good correlation, the grayscale mean of the image remains relatively stable, the variance does not change much, and the contrast between the target image and the background is low, the edges are blurred, the whole image has large noise and complex components. Compared with optical images, it is difficult for infrared bands to reflect the texture information of the target surface. Therefore, the difficulty of using infrared images for target recognition is much greater than that of using optical images for target recognition. The specific performance is as follows:

[0003] 1) Characteristics of vehicle infrared targets: Infrared vehicle targets themselves lack features such as color and texture, and are small in size. Most methods can only use their grayscale distribution characteristics and motion characteristics. In addition, the signal-to-noise ratio of infrared vehicle targets is low, and they are easily submerged by complex and changing backgrounds and noise in the atmosphere.

[0004] 2) Characteristics of infrared background: the background is similar, the distribution is continuous, and it is complex and changeable; the complex background characteristics of infrared images are relatively uneven, and the grayscale is darker than that of the target area.

[0005] 3) Difficulties in infrared vehicle target detection: (1) Few available target features. Since the vehicle target is small in size and its total radiation energy is less than the background radiation energy, the grayscale distribution in the image is variable, making it difficult to describe it using a unified mathematical model. In addition, there is no fine texture, shape and other structural information, which makes it impossible to directly use the target detection method of traditional visible light images for infrared weak target detection. (2) Low image signal-to-noise ratio. Due to the long imaging distance, the vehicle target has similar characteristics to clutter and noise, and is easily submerged and interfered with, resulting in low intensity of the vehicle target and low signal-to-noise ratio of the image. The target signal is almost submerged in the unpredictable background, making it even more difficult to detect. (3) There are few public data sets, which makes it difficult to build an algorithm model for vehicle recognition, which also restricts the development of vehicle target detection algorithms to a certain extent.

[0006] At present, infrared remote sensing image target recognition models mainly include traditional pattern recognition models and deep learning models. When there are sufficient samples, the deep learning model can well learn the deep features of the target and achieve high recognition accuracy, but when the sample size is small, the effect of the deep learning model is poor; while the pattern recognition model can achieve a more robust effect than the deep learning model when the sample size is small, but it is difficult to obtain the deep features of the target. Summary of the invention

[0007] The purpose of the present invention is to solve the problem of difficulty in building a deep learning model for target recognition due to the small number of vehicle samples in infrared remote sensing images, and propose a vehicle recognition method for infrared remote sensing images based on semi-supervised learning. This method can reselect some samples from unlabeled infrared remote sensing images and add them to the sample library, effectively increasing the number of vehicle samples in infrared remote sensing images. By comprehensively utilizing the deep learning model and the SVM model, the problem of low reliability of the deep learning model when the sample size is small is solved, and the recognition accuracy of the deep learning model is improved.

[0008] The object of the present invention is achieved in that:

[0009] A method for vehicle recognition in infrared remote sensing images based on semi-supervised learning comprises the following steps:

[0010] (1) Construct an infrared remote sensing image vehicle sample library. The initial samples in the infrared remote sensing image vehicle sample library are manually annotated infrared remote sensing image vehicle samples. The infrared remote sensing image vehicle sample library is used to train the SVM target recognition model.

[0011] (2) Use the infrared remote sensing image vehicle sample library to train the deep learning target recognition model;

[0012] (3) Using the current deep learning target recognition model and the SVM target recognition model to obtain new infrared remote sensing image vehicle samples, and adding the new infrared remote sensing image vehicle samples to the infrared remote sensing image vehicle sample library;

[0013] (4) Determine whether the current loop has reached the stop condition. If so, use the current deep learning target recognition model as the final infrared remote sensing image vehicle recognition model; if not, repeat steps (2) to (4).

[0014] Furthermore, the stopping condition in step (4) is that the number of new infrared remote sensing image vehicle samples acquired in the current cycle is less than 10.

[0015] Furthermore, step (3) specifically includes the following steps:

[0016] (301) using the deep learning target recognition model trained in step (2) to identify vehicles in the infrared image, and giving the probability that the identified target belongs to each category;

[0017] (302) All identified vehicles are divided according to whether they have been marked or not. When the probability of a vehicle belonging to a certain type is greater than a threshold a, the target is set as an unmarked vehicle target;

[0018] (303) The unlabeled vehicle target predicted in step (302) is judged using the SVM target recognition model trained in step (1), the probability distribution of the target belonging to each category is given, and the information entropy H(X) of the target is calculated based on the probability distribution:

[0019]

[0020] Among them, X represents the unlabeled vehicle target, n is the number of categories that the target may belong to, and p(x i ) is the target belonging to category x i probability;

[0021] (304) The targets with the smallest information entropy are taken as new infrared remote sensing image vehicle samples and added into the infrared remote sensing image vehicle sample library.

[0022] Furthermore, in step (302), the threshold a=0.8, and in step (304), the first 20% of targets with the smallest information entropy are added to the infrared remote sensing image vehicle sample library.

[0023] Compared with the background technology, the present invention has the following advantages:

[0024] 1. The present invention can effectively increase vehicle samples in infrared remote sensing images and improve the target recognition accuracy of deep learning models.

[0025] 2. The present invention can reselect some samples from unlabeled infrared remote sensing images and add them to the sample library, thereby effectively increasing the number of vehicle samples in infrared remote sensing images.

[0026] 3. The present invention solves the problem of low reliability of the deep learning model when the sample size is small by comprehensively utilizing the deep learning model and the SVM model, thereby improving the recognition accuracy of the deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0029] like Figure 1 As shown, a method for infrared remote sensing image vehicle recognition based on semi-supervised learning includes the following steps:

[0030] (1) The SVM target recognition model is trained using infrared remote sensing image samples, and the final output is the probability that the target belongs to a category.

[0031] (2) The FasterRCNN target recognition model is trained using infrared remote sensing image samples, and the final output is the probability that the target belongs to a category.

[0032] (3) Use FasterRCNN to predict infrared images. Use the FasterRCNN model trained in step 2 to predict all infrared remote sensing images, and divide the objects in the prediction results into categories according to whether they have been labeled. If the probability that the object belongs to a certain type of vehicle is greater than 0.8, then put the object into the predicted unlabeled vehicle object library.

[0033] (4) Use the SVM model trained in step 1 to calculate the probability distribution of unlabeled vehicles in step 3, and use the probability distribution to calculate the information entropy of the target.

[0034] (5) Sort the information entropy of all unlabeled vehicles from small to large, and take the top 20% of unlabeled vehicles and place them in the infrared remote sensing image vehicle sample library.

[0035] (6) Determine whether the loop stop condition is met. If the number of new samples in the current loop is less than 10, the loop is stopped and the current deep learning target recognition model is used as the final infrared remote sensing image vehicle recognition model. Otherwise, return to step 2 and start a new loop.

[0036] In order to verify the effectiveness of this method, 10,000 drone infrared remote sensing images were used for vehicle recognition. Before the experiment, 500 infrared image vehicle samples were manually annotated, of which 400 were used for training, 100 were used for verification, and another 1,000 samples were annotated for testing; the FasterRCNN method and the method of this embodiment were used for vehicle recognition, and statistics were taken. The verification results show that the vehicle recognition accuracy of the FasterRCNN model alone is 86.6%, and the vehicle recognition accuracy of the method of this embodiment is 92.3%.

[0037] In summary, the present invention realizes infrared remote sensing image vehicle recognition based on semi-supervised learning, and comprehensively utilizes the deep learning vehicle recognition model and the SVM vehicle recognition model. When the deep learning vehicle recognition probability is greater than a certain threshold and the SVM model recognizes the vehicle information entropy value less than a certain threshold, the target is added to the infrared remote sensing image vehicle sample library for retraining the deep learning vehicle recognition model. This method effectively increases the infrared remote sensing image vehicle samples, and by comprehensively utilizing the deep learning model and the SVM model, it solves the problem of low reliability of the deep learning model when the sample size is small, and improves the recognition accuracy of the deep learning recognition model.

[0038] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for vehicle recognition in infrared remote sensing images based on semi-supervised learning, characterized in that: The following steps are involved: (1) Construct an infrared remote sensing image vehicle sample library. The initial samples in the infrared remote sensing image vehicle sample library are manually annotated infrared remote sensing image vehicle samples. The infrared remote sensing image vehicle sample library is used to train the SVM target recognition model. (2) Use the infrared remote sensing image vehicle sample library to train the deep learning target recognition model; (3) Using the current deep learning target recognition model and the SVM target recognition model to obtain new infrared remote sensing image vehicle samples, and adding the new infrared remote sensing image vehicle samples to the infrared remote sensing image vehicle sample library; The specific steps include: (301) using the deep learning target recognition model trained in step (2) to identify vehicles in the infrared image, and giving the probability that the identified target belongs to each category; (302) All identified vehicles are divided according to whether they have been marked or not. When the probability of a vehicle belonging to a certain type is greater than a threshold a, the target is set as an unmarked vehicle target; (303) The unlabeled vehicle target predicted in step (302) is judged using the SVM target recognition model trained in step (1), the probability distribution of the target belonging to each category is given, and the information entropy H(X) of the target is calculated based on the probability distribution: Among them, X represents the unlabeled vehicle target, n is the number of categories to which the target belongs, and p(x i ) is the target belonging to category x i probability; (304) taking the part of the targets with the smallest information entropy as new infrared remote sensing image vehicle samples and adding them into the infrared remote sensing image vehicle sample library; (4) Determine whether the current loop has reached the stop condition. If so, use the current deep learning target recognition model as the final infrared remote sensing image vehicle recognition model; if not, repeat steps (2) to (4).

2. The method for vehicle recognition based on infrared remote sensing images based on semi-supervised learning according to claim 1, characterized in that: The stopping condition in step (4) is that the number of new infrared remote sensing image vehicle samples acquired in the current cycle is less than 10.

3. The method for vehicle recognition based on infrared remote sensing images based on semi-supervised learning according to claim 1, characterized in that: In step (302), the threshold a=0.8, and in step (304), the first 20% of targets with the smallest information entropy are added to the infrared remote sensing image vehicle sample library.