An infrared aerial video environment perception algorithm based on scene classification

Abstract

The application relates to an infrared aerial video environment perception algorithm based on scene classification, which comprises the following steps: (1) region segmentation of an infrared image; (2) input of the image of each region into a scene classification model respectively to obtain the scene classification probability of each region; (3) histogram statistics of each region respectively to obtain the histogram statistical information of each region; (4) calculation of the histogram weighted statistical result of each category of each region and the histogram statistical information of each category of a frame image; (5) setting of the weight of each category, weighted summation of the histogram statistical results of all categories to obtain a full image histogram; and weighted updating of the full image histogram in the time domain according to the set updating rate to obtain the environment histogram statistical result. The application realizes histogram statistics of the aerial environment and provides good and comprehensive environment information for subsequent image processing.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing technology, and to an infrared aerial video environment perception algorithm based on scene classification. The algorithm performs perception modeling of the aerial environment through scene classification and establishes an environment histogram statistical model. Background Technology

[0002] Infrared aerial video has the advantage of a large dynamic range, but it also often faces situations where the scene changes rapidly. However, most current image processing algorithms do not provide special processing for these situations caused by the rapid changes in the field of view in rich scenes, or they simply perform simple processing through temporal smoothing operations, resulting in problems such as unstable brightness of the output image and loss of some scene information after mapping. Summary of the Invention

[0003] The purpose of this invention is to address the aforementioned problems by providing an infrared aerial video environment perception algorithm based on scene classification. This algorithm establishes a histogram statistical model of the aerial environment through scene classification information, providing environmental perception information for the output infrared image, thereby improving the quality and stability of the output image.

[0004] The technical solution of the present invention is as follows:

[0005] An infrared aerial video environment perception algorithm based on scene classification, characterized by the following steps:

[0006] (1) Infrared image region segmentation: The infrared image is segmented according to the video frame size, field of view size, etc.;

[0007] (2) Scene classification of infrared image regions: Select an appropriate scene classification network and scene classification model, and input the images of each region segmented in step (1) into the scene classification model according to scene classification to obtain the scene classification probability of each region;

[0008] (3) Histogram statistics of infrared image regions: Histogram statistics are performed on each region segmented in step (1) to obtain histogram statistics information of each region;

[0009] (4) Weighted cumulative statistical analysis of regional histograms:

[0010] For each region, the classification probability from step (2) and the histogram statistics from step (3) are weighted to obtain the histogram weighted statistics for each category in that region.

[0011] The weighted statistical results of histograms for the same category in each region are summed to obtain the histogram statistical information for each category of the current frame image;

[0012] (5. Statistical weighted calculation of environmental histogram:)

[0013] The weight of each category is set according to the actual application requirements, and the full-image histogram statistical results of all categories are weighted and summed to obtain the full-image histogram.

[0014] Based on the set update rate, the full histogram is weighted and updated in the time domain to obtain the environmental histogram statistics.

[0015] Compared with existing simple temporal smoothing methods, this invention introduces probability information of different scenes by classifying the scene in the region, and then realizes histogram statistics for different scenes by weighted accumulation of regional histograms. In the process of fusion of the whole image histogram, different weights are set for the observed target and the background to realize histogram weighting for different scenes. Finally, the histogram accumulation of the overall environment is completed by temporal update, thereby realizing histogram statistics of the aerial photography environment.

[0016] This invention can create a histogram statistical model of the aerial photography environment according to actual application needs, providing good and comprehensive environmental information for subsequent image processing. For example, based on the environmental histogram information, the target contrast can be improved by compressing the proportion of non-target histograms, or the grayscale stability of the video when switching between different scenes can be improved by fusing the current frame image information with the environmental histogram. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention;

[0018] Figure 2 These are examples of proportionally segmented images according to the present invention. Detailed Implementation

[0019] like Figure 1 As shown, this invention mainly includes five steps: infrared image region segmentation, region scene classification, region histogram statistics, region histogram weighted accumulation, and environment histogram weighted calculation. The specific implementation methods are as follows:

[0020] (1) Infrared image region segmentation: The infrared image is segmented according to the video frame size, field of view size, etc.;

[0021] like Figure 2 As shown, the image can be segmented into Figure 2 (a) shows the 4 equal parts, Figure 2 (b) shows 8 equal parts, Figure 2 The 16-equal division shown in (c) can be further divided according to the above multiples, depending on the actual application.

[0022] (2) Scene classification of infrared image regions: Select an appropriate scene classification network and scene classification model, and input the images of each region segmented in step (1) into the scene classification model according to scene classification to obtain the scene classification probability of each region.

[0023] For example, MobileNet can be used as the backbone of the scene classification network to classify the target into four categories: sky, water, buildings and others. MobileNet can then be transferred and trained under the target scene and classification conditions to obtain the classification model used later.

[0024] In step (1), each image in the video stream is divided into N regions, and each image contains K categories. The N regions are then input into the scene classification model, and the model outputs the probabilities corresponding to the K categories. The probability of the i-th region being of the j-th class is .

[0025] (3) Histogram statistics of infrared image regions: Histogram statistics are performed on each region segmented in step (1) to obtain histogram statistics information of each region.

[0026] After each image in the video stream undergoes region segmentation in step (1), the corresponding histogram information for each of the N regions is calculated, thus obtaining the statistical results of the N histograms. .

[0027] (4) Weighted cumulative statistical analysis of regional histograms:

[0028] For each region, the classification probability from step (2) and the histogram statistics from step (3) are weighted to obtain the histogram weighted statistics for each category in that region.

[0029] The weighted statistical results of histograms for the same category in each region are summed to obtain the histogram statistical information for each category of the current frame image;

[0030] As in the above embodiment, the weighted statistical histogram corresponding to the j-th category in the i-th region is as follows:

[0031] ,

[0032] By summing the histograms of each category, we can obtain the histogram statistics for each scene category.

[0033] By summing the weighted statistical histograms of each category in each region, the histogram statistics of each category in the current frame image are obtained. Then, the histogram statistics of the entire image for the j-th category are:

[0034]

[0035] (5) Statistical weighted calculation of environmental histogram: Based on the actual observation target, the histograms of each category are weighted and then merged.

[0036] The weight of each category is set according to the actual application requirements, and the full-image histogram statistical results of all categories are weighted and summed to obtain the full-image histogram.

[0037] Based on the set update rate, the full histogram is weighted and updated in the time domain to obtain the environmental histogram statistics.

[0038] As in the above embodiments, corresponding weights are assigned to each category according to actual application requirements. The higher the weight, the more important the corresponding category information. The weighted sum of the full-image histogram results for all categories yields the following full-image histogram result:

[0039] ;

[0040] To obtain the histogram information of the aerial environment, a weighted update of the full-image histogram is needed in the time domain. Let the update rate be α, and the full-image histogram at time t be... The updated formula is: .

[0041] Based on the above formula, the environmental histogram statistics can be obtained.

Claims

1. An infrared aerial video environment perception algorithm based on scene classification, characterized in that, Includes the following steps: (1) Infrared image region segmentation: The infrared image is segmented according to the video frame size and field of view size; (2) Scene classification of infrared image regions: MobileNet is used as the scene classification network to classify the target into at least four categories: sky, water, buildings and others. MobileNet is transferred and trained under the target scene and classification conditions to obtain the scene classification model used later. According to scene classification, the images of each region segmented in step (1) are input into the scene classification model to obtain the scene classification probability of each region; (3) Histogram statistics of infrared image regions: Histogram statistics are performed on each region segmented in step (1) to obtain histogram statistics information of each region; (4) Weighted cumulative statistical analysis of regional histograms: For each region, the classification probability from step (2) and the histogram statistics from step (3) are weighted to obtain the histogram weighted statistics for each category in that region. The weighted statistical results of histograms for the same category in each region are summed to obtain the histogram statistical information for each category of the current frame image; (5) Statistical weighted calculation of environmental histogram: The weight of each category is set according to the actual application requirements, and the full-image histogram statistical results of all categories are weighted and summed to obtain the full-image histogram. Based on the set update rate, the full histogram is weighted and updated in the time domain to obtain the environmental histogram statistics.

2. The infrared aerial video environment perception algorithm based on scene classification according to claim 1, characterized in that: In step (1), the infrared image region is segmented by dividing it into multiples of 4, 8, or 16 equal parts.

Images