An optical smoke detection method based on feature extraction in sea-sky background

Abstract

The application relates to an optical smoke detection method based on feature extraction in a sea-sky background, which comprises the following steps: establishing a sample library of smoke targets, other targets and background targets for classifier learning; extracting multi-dimensional feature descriptors of all samples in the sample library based on the gray level features, texture features and gradient features of images; learning and training a classifier by using the multi-dimensional feature descriptors of all samples, so that the classifier can identify smoke targets; extracting multiple smoke suspected target area blocks from a target image containing optical smoke interference in the sea-sky background to be identified; extracting the multi-dimensional feature descriptors of each smoke suspected target area block respectively, and inputting the multi-dimensional feature descriptors into the trained classifier for judgment to detect smoke targets. The smoke feature descriptor construction and classification algorithm adopted by the application is simple and effective, occupies few hardware resources, has high running speed, and is suitable for a hardware platform with limited resources.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an optical smoke detection method in a sea-sky background based on feature extraction. BACKGROUND

[0002] At present, smoke detection and recognition technology mainly focuses on fire detection field, and the purpose of detecting smoke is to discover fire earlier to win valuable time for fire extinguishing. And the application scene is mainly video monitoring scene in fixed background, and research mainly focuses on video smoke detection and recognition technology using color, texture, turbulence, floating and other characteristics of smoke.

[0003] However, for the sea-sky background, there is no effective detection method for the detection of smoke interference close to the water surface, and the general video smoke detection and recognition technology based on color, texture, turbulence, floating and other characteristics cannot be applied to optical smoke detection in the sea-sky background due to different image features, for example, the collected image is only a gray image, and color information cannot be used. And the feature extraction and classification method in the existing smoke recognition algorithm has high calculation complexity, long calculation time and needs to occupy a large storage space. And the hardware resource limited platform cannot meet the operation requirement of the existing smoke recognition algorithm. SUMMARY

[0004] In view of the above analysis, the application aims to disclose an optical smoke detection method in a sea-sky background, which solves the technical problem of optical smoke detection in a sea-sky background.

[0005] The application discloses an optical smoke detection method in a sea-sky background based on feature extraction, comprising:

[0006] establishing a sample library of smoke targets, other targets and background targets for classifier learning;

[0007] extracting multi-dimensional feature descriptors of all samples in the sample library based on the gray scale features, texture features and gradient features of the image;

[0008] training the classifier using the multi-dimensional feature descriptors extracted from all samples, so that the classifier can recognize smoke targets;

[0009] for a target image containing optical smoke interference in the sea-sky background to be recognized, the target image is segmented based on smoke gray scale features, and a plurality of smoke suspected target region blocks are extracted;

[0010] for each smoke suspected target region block, the multi-dimensional feature descriptors are extracted respectively and sent to the trained classifier for judgment to detect smoke targets.

[0011] Further, the multi-dimensional feature descriptor based on the image's gray level feature extraction is the image's gray level mean, variance and maximum value features.

[0012] Further, the multi-dimensional feature descriptor based on the image's texture feature extraction is the image's gray level difference mean, contrast and entropy features.

[0013] Further, the multi-dimensional feature descriptor based on the image's gradient feature extraction is the image's vertical gradient mean and vertical gradient variance features.

[0014] Further, the classification algorithm adopted by the classifier is a tree classification algorithm.

[0015] Further, the target image is segmented based on the smoke gray level feature, and a plurality of smoke suspected target region blocks are extracted.

[0016] 1) According to the histogram information of the image, the gray level estimation value of the sea surface and the sky is obtained, and the initial seed points of the sea surface and the sky are acquired;

[0017] 2) Based on the white highlight gray level statistical experience value of the smoke region and the gray level maximum value of the image, the initial seed points of the smoke are acquired;

[0018] 3) Based on the initial seed points of the sea surface, the sky and the smoke, the clustering algorithm is adopted to cluster the image gray level, the potential region of the image belonging to the smoke target is segmented and extracted, and the smoke candidate region is obtained.

[0019] Further, the histogram is smoothed and the peak point is detected, the gray level estimation value of the sea surface and the sky is obtained, and the initial seed points of the sea surface and the sky are acquired;

[0020] The initial seed points of the sky and the sea surface are the first extreme value point and the second extreme value point of the histogram of the smoothed image.

[0021] Further, the smoothing of the histogram adopts Gaussian smoothing.

[0022] The Gaussian smoothing calculation formula is Wherein h i represents the calculated image histogram, r is the smoothing scale size, and σ is the standard deviation.

[0023] Further, the initial seed points of the smoke are Wherein, V s is the statistical experience value of the smoke gray level, and max(img) is the gray level maximum value of the image.

[0024] Further, the Kmeans gray level clustering algorithm is adopted to cluster the image gray level, the potential region of the image belonging to the smoke target is obtained, and after the segmentation and extraction, a plurality of smoke suspected target region blocks are obtained.

[0025] The present application can achieve at least one of the following beneficial effects:

[0026] The optical smoke detection method based on feature extraction in the sea-sky background designed by the present application combines the smoke scene in the sea-sky background, and constructs a feature descriptor for the gray scale feature, texture feature and gradient feature of the smoke. The feature descriptor construction method is simple and effective, and can quickly realize smoke feature extraction. The tree classification algorithm is used to realize the fast classification of the constructed smoke feature descriptor. The classification algorithm has low complexity, occupies less hardware resources, and has high running speed. In summary, the smoke feature descriptor construction and classification algorithm used by the present application is simple and effective, occupies less hardware resources, has high running speed, and is suitable for hardware platforms with limited resources. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application, and together with the description serve to explain the principles of the application.

[0028] Figure 1 The flow chart of the optical smoke detection method based on feature extraction in the sea-sky background in the embodiment of the present application.

[0029] Figure 2 The flow chart of the smoke candidate region segmentation method in the embodiment of the present application.

[0030] Figure 3 The flow chart of the edge extraction and fusion method for the smoke candidate region in the embodiment of the present application. DETAILED DESCRIPTION

[0031] The preferred embodiments of the present application will be described in detail below with reference to the accompanying drawings, wherein the accompanying drawings constitute a part of this application, and are used to explain the principles of the embodiments of the present application.

[0032] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0033] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0034] The relative arrangement of parts and steps, numerical expressions, and numerical values set forth in the examples are not intended to limit the scope of the application unless otherwise specifically stated. It is to be understood that the drawings are not necessarily to scale as the dimensions of the parts shown are for the purpose of illustration and description only. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail but are intended to be understood as a part of the specification when appropriate. In all examples shown and discussed herein, any specific values are to be interpreted as illustrative only and not as a limitation. Thus, other examples of the example embodiments can have different values. It is noted that like numbers and letters on the figures identify like parts throughout the disclosure, and thus, once defined, no further discussion of such parts is necessary.

[0035] One embodiment of the present application discloses a method for optical smoke detection in sea-sky background based on feature extraction, as shown in Figure 1 The method comprises the following steps:

[0036] Step S101, establishing a sample library of smoke targets, other targets and background targets for classifier learning;

[0037] Step S102, extracting multi-dimensional feature descriptors for all samples in the sample library based on the gray feature, texture feature and gradient feature of the image;

[0038] Step S103, training the classifier using the multi-dimensional feature descriptors extracted for all samples, so that the classifier can recognize the smoke target;

[0039] Step S104, for a target image containing optical smoke interference in sea-sky background to be identified, segmenting the target image based on the smoke gray feature, and extracting a plurality of smoke suspected target region blocks;

[0040] Step S105, for each smoke suspected target region block, respectively extracting the multi-dimensional feature descriptors and sending them into the trained classifier for judgment, and detecting the smoke target.

[0041] In step S101, for all training images containing smoke, the smoke image block is cut out to construct a smoke image as a positive sample library, and other targets (such as ships, etc.) and background targets are cut out as a negative sample library.

[0042] Specifically, the color characteristics of the smoke target processed in the embodiment present in the optical image are that there is a high gray highlight area, the gray level gradually changes, and the overall gray value is high. The shape is variable, and there are shapes similar to an ellipse and shapes similar to a cloud. Moreover, the texture of the smoke is relatively simple compared with other targets such as a ship, and there is strong directionality, and the horizontal edge is weak, while the ship body of the ship target has a strong horizontal edge. Based on the analysis of the above smoke characteristics, the gray feature, the texture feature, and the gradient feature of the smoke image can be extracted, thereby serving as the final features for classification.

[0043] Specifically, in step S102, the multi-dimensional feature descriptor based on the gray feature extraction is three-dimensional features of the gray mean, variance, and maximum value of the image.

[0044] Based on the characteristics that the smoke image has a high gray highlight and the gray level gradually changes, and the overall gray value of the ship target is low, the embodiment mainly extracts the three-dimensional features of the mean, variance, and maximum value of the image on the gray feature.

[0045] The mean value refers to the average value of the image gray level, the variance refers to the variance of the image gray value, and the maximum value refers to the maximum value of the image gray level. The definitions are as follows.

[0046] Mean value:

[0047]

[0048] Variance:

[0049]

[0050] Maximum value:

[0051]

[0052] Wherein, M and N are the width and height of the image, and P ij is the gray value of the pixel.

[0053] Specifically, in step S102, the multi-dimensional feature descriptor based on the texture feature extraction is three-dimensional features of the gray difference average, contrast, and entropy of the image.

[0054] The texture feature describes the surface properties of the object corresponding to the image or image region, and quantifies the features of the internal gray level change of the region.

[0055] Based on the fact that the texture of the smoke is relatively simple compared with the ship target, the embodiment extracts the texture feature based on the gray difference method.

[0056] Specifically, assuming a pixel point (m, n) in the image, the gray difference value between the point and its neighborhood points is:

[0057]

[0058] wherein, is called the gray difference. Assuming that all possible values of the gray difference value have m levels, the histogram of is obtained. The histogram can be used to calculate the probability p(k) of the value of k, which is the gray difference value. The greater p(k) is, the coarser the texture is, and the smaller p(k) is, the more detailed the texture is.

[0059] The embodiment mainly extracts the average value of the gray difference, the contrast, and the entropy as the texture features of the image. The definitions are as follows.

[0060] Average value of the gray difference:

[0061]

[0062] Contrast:

[0063]

[0064] Entropy:

[0065]

[0066] In step S102, the multi-dimensional feature descriptor based on the gradient feature extraction is a two-dimensional gradient feature of the vertical gradient mean and the vertical gradient variance of the image.

[0067] Based on the fact that the smoke image has strong directionality and weak horizontal edges, and the ship target has strong horizontal edges, the embodiment uses the vertical gradient feature of the image as the gradient feature. The vertical gradient mean and the vertical gradient variance of the image are calculated as the extracted two-dimensional gradient feature.

[0068] The template for calculating the vertical gradient is a sobel operator, as shown in the formula.

[0069]

[0070] The vertical gradient G y is obtained, and the vertical gradient mean and the variance of the vertical gradient of the image are calculated, as shown in the formula.

[0071] Vertical gradient mean:

[0072]

[0073] In the formula, Gy (i,j) is the vertical gradient of coordinate point (i,j); N is the width of the image;

[0074] Vertical gradient variance:

[0075]

[0076] The two-dimensional features of the vertical gradient mean and variance are taken as the gradient features.

[0077] The multi-dimensional feature descriptor finally extracted for each sample in the sample library contains 8 dimensions, which are the gray mean, gray variance, gray maximum, gray difference mean, contrast, entropy, vertical gradient mean, and vertical gradient variance.

[0078] Specifically, in step S103, the 8 multi-dimensional feature descriptors extracted by all samples are used to select a classification algorithm to complete the learning and training of the classifier, so as to obtain a smoke classification model for identification.

[0079] The classification algorithm can use a tree classification algorithm, for example, a CART binary tree classification algorithm. The tree classification algorithm realizes fast classification, has low complexity, occupies less hardware resources, and runs fast, and is suitable for hardware platforms with limited resources. After training, the smoke target classifier is obtained.

[0080] In the application scenario of the embodiment, the image content in the sea-sky background is relatively single, mainly including sea surface, sky, and target 3-part regions, and the 3-part regions have large differences in gray scale. In order to more accurately perform smoke segmentation, a clustering segmentation algorithm based on sea surface, sky, and smoke 3-classification can be used,

[0081] In order to more quickly and accurately cluster the image, the initial segmentation points (i.e., seed points) of the sea surface, sky, and smoke 3-class targets need to be accurately obtained. Therefore, the approximate gray scale range of the sea surface, sky, and smoke needs to be determined.

[0082] For the image in the sea-sky background, because the background in the image is simple, the histogram has a double-peak characteristic, and the clustering initial segmentation points of the sea surface and sky can be obtained through histogram peak point detection.

[0083] Preferably, in step S104, the gray clustering algorithm is used to segment the target image based on the gray highlight feature of the smoke, and a plurality of smoke suspected target region blocks are extracted.

[0084] Specifically, as shown in Figure 2 segmenting the target image to extract a plurality of smoke suspected target region blocks includes:

[0085] Step S201, obtaining the gray scale estimation value of the sea surface and the sky according to the histogram information of the image, and obtaining the initial seed point of the sea surface and the sky;

[0086] Preferably, the gray scale estimation value of the sea surface and the sky is obtained by smoothing the histogram of the image and detecting the peak point; and the initial seed point of the sky and the sea surface is the first extreme point and the second extreme point of the histogram of the smoothed image.

[0087] The histogram of the image is calculated, and since the histogram has burrs (i.e. local peaks), the histogram needs to be smoothed to eliminate the burrs, and the histogram smoothing method can be selected from Gaussian smoothing, mean smoothing, etc.

[0088] The Gaussian smoothing calculation formula of the histogram is as follows: Wherein h i represents the calculated histogram of the image, r is the smoothing scale size, and σ is the standard deviation.

[0089] The first two extreme points (i.e. the first two maximum peaks) of the smoothed histogram are calculated, and the extreme point calculation formula is H max = max(H i-k ,…,H i ,…,H i+k ), i = 0,…, 255; wherein H i represents the smoothed histogram, and k is the scale interval for obtaining the maximum value.

[0090] According to the extreme point calculation formula, two maximum values can be obtained, and according to the prior knowledge that the sea surface has a lower gray scale and the sky has a higher gray scale, the gray scale values of the first two extreme points are the gray scale estimation values of the sea surface and the sky, the first extreme point (i.e. the maximum extreme point) is the initial seed point of the sky val_sky, and the second extreme point (i.e. the second maximum extreme point) is the initial seed point of the sea surface val_sea.

[0091] Step S202, obtaining the initial seed point of the smoke based on the statistical experience value of the white highlight gray scale of the smoke area and the maximum gray scale of the image;

[0092] If there is smoke in the image, according to the imaging characteristics of the smoke, there must be a white highlight area, and through statistical analysis of the white highlight gray scale range of the smoke area, it can be known that the gray scale value is basically in V s (V s is the statistical value), by calculating the maximum gray scale of the image, and comparing the statistical value of the maximum gray scale with the statistical value of the smoke gray scale, the seed point of the smoke is confirmed. Combined with the gray scale estimation value of the sea surface and the sky obtained in step 1), the initial seed points of the sea surface, the sky and the smoke are confirmed as follows:

[0093]

[0094] wherein, V s is the statistical experience value of the smoke gray scale, and max(img) is the maximum gray scale of the image.

[0095] The statistical experience value V of the smoke gray scale s is calculated by constructing a smoke image library with all the smoke images to be detected, and calculating the average value V of the gray scale of the smoke area in the image library. s

[0096] Step S203, based on the initial seed points of the sea surface, the sky and the smoke, the clustering algorithm is used to cluster the image gray scale, the potential area of the image belonging to the smoke target is segmented and extracted, and the smoke candidate area is obtained.

[0097] Specifically, the Kmeans gray scale clustering algorithm is used to cluster the image gray scale, and the potential area of the image belonging to the smoke target is obtained. After segmentation and extraction, multiple smoke suspected target area blocks are extracted.

[0098] In this embodiment, since the initial seed point is obtained by calculation, the value is relatively accurate, and therefore a smaller number of iterations can be used to obtain an accurate clustering result. According to the clustering result, the potential area of the image belonging to the smoke category is segmented and extracted, and multiple smoke suspected target area blocks are extracted.

[0099] In step S105, for each smoke suspected target area block, the 8 multi-dimensional feature descriptors are extracted respectively, and are sent into the trained classifier to obtain the discrimination result of the classifier. The discrimination method is that the target is output as 1 and the non-target is output as 0. Finally, the target area recognized as smoke is displayed and output with a colored frame.

[0100] In this embodiment, the smoke segmentation method based on gray scale clustering does not consider the information such as the boundary and contrast of the target, and therefore false alarms may occur when the camera is in transition exposure and the sky background is bright. At the same time, since the shape of the smoke is not uniform, the gray scale of the smoke area changes unevenly, and the phenomenon of high gray scale in some areas and low gray scale in other areas occurs. The use of the clustering segmentation algorithm alone will lead to the situation that the same target is segmented into multiple sub-blocks. Based on the above reasons, preferably, in this embodiment, the target area segmented by clustering is further supplemented and connected through edge extraction and fusion.

[0101] Specifically, as Figure 3 shown, the method of edge extraction and fusion for the confirmed smoke candidate area includes:

[0102] Step S301, the edge detection algorithm is used to extract the edge map of the smoke;

[0103] ​Since the optical smoke usually only appears near the sky and the horizon, the contrast between the smoke and the background is high, and there is edge information. Therefore, an edge detection algorithm (such as the Canny edge extraction method) is used to extract the edges of the image to obtain the edge graph of the smoke.

[0104] Specifically, when the edges of the image are extracted, the hough transform is used to extract the horizon, and after the extracted horizon is removed, the remaining edges are used as the edge graph of the smoke.

[0105] Step S302, based on the edge graph information, the smoke is confirmed and the segmentation result is supplemented, and the segmentation graph after fusion edges is obtained.

[0106] For each smoke candidate region, the number of edge points in the position corresponding to the edge graph is counted to determine the matching degree thereof with the smoke candidate region. When the overlap rate condition is met, it is considered that the two have an intersection, and it is determined that the current smoke candidate region is a smoke target region. The union of the clustered and segmented smoke candidate region and the edge graph corresponding to the position thereof is obtained to obtain the segmentation graph after fusion edges.

[0107] Specifically, the segmentation graph after fusion edges is:

[0108]

[0109] In the formula, Seg(i,j) is the clustered and segmented graph, Edge(i,j) is the edge extraction graph, Match(m) is the matching degree of the edge graph corresponding to the position of the mth smoke candidate region and the current mth smoke candidate region,

[0110] Overlap(m) is the overlap rate of the edge graph corresponding to the position of the mth smoke candidate region and the current mth smoke candidate region; m = 1,..., boxnum; boxnum is the total number of smoke candidate regions in the clustered and segmented graph; i, j are the longitudinal and transverse coordinates of the image.

[0111] Specifically, the calculation formula of the matching degree is:

[0112]

[0113] The calculation formula of the overlap rate is:

[0114] Overlap(m) = box[m].w * box[m].h * β

[0115] Wherein, the box[m] is the coordinate information corresponding to the mth smoke candidate area in the clustering segmentation map expressed by a rectangular frame, wherein the box[m].bottom, the box[m].top, the box[m].left and the box[m].right respectively represent the coordinates of the upper boundary, the lower boundary, the left boundary and the right boundary of the detection frame, and the box[m].w and the box[m].h represent the width and the height of the mth rectangular frame; and β is a settable proportion coefficient.

[0116] Step S303, morphological dilation is performed on the segmentation map after the fusion of the edges to obtain a final edge feature fusion map.

[0117] Step S304, connected domain labeling is performed on the edge feature fusion map to obtain a final smoke detection result output.

[0118] In summary, the embodiment of the present application combines the smoke scene under the sky background, and constructs a feature descriptor for the gray scale feature, the texture feature and the gradient feature of the smoke. The feature descriptor construction method is simple and effective, and can quickly realize the smoke feature extraction. The tree classification algorithm is used to realize the fast classification of the constructed smoke feature descriptor. The classification algorithm has low complexity, occupies less hardware resources, and has high running speed. The smoke feature descriptor construction and the classification algorithm used in the present application are simple and effective, occupy less hardware resources, have high running speed, and are suitable for the hardware platform with limited resources.

[0119] The above merely describes the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for optical smoke detection in sea-sky background based on feature extraction, characterized in that, The method comprises the following steps: a sample library of smoke targets, other targets and background targets is established for classifier learning; multi-dimensional feature descriptors are extracted from all samples in the sample library based on image gray features, texture features and gradient features; a classifier is trained using the multi-dimensional feature descriptors extracted from all samples, so that the classifier can recognize smoke targets; for a target image containing optical smoke interference in a sea-sky background to be recognized, the target image is segmented based on smoke gray features, and multiple smoke suspected target region blocks are extracted; for each smoke suspected target region block, the multi-dimensional feature descriptors are extracted and input into the trained classifier for judgment to detect smoke targets; the multi-dimensional feature descriptors extracted based on image gray features are image gray mean value, variance and maximum value features; the multi-dimensional feature descriptors extracted based on image texture features are image gray difference average value, contrast and entropy features; the multi-dimensional feature descriptors extracted based on image gradient features are image vertical gradient mean value and vertical gradient variance features; the target image is segmented based on smoke gray features, and multiple smoke suspected target region blocks are extracted; the method comprises the following steps: 1) gray estimation values of the sea surface and the sky are obtained according to histogram information of the image, and initial seed points of the sea surface and the sky are obtained; 2) initial seed points of smoke are obtained based on statistical experience values of white highlight gray of the smoke region and the maximum value of the gray of the image; 3) the initial seed points of the sea surface, the sky and the smoke are used to perform clustering on the gray of the image by using a clustering algorithm, the potential region of the image belonging to the smoke target is segmented and extracted, and smoke candidate regions are obtained.

2. The method for optical smoke detection in sky background based on feature extraction according to claim 1, characterized in that, The classification algorithm used by the classifier is a tree classification algorithm.

3. The method for optical smoke detection in sky background based on feature extraction as claimed in claim 1, wherein, The histogram is smoothed and peak point detection is performed to obtain the gray estimation values of the sea surface and the sky, and the initial seed points of the sea surface and the sky are obtained; the initial seed points of the sky and the sea surface are the first extreme point and the second extreme point of the histogram of the smoothed image.

4. The method for optical smoke detection in sky background based on feature extraction according to claim 3, characterized in that, Gaussian smoothing is used for the smoothing of the histogram; The Gaussian smoothing calculation formula where h i represents the calculated image histogram, r is the smoothing scale size, and σ is the standard deviation.

5. The optical smoke detection method in a sea-sky background based on feature extraction according to claim 1, wherein The initial seed point of the smoke is wherein, V s is a statistical experience value of the smoke gray scale, and max(img) is a maximum value of the gray scale of the image.

6. The method for optical smoke detection in sky background based on feature extraction as claimed in claim 1, wherein, a Kmeans gray clustering algorithm is used to cluster the gray of the image to obtain the potential region of the image belonging to the smoke target, and multiple smoke suspected target region blocks are obtained after segmentation and extraction.

Images