Foggy day wide-field-of-view video image change detection method and device

The difference graph is generated by the arctangent ratio operator and demean normalization process, and the EN-K-Means clustering algorithm is used to solve the problem of wide-field video image change detection in foggy weather, and the weak changes are achieved Accurate detection and stability improvement.

CN119048463AActive Publication Date: 2024-11-29XINJIANG UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411140469.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-29
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

The existing technology cannot effectively detect weak changes in wide-field video images in foggy weather, resulting in safety hazards and waste of human resources. The existing change detection methods have failed to adapt to the needs of weak change detection in foggy weather environments.

Method used

The arctangent ratio operator is used to generate a difference graph, combined with demean normalization and median filtering, and cluster analysis is performed using the EN-K-Means clustering algorithm based on the pixel's closest relationship, and the changing and invariant classes are segmented.

Benefits of technology

It improves the contrast and noise resistance of the difference map, reduces false alarm situations, and can effectively detect weak changes in wide field of view video images in foggy days, improving the accuracy and stability of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048463A_ABST
    Figure CN119048463A_ABST
Patent Text Reader

Abstract

The invention discloses a foggy day wide-view-field video image change detection method and device, and the method comprises the steps: carrying out the defogging of a single image based on color attenuation prior, and obtaining a defogged clear image; the defogged clear image is processed through an arc tangent ratio operator and an extreme value pixel ratio operator, and two difference images are obtained respectively; performing adaptive fusion on the two obtained difference images by combining the image energy features and the Laplacian pyramid to obtain a fused difference image, and further denoising by combining median filtering; carrying out mean value removal normalization operation on the denoised fusion difference image, and compressing the difference between the changed pixel and the unchanged pixel caused by the noise of the sensor to obtain a final difference image; and an EN-K-Means clustering algorithm of a clustering prototype is initialized based on the pixel nearest neighbor relation to perform clustering analysis on the final difference image to obtain a final binary image. The device comprises a processor and a memory. According to the invention, false alarm conditions can be effectively reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video image change detection, and in particular to a method and device for detecting changes in foggy wide-field video images. Background Art

[0002] The main purpose of video image change detection is to detect the slight changes between different video image frames collected by the Hawkeye vision sensor in the same scene. In foggy weather, the wide-field surveillance video images have low visibility and low contrast. The moving targets are very small and move slowly. For the staff, it is difficult to maintain a high degree of concentration at all times. It is difficult to detect slight changes in the surveillance video when watching multiple surveillances at the same time, and safety hazards cannot be eliminated in time, which easily leads to dangerous incidents. [1] In addition, in reality, the possibility of abnormal situations in most monitoring scenes is small, and the use of a large number of manual monitoring methods will cause a waste of human resources. Therefore, the research on change detection technology based on wide-field video images in foggy weather is particularly important.

[0003] There is no research report on change detection of two multi-temporal monitoring images under foggy weather, but there are some studies on change detection in remote sensing images and lesion detection in medical images. [2] , Video Image Anomaly Detection [3] , Video Salient Object Detection [4] Many studies have been conducted in other areas. Change detection has always been a hot issue in the field of remote sensing images. Its purpose is to identify the differences in the same land cover area from the same sensor (or different sensors) at different times. [5] In order to better identify the changes between two multi-temporal remote sensing images, many methods have been proposed in recent years. Traditional change detection methods are mainly based on pixel-level statistical methods, including direct comparison method, image transformation method, post-classification comparison method, etc. [6-8] Using independent pixels as detection units, the change information is extracted by analyzing the pixel spectral differences pixel by pixel. The main methods for analyzing pixel differences are logarithmic ratio operators. [9] and mean ratio operator

[10] In order to preserve details and suppress noise, many researchers use weight fusion to generate difference maps. Then, threshold methods and cluster analysis are applied to extract variable pixels. Since clustering algorithms such as K-means and FCM do not require the establishment of statistical models, they are popular among many researchers in image change detection.

[0004] Common image change detection methods are not suitable for monitoring video image change detection in foggy weather. Reason 1) Most current change detection methods focus on feature extraction of multi-temporal images, and do not consider the impact of other factors on the detection results, such as: diverse weather conditions or different lighting conditions; Reason 2) At present, most change detection algorithms are often object-oriented and extract change areas based on the spectrum, texture and structure information of adjacent pixels in the image. To a certain extent, they provide inherent noise resistance and can identify changes in large areas, but they are prone to losing details of the change area and ignoring changes in smaller areas. They are not suitable for detecting weakly changing targets in wide-field monitoring scenes.

[0005] Combined with the above analysis, the existing change detection methods have not explored the change detection of wide field of view video images in foggy environment. Therefore, studying the change detection of video images in foggy wide field of view environment has very important practical significance and application value.

[0006] References

[0007] [1]Shi B,Jia Z,Yang J,et al.Unsupervised change detection in wide-field video images under low illumination[J].IEEE Transactions on Circuitsand Systems for Video Technology,2022,33(4):1564-1576.

[0008] [2]Adal KM, Van Etten PG, Martinez JP, et al. An automated system for the detection and classification of retinal changes due to red lesions in longitudinal fundus images [J]. IEEE transactions on biomedical engineering, 2017, 65(6): 1382-1390.

[0009] [3]Zhou J T,Zhang L,Fang Z,et al.Attention-driven loss for anomalydetection in video surveillance[J].IEEE transactions on circuits and systemsfor video technology,2019,30(12):4639-4647.

[0010] [4]Xu M,Liu B,Fu P,et al.Video salient object detection via robustseeds extraction and multi-graphs manifold propagation[J].IEEE Transactionson Circuits and Systems for Video Technology,2019,30(7):2191-2206.

[0011] [5]Lu X,Yuan Y,Zheng X.Joint dictionary learning for multispectralchange detection[J].IEEE transactions on cybernetics,2016,47(4):884-897.

[0012] [6]Hussain M,Chen D,Cheng A,et al.Change detection from remotelysensed images:From pixel-based to object-based approaches[J].ISPRS Journal ofphotogrammetry and remote sensing,2013,80:91-106.

[0013] [7]Karantzalos K.Recent advances on 2D and 3D change detection inurban environments from remote sensing data[J].Computational Approaches forUrban Environments,2015:237-272.

[0014] [8]Lu D,Mausel P,Brondizio E,et al.Change detection techniques[J].International journal of remote sensing,2004,25(12):2365-2401.

[0015] [9]Wang J, Yang X, Yang

[0016]

[10] Li Z, Jia Z, Yang J, et al. A method to improve the accuracy of SARimage change detection by using an image enhancement method [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 163: 137-151. Summary of the invention

[0017] The present invention provides a method and device for detecting changes in a wide-field video image in foggy weather. The present invention proposes an inverse tangent ratio operator for generating a multi-phase image difference map in a foggy environment. The present invention proposes to use a de-mean normalization process for image fusion difference maps to compress the difference between changed and unchanged pixels caused by noise, thereby effectively reducing the occurrence of false alarms. Inspired by the relationship between pixel distance and the probability of pixel and cluster prototype assignment, a new EN-K-Means clustering algorithm based on pixel nearest neighbor relationship initialization cluster prototype is proposed to process the compressed difference map, and the difference map is divided into a change class and an unchanged class, as described below:

[0018] In a first aspect, a method for detecting changes in a wide-field video image in foggy weather is provided, the method comprising:

[0019] Dehaze a single image based on color attenuation prior to obtain a clear image after dehazing;

[0020] The clear image after dehazing is processed by the inverse tangent ratio operator and the extreme pixel ratio operator to obtain two difference maps respectively;

[0021] The two acquired difference images are adaptively fused by combining image energy features and Laplacian pyramid to obtain a fused difference map, which is further denoised by combining median filtering.

[0022] The denoised fused difference image is subjected to a de-mean normalization operation to compress the difference between the changed pixels caused by the sensor noise and the unchanged pixels to obtain the final difference map;

[0023] The EN-K-Means clustering algorithm based on the nearest neighbor relationship of pixels to initialize the clustering prototype is used to perform cluster analysis on the final difference map to obtain the final binary image.

[0024] Wherein, the inverse tangent ratio operator is:

[0025]

[0026] Among them, I1(i, j) and I2(i, j) are the pixel values ​​of two multi-phase video images respectively.

[0027] Among them, the final difference map is:

[0028]

[0029] DI * (i, j) = DI (i, j) - DI mean (i, j)

[0030]

[0031] Where W and H are the width and height of the difference image respectively.

[0032] The EN-K-Means clustering algorithm based on the nearest neighbor relationship of pixels to initialize the clustering prototype performs clustering analysis on the final difference map as follows:

[0033] Find the nearest pixel of each data point based on the Euclidean distance;

[0034]

[0035]

[0036]

[0037] Construct an adjacency matrix based on the nearest neighbor relationship of the data sample:

[0038]

[0039] Z∈R n×n is the adjacency matrix constructed based on the nearest neighbor relationship of the data sample. According to the adjacency matrix, an undirected graph G = (V, E) is constructed. W is recorded (G) is a connected branch of the graph G. The proposed algorithm takes the mean of all data samples in the connected branch of the graph as the initial clustering center:

[0040]

[0041] Among them, W (G) represents the connected branches of its graph, |W (G) | represents the number of data samples in its connected branches;

[0042] The overall strategy of initializing cluster prototypes according to the nearest neighbor relationship depends on the relationship between q and cluster K, where |W| represents the number of data samples in the connected branch after prototype initialization;

[0043]

[0044] Among them, when q=k, the mean of the data samples in the q connected branches is taken as the initial cluster center of the EN-k-means clustering algorithm for iteration; when q>k, that is, the obtained initial cluster prototypes are more than the set k value, the initial cluster prototypes are merged to construct k cluster prototypes; after obtaining the q initial cluster prototype centers, the Euclidean distance is also used to calculate the weights between the q initial mass points in sequence, and the weights are recorded as ω, and the weights between the q mass points are calculated;

[0045]

[0046] Among them, a i represents the i-th cluster center, 1≤i≤q, a j Represents the jth sample, 1≤j≤q, sorts the weight ω in descending order, resets ω, and takes the mean of the data samples in its k cluster prototypes as the initial cluster centroid of the cluster;

[0047]

[0048] When q < k, that is, the number of initial cluster prototypes is less than the set k value, the initial cluster prototype is decomposed to construct k cluster prototypes, the weights σ between the calculated nearest data samples are sorted in descending order, and σ is reset. The sample mean of the k cluster prototypes is used as the centroid of the cluster prototype:

[0049]

[0050] In a second aspect, a device for detecting changes in a wide-field video image in foggy weather comprises: a processor and a memory, wherein the memory stores program instructions, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods described in the first aspect.

[0051] In a third aspect, a computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes any one of the methods described in the first aspect.

[0052] The beneficial effects of the technical solution provided by the present invention are:

[0053] 1. The present invention proposes an inverse tangent pixel ratio operator, which can standardize the pixel values ​​of the difference map generated by it within a limited value range. Its strong nonlinear property makes the difference more obvious and prominent. For relatively small input differences, the difference map will have a larger contrast; compared with other difference operators, its anti-noise performance is stronger;

[0054] 2. The present invention proposes to perform a de-meaning normalization operation on the fused difference image to normalize the fused difference image to the range of [0.1], which can effectively compress the difference between the changed pixels and the unchanged pixels caused by the sensor noise, suppress the noise and reduce the occurrence of false alarms;

[0055] 3. In the final detection stage, the present invention proposes a new EN-K-Means clustering algorithm based on the nearest neighbor relationship of pixels to initialize the clustering prototype to process the compressed difference map. Compared with the general K-means clustering algorithm, it is more stable and stronger, and is applied to the final clustering to obtain the change detection segmentation map. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flow chart of a method for detecting changes in a wide-field video image in foggy weather;

[0057] Figure 2 Schematic diagram of initializing clustering prototypes based on the nearest neighbor relationship of pixels for the EN-K-Means clustering algorithm;

[0058] Among them, (a) is the original pixel distribution map; (b) is the pixel nearest neighbor relationship map; (c) is the updated cluster center map; (d) is the cluster prototype map.

[0059] Figure 3 Schematic diagram of the change detection results of fusing different difference operators;

[0060] Among them, (a) is the time phase Figure 1 ; (b) is the time phase Figure 2; (c) is the reference image; (d) is the result of the Log+Pixel algorithm; (e) is the result of the Arctan+Pixel algorithm.

[0061] Figure 4 This is a schematic diagram of the change detection results under different foggy environments;

[0062] Among them, (a) is the time phase Figure 1 ; (b) is the time phase Figure 2 ; (c) is the reference image; (d) is the algorithm result.

[0063] Figure 5 Schematic diagram of detecting the robustness of the algorithm at different times and in an unchanged environment in a foggy scene.

[0064] Among them, (a) is the phase Figure 1 ; (b) is the time phase Figure 2 ; (c) is the reference image; (d) is the algorithm result. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0066] 1. Image Dehazing Preprocessing

[0067] The surveillance video images captured in foggy environment are often interfered by haze, which easily affects the subsequent change detection effect. It is necessary to perform a defogging operation on the obtained multi-phase surveillance images. In order to improve the accuracy of change detection, the embodiment of the present invention selects to defog a single image based on color attenuation prior. As shown in formula (1), the color attenuation prior, that is, the depth of the scene is positively correlated with the concentration of fog.

[0068] d(x)∝c(x)∝v(x)-s(x) (1)

[0069] Where d is the scene depth, c is the haze concentration, v is the scene brightness, and s is the saturation. The final restored image J is expressed as shown in formula (2).

[0070]

[0071] Among them, I(x) is the video image before defogging, A is the air atmospheric light, β is the atmospheric scattering coefficient, d(x) is the scene depth, and J(x) is the clear image after defogging.

[0072] 2. Difference map generation process

[0073] Among the methods for generating difference maps, the mean ratio operator and the logarithmic ratio operator are more common. The mean ratio operator utilizes the neighborhood information of pixels and has a certain degree of suppression effect on isolated noise points, but lacks scaling transformation and is not easy to effectively suppress noise that appears in the form of patches. The logarithmic ratio operator has the advantage of converting multiplicative coherent noise into additive noise, and after logarithmic transformation, the difference map undergoes nonlinear shrinkage, which enhances the contrast between the change class and the non-change class and has strong anti-noise performance. Inspired by the logarithmic ratio operator, a new inverse tangent ratio operator is proposed, as shown in formula (3):

[0074]

[0075]

[0076] Among them, I1(i, j) and I2(i, j) are the pixel values ​​of two multi-phase video images, DI1(i, j) is the difference map obtained by the inverse tangent ratio operator, and DI2(i, j) is the difference map obtained by the extreme pixel ratio operator. The inverse tangent pixel ratio operator can standardize the pixel values ​​of the difference map generated by it within a limited value range. At the same time, it is a nonlinear function and has a saturation property in response to the input value, which means that for relatively small input differences, the generated difference map will have a larger contrast, making the difference more obvious and prominent. In order to suppress the influence of extreme values, a constant (here is 1) is added when adding or subtracting pixel values, which can reduce the influence of noise or outliers generated by pixel value operations on the difference map. The absolute value of each pixel value of the calculated difference map is taken to ensure that all difference values ​​in the difference map are non-negative, which helps to maintain the contrast of the image and can show the difference between images more clearly. Experiments have shown that compared with the logarithmic ratio and mean ratio operators, it has stronger noise resistance.

[0077] A single difference map generation method cannot take into account both noise suppression and detail preservation. The embodiment of the present invention simultaneously selects an extreme pixel ratio operator to generate a difference map, as shown in formula (4). Although the inverse tangent pixel ratio operator can map pixel values ​​to a bounded range, it can enhance contrast and reduce dynamic range, but when there are large-scale brightness changes or subtle differences in the image, some information of the changed areas in the difference map may be compressed or lost. The extreme pixel ratio operator does not have such a problem. Compared with the inverse tangent pixel ratio operator, the extreme pixel ratio operator is more sensitive to noise, but can retain more detailed information in the changed area. Therefore, the DI obtained by the inverse tangent pixel ratio operator and the extreme pixel ratio operator is merged to maximize the balance between noise resistance and detail extraction.

[0078] Considering that the difference map is a typical gradient image, except for the larger pixel values ​​in the difference area, the pixel values ​​in other areas are close to 0, and the Laplacian pyramid can extract more detailed information at multiple scales, the adaptive fusion difference map method in the LUCD algorithm is selected to combine the energy features of the difference map with the Laplacian pyramid to achieve a better fusion effect.

[0079]

[0080]

[0081]

[0082] Among them, F k represents the image after k-layer horizontal fusion, D1 and D2 represent the difference maps generated by the inverse tangent pixel ratio operator and the extreme pixel ratio operator respectively, and represents the energy of the difference map, Represents the local area energy of the k-th layer of the Laplacian pyramid. M represents the local area matching degree of the k-th layer of the Laplacian pyramid. and Respectively represent the fusion weights of the maximum energy and the minimum energy of the kth layer of the pyramid. In a foggy environment, the purpose of an embodiment of the present invention is to obtain the change area of ​​the multi-phase video image captured by the Hawkeye monitoring device. The noise that affects the image quality in the monitoring images collected by the camera under foggy conditions is mainly the random noise generated by the internal circuit. If these images are directly subjected to change detection, the accuracy of the detection results will be reduced to a certain extent. When there are changes between two multi-phase monitoring images, these changes increase the difference between the change area and the unchanged area, and to a certain extent suppress the noise interference in the change detection process. However, the scenes reflected between the multi-phase monitoring images in video monitoring do not always change. When the scenes reflected between the multi-phase images do not change, the difference in sensor noise between the two multi-phase images will become more significant, which can easily lead to false alarms in the monitoring system.

[0083] Based on the above analysis, the embodiment of the present invention selects to use median filtering to denoise the image, eliminate the isolated noise points in the image, and then perform a demean normalization operation on the fused difference image to normalize the fused difference image to the range of [0.1], in order to compress the difference between the changed pixels and the unchanged pixels caused by the sensor noise. This method suppresses the noise to a certain extent and improves the performance of subsequent clustering.

[0084] First, the pixel mean of the image is calculated, and the mean is subtracted from each value of the data to centralize the data, as shown in formulas (8) and (9), where W and H are the width and height of the difference image, respectively. The final normalized fused difference image can be obtained by formula (10).

[0085]

[0086] DI * (i, j) = DI (i, j) - DI mean (i,j) (9)

[0087]

[0088] 3. Obtaining change detection graph based on EN-K-Means clustering algorithm

[0089] The traditional K-means clustering algorithm can usually complete the clustering task in a relatively short time, and is suitable for real-time processing or large-scale data processing scenarios. However, it is sensitive to the selection of the initial centroid, and the clustering effect depends on the initialization of the cluster center. Different selections of the initial centroid may lead to different clustering results. In order to improve the clustering effect and enhance the stability of its clustering, the embodiment of the present invention combines graph theory to propose an enhanced K-means clustering algorithm based on pixel nearest neighbors, and is named EN-K-means.

[0090] Cluster analysis can automatically divide data objects into multiple categories or clusters by mining the intrinsic structure and regularities in the data without prior knowledge or label information. The division results meet the characteristics of "maximizing similarities within clusters and maximizing differences between clusters". Therefore, generally speaking, the distance between data samples belonging to the same cluster is smaller than the distance between data samples belonging to different clusters. From a probability perspective, the probability that a data sample belongs to the same cluster as its nearest neighbor data sample is the highest. Based on the above analysis, the EN-K-means algorithm proposed in the embodiment of the present invention selects the initial cluster prototype according to the nearest neighbor relationship between pixels.

[0091] Assume that there is a data set X = {x1, x2, ..., x n}, which contains n samples, and each sample has attributes of m dimensions. The proposed algorithm finds the nearest pixel of each data sample point according to the Euclidean distance, as shown in formula (11).

[0092]

[0093] Among them, xi represents the i-th data sample, 1≤i≤n, x j represents the jth sample, 1≤j≤n; x it represents the mth dimension of the ith data sample, x jt represents the mth dimension of the jth data sample. Represents the Euclidean distance between two samples. For each data sample x i , the sample with the shortest distance to it, that is, the nearest sample, can be found through formula (1), as shown in formula (12).

[0094]

[0095]

[0096] Let its nearest neighbor sample As shown in formula (13). Similarly, if x j is x i The nearest sample of x i The same is x j The nearest sample of So The adjacency matrix is ​​constructed based on the nearest neighbor relationship of the data samples. The adjacency matrix construction formula is shown in (14).

[0097]

[0098] Z∈R n×n is an adjacency matrix constructed based on the nearest neighbor relationship of data samples, and an undirected graph G = (V, E) is constructed based on the adjacency matrix. The main diagonal elements of the adjacency matrix constructed based on the nearest neighbor relationship of data samples are all 0, and it is a symmetric matrix. Where V is the set of vertices (data samples), E is the set of edges, and z(i, j) = σ i,j It represents that the i-th data sample is connected to the j-th data sample, and the weight between them is the Euclidean distance calculated by them. (G) is a connected branch of the graph G. The proposed algorithm takes the mean of all data samples in the connected branch of the graph as the initial clustering center. Then it can be obtained according to formula (15).

[0099]

[0100] Among them, W (G) represents the connected branches of its graph, |W (G)| represents the number of data samples in its connected branches. In actual situations, the number of q initial clustering prototypes obtained according to the above steps is often not equal to the number of clusters K set before clustering, so it is necessary to further judge the relationship between q and the number of clusters K. When q=k, directly take the mean of the data samples in the q connected branches as the initial clustering center of the EN-k-means clustering algorithm for iteration. When q>k, that is, the number of initial clustering prototypes obtained is more than the set k value, at this time, the initial clustering prototypes are merged to construct k clustering prototypes. After obtaining the q initial clustering prototype centers, the Euclidean distance is also used to calculate the weights between the q initial particle points in sequence, and the weights are recorded as ω. The weights between the q particle points are calculated according to formula (16).

[0101]

[0102] Among them, a i represents the i-th cluster center, 1≤i≤q, a j Denotes the jth sample, 1≤j≤q. Sort the weights ω in descending order and reset ω according to formula (17). Take the mean of the data samples in the k cluster prototypes as the initial cluster centroid of the cluster.

[0103]

[0104] When q < k, that is, the number of initial cluster prototypes obtained is less than the set k value, then the initial cluster prototype is decomposed to construct k cluster prototypes. The weights σ between the calculated nearest neighbor data samples are sorted in descending order, and σ is reset according to formula (18). The sample mean of the k cluster prototypes is taken as the centroid of the cluster prototype.

[0105]

[0106] The strategy of initializing the cluster prototype according to the nearest neighbor relationship is shown in formula (19). Where |W| represents the number of data samples in the connected branch after the prototype is initialized.

[0107]

[0108] In order to verify that the proposed change detection algorithm is effective in low-illumination and wide-field-of-view environments, experiments are carried out in different environments.

[0109] 4. Difference Map Quality Analysis

[0110] from Figure 3 It can be seen that the proposed difference map operator and difference map fusion method obtain the optimal difference map.

[0111] 5. Scene Change Detection in Foggy Weather

[0112] 1) Change detection in simple foggy scenes

[0113] from Figure 4 It can be seen from the first two or three scenes that the detection result of the embodiment of the present invention is very close to the real reference image, and can better preserve the edge contour of the changed area.

[0114] 2) Change detection in complex foggy scenes

[0115] In order to further test the anti-noise performance of the present invention in a complex environment, Figure 4 As shown in the fourth and fifth scenes in Figure 3, although the background is more complex than the first two scenes, the detection results are still close to the actual changes and can accurately detect the changed areas.

[0116] 3) Change detection in multi-target foggy scenes

[0117] Figure 4 In the sixth scene, there is not only one target but multiple targets. From the change detection results, it can be seen that the present invention can accurately detect the change area, whether it is single target change detection or multi-target change detection, and can better preserve the edge contour of the change area.

[0118] 4) Change detection in non-changing foggy scenes

[0119] In order to test the robustness of the invented algorithm, four groups of multi-phase video images of two scenes with different time intervals were selected for testing. Figure 5 As shown in the figure, it can be found from the detection results that there is no extra noise influence and no false alarm occurs, which shows that the algorithm has strong robustness.

[0120] A device for detecting changes in a wide-field video image in foggy weather, the device comprising: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute the following method steps in Example 1:

[0121] Dehaze a single image based on color attenuation prior to obtain a clear image after dehazing;

[0122] The clear image after dehazing is processed by the inverse tangent ratio operator and the extreme pixel ratio operator to obtain two difference maps respectively;

[0123] The two acquired difference images are adaptively fused by combining image energy features and Laplacian pyramid to obtain a fused difference map, which is further denoised by combining median filtering.

[0124] The denoised fused difference image is subjected to a de-mean normalization operation to compress the difference between the changed pixels caused by the sensor noise and the unchanged pixels to obtain the final difference map;

[0125] The EN-K-Means clustering algorithm based on the nearest neighbor relationship of pixels to initialize the clustering prototype is used to perform cluster analysis on the final difference map to obtain the final binary image.

[0126] Among them, the inverse tangent ratio operator is:

[0127]

[0128] Among them, I1(i, j) and I2(i, j) are the pixel values ​​of two multi-phase video images respectively.

[0129] The final difference map is:

[0130]

[0131] DI * (i, j) = DI (i, j) - DI mean (i, j)

[0132]

[0133] Where W and H are the width and height of the difference image respectively.

[0134] Among them, the EN-K-Means clustering algorithm based on the nearest neighbor relationship of pixels to initialize the clustering prototype performs clustering analysis on the final difference map as follows:

[0135] Find the nearest pixel of each data point based on the Euclidean distance;

[0136]

[0137]

[0138]

[0139] Construct an adjacency matrix based on the nearest neighbor relationship of the data sample:

[0140]

[0141] Z∈R n×n is the adjacency matrix constructed based on the nearest neighbor relationship of the data sample. According to the adjacency matrix, an undirected graph G = (V, E) is constructed. W is recorded (G) is a connected branch of the graph G. The proposed algorithm takes the mean of all data samples in the connected branch of the graph as the initial clustering center:

[0142]

[0143] Among them, W (G) represents the connected branches of its graph, |W (G)| represents the number of data samples in its connected branches;

[0144] The overall strategy of initializing cluster prototypes according to the nearest neighbor relationship depends on the relationship between q and cluster K, where |W| represents the number of data samples in the connected branch after prototype initialization;

[0145]

[0146] Among them, when q=k, the mean of the data samples in the q connected branches is taken as the initial cluster center of the EN-k-means clustering algorithm for iteration; when q>k, that is, the obtained initial cluster prototypes are more than the set k value, the initial cluster prototypes are merged to construct k cluster prototypes; after obtaining the q initial cluster prototype centers, the Euclidean distance is also used to calculate the weights between the q initial mass points in sequence, and the weights are recorded as ω, and the weights between the q mass points are calculated;

[0147]

[0148] Among them, a i represents the i-th cluster center, 1≤i≤q, a j Represents the jth sample, 1≤j≤q, sorts the weight ω in descending order, resets ω, and takes the mean of the data samples in its k cluster prototypes as the initial cluster centroid of the cluster;

[0149]

[0150] When q < k, that is, the number of initial cluster prototypes is less than the set k value, the initial cluster prototype is decomposed to construct k cluster prototypes, the weights σ between the calculated nearest data samples are sorted in descending order, and σ is reset. The sample mean of the k cluster prototypes is used as the centroid of the cluster prototype:

[0151]

[0152] It should be pointed out here that the device description in the above embodiment corresponds to the method description in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0153] The execution subjects of the above-mentioned processor and memory can be devices with computing functions such as computers, single-chip microcomputers, and microcontrollers. In specific implementation, the embodiments of the present invention do not limit the execution subjects and are selected according to the needs of actual applications.

[0154] The data signal is transmitted between the memory and the processor via a bus, which is not described in detail in the embodiment of the present invention.

[0155] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, the storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the method steps in the above embodiment.

[0156] The computer-readable storage medium includes but is not limited to a flash memory, a hard disk, a solid-state drive, and the like.

[0157] It should be pointed out here that the description of the readable storage medium in the above embodiment corresponds to the description of the method in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0158] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present invention are generated.

[0159] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by the computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium or a semiconductor medium, etc.

[0160] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0161] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0162] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting changes in a wide-field video image in foggy weather, characterized in that: The method comprises: Dehaze a single image based on color attenuation prior to obtain a clear image after dehazing; The clear image after dehazing is processed by the inverse tangent ratio operator and the extreme pixel ratio operator to obtain two difference maps respectively; The two acquired difference images are adaptively fused by combining image energy features and Laplacian pyramid to obtain a fused difference map, which is further denoised by combining median filtering. The denoised fused difference image is subjected to a de-mean normalization operation to compress the difference between the changed pixels caused by the sensor noise and the unchanged pixels to obtain the final difference map; The EN-K-Means clustering algorithm based on the nearest neighbor relationship of pixels to initialize the clustering prototype is used to perform cluster analysis on the final difference map to obtain the final binary image.

2. The method for detecting changes in a wide-field video image in foggy weather according to claim 1, characterized in that: The inverse tangent ratio operator is: Among them, I1(i, j) and I2(i, j) are the pixel values ​​of two multi-phase video images respectively.

3. The method for detecting changes in foggy wide-field video images according to claim 1, characterized in that: The final difference map is: IN * (i, j)=DI(i, j)-DI mean (i, j) Where W and H are the width and height of the difference image respectively.

4. The method for detecting changes in foggy wide-field video images according to claim 1, characterized in that: The EN-K-Means clustering algorithm based on the nearest neighbor relationship of pixels to initialize the clustering prototype performs clustering analysis on the final difference map as follows: Find the nearest pixel of each data point based on the Euclidean distance; Construct an adjacency matrix based on the nearest neighbor relationship of the data sample: Z∈R n×n is the adjacency matrix constructed based on the nearest neighbor relationship of the data sample. According to the adjacency matrix, an undirected graph G = (V, E) is constructed. W is recorded (G) is a connected branch of the graph G. The proposed algorithm takes the mean of all data samples in the connected branch of the graph as the initial clustering center: Among them, W (G) represents the connected branches of its graph, |W (G) | represents the number of data samples in its connected branches; The overall strategy of initializing cluster prototypes according to the nearest neighbor relationship depends on the relationship between q and cluster K, where |W| represents the number of data samples in the connected branch after prototype initialization; Among them, when q = k, the mean of the data samples in the q connected branches is taken as the initial cluster center of the EN-k-means clustering algorithm for iteration; when q>k, that is, the initial cluster prototypes obtained are more than the set k value, the initial cluster prototypes are merged to construct k cluster prototypes; after obtaining the q initial cluster prototype centers, the Euclidean distance is also used to calculate the weights between the q initial mass points in sequence, and the weights are recorded as ω, and the weights between the q mass points are calculated; Among them, a i represents the i-th cluster center, 1≤i≤q, a j Represents the jth sample, 1≤j≤q, sorts the weight ω in descending order, resets ω, and takes the mean of the data samples in its k cluster prototypes as the initial cluster centroid of the cluster; When q < k, that is, the number of initial cluster prototypes is less than the set k value, the initial cluster prototype is decomposed to construct k cluster prototypes, the weights σ between the calculated nearest data samples are sorted in descending order, and v is reset. The sample mean of the k cluster prototypes is used as the centroid of the cluster prototype:

5. A device for detecting changes in foggy wide-field video images, characterized in that: The device comprises: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute the method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is enabled to perform the method according to any one of claims 1 to 4.