A method and device for detecting changes in wide field of view video images in foggy weather

Abstract

The application discloses a kind of fog wide field of view video image change detection method and device, method includes: based on color attenuation priori to single image is defogged, obtains clear image after defogging;By arctangent ratio operator and extreme pixel ratio operator, clear image after defogging is handled, respectively obtain two difference maps;Combining image energy features and Laplacian pyramid, the two difference images obtained are adaptively fused to obtain fused difference map and further denoising is combined with median filtering;To denoised fusion difference image, difference between change and unchanged pixel caused by sensor noise is compressed to obtain final difference map;Based on the nearest neighbor relationship of pixel, the EN-K-Means clustering algorithm of clustering prototype is initialized to carry out clustering analysis on final difference map to obtain final binary image.The device includes: processor and memory.The present application can effectively reduce the occurrence of false alarm.

Description

Technical Field

[0001] This invention relates to the field of video image change detection, and more particularly to a method and apparatus for detecting changes in wide-field-of-view video images in foggy weather. Background Technology

[0002] The primary purpose of video image change detection is to detect subtle changes between different video image frames captured by an eagle-eye vision sensor in the same scene. In foggy weather, wide-field-of-view surveillance video images have low visibility and contrast, and moving targets are very small and slow-moving. For staff, it is difficult to maintain a high level of concentration at all times, and viewing multiple monitors simultaneously makes it difficult to detect subtle changes in the video in a timely manner, thus failing to eliminate potential safety hazards and easily leading to dangerous incidents. [1] Furthermore, in reality, the likelihood of anomalies in most monitoring scenarios is relatively low, and relying heavily on manual monitoring would be a waste of human resources. Therefore, research on wide-field-of-view video image change detection technology in foggy conditions is particularly important.

[0003] There are currently no research reports on change detection between two multi-temporal monitoring images under foggy weather conditions, but there are applications in remote sensing image change detection and lesion detection in medical images. [2] Video image anomaly detection [3] Video salient object detection [4] Numerous studies have been conducted in other areas as well. Change detection has always been a hot topic in the field of remote sensing imagery, with the aim of identifying differences in the same land cover area at different times using the same sensor (or different sensors). [5] To better identify changes between two multi-temporal remote sensing images, many methods have been proposed in recent years. Traditional change detection methods mainly rely on pixel-level statistical methods, including direct comparison methods, image transformation methods, and post-classification comparison methods. [6-8] Using individual pixels as detection units, this method extracts change information by analyzing pixel-by-pixel spectral differences. The main methods for analyzing pixel differences include the logarithmic ratio operator. [9] Ratio to Mean Operator

[10] To preserve details and suppress noise, many researchers use weighted fusion to generate difference maps. They then apply thresholding methods and cluster analysis to extract variable pixels. Since clustering algorithms such as K-means and FCM do not require statistical models, they are popular in image change detection.

[0004] Common image change detection methods are not suitable for detecting changes in surveillance video images in foggy weather. Reason 1) Most current change detection methods focus on feature extraction from multi-temporal images, without considering the impact of other factors on the detection results, such as diverse weather conditions or different lighting conditions. Reason 2) Currently, most change detection algorithms are object-oriented, extracting changed regions based on the spectral, texture, and structural information of neighboring pixels. While this provides some inherent noise resistance and can identify changes in large areas, it easily loses details in the changed areas and ignores changes in smaller regions, making it unsuitable for detecting subtle changes in targets in wide-field-of-view surveillance scenarios.

[0005] Based on the above analysis, existing change detection methods have not explored suitable methods for change detection in wide-field-of-view video images under foggy conditions. Therefore, researching change detection in video images under wide-field-of-view conditions in foggy weather has significant practical implications and application value.

[0006] References

[0007] [1]Shi B,Jia Z,Yang J,et al.Unsupervised change detection in wide-field video images under low illumination[J].IEEE Transactions on Circuitsand Systems for Video Technology,2022,33(4):1564-1576.

[0008] [2]Adal KM, Van Etten PG, Martinez JP, et al. An automated system for the detection and classification of retinal changes due to red lesions in longitudinal fundus images [J]. IEEE transactions on biomedical engineering, 2017, 65(6): 1382-1390.

[0009] [3]Zhou J T,Zhang L,Fang Z,et al.Attention-driven loss for anomalydetection in video surveillance[J].IEEE transactions on circuits and systemsfor video technology,2019,30(12):4639-4647.

[0010] [4]Xu M,Liu B,Fu P,et al.Video salient object detection via robustseeds extraction and multi-graphs manifold propagation[J].IEEE Transactionson Circuits and Systems for Video Technology,2019,30(7):2191-2206.

[0011] [5]Lu X,Yuan Y,Zheng X.Joint dictionary learning for multispectralchange detection[J].IEEE transactions on cybernetics,2016,47(4):884-897.

[0012] [6]Hussain M,Chen D,Cheng A,et al.Change detection from remotelysensed images:From pixel-based to object-based approaches[J].ISPRS Journal ofphotogrammetry and remote sensing,2013,80:91-106.

[0013] [7]Karantzalos K.Recent advances on 2D and 3D change detection inurban environments from remote sensing data[J].Computational Approaches forUrban Environments,2015:237-272.

[0014] [8]Lu D,Mausel P,Brondizio E,et al.Change detection techniques[J].International journal of remote sensing,2004,25(12):2365-2401.

[0015] [9]Wang J, Yang X, Yang

[0016]

[10] Li Z, Jia Z, Yang J, et al. A method to improve the accuracy of SARimage change detection by using an image enhancement method [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 163: 137-151. Summary of the Invention

[0017] This invention provides a method and apparatus for detecting changes in wide-field-of-view video images in foggy conditions. It proposes an arctangent ratio operator to generate multi-temporal image difference maps under foggy conditions. The invention also proposes applying mean-free normalization to the image fusion difference map, compressing the differences between noise-induced changes and unchanged pixels, effectively reducing false alarms. Inspired by pixel distance and the probability relationship between pixels and clustering prototypes, a novel EN-K-Means clustering algorithm based on pixel nearest neighbor relationships is proposed to process the compressed difference map, dividing it into changed and invariant classes, as detailed below.

[0018] A first aspect: a method for detecting changes in wide-field-of-view video images in foggy weather, the method comprising:

[0019] Dehazing is performed on a single image based on color attenuation prior, resulting in a clear image after dehazing.

[0020] The clear image after dehazing is processed by the arctangent ratio operator and the extreme pixel ratio operator to obtain two difference images.

[0021] The two acquired difference images are adaptively fused by combining image energy features and Laplacian pyramid to obtain a fused difference map, and then further denoised by median filtering;

[0022] The denoised fused difference image is subjected to mean normalization to compress the difference between the changes caused by sensor noise and the unchanged pixels to obtain the final difference map.

[0023] The EN-K-Means clustering algorithm, which initializes the clustering prototype based on the pixel nearest neighbor relationship, performs clustering analysis on the final difference map to obtain the final binary image.

[0024] The arctangent ratio operator is:

[0025]

[0026] Where I1(i,j) and I2(i,j) are the pixel values ​​of the two multi-temporal video images, respectively.

[0027] The final difference diagram is as follows:

[0028]

[0029] DI * (i,j)=DI(i,j)-DI mean (i, j)

[0030]

[0031] Where W and H are the width and height of the difference image, respectively.

[0032] The EN-K-Means clustering algorithm, which initializes the clustering prototype based on pixel nearest neighbor relationships, performs clustering analysis on the final difference map as follows:

[0033] Find the nearest neighbor pixel for each data point using Euclidean distance;

[0034]

[0035]

[0036]

[0037] Construct an adjacency matrix based on the nearest neighbor relationships of the data samples:

[0038]

[0039] Z∈R n×n Let W be an adjacency matrix constructed based on the nearest neighbor relationships of the data samples. Then, construct an undirected graph G = (V, E) based on the adjacency matrix, and denote W as... (G) Given a connected component of graph G, the proposed algorithm uses the mean of all data samples in the connected component as the initial cluster center:

[0040]

[0041] Among them, W (G) Denotes the connected components of its graph, |W (G) | indicates the number of data samples within its connected component;

[0042] The overall strategy of initializing the clustering prototype based on the nearest neighbor relationship depends on the relationship between q and cluster K, where |W| represents the number of data samples in the connected component after prototype initialization;

[0043]

[0044] When q = k, the mean of the data samples within the q connected components is taken as the initial cluster center of the EN-k-means clustering algorithm for iteration; when q > k, that is, the number of initial cluster prototypes obtained is greater than the set value of k, the initial cluster prototypes are merged to construct k cluster prototypes; after obtaining the q initial cluster prototype centers, the weights between the q initial points are calculated sequentially using Euclidean distance, and the weights are denoted as ω, and the weights between the q points are calculated.

[0045]

[0046] Among them, a i Let a represent the i-th cluster center, 1≤i≤q. j Let represent the j-th sample, 1≤j≤q. Sort the samples in descending order of weight ω, reset ω, and take the mean of the data samples within the k cluster prototypes as the initial cluster centroids.

[0047]

[0048] When q < k, meaning the initial cluster prototypes are fewer than the set value k, the initial cluster prototypes are decomposed to construct k cluster prototypes. The weights σ between the nearest neighbor data samples that have been calculated are sorted in descending order, and σ is reset. The mean of the samples in the k cluster prototypes is used as the centroid of the cluster prototypes.

[0049]

[0050] In a second aspect, a wide-field-of-view video image change detection device in foggy weather, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the method described in any one of the first aspects.

[0051] Third aspect, a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of the first aspects.

[0052] The beneficial effects of the technical solution provided by this invention are:

[0053] 1. This invention proposes an arctangent pixel ratio operator, which can standardize the pixel values ​​of the generated difference map within a limited range. Its strong nonlinear properties make the differences more obvious and prominent. For relatively small input differences, the difference map will have a large contrast. Compared with other difference operators, it has stronger noise resistance.

[0054] 2. The present invention proposes to perform mean normalization on the fused difference image to normalize the fused difference image to the range of [0.1], which can effectively compress the difference between changed pixels and unchanged pixels caused by sensor noise, suppress noise and reduce the occurrence of false alarms.

[0055] 3. In the final detection stage, this invention proposes a new EN-K-Means clustering algorithm based on pixel nearest neighbor relationship initialization to process the compressed difference map. It is more stable than the general K-means clustering algorithm and is applied to the final clustering to obtain the change detection segmentation map. Attached Figure Description

[0056] Figure 1 A flowchart of a method for detecting changes in wide-field-of-view video images in foggy weather;

[0057] Figure 2 A schematic diagram illustrating the initialization of the clustering prototype based on pixel nearest neighbor relationships for the EN-K-Means clustering algorithm;

[0058] Among them, (a) is the original pixel distribution map; (b) is the pixel nearest neighbor relationship map; (c) is the updated cluster center map; and (d) is the cluster prototype map.

[0059] Figure 3 A schematic diagram illustrating the fusion of change detection results from different difference operators;

[0060] Where (a) represents the time phase. Figure 1 (b) is the phase. Figure 2(c) is the reference image; (d) is the result of the Log+Pixel algorithm; (e) is the result of the Arctan+Pixel algorithm.

[0061] Figure 4 This is a schematic diagram showing the changes in detection results under different foggy conditions;

[0062] Where (a) represents the time phase. Figure 1 (b) is the phase. Figure 2 (c) is the reference diagram; (d) is the algorithm result.

[0063] Figure 5 This diagram illustrates the robustness of the algorithm for detecting foggy weather conditions at different times while remaining constant.

[0064] Where (a) is the phase Figure 1 (b) is the phase. Figure 2 (c) is the reference diagram; (d) is the algorithm result. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0066] I. Image Dehazing Preprocessing

[0067] Surveillance video images captured in foggy conditions are often affected by haze, which can easily affect the subsequent change detection effect. Therefore, it is necessary to perform dehazing on the acquired multi-temporal surveillance images. In order to improve the accuracy of change detection, this embodiment of the invention selects to perform dehazing on a single image based on color attenuation prior. As shown in formula (1), the color attenuation prior means that the depth of the scene is positively correlated with the concentration of fog.

[0068] d(x)∝c(x)∝v(x)-s(x) (1)

[0069] Where d is the scene depth, c is the haze concentration, v is the scene brightness, and s is the saturation. The final restored image J is expressed as shown in formula (2).

[0070]

[0071] Where I(x) is the video image before defogging, A is atmospheric light, β is the atmospheric scattering coefficient, d(x) is the scene depth, and J(x) is the clear image after defogging.

[0072] II. Difference Map Generation Process

[0073] Among the methods for generating difference maps, the mean ratio operator and the logarithmic ratio operator are more common. The mean ratio operator utilizes the neighborhood information of pixels and has a certain degree of suppression effect on noise points that appear alone, but it lacks scaling transformation and is not easy to effectively suppress noise that appears in patches. The logarithmic ratio operator has the advantage of converting multiplicative coherent noise into additive noise, and after logarithmic transformation, the difference map undergoes nonlinear shrinkage, which enhances the contrast between the changed and unchanged classes and has strong anti-noise performance. Inspired by the logarithmic ratio operator, a novel arctangent ratio operator is proposed, as shown in formula (3):

[0074]

[0075]

[0076] Here, I1(i,j) and I2(i,j) are the pixel values ​​of two multi-temporal video images, DI1(i,j) is the difference map obtained by the arctangent ratio operator, and DI2(i,j) is the difference map obtained by the extreme pixel ratio operator. Using the arctangent pixel ratio operator can normalize the pixel values ​​of the generated difference map within a finite range. Furthermore, it is a non-linear function with a saturation property in its response to input values. This means that for relatively small input differences, the generated difference map will have greater contrast, making the differences more obvious and prominent. To suppress the influence of extreme values, a constant (here, 1) is added when adding or subtracting pixel values, which can reduce the impact of noise or outliers generated by pixel value operations on the difference map. Taking the absolute value of each pixel value in the calculated difference map ensures that all difference values ​​in the difference map are non-negative, which helps maintain image contrast and more clearly displays the differences between images. Experiments show that it has stronger noise resistance than the logarithmic ratio and mean ratio operators.

[0077] A single method of generating a difference map cannot simultaneously suppress noise and preserve details. Therefore, this embodiment of the invention simultaneously selects the extreme pixel ratio operator to generate the difference map, as shown in formula (4). Although the arctangent pixel ratio operator can map pixel values ​​to a bounded range, enhancing contrast and reducing dynamic range, when there are large-scale brightness changes or subtle differences in the image, some information in the change areas of the difference map may be compressed or lost. The extreme pixel ratio operator does not have this problem. Compared to the arctangent pixel ratio operator, the extreme pixel ratio operator, while more sensitive to noise, can retain more detailed information in the change areas. Therefore, fusing the DI obtained from the arctangent pixel ratio operator and the extreme pixel ratio operator can maximize both noise resistance and detail extraction.

[0078] Considering that the difference map is a typical gradient image, with pixel values ​​close to 0 in areas other than the difference region, and that the Laplacian pyramid can extract more detailed information at multiple scales, the adaptive fusion difference map method in the LUCD algorithm is chosen to combine the energy features of the difference map with the Laplacian pyramid to achieve better fusion results.

[0079]

[0080]

[0081]

[0082] Among them, F k This represents the image after k-layer horizontal fusion, where D1 and D2 represent the difference maps generated by the arctangent pixel ratio operator and the extreme pixel ratio operator, respectively. and The energy representing the difference plot. represents the local energy of the k-th layer of the Laplace pyramid. M represents the local matching degree of the k-th layer of the Laplace pyramid. and These represent the fusion weights of the maximum and minimum energies at the k-th layer of the pyramid, respectively. In foggy conditions, the purpose of this invention is to obtain the change areas in multi-temporal video images captured by the eagle-eye monitoring device. In foggy conditions, the noise affecting image quality in the monitoring images acquired by the camera is mainly random noise generated by internal circuitry. Directly performing change detection on these images can reduce the accuracy of the detection results to some extent. When there are changes between two multi-temporal monitoring images, these changes increase the difference between the changed and unchanged areas, suppressing noise interference during change detection to some extent. However, the scenes reflected in multi-temporal monitoring images in video surveillance do not always change. When the scenes reflected in multi-temporal images do not change, the difference in sensor noise between the two multi-temporal images becomes more significant, easily leading to false alarms in the monitoring system.

[0083] Based on the above analysis, this embodiment of the invention selects median filtering to denoise the image, eliminating isolated noise points in the image. Then, a mean-free normalization operation is performed on the fused difference image to normalize it to the range [0.1]. The purpose is to compress the difference between changed and unchanged pixels caused by sensor noise. This method suppresses noise to a certain extent and improves the performance of subsequent clustering.

[0084] First, calculate the pixel mean of the image, subtract the mean from each value of the data to normalize the data, as shown in formulas (8) and (9), where W and H are the width and height of the difference image, respectively. Finally, the normalized fused difference image can be obtained by formula (10).

[0085]

[0086] DI * (i,j)=DI(i,j)-DI mean (i, j) (9)

[0087]

[0088] III. Change detection map obtained based on EN-K-Means clustering algorithm

[0089] Traditional K-means clustering algorithms can typically complete clustering tasks in a short time, making them suitable for real-time or large-scale data processing scenarios. However, they are sensitive to the choice of initial centroids, and the clustering effect depends on the initialization of the cluster centers; different choices of initial centroids may lead to different clustering results. To improve the clustering effect and enhance its stability, this invention proposes an enhanced K-means clustering algorithm based on pixel nearest neighbors, named EN-K-means, incorporating graph theory.

[0090] Cluster analysis can automatically divide data objects into multiple categories or clusters by mining the inherent structure and patterns in data, without prior knowledge or label information. The division results conform to the characteristics of "maximizing similarity within clusters and maximizing differences between clusters." Therefore, generally speaking, the distance between data samples belonging to the same cluster is smaller than the distance between data samples belonging to different clusters. From a probabilistic perspective, a data sample has the highest probability of belonging to the same cluster as its nearest neighbor. Based on the above analysis, the EN-K-means algorithm proposed in this embodiment selects the initial cluster prototype based on the nearest neighbor relationship between pixels.

[0091] Suppose we have a dataset X = {x1, x2, ..., x...} n The dataset contains n samples, each with m-dimensional attributes. The proposed algorithm finds the nearest neighbor pixel for each data sample point based on Euclidean distance, as shown in Equation (11).

[0092]

[0093] Where, xi Let x represent the i-th data sample, 1≤i≤n. j Let x represent the j-th sample, 1≤j≤n; it Let x represent the m-th dimension of the i-th data sample. jt This represents the m-th dimension of the j-th data sample. This represents the Euclidean distance between two samples. For each data sample x... i The nearest sample, i.e. the closest sample, can be found by formula (1), as shown in formula (12).

[0094]

[0095]

[0096] Let its nearest neighbor sample As shown in formula (13). Similarly, if x j It is x i The nearest neighbor sample of x, then i Also x j The nearest neighbor sample, i.e. So The adjacency matrix is ​​constructed based on the nearest neighbor relationship of the data samples. The formula for constructing the adjacency matrix is ​​shown in (14).

[0097]

[0098] Z∈R n×n Let G = (V, E) be an undirected graph constructed based on the adjacency matrix constructed from the nearest neighbor relationships of the data samples. The adjacency matrix constructed from the nearest neighbor relationships of the data samples has all zero elements on its main diagonal and is a symmetric matrix. Here, V is the set of vertices (data samples), E is the set of edges, and z(i, j) = σ i，j This represents the connection between the i-th and j-th data samples, with their weights being the calculated Euclidean distance. Let W... (G) Let G be a connected component of the graph. The proposed algorithm uses the mean of all data samples in the connected component of the graph as the initial cluster center. Then, it can be obtained according to formula (15).

[0099]

[0100] Among them, W (G) Denotes the connected components of its graph, |W (G)| represents the number of data samples within its connected components. In practice, the number of q initial cluster prototypes obtained from the above steps is often not equal to the number of clusters K set before clustering. Therefore, it is necessary to further determine the relationship between q and the number of clusters K. When q = k, the mean of the data samples within the q connected components is directly taken as the initial cluster center of the EN-k-means clustering algorithm for iteration. When q > k, that is, the number of initial cluster prototypes obtained is more than the set value of k, the initial cluster prototypes are merged to construct k cluster prototypes. After obtaining the centers of the q initial cluster prototypes, the weights between the q initial points are calculated sequentially using Euclidean distance, and the weights are denoted as ω. The weights between the q points are calculated according to formula (16).

[0101]

[0102] Among them, a i Let a represent the i-th cluster center, 1≤i≤q. j Let represent the j-th sample, 1≤j≤q. Sort the weights ω in descending order and reset ω according to formula (17). Take the mean of the data samples within its k cluster prototypes as the initial cluster centroids.

[0103]

[0104] When q < k, meaning the initial cluster prototype is less than the set value k, the initial cluster prototype is decomposed to construct k cluster prototypes. The weights σ between the nearest neighbor data samples that have been calculated are sorted in descending order, and σ is reset according to formula (18). The mean of the samples in the k cluster prototypes is used as the centroid of the cluster prototype.

[0105]

[0106] The strategy for initializing the cluster prototype based on the nearest neighbor relationship is shown in formula (19). Where |W| represents the number of data samples in the connected component after prototype initialization.

[0107]

[0108] To verify the effectiveness of the proposed change detection algorithm in low-light wide field-of-view environments, experiments were conducted in different environments.

[0109] IV. Quality Analysis of Difference Charts

[0110] from Figure 3 It can be seen that the proposed difference graph operator and difference graph fusion method yielded the optimal difference graph.

[0111] V. Detection of changes in foggy weather scenes

[0112] 1) Change detection in simple foggy scenes

[0113] from Figure 4 As can be seen from the first two or three scenarios, the detection results of this embodiment of the invention are very close to the real reference image, and at the same time, they can better preserve the edge contour of the changing area.

[0114] 2) Change detection in complex foggy scenes

[0115] To further test the noise immunity performance of the present invention in complex environments, such as Figure 4 As shown in the fourth and fifth scenarios, although the background is more complex than the first two scenarios, the detection results are still close to the real changes and can accurately detect the changed areas.

[0116] 3) Change detection in multi-target foggy scenes

[0117] Figure 4 In the sixth scenario, there are multiple changing targets, not just one. The change detection results show that, whether it's single-target or multi-target change detection, this invention can accurately detect the changed areas and effectively preserve their edge contours.

[0118] 4) Change detection in unchanging foggy scenes

[0119] To test the robustness of the invented algorithm, four sets of multi-temporal video images from two scenes at different time intervals were selected for detection. For example... Figure 5 As shown in the results, there is no extra noise or false alarms, indicating that the algorithm has strong robustness.

[0120] A wide-field-of-view video image change detection device in foggy weather, the device includes: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to execute the following method steps in Embodiment 1:

[0121] Dehazing is performed on a single image based on color attenuation prior, resulting in a clear image after dehazing.

[0122] The clear image after dehazing is processed by the arctangent ratio operator and the extreme pixel ratio operator to obtain two difference images.

[0123] The two acquired difference images are adaptively fused by combining image energy features and Laplacian pyramid to obtain a fused difference map, and then further denoised by median filtering;

[0124] The denoised fused difference image is subjected to mean normalization to compress the difference between the changes caused by sensor noise and the unchanged pixels to obtain the final difference map.

[0125] The EN-K-Means clustering algorithm, which initializes the clustering prototype based on the pixel nearest neighbor relationship, performs clustering analysis on the final difference map to obtain the final binary image.

[0126] The arctangent ratio operator is:

[0127]

[0128] Where I1(i,j) and I2(i,j) are the pixel values ​​of the two multi-temporal video images, respectively.

[0129] The final difference graph is as follows:

[0130]

[0131] DI * (i,j)=DI(i,j)-DI mean (i, j)

[0132]

[0133] Where W and H are the width and height of the difference image, respectively.

[0134] Among them, the EN-K-Means clustering algorithm, which initializes the clustering prototype based on the nearest neighbor relationship of pixels, performs clustering analysis on the final difference map as follows:

[0135] Find the nearest neighbor pixel for each data point using Euclidean distance;

[0136]

[0137]

[0138]

[0139] Construct an adjacency matrix based on the nearest neighbor relationships of the data samples:

[0140]

[0141] Z∈R n×n Let W be an adjacency matrix constructed based on the nearest neighbor relationships of the data samples. Then, construct an undirected graph G = (V, E) based on the adjacency matrix, and denote W as... (G) Given a connected component of graph G, the proposed algorithm uses the mean of all data samples in the connected component as the initial cluster center:

[0142]

[0143] Among them, W (G) Denotes the connected components of its graph, |W (G)| indicates the number of data samples within its connected component;

[0144] The overall strategy of initializing the clustering prototype based on the nearest neighbor relationship depends on the relationship between q and cluster K, where |W| represents the number of data samples in the connected component after prototype initialization;

[0145]

[0146] When q = k, the mean of the data samples within the q connected components is taken as the initial cluster center of the EN-k-means clustering algorithm for iteration; when q > k, that is, the number of initial cluster prototypes obtained is greater than the set value of k, the initial cluster prototypes are merged to construct k cluster prototypes; after obtaining the q initial cluster prototype centers, the weights between the q initial points are calculated sequentially using Euclidean distance, and the weights are denoted as ω, and the weights between the q points are calculated.

[0147]

[0148] Among them, a i Let a represent the i-th cluster center, 1≤i≤q. j Let represent the j-th sample, 1≤j≤q. Sort the samples in descending order of weight ω, reset ω, and take the mean of the data samples within the k cluster prototypes as the initial cluster centroids.

[0149]

[0150] When q < k, meaning the initial cluster prototypes are fewer than the set value k, the initial cluster prototypes are decomposed to construct k cluster prototypes. The weights σ between the nearest neighbor data samples that have been calculated are sorted in descending order, and σ is reset. The mean of the samples in the k cluster prototypes is used as the centroid of the cluster prototypes.

[0151]

[0152] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0153] The execution entities of the aforementioned processor and memory can be devices with computing functions such as computers, microcontrollers, and single-chip microcomputers. In specific implementations, the embodiments of the present invention do not limit the execution entities and can select them according to the needs of actual applications.

[0154] Data signals are transmitted between the memory and the processor via a bus, which will not be elaborated upon in this embodiment of the invention.

[0155] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0156] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0157] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0158] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0159] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0160] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0161] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0162] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting changes in wide-field-of-view video images in foggy weather, characterized in that, The method includes: Dehazing is performed on a single image based on color attenuation prior, resulting in a clear image after dehazing. The clear image after dehazing is processed by the arctangent ratio operator and the extreme pixel ratio operator to obtain two difference images. The two acquired difference images are adaptively fused by combining image energy features and Laplacian pyramid to obtain a fused difference map, and then further denoised by median filtering; The denoised fused difference image is subjected to mean normalization to compress the difference between the changes caused by sensor noise and the unchanged pixels to obtain the final difference map. The EN-K-Means clustering algorithm, which initializes the clustering prototype based on the pixel nearest neighbor relationship, performs clustering analysis on the final difference map to obtain the final binary image. The EN-K-Means clustering algorithm, which initializes the clustering prototype based on pixel nearest neighbor relationships, performs clustering analysis on the final difference map as follows: Find the nearest neighbor pixel for each data point using Euclidean distance; ； ； ； Construct an adjacency matrix based on the nearest neighbor relationships of the data samples: ； Given an adjacency matrix constructed based on the nearest neighbor relationships of data samples, construct an undirected graph based on the adjacency matrix. ,remember Given a connected component of graph G, the proposed algorithm uses the mean of all data samples in the connected component as the initial cluster center: ； in, Indicate the connected components of its graph. This indicates the number of data samples within its connected component; The overall strategy for initializing cluster prototypes based on nearest neighbor relationships depends on the relationship between q and cluster K, where This indicates the number of data samples within the connected component after prototype initialization. ； Among them, when When, the mean of the data samples within q connected components is taken as the initial cluster centers for the EN-k-means clustering algorithm and iterated; when If the number of initial cluster prototypes obtained exceeds the set value of k, the initial cluster prototypes are merged to construct k cluster prototypes. After obtaining q initial cluster prototype centers, the weights among the q initial points are calculated sequentially using Euclidean distance, and these weights are denoted as... Calculate the weights among q particles; ； in, Indicates the first Cluster centers, , Indicates the first One sample, For weights Sort in descending order and reset. The mean of the data samples within the k cluster prototypes is taken as the initial cluster centroids. ； when If the initial cluster prototype is less than the set k value, then the initial cluster prototype is decomposed to construct k cluster prototypes, and the weights between the already calculated nearest neighbor data samples are used. Sort in descending order and reset. The mean of the samples in the k cluster prototypes is used as the centroid of the cluster prototype: 。 2. The method for detecting changes in wide-field-of-view video images in foggy weather according to claim 1, characterized in that, The arctangent ratio operator is: ； in, and These are the pixel values ​​of two multi-temporal video images.

3. The method for detecting changes in wide-field-of-view video images in foggy weather according to claim 1, characterized in that, The final difference diagram is as follows: ； ； ； Where W and H are the width and height of the difference image, respectively.

4. A wide-field-of-view video image change detection device in foggy weather, characterized in that, The device includes a processor and a memory, the memory storing program instructions, the processor invoking the program instructions stored in the memory to cause the device to perform the method according to any one of claims 1-3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1-3.

Images