Method and apparatus for salient change detection for low-light wide-field video images

Abstract

The application discloses a kind of for low-illumination wide field of view video image significant change detection method and device, method includes: using logarithm ratio operator and subtraction operator respectively obtains global difference map and carries out fusion;Using frequency domain attention mechanism obtains the significant change area of global difference map;Combining morphological filtering, and energy feature and density feature are handled to noise, and the local change area pair where target is located is extracted;Using logarithm ratio operator extracts the difference feature of local change area pair, obtains local change difference map;Weighted fusion is used to carry out feature optimization to local change difference map, obtains local feature fusion map;Local feature fusion map is classified using k-means clustering algorithm to generate corresponding local change map, and all local change map is integrated into global change map, obtains the final change detection result.The device includes: processor and memory.

Description

Technical Field

[0001] This invention relates to the field of image change detection, and more particularly to a method and apparatus for detecting significant changes in low-light wide-field-of-view video images. Background Technology

[0002] Due to insufficient light, the distinction between targets and backgrounds is unclear in low-light images. Simultaneously, the dramatic increase in the sensitivity of the imaging sensor leads to a significant decrease in the signal-to-noise ratio, resulting in a decline in overall image quality, particularly in detail loss and image sharpness. Low contrast and strong noise greatly increase the difficulty of target detection. Target detection algorithms developed and trained in well-lit environments are unable to adapt to low-light scenes, resulting in a large number of missed detections.

[0003] Unlike detection methods based on the object's original features, change detection methods can remove background information and preserve the contours of moving targets, rather than simply returning their bounding boxes. Change detection algorithms are widely used in remote sensing images. Their core function is to analyze and extract the difference features between two temporal images, and then classify these differences according to the specific research object. In low-light environments, the spatial features of moving targets degrade, but their differences relative to the background remain. Therefore, in video sequences captured by monitoring equipment at a fixed angle, the feature differences between the input and reference frames can serve as a basis for determining the presence of moving targets. Change detection algorithms detect targets by extracting the temporal difference features of the compared sequences. In recent years, deep learning-based change detection algorithms have developed rapidly. Deep learning models typically require large amounts of labeled data for effective training. This can be a significant obstacle when data is scarce or labeling is costly.

[0004] For change detection algorithms for low-light image sequences, noise interference is the main problem that needs to be overcome. The reduced signal-to-noise ratio of low-light images makes it more difficult to remove noise, and noise reduction will further reduce image details. Unlike remote sensing images, noise in low-light images manifests as granular or speckled anomalies caused by random noise, as well as blocky changes caused by illumination fluctuations. In low-light wide-field video images, change detection algorithms that directly process global input are easily interfered with by high-density random noise in unchanged background areas. For example, Reference [1] proposed a video image change detection algorithm based on morphological structure filtering and normalized fusion of difference images, referred to as Zhu. This algorithm attempts to eliminate noise interference by performing morphological filtering on the global input image. Reference [2] proposed a video image change detection method based on optimized kmedoids clustering and adaptive fusion of difference images, referred to as Shi. At present, there is relatively little research on low-light image change detection technology, and there are limitations in the universality of detection technology for low-light images and remote sensing images. Reference [3] uses a Nonlocal Patch Similarity based Graph (NPSG) to measure the structural consistency between heterogeneous images. Reference [4] proposes a novel spatial context information exploration algorithm, ASEA, to explore the spatial context information around each pixel in a VHR remote sensing image, and then defines B2B distance to measure the magnitude of change between paired adaptive regions in a two-time image. Reference [5] proposes a technique for unsupervised change detection in multi-temporal satellite images using Principal Components Analysis (PCA) and K-means clustering, abbreviated as PCAK. Reference [6] proposes a SAR image change detection method based on Neighborhood Ratio (NR) and Extreme Learning Machine (ELM), abbreviated as NR_ELM. Reference [7] proposes a simple combined framework CDIK. First, it applies the subtraction operator and the logarithmic ratio operator to generate different difference images. Then, it uses the mean filter and the median filter to process the images to generate a better combined difference image (CDI). Finally, it uses the K-means clustering algorithm to divide the images into changed regions and unchanged regions. Reference [8] proposes a change detection method based on convolutional-wavelet neural networks (CWNN). It introduces dual-tree complex wavelet transform into the convolutional neural network to classify changed pixels and unchanged pixels, thereby effectively reducing the influence of speckle noise.Reference [9] proposes a CD method based on structural consistency, which achieves structural comparison by constructing and mapping an improved nonlocal patch-based graph (INLPG) instead of comparing the pixel values ​​of the image to detect changes. Reference

[10] proposes a SAR image change detection method based on frequency-domain analysis and random multigraphs (FDA_RMG). Among the above algorithms, Shi, ASEA, PCAK, CDIK and FDA_RMG have relatively good detection performance, while other algorithms are more sensitive to low-light environmental noise. For example, Zhu is affected by global high-density random noise, NPSG and INLPG are greatly affected by illumination fluctuations, and CWNN produces many false detections caused by dark spot noise. Summary of the Invention

[0005] This invention provides a method and apparatus for detecting significant changes in low-light wide-field-of-view video images. This invention combines significant change region localization and local change detection, effectively detecting moving targets in low-light environments while reducing computational redundancy. This invention introduces a frequency domain attention mechanism to focus on the Local Region of Change (LRC) and combines multiple feature information from the frequency and spatial domains to improve the reliability of localization. This invention uses an improved ratio operator to extract differential features and optimizes the differential information through feature reconstruction, improving the accuracy of clustering. See the description below for details.

[0006] A method for detecting significant changes in low-light wide-field-of-view video images, the method comprising:

[0007] The global difference map is obtained and fused using the logarithmic ratio operator and the subtraction operator respectively; the significant change region of the global difference map is obtained using the frequency domain attention mechanism.

[0008] Noise processing is performed by combining morphological filtering, energy features, and density features, and pairs of local variation regions where the target is located are extracted;

[0009] The logarithmic ratio operator is used to extract the difference features of pairs of locally changed regions to obtain a local change difference map; weighted fusion is used to optimize the features of the local change difference map to obtain a local feature fusion map;

[0010] The k-means clustering algorithm is used to classify the local feature fusion map to generate the corresponding local change map, and all the local change maps are combined into a global change map to obtain the final change detection result.

[0011] The method further includes: using Fourier transform to convert the global difference map generated by the subtraction operator from the spatial domain to the frequency domain, concentrating the low-frequency components at the center of the spectrum, extracting significant features through a high-pass filter, performing an inverse transform, and retaining the real part information as a mask for subsequent LRC retrieval.

[0012]

[0013] F shifted (μ, v) = F(μ, v)·(-1) i+j

[0014]

[0015] S(μ, v) = F shifted (μ, v)·H(μ, v)

[0016]

[0017] s mask (i, j) = |s(i, j)|

[0018] Among them, F shifted (μ, v) represents the result of spectrum centering, H(μ, v) is a high-pass filter, S(μ, v) is the frequency domain representation after high-pass filtering, and s(i, j) is the spatial domain representation obtained by inverse transform after spectrum centering and high-pass filtering. mask (i, j) are the real part information that is retained.

[0019] The local change regions where the extracted target is located are: using energy features to roughly screen for LRCs with potential change targets in the significant change regions of the global difference map, and combining the energy peak of the LRC with the adaptive energy threshold E obtained based on the global energy features. ave Comparison:

[0020]

[0021] In the formula, G d This represents the global difference map after morphological processing, where δ represents the energy parameter, X represents the number of pixels in the horizontal direction of the global difference map, and Y represents the number of pixels in the vertical direction of the global difference map.

[0022] Based on the energy characteristics, the LRC of objects with potential changes was initially determined, and then normalized and binarized.

[0023] Based on the statistical count of changing pixels in the LRC, potential change targets are further filtered:

[0024]

[0025] In the formula, L e Indicates L b The corrosion results are shown, where H and W represent the height and width of the anchor frame used to extract the LRC, respectively.

[0026] The logarithmic ratio operator is:

[0027]

[0028] In the formula, denoted by β, where β is the correction coefficient, h represents the row index of the image to be processed, and w represents the column index of the image to be processed. and D represents image sequences at different time points. P To obtain the difference map.

[0029] A second aspect is a device for detecting significant changes in low-light wide-field-of-view video images, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the method described in any of the first aspects.

[0030] Third aspect, a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of the first aspects.

[0031] The beneficial effects of the technical solution provided by this invention are:

[0032] 1. This invention focuses on the detection of regions with significant changes, avoiding the complex noise and computational redundancy of global analysis; this invention solves the problem of false detection caused by noise by integrating frequency domain and spatial attention.

[0033] 2. In the process of local change detection, this invention introduces a pixel extremum log-ratio operator (PELRO), which effectively improves the contrast between the changing area where the target is located and the unchanged area where the background is located.

[0034] 3. In order to further enhance the target of change and suppress interference features, this invention proposes a weighted fusion based on multidimensional feature reconstruction (WF-MFR) technology, which can effectively improve the reliability of change classification. Attached Figure Description

[0035] Figure 1 This is a flowchart of a method for detecting significant changes in low-light wide-field-of-view video images;

[0036] Figure 2 A schematic diagram of the input dual-temporal video sequence;

[0037] Figure 3 A change graph for directly detecting changes in the input sequence;

[0038] Figure 4 This is a change diagram of the local change detection method adopted in this invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0040] Example 1

[0041] A method for detecting significant changes in low-light wide-field-of-view video images, the method comprising the following steps:

[0042] Step 101: Use the logarithmic ratio operator and the subtraction operator to obtain the global difference map and then merge them;

[0043] Step 102: Use the frequency domain attention mechanism to obtain the significant change regions in the global difference map;

[0044] Step 103: Combine morphological filtering with spatial attributes such as energy and density features to quickly and effectively eliminate noise interference and extract the LRC pair where the target is located;

[0045] Step 104: Use PELRO to extract the difference features of LRC pairs to obtain the Local Change Difference Image (LCDI);

[0046] Step 105: Use WF-MFR to optimize the features of LCDI to obtain a Local Feature Fusion Image (LFFI);

[0047] Step 106: Use the k-means clustering algorithm to generate the corresponding Local Change Image (LCI) and combine them into a Global Change Image (GCI) to obtain the final change detection result.

[0048] In summary, the embodiments of the present invention focus on the detection of significantly changing regions through steps 101-106, avoiding the complex noise and computational redundancy of global analysis; the embodiments of the present invention solve the problem of false detection caused by noise by integrating frequency domain and spatial attention.

[0049] Example 2

[0050] The following is combined Figure 1 The specific calculation formulas for Example 1 will be described in more detail below:

[0051] Step 201: Use the logarithmic ratio operator and the subtraction operator to generate the initial global change feature map respectively, and obtain a difference map with richer features through weighted fusion. The calculation method is shown in formulas (1)-(3).

[0052]

[0053] G s (i,j)=|T1(i,j)-T2(i,j)| (2)

[0054] G=λG l +(1-λ)G s (3)

[0055] Where T1 is the video sequence at time 1, and T2 is the video sequence at time 2. G l G represents the global difference map generated by LRO. s Let G represent the global difference map generated by the subtraction operator, λ represent the weight coefficients, and G be the generated fused difference map. In this embodiment of the invention, λ is set to 0.5 for equal-weighted fusion.

[0056] Step 202: Use Fourier transform to transform G s The process involves transforming from the spatial domain to the frequency domain, concentrating low-frequency components at the center of the spectrum, extracting significant features through a high-pass filter, performing an inverse transform and retaining the real part information as a mask for subsequent LRC retrieval. The calculation principle is shown in formulas (4)-(9).

[0057]

[0058] F shifted (μ, v) = F(μ, v)·(-1) i+j (5)

[0059]

[0060] S(μ, v) = F shifted (μ, v)·H(μ, v) ( 7 )

[0061]

[0062] s mask (i, j) = |s(i, j)| (9)

[0063] Among them, F shifted (μ, v) represents the result of spectrum centering, H(μ, v) is a high-pass filter, S(μ, v) is the frequency domain representation after high-pass filtering, and s(i, j) is the spatial domain representation obtained by inverse transform after spectrum centering and high-pass filtering. mask (i, j) are the real part information that is retained.

[0064] Step 203: In low-light environments, noise features manifest in various forms. In particular, the presence of point-like bright noise can seriously interfere with the accurate positioning of real targets with similar energy features. Morphological filtering provides a fast and effective solution to mitigate this interference. This process includes expansion and erosion operations, the principles of which are shown in formulas (10)-(11).

[0065]

[0066] (UΘV)(i,j)=inf{U(i V j V ):(i V j V )∈(i,j)+V} (11)

[0067] In the formula, Θ represents the expansion operation, θ represents the erosion operation, sup represents the maximum value (the largest element in the set), and inf represents the minimum value (the minimum value among all elements in the set). The translation of the structuring element V at each point (i, j) is represented as (i, j) - V.

[0068] Morphological erosion and dilation can respectively constitute opening and closing operations, as shown in formulas (12)-(13). The embodiments of the present invention adopt the processing method of opening first and then closing, which eliminates the interference of isolated noise points on the localization of the real target while protecting the target with weak energy characteristics, as shown in formula (14).

[0069]

[0070]

[0071]

[0072] In the formula, · represents the opening operation, and · represents the closing operation. V1 and V2 represent structure elements of different sizes.

[0073] After eliminating significant noise, energy features are used to roughly screen for potential change targets in the significant change regions of the global difference map. The energy peak value of the LRC and the adaptive energy threshold E obtained based on the global energy features are then compared. ave The comparisons are shown in formula (15).

[0074]

[0075] In the formula, G d The diagram represents the global difference map after morphological processing, and δ represents the energy parameter, which is set to 35 through experimental analysis in this embodiment of the invention.

[0076] Based on the energy characteristics, the LRC of the potential change object is initially determined, and then normalized and binarized as shown in formulas (16)-(17).

[0077]

[0078]

[0079] In the formula, L(h, w) corresponds to the grayscale value of the retrieved LRC at index (h, w), and L... min and L max L represents the minimum and maximum values ​​of a local region, respectively. n L represents the result of the normalization operation. b The result of the binarization operation is represented by θ, which is the reference threshold for performing the binarization operation. In this embodiment of the invention, θ is set to 63, which is applicable to most low-light variation scenarios.

[0080] After the above series of processing operations for potential changes in the spatial domain, the embodiments of the present invention further filter potential change objects based on the statistical number of changing pixels in LRC, reducing the probability of misjudging point noise with strong change characteristics as targets, as shown in formula (18).

[0081]

[0082] In the formula, L e Indicates L b The corrosion results are shown, where H and W represent the height and width of the anchor frame used to extract the LRC, respectively.

[0083] If N equals or exceeds the density threshold χ, it is inferred that the neighborhood of the peak pixel contains a target. At the end of each cycle, pixels in these local regions that are identified as being associated with the target are removed from the global difference map for that cycle, thereby eliminating the influence of the detected target on the remaining targets to be detected.

[0084] Finally, after a finite number of iterations, the neighborhood information of the peak pixel where the target is located is retained, and LRC pairs are extracted from the original input bi-temporal video sequence.

[0085] Step 204: An ideal difference operator should preserve the details of the real target within the changed region while eliminating various interferences caused by noise in the unchanged region. Therefore, the feature difference between the changed and unchanged regions should be as large as possible to maximize the contrast and separation between the target and the background.

[0086] In this embodiment of the invention, the designed PELRO operator is used to extract the difference features in the LRC pair to obtain LCDI, and its calculation formula is shown in formula (19).

[0087]

[0088] In the formula, β is the correction coefficient, which is set to 0.95 in this embodiment. The numerator is the minimum value of the corresponding pixel in the dual-phase image, and the denominator is the maximum value. It can be seen that in areas with no change or small change, the ratio tends to be closer to 1. The closer the extreme pixel ratio of the changing region is to 0, the closer the absolute value of the logarithmic transformation function tends to infinity, thus amplifying the pixel energy of the changing region. PELRO also utilizes the extreme ratio characteristics between the changing and unchanged regions and the gain effect of the logarithmic ratio curve to further improve the separation between the real changes caused by the motion of the foreground object and the interference changes caused by background noise, thereby improving the stability of subsequent foreground and background segmentation.

[0089] Step 205: Dimensionality reduction of the extracted differential features is performed using WF-MFR to reduce the correlation between interfering features and the main structural features of the changing target. Feature decomposition of the extracted LCDI can be performed according to formula (20).

[0090]

[0091] In the above formula, U and V are orthogonal matrices composed of eigenvectors, and they are mutually orthogonal. Si represents a diagonal matrix with diagonal elements arranged in descending order. It describes the importance of the original matrix in each direction; the earlier singular values ​​correspond to more prominent features and contribute more to the reconstruction of the original matrix, while the contributions of later singular values ​​gradually decrease. S0 represents the two-dimensional image matrix corresponding to LCDI.

[0092] The principal components of LCDI are extracted and retained according to formula (21) to obtain the dimension-reduced local feature map.

[0093]

[0094] in, σ1≥…≥σ gk ≥0. F k Indicates g before extraction k The feature map is obtained from the main feature components.

[0095] Then, the difference feature maps of different dimensions are fused to obtain LFFI, as shown in formula (22).

[0096]

[0097] Where S represents the fused feature map, ω k This is the weighting coefficient, and N represents the number of subsets used for reconstruction. For example, if g0 = 3, g1 = 5, g2 = 7, g3 = 9, and g4 = 11, then N is 5.

[0098] Among them, WF-MFR improves the distinguishability between foreground and background features by rapidly and effectively extracting the main components of LCDI in different dimensions and suppressing related interference features, and then performing fusion enhancement.

[0099] Step 206: Use the K-means clustering algorithm to perform change classification on the LFFI to obtain LCI, which is a local binary map reflecting the changed and unchanged classes. Then, based on the orientation information obtained during retrieval, all LCIs are synthesized into a complete GCI. Before clustering, the local feature maps are first converted into a one-dimensional vector representation Z, as shown below:

[0100] Z = [l1, ..., l] i ,...,l HW ] T (twenty three)

[0101] Among them, l i ={l i0 , ..., l iq}, q=1, represents a 1×2 row vector. HW represents the total number of feature points in the local feature map. The Euclidean distance R between different feature points is calculated as shown in formula (24).

[0102]

[0103] Among them, l ik and l jk The corresponding element values ​​of different feature vectors are represented by q, which controls the upper limit of the summation, thereby calculating the Euclidean distance between different feature points.

[0104] In the initial stage of clustering, cluster center C x(x = 1, 2, ..., J) are randomly assigned based on J predefined categories. All feature points l in Z are traversed. i To re-cluster and obtain l i and C x The Euclidean distance between them. Update C according to formula (25). x .

[0105]

[0106] Among them, O x Let |O| represent the set of feature points of the x-th category. x | represents the statistical number of feature points in the x-th cluster.

[0107] Calculate the tolerance ΔQ between the two iterations, as shown in formulas (26)-(27).

[0108]

[0109] ΔQ=Q n -Q n-1 (27)

[0110] Among them, Q n Q represents the sum of the Euclidean distances between each feature point and its corresponding cluster center in different clusters after the nth clustering; n-1 This represents the summation of the previous clustering.

[0111] In practice, the loop ends and a local change map is output when the tolerance is less than the preset value or the number of iterations exceeds the upper limit. The merging of all local change maps will be performed in a background where there are no interfering changes by default.

[0112] In summary, the embodiments of the present invention focus on the detection of significantly changing regions through steps 201-206, avoiding the complex noise and computational redundancy of global analysis; the embodiments of the present invention solve the problem of false detection caused by noise by integrating frequency domain and spatial attention.

[0113] Example 3

[0114] The feasibility of the schemes in Examples 1 and 2 is verified below with specific examples, as detailed in the following description:

[0115] To evaluate the performance differences between the proposed operator and existing operators, tests were conducted in five sets of complex bi-temporal local variation regions. Table 1 lists the evaluation metrics for the Log-ratio Operator (LRO), Extremum Pixel Ratio Operator (EPRO), and PELRO. Compared to EPRO, PELRO improves the average Kappa metric by 28.5%. Compared to LRO, PELRO improves the average Kappa metric by 33.32%.

[0116] Table 1. Evaluation values ​​of different operators in 5 groups of LRC.

[0117]

[0118] To further evaluate the feasibility of the change detection system designed in this embodiment of the invention, and its performance compared with existing change detection algorithms, comparative tests were conducted in three challenging low-light scenarios. Table 2 shows the Kappa evaluation values ​​and running times T for different change detection algorithms. In scenario 1, the Kappa index of this embodiment is slightly lower than FDA_RMG, but its detection efficiency is the highest. In scenarios 2 and 3, the Kappa index of this embodiment is better than all other change detection methods. Due to its sensitivity to significant changes, FDA_RMG cannot operate normally in scenarios 2 and 3, where the interference features are more complex. Shi has the shortest running time in scenario 2, but its Kappa index is much lower than that of this method and CDIK. In addition, CDIK has excellent detection performance among the comparison algorithms, but it shows significant performance degradation in scenario 3. Overall, the detection method designed in this embodiment of the invention achieves relatively good performance in different low-light scenarios, balancing detection accuracy and efficiency.

[0119] Table 2 Evaluation values ​​of different change detection algorithms

[0120]

[0121]

[0122] Example 4

[0123] A device for detecting significant changes in low-light wide-field-of-view video images, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the following method steps in Embodiment 1:

[0124] The global difference map is obtained and fused using the logarithmic ratio operator and the subtraction operator respectively; the significant change region of the global difference map is obtained using the frequency domain attention mechanism.

[0125] Noise processing is performed by combining morphological filtering, energy features, and density features, and pairs of local variation regions where the target is located are extracted;

[0126] The logarithmic ratio operator is used to extract the difference features of pairs of locally changed regions to obtain a local change difference map; weighted fusion is used to optimize the features of the local change difference map to obtain a local feature fusion map;

[0127] The k-means clustering algorithm is used to classify the local feature fusion map to generate the corresponding local change map, and all the local change maps are combined into a global change map to obtain the final change detection result.

[0128] This also includes: using Fourier transform to convert the global difference map generated by the subtraction operator from the spatial domain to the frequency domain, concentrating low-frequency components at the center of the spectrum, extracting significant features through a high-pass filter, performing an inverse transform, and retaining the real part information as a mask for subsequent LRC retrieval.

[0129]

[0130] F shifted (μ, v) = F(μ, v)·(-1) i+j

[0131]

[0132] S(μ, v) = F shifted (μ, v)·H(μ, v)

[0133]

[0134] s mask (i, j) = |s(i, j)|

[0135] Among them, F shifted (μ, v) represents the result of spectrum centering, H(μ, v) is a high-pass filter, S(μ, v) is the frequency domain representation after high-pass filtering, and s(i, j) is the spatial domain representation obtained by inverse transform after spectrum centering and high-pass filtering. mask (i, j) are the real part information that is retained.

[0136] Specifically, the extraction of the local variation region pair where the target is located involves: using energy features to roughly screen for LRCs with potential changing targets in the significant variation regions of the global difference map, and then combining the energy peak value of the LRC with the adaptive energy threshold E obtained based on the global energy features. ave Comparison:

[0137]

[0138] In the formula, G d This represents the global difference map after morphological processing, where δ represents the energy parameter, X represents the number of pixels in the horizontal direction of the global difference map, and Y represents the number of pixels in the vertical direction of the global difference map.

[0139] Based on the energy characteristics, the LRC of objects with potential changes was initially determined, and then normalized and binarized.

[0140] Based on the statistical count of changing pixels in the LRC, potential change targets are further filtered:

[0141]

[0142] In the formula, L e Indicates L b The corrosion results are shown, where H and W represent the height and width of the anchor frame used to extract the LRC, respectively.

[0143] The logarithmic ratio operator is:

[0144]

[0145] In the formula, β represents the positive minimum value approaching zero, h represents the row index of the image to be processed, and w represents the column index of the image to be processed. and D represents image sequences at different time points. P To obtain the difference map.

[0146] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0147] The execution entities of the aforementioned processor and memory can be devices with computing functions such as computers, microcontrollers, and single-chip microcomputers. In specific implementations, the embodiments of the present invention do not limit the execution entities and can select them according to the needs of actual applications.

[0148] Data signals are transmitted between the memory and the processor via a bus, which will not be elaborated upon in this embodiment of the invention.

[0149] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0150] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0151] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0152] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0153] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0154] References

[0155] [1]Zhu Y,Jia Z,Yang J,et al.Change detection in multitemporalmonitoring images under lowillumination[J].IEEE Access,2020,8:126700-126712.

[0156] [2]Shi B,Jia Z,Yang J,et al.Unsupervised change detection in wide-field video images underlow illumination[J].IEEE Transactions on Circuits andSystems for Video Technology,2022,33(4):1564-1576.

[0157] [3] Sun Y, Lei L, Li X, et al. Nonlocal patch similarity based heterogeneous remote sensing change detection [J]. Pattern Recognition, 2021, 109: 107598.

[0158] [4]Lv Z,Wang F J,Liu T,et al.Novel automatic approach for land coverchange detection byusing VHR remote sensing images[J].IEEE Geoscience andRemote Sensing Letters,2021,19:1-5.

[0159] [5]Celik T.Unsupervised change detection in satellite images usingprincipal component analysisand$k$-means clustering[J].IEEE geoscience andremote sensing letters,2009,6(4):772-776.[6]Gao F,Dong J,Li B,et al.Changedetection from synthetic aperture radar images based onneighborhood-basedratio and extreme learning machine[J].Journal of Applied Remote Sensing,2016,10(4):046019-046019.

[0160] [7]Zheng Y,Zhang X,Hou B,et al.Using combined difference image and$k$-means clusteringfor SAR image change detection[J].IEEE Geoscience andRemote Sensing Letters,2013,11(3):691-695.

[0161] [8]Gao F,Wang X,Gao Y,et al.Sea ice change detection in SAR imagesbased onconvolutional-wavelet neural networks[J].IEEE Geoscience and RemoteSensing Letters,2019,16(8):1240-1244.

[0162] [9] Sun Y, Lei L, Li X, et al. Structure consistency-based graph for unsupervised change detection with homogeneous and heterogeneous remote sensing images [J]. IEEE transactions ongeoscience and remote sensing, 2021, 60: 1-21.

[0163]

[10] Gao F, Wang

[0164] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0165] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0166] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting significant changes in low-light wide-field-of-view video images, characterized in that, The method includes: The global difference map is obtained and fused using the logarithmic ratio operator and the subtraction operator respectively; the significant change region of the global difference map is obtained using the frequency domain attention mechanism. Noise processing is performed by combining morphological filtering, energy features, and density features, and pairs of local variation regions where the target is located are extracted; The logarithmic ratio operator is used to extract the difference features of pairs of locally changed regions to obtain a local change difference map; weighted fusion is used to optimize the features of the local change difference map to obtain a local feature fusion map; The k-means clustering algorithm is used to classify the local feature fusion map to generate the corresponding local change map, and all the local change maps are combined into a global change map to obtain the final change detection result. The logarithmic ratio operator is: In the formula, For correction factors, represents the row index of the image to be processed, and w represents the column index of the image to be processed. and These represent image sequences from different time phases. To obtain the difference map.

2. The method for detecting significant changes in low-light wide-field-of-view video images according to claim 1, characterized in that, The method further includes: using Fourier transform to transform the global difference map generated by the subtraction operator from the spatial domain to the frequency domain, concentrating low-frequency components at the center of the spectrum, extracting significant features through a high-pass filter, performing an inverse transform, and retaining the real part information as a mask for subsequent LRC retrieval. in, This represents the result of spectrum centering. It is a high-pass filter. This is the frequency domain representation after high-pass filtering. This is the spatial domain representation obtained by inverse transform after spectral centering and high-pass filtering. It is the retained real information.

3. The method for detecting significant changes in low-light wide-field-of-view video images according to claim 1, characterized in that, The local variation regions where the extracted target is located are: using energy features to roughly screen for LRCs with potential variation targets in the significant variation regions of the global difference map, and then combining the energy peak value of the LRC with an adaptive energy threshold calculated based on the global energy features. Comparison: In the formula, This represents the global difference map after morphological processing. The values ​​represent energy parameters, X represents the number of pixels in the global difference map in the horizontal direction, and Y represents the number of pixels in the global difference map in the vertical direction. Based on the energy characteristics, the LRC of objects with potential changes was initially determined, and then normalized and binarized. Based on the statistical count of changing pixels in the LRC, potential change targets are further filtered: In the formula, Indicates to The corrosion results are shown, where H and W represent the height and width of the anchor frame used to extract the LRC, respectively.

4. A device for detecting significant changes in low-light wide-field-of-view video images, characterized in that, The device includes a processor and a memory, the memory storing program instructions, the processor invoking the program instructions stored in the memory to cause the device to perform the method according to any one of claims 1-3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1-3.

Images