Video remote sensing image moving target detection method based on fusion of biomimetic vision

Abstract

The application discloses a video remote sensing image moving target detection method fusing bionic vision. First, a saliency network simulating a human visual system is constructed to extract visual saliency features of a video remote sensing image to obtain a saliency map; then, a target detector simulating a fruit fly visual system is designed to detect moving targets in the saliency map, and the specific process is as follows: first, a small eye receiver simulating a fruit fly compound eye is constructed to collect information in the saliency image, then a fruit fly ocellar nerve unit is simulated to further acquire moving features of the target, and finally a fruit fly visual information processing center is simulated to calculate a final target detection result. The application is based on the bionic theory, simulates different biological visual systems, can effectively extract moving targets in the video remote sensing image, suppresses a cluttered background at the same time, and improves the accuracy and robustness of the moving target detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and in particular relates to a method for detecting moving targets in video remote sensing images that integrates biomimetic vision. Background Technology

[0002] Moving target detection in video remote sensing imagery is a key research area in computer vision, primarily involving the localization, detection, and extraction of moving targets of interest in remote sensing image sequences or videos. Moving target detection has wide applications in autonomous driving, intelligent transportation, and video surveillance. In the early stages of this research, most moving target detection algorithms were based on handcrafted features, heavily reliant on specialized knowledge and techniques in feature design, and thus had limitations in application, resulting in insufficient adaptability and expressive power. Representative algorithms include inter-frame differencing, background differencing, optical flow, and template matching-based algorithms.

[0003] Later, with the rapid development of deep learning, a large number of deep learning-based moving target detection algorithms emerged. Well-trained models based on deep learning algorithms have a high recognition rate and strong adaptability for moving targets. However, they rely on expensive and large amounts of training samples, and the algorithm structure is very complex, requiring high-end hardware and taking a long time to train.

[0004] Most organisms in nature possess excellent visual neural networks. These biological visual systems exhibit strong adaptability and stability, enabling them to quickly and accurately analyze and understand complex visual environments. Their superior ability to detect moving targets, in particular, allows them to perform behaviors such as hunting, evading predators, and courtship. Current research on biomimetic vision mainly includes: mimicking human eye vision, mimicking mammalian vision (e.g., cats, macaques), and mimicking insect vision (e.g., locusts, dragonflies, fruit flies). Driven by advances in research related to biological visual mechanisms, moving target detection in video remote sensing images driven by biomimetic vision mechanisms has attracted significant attention. Compared to traditional moving target detection algorithms and deep learning-based moving target detection algorithms, biomimetic vision-based moving target detection algorithms offer advantages such as simple principles, ease of implementation, and accurate results.

[0005] Announcement No. CN115359094B describes a deep learning-based moving target detection method, including steps such as dataset creation, moving target detection model training, and on-device application. This method, against a fixed background, extracts different categories of targets as moving targets based on labeled information. Then, it merges the extracted targets with the current fixed background to form a single image. Next, it uses pixel calculations to make the moving targets move randomly in the background, generating a new image. Finally, it uses background subtraction to update the background information of the image in real time, completing the dataset creation. The YOLOX network is used to train the moving target detection model. This method avoids the problem of high time consumption due to background updates during target detection and reduces false positives in complex scenes. However, this method uses background subtraction to construct the dataset, which has poor anti-interference performance and requires determining the moving background before application, complicating dataset creation and affecting the algorithm's universality. Using a deep learning-based method to train the moving target detection model also results in a long overall processing time.

[0006] Announcement No. CN115690190B describes a moving target detection and localization method based on optical flow imaging and pinhole imaging. This method includes steps such as video image preprocessing, optical flow calculation, coarse segmentation of foreground targets, fine segmentation of foreground targets, capture of moving targets, and target localization. The method captures moving targets by performing enhancement and denoising, improved optical flow calculation, and segmentation extraction on two consecutive frames acquired by a monocular vision sensor. It also constructs a simple localization mathematical model to obtain the coordinates of the current moving target and its orientation and distance relative to the camera in a timely and accurate manner. This method can effectively improve the robustness and accuracy of moving target detection and localization systems with low cost and low information content. However, due to the inherent characteristics of the optical flow method, its calculation process is relatively complex and computationally intensive, consuming a considerable amount of time, which somewhat affects the real-time performance of moving target detection and localization.

[0007] In summary, the limitations of traditional video remote sensing image moving target detection algorithms and deep learning-based video remote sensing image moving target detection algorithms are mainly reflected in the following aspects:

[0008] (1) Poor universality: Traditional moving target detection algorithms either require prior determination of the motion background and have poor anti-interference ability, or have a narrow application range and can only identify moving targets within the ground range, resulting in weak universality and poor applicability of the algorithm.

[0009] (2) The feature representation and calculation process is relatively complex: Most traditional moving target detection algorithms are based on hand-constructed features, which are relatively complex and heavily rely on professional knowledge and domain experience when designing features. Currently, the mainstream deep learning-based methods also have very complex feature extraction processes and overall model structures.

[0010] (3) The algorithm takes a long time: Deep learning-based algorithms require a lot of time to train the model, resulting in a long overall time consumption, a complex calculation process, and a large amount of computation, making it difficult to obtain real-time results. Summary of the Invention

[0011] Objective: To address the problems existing in the prior art, this invention provides a method for moving target detection in video remote sensing images that integrates biomimetic vision. This method avoids the problems of complex algorithm structures, high hardware requirements, and long training times inherent in mainstream deep learning-based algorithms. It can shorten the time spent on moving target detection while maintaining high detection accuracy. Simultaneously, it uses a saliency network mimicking the human visual system for the extraction and fusion of multiple features. During the moving target detection process, a moving target detector mimicking the fruit fly visual system can effectively extract moving targets from video remote sensing images while suppressing background clutter, thus improving the accuracy and robustness of moving target detection.

[0012] Technical Solution: To achieve the objectives of this invention, the technical solution adopted is: a method for detecting moving targets in video remote sensing images that integrates bionic vision, the specific steps of which are as follows:

[0013] (1) Construct a saliency network that mimics the human visual system. In this network, first extract the saliency maps of three types of features: brightness, color, and orientation at different scales of video remote sensing images.

[0014] (2) Normalize the saliency maps of each type of feature at different scales to obtain the saliency map corresponding to that type of feature;

[0015] (3) The saliency maps corresponding to all features are fused together, and the winner-takes-all mechanism and return inhibition mechanism are used to shift the focus of attention to obtain the final saliency map;

[0016] (4) Construct a motion target detector that mimics the visual system of fruit flies. First, construct a small eye receiver that mimics the compound eye of fruit flies to further integrate and smooth the information of the salient image.

[0017] (5) Simulate the visual lobe neural unit of fruit flies to further obtain the motion characteristics of the target;

[0018] (6) The visual information processing center of the fruit fly is simulated to calculate the final target detection result.

[0019] In step (1), the method for extracting the saliency maps of brightness, color, and orientation features at different scales of the video remote sensing image is as follows:

[0020] (1.1) Extracting luminance features; Gaussian pyramid decomposition is performed using a Gaussian difference filter bank, and the luminance features are expressed as:

[0021]

[0022] Where r, g, and b represent the red, green, and blue channel components of the original RGB remote sensing image, respectively.

[0023] The luminance features I(c) and I(s) of the c-th and s-th layers of the feature pyramid are fused to obtain the corresponding fused luminance feature map I(c, s):

[0024]

[0025] in, This indicates that subtraction is performed between pixels.

[0026] (1.2) Extracting color features: Using the same Gaussian pyramid decomposition method, the original three primary color RGB channels are converted into the color features of generalized tuned red (R), green (G), blue (B), and yellow (Y):

[0027]

[0028]

[0029]

[0030]

[0031] Calculate the red-green channel color feature map RG(c,s) and the blue-yellow channel color feature map BY(c,s):

[0032]

[0033]

[0034] in, This indicates subtraction between pixels, where c and s represent the c-th and s-th layers of the color feature pyramid.

[0035] (1.3) Extracting Orientation Features: The orientation feature map D(c, s, θ) is calculated using the Gabor pyramid method:

[0036]

[0037] Where θ represents the direction of the output. This indicates subtraction between pixels, where c and s represent the c-th and s-th layers of the feature pyramid.

[0038] In step (2), the method for normalizing the saliency maps of each type of feature at different scales to obtain the saliency map corresponding to that type of feature is as follows:

[0039] (2.1) Normalize the feature maps obtained at different scales for the same type of features in step (1). Let... and Let represent the saliency maps of the luminance component, chromaticity component, and orientation component, respectively. Then:

[0040]

[0041]

[0042]

[0043] Where N(·) denotes the normalization operator, This represents cross-scale addition, which involves scaling each feature map to the size of the fourth layer and then adding the pixels together.

[0044] (2.2) Linearly fuse the saliency maps of various features to obtain the overall saliency map S:

[0045]

[0046] Where N(·) represents the normalization operator.

[0047] In step (3), the saliency maps corresponding to all features are fused, and the winner-takes-all mechanism and return inhibition mechanism are used to shift the focus of attention. The method to obtain the final saliency map is as follows:

[0048] (3.1) The winner-takes-all mechanism is used to compete for the total saliency map S obtained in step (2). The final winner of the competition is the focus of attention, that is, the neuron with the largest saliency value in the saliency map is the focus of attention.

[0049] (3.2) A return inhibition mechanism is used to suppress the already noticed locations and their salient regions, causing the focus of attention to shift back to the unnoticed regions. Specifically, after obtaining the focus of attention and the salient regions, a Gaussian difference function with central inhibition and peripheral excitation is used, where the inhibition center of this function is the focus of attention in step (3.1). Finally, the final salient map can be obtained.

[0050] In step (4), the method for constructing the microscopic receptors that mimic the compound eye of a fruit fly is as follows:

[0051] (4.1) Multiple small-eye receivers are simulated using H Gaussian kernel functions of different sizes:

[0052]

[0053] in, This represents the final saliency map obtained in step (3). Let i represent the i-th Gaussian kernel function, i = 1, ..., H, O. i This represents the output of the i-th small eye.

[0054] (4.2) Spatial integration of all small-eye outputs is performed according to the following rules: Align overlapping sub-regions of a scene and combine them into a larger region. This biomimetic compound eye method effectively eliminates redundant information, thereby representing the target more effectively. The integration result is...

[0055] In step (5), the method for simulating the visual lobe neural unit of a fruit fly to further obtain the motion characteristics of the target is as follows:

[0056] (5.1) Design a bandpass filter to simulate the visual lobe neural unit of a fruit fly:

[0057] φ(t) = B1(t) - B2(t)

[0058] Here, B1(t) and B2(t) represent two two-dimensional time filters.

[0059] (5.2) Based on the bandpass filter, the small-eye output obtained in step (4) Extracting brightness change features:

[0060]

[0061] in, This represents the output of the small eye at time t.

[0062] (5.3) Simulate neurons T4 and T5 in the fruit fly visual system to detect motion information and extract motion information from brightness change features:

[0063] T4(t)=ζ[M(t)]

[0064] T5(t) = ζ[-M(t)]

[0065] Where ζ[k] represents max(0, k), and T4(t) and T5(t) represent the outputs of neurons T4 and T5, respectively.

[0066] In step (6), the method for target detection using the simulated fruit fly visual information processing center is as follows:

[0067] (6.1) Integrate the outputs T4(t) and T5(t) from step (5.3) to obtain the possible motion targets:

[0068] T(t) = T4(t) - T5(t)

[0069] (6.2) Use the integrated results to determine the authenticity of the detected possible targets, as follows: when T(t) is greater than a certain threshold, it indicates that the target is a true target; otherwise, it is a false target.

[0070] Beneficial effects: The present invention, by adopting the above technical solution, has the following beneficial effects:

[0071] (1) The method of this invention integrates a biomimetic vision system to establish a moving target detection model for video remote sensing images. It can be applied not only to moving target detection in simple backgrounds but also to moving target detection in complex moving backgrounds. It can detect moving targets not only generated random bright spots but also real moving objects. The model has good universality.

[0072] (2) The present invention uses a saliency network that mimics the human visual system to extract the brightness, color and orientation features of the image and performs multi-feature fusion. The fused visual saliency features can effectively improve the accuracy of the network in detecting moving targets.

[0073] (3) The method of this invention designs a target detector based on a fruit fly-like vision system for moving target detection. Currently, mainstream deep learning-based methods have very complex algorithm structures, high hardware requirements, and long training time. In contrast, bionic vision systems are relatively simple in structure, easy to implement, and can quickly and accurately capture key visual information, thus efficiently and reliably solving the problem of moving target detection. Attached Figure Description

[0074] Figure 1 This is a diagram of the network architecture for detecting moving targets in video remote sensing images that integrates biomimetic vision. Detailed Implementation

[0075] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0076] like Figure 1 As shown, the technical solution of the present invention is further described in detail below:

[0077] (1) Construct a saliency network that mimics the human visual system. In this network, first extract the saliency maps of three types of features: brightness, color, and orientation at different scales of video remote sensing images.

[0078] (1.1) Extracting luminance features: Gaussian pyramid decomposition is performed using a Gaussian difference filter bank, and the luminance features are expressed as:

[0079]

[0080] Where r, g, and b represent the red, green, and blue channel components of the original RGB remote sensing image, respectively.

[0081] The luminance features I(c) and I(s) of the c-th and s-th layers of the feature pyramid are fused to obtain the corresponding fused luminance feature map I(c, s):

[0082]

[0083] in, This indicates that subtraction is performed between pixels.

[0084] (1.2) Extracting color features: Using the same Gaussian pyramid decomposition method, the original three primary color RGB channels are converted into the color features of generalized tuned red (R), green (G), blue (B), and yellow (Y):

[0085]

[0086]

[0087]

[0088]

[0089] Calculate the red-green channel color feature map RG(c,s) and the blue-yellow channel color feature map BY(c,s):

[0090]

[0091]

[0092] in, This indicates subtraction between pixels, where c and s represent the c-th and s-th layers of the color feature pyramid.

[0093] (1.3) Extracting Orientation Features: The orientation feature map D(c, s, θ) is calculated using the Gabor pyramid method:

[0094]

[0095] Where θ represents the direction of the output. This indicates subtraction between pixels, where c and s represent the c-th and s-th layers of the feature pyramid.

[0096] (2) Normalize the saliency maps of each type of feature at different scales to obtain the saliency map corresponding to that type of feature.

[0097] (2.1) Normalize the feature maps obtained at different scales for the same type of features in step (1). Let... and Let represent the saliency maps of the luminance component, chromaticity component, and orientation component, respectively. Then:

[0098]

[0099]

[0100]

[0101] Where N(·) denotes the normalization operator, This represents cross-scale addition, which involves scaling each feature map to the size of the fourth layer and then adding the pixels together.

[0102] (2.2) Linearly fuse the saliency maps of various features to obtain the overall saliency map S:

[0103]

[0104] Where N(·) represents the normalization operator.

[0105] (3) The saliency maps corresponding to all features are merged, and the winner-takes-all mechanism and return inhibition mechanism are used to achieve the shift of attention focus to obtain the final saliency map.

[0106] (3.1) The winner-takes-all mechanism is used to compete for the total saliency map S obtained in step (2). The final winner of the competition is the focus of attention, that is, the neuron with the largest saliency value in the saliency map is the focus of attention.

[0107] (3.2) A return inhibition mechanism is used to suppress the already noticed locations and their salient regions, causing the focus of attention to shift back to the unnoticed regions. Specifically, after obtaining the focus of attention and the salient regions, a Gaussian difference function with central inhibition and peripheral excitation is used, where the inhibition center of this function is the focus of attention in step (3.1). Finally, the final salient map can be obtained.

[0108] (4) Construct a motion target detector that mimics the visual system of fruit flies. First, construct a small eye receiver that mimics the compound eye of fruit flies to further integrate and smooth the information of salient images.

[0109] (4.1) Multiple small-eye receivers are simulated using H Gaussian kernel functions of different sizes:

[0110]

[0111] in, This represents the final saliency map obtained in step (3). Let i represent the i-th Gaussian kernel function, i = 1, ..., H, O. i This represents the output of the i-th small eye.

[0112] (4.2) Spatial integration of all small-eye outputs is performed according to the following rules: Align overlapping sub-regions of a scene and combine them into a larger region. This biomimetic compound eye method effectively eliminates redundant information, thereby representing the target more effectively. The integration result is...

[0113] (5) Simulate the visual lobe nerve unit of fruit fly to further obtain the motion characteristics of the target.

[0114] (5.1) Design a bandpass filter to simulate the visual lobe neural unit of a fruit fly:

[0115] φ(t) = B1(t) - B2(t)

[0116] Here, B1(t) and B2(t) represent two two-dimensional time filters.

[0117] (5.2) Based on the bandpass filter, the small-eye output obtained in step (4) Extracting brightness change features:

[0118]

[0119] in, This represents the output of the small eye at time t.

[0120] (5.3) Simulate neurons T4 and T5 in the fruit fly visual system to detect motion information and extract motion information from brightness change features:

[0121] T4(t)=ζ[M(t)]

[0122] T5(t) = ζ[-M(t)]

[0123] Where ζ[k] represents max(0, k), and T4(t) and T5(t) represent the outputs of neurons T4 and T5, respectively.

[0124] (6) The visual information processing center of the fruit fly is simulated to calculate the final target detection result.

[0125] (6.1) Integrate the outputs T4(t) and T5(t) from step (5.3) to obtain the possible motion targets:

[0126] T(t) = T4(t) - T5(t)

[0127] (6.2) Use the integrated results to determine the authenticity of the detected possible targets, as follows: when T(t) is greater than a certain threshold, it indicates that the target is a true target; otherwise, it is a false target.

Claims

1. A method for detecting moving targets in video remote sensing images that integrates biomimetic vision, characterized in that, The method includes the following steps: (1) Construct a saliency network that mimics the human visual system. In this network, first extract the saliency maps of three types of features: brightness, color, and orientation at different scales of video remote sensing images. (2) Normalize the saliency maps of each type of feature at different scales to obtain the saliency map corresponding to that type of feature; (3) The saliency maps corresponding to all features are fused together, and the winner-takes-all mechanism and return inhibition mechanism are used to shift the focus of attention to obtain the final saliency map; (4) Construct a motion target detector that mimics the visual system of fruit flies. First, construct a small eye receiver that mimics the compound eye of fruit flies to further integrate and smooth the information of the salient image. (5) Simulate the visual lobe neural unit of fruit flies to further obtain the motion characteristics of the target; (6) The visual information processing center, which is modeled after the fruit fly, is used to calculate the final target detection result; In step (4), the method for constructing the microscopic receptors that mimic the compound eye of a fruit fly is as follows: (4.1) Multiple small-eye receivers are simulated using H Gaussian kernel functions of different sizes: in, This represents the final saliency map obtained in step (3). Let i represent the i-th Gaussian kernel function, i = 1, ..., H, O. i This represents the output of the i-th small eye; (4.2) Spatial integration of all small-eye outputs is performed according to the following rules: Overlapping sub-regions of a scene are aligned and combined into a larger region. This biomimetic compound eye method effectively eliminates redundant information, thereby representing the target more effectively. The integration result is as follows: In step (5), the method for simulating the visual lobe neural unit of a fruit fly to further obtain the motion characteristics of the target is as follows: (5.1) Design a bandpass filter to simulate the visual lobe neural unit of a fruit fly: φ(t) = B1(t) - B2(t) Where B1(t) and B2(t) represent two two-dimensional time filters; (5.2) Based on the bandpass filter, the small-eye output obtained in step (4) Extracting brightness change features: in, This indicates the output of the small eye at time t; (5.3) Simulate neurons T4 and T5 in the fruit fly visual system to detect motion information and extract motion information from brightness change features: T4(t)=ζ[M(t)] T5(t) = ζ[-M(t)] Where ζ[k] represents max(0, k), and T4(t) and T5(t) represent the outputs of neurons T4 and T5, respectively.

2. According to the method for detecting moving targets in video remote sensing images that integrates bionic vision as described in claim 1, the method for extracting the three types of feature saliency maps of brightness, color, and orientation at different scales of the video remote sensing image in step (1) is as follows: (1.1) Extracting luminance features: Gaussian pyramid decomposition is performed using a Gaussian difference filter bank, and the luminance features are expressed as: in, r, g, and b represent the red, green, and blue channel components of the original RGB remote sensing image, respectively. The luminance features I(c) and I(s) of the c-th and s-th layers of the feature pyramid are fused to obtain the corresponding fused luminance feature map I(c, s): in, This indicates that subtraction is performed between pixels; (1.2) Extracting color features: Using the same Gaussian pyramid decomposition method, the original three primary color RGB channels are converted into the color features of generalized tuned red (R), green (G), blue (B), and yellow (Y): Calculate the red-green channel color feature map RG(c,s) and the blue-yellow channel color feature map BY(c,s): in, This indicates subtraction between pixels, where c and s represent the c-th and s-th layers of the color feature pyramid; (1.3) Extracting Orientation Features: The orientation feature map D(c, s, θ) is calculated using the Gabor pyramid method: Where θ represents the direction of the output. This indicates subtraction between pixels, where c and s represent the c-th and s-th layers of the feature pyramid.

3. According to claim 2, the method for detecting moving targets in video remote sensing images that integrates bionic vision, in step (2), the method for normalizing the saliency maps of each type of feature at different scales to obtain the saliency map corresponding to that type of feature is as follows: (2.1) Normalize the feature maps obtained at different scales for the same type of feature in step (1); let and Let represent the saliency maps of the luminance component, chromaticity component, and orientation component, respectively. Then: in, N(·) represents the normalization operator. This represents cross-scale addition, which involves scaling each feature map to the size of the fourth layer and then adding the pixels together. (2.2) Linearly fuse the saliency maps of various features to obtain the overall saliency map S: Where N(·) represents the normalization operator.

4. According to claim 3, in the method for detecting moving targets in video remote sensing images by fusing bionic vision, in step (3), the saliency maps corresponding to all features are fused, and the winner-takes-all mechanism and return suppression mechanism are used to realize the shift of attention focus, and the method for obtaining the final saliency map is as follows: (3.1) The winner-takes-all mechanism is used to compete for the total saliency map S obtained in step (2). The final winner of the competition is the focus of attention, that is, the neuron with the largest saliency value in the saliency map is the focus of attention. (3.2) A return inhibition mechanism is used to suppress the already noticed locations and their salient regions, causing the focus of attention to shift back to the unnoticed regions. Specifically, after obtaining the focus of attention and the salient regions, a Gaussian difference function with central inhibition and peripheral excitation is used, where the inhibition center of this function is the focus of attention in step (3.1). Finally, the final salient map can be obtained.

5. According to claim 1, in the method for detecting moving targets in video remote sensing images fused with biomimetic vision, the method for target detection by the fruit fly-inspired visual information processing center in step (6) is as follows: (6.1) Integrate the outputs T4(t) and T5(t) from step (5.3) to obtain the possible motion targets: T(t) = T4(t) - T5(t) (6.2) Use the integrated results to determine the authenticity of the detected possible targets, as follows: when T(t) is greater than a certain threshold, it indicates that the target is a true target; otherwise, it is a false target.

Images