A spatio-temporal information fusion-based infrared video stream body target detection method and system

Abstract

The application discloses a kind of infrared video stream fluid target detection method and system of space-time information fusion, the method of the present application includes for each current frame I raw The processing includes: for current frame I raw Carrying out foreground segmentation and extracting foreground as initial fluid target mask, then in turn carrying out filtering and morphological processing, filtering based on centroid distance, I raw As guide image, guide filtering is carried out, I raw Dot multiplication I raw Pseudo-color processing obtains pseudo-color image and is synthesized into detection video.The present application aims to combine the change information of single pixel point in time domain and the adjacent information of overall target in space domain, realize a kind of high feasibility, strong robustness, low computing cost infrared video fluid target detection technology, the target area of water surface temperature difference change is segmented, water surface temperature difference change is effectively monitored, and data support and guidance are provided for the research and application of relevant fields.

Description

Technical Field

[0001] This invention belongs to the field of video image processing target segmentation technology, specifically relating to an infrared video fluid target detection method and system based on spatiotemporal information fusion. Background Technology

[0002] Changes in water surface temperature can provide information about aquatic ecosystems, water health, and climate change. It also helps determine relevant parameters for water engineering applications and assess water resource conditions, making it significant for ecology, environment, climate, water resources, and engineering. An infrared camera is a device that senses and records infrared radiation. It utilizes the characteristic that all objects emit infrared thermal radiation, converting this radiation into visible images or thermal maps to achieve infrared imaging. Because infrared cameras can operate at night or in low-light conditions and offer the advantage of non-contact measurement, they are well-suited for monitoring changes in water surface temperature.

[0003] However, the long wavelength of infrared light and the limited size of detectors based on thermal effects restrict the ability of infrared cameras to perceive details and detect subtle changes in the real world. Furthermore, atmospheric factors such as humidity, fog, and smoke affect the propagation and detection of infrared radiation, resulting in high noise levels, low image clarity, and poor reliability in infrared images and videos. Therefore, conventional target segmentation methods struggle to achieve precise extraction of targets from infrared videos. Deep learning-based target segmentation methods require a large number of labeled samples, which is labor-intensive. Due to the limited availability of existing infrared target segmentation datasets, research on using deep learning for infrared video target detection is currently limited. Among traditional methods, background subtraction is easily affected by changes in lighting conditions, dynamic backgrounds, and partial target occlusion, leading to high false positives and false negatives. Optical flow methods calculate object motion based on pixel brightness changes in an image, but they are also highly sensitive to lighting changes. Image noise can easily blur brightness changes, making optical flow estimation difficult and inaccurate. Additionally, optical flow methods are computationally intensive, requiring high-performance computing resources and demanding high-performance data processing equipment. Considering the practical application of infrared video monitoring of water surface temperature differences, none of the above target segmentation methods are suitable for this problem. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an infrared video fluid target detection method and system based on spatiotemporal information fusion, addressing the aforementioned problems of the prior art. This invention aims to combine the temporal variation information of individual pixels with the adjacency information of the overall target in the spatial domain to achieve a highly feasible, robust, and computationally low infrared video fluid target detection technology. It can segment target areas with large water surface temperature differences, effectively monitor water surface temperature differences, and provide data support and guidance for research and applications in related fields.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A spatiotemporal information fusion-based infrared video fluid target detection method, comprising detecting each current frame I in the infrared video. raw The processing includes: S1: For the current frame I raw Perform foreground segmentation, and superimpose the foreground of the current frame and the foreground of the previous frame as the initial fluid target mask M1; S2: Filter and morphologically process the initial fluid target mask M1 to suppress video noise, remove small patches and smooth the target outline to obtain the denoised target mask M2. S3: Perform connectivity analysis on the denoised target mask M2, mark the connected regions and determine the centroid positions, calculate the distance from the centroid of each connected region to the specified fluid center, and retain the target mask M3 corresponding to the connected regions whose distance from the centroid to the fluid center is less than a set threshold. S4: Using the target mask M3 as the input image, and the original current frame I... raw As a guiding image, combined with the original current frame I raw The spatial information is used for guided filtering to output a target mask M4 with fine edges; S5: Multiply the target mask M4 with fine edges by the original current frame I. raw The target region R carrying the original grayscale value change information is obtained. T For the target region R T Pseudocolor processing is performed to obtain pseudocolor image I processed and the pseudo-color image I processed Synthesized into the detection video V at the same frame rate processed middle.

[0006] Optionally, in step S1, for the current frame I raw Foreground segmentation refers to using a Gaussian Mixture Model (GMM) to segment the current frame I raw Foreground and background recognition is performed on each pixel in the image, and then all foreground pixels are extracted to obtain the foreground of the current frame.

[0007] Optionally, when performing filtering and morphological processing on the initial fluid target mask M1 in step S2, the filtering refers to performing a median filter on the initial fluid target mask M1 to filter out instantaneous video noise. The median filter refers to selecting a local window centered on the current pixel for each pixel and using the median value within the window as the result of filtering that pixel. The morphological processing refers to performing a morphological "opening" operation on the filtered initial fluid target mask M1. The morphological "opening" operation includes first eroding and then dilating to break the narrow connections between targets and removing small protruding areas to smooth the target contour.

[0008] Optionally, in step S4, the original current frame I is combined. raw Guided filtering based on spatial information refers to filtering based on a set local window, combined with the original current frame I. raw The spatial information is used for guided filtering, and the function expression of the guided filtering is: , In the above formula, This represents a local filtering window centered at k. For the target mask M4 in the local window The i-th pixel, For the current frame I raw In local window The i-th pixel, and For local windows The two corresponding constants, and The expressions for the computation functions are as follows: , , In the above formula, For local windows The total number of pixels in the image. For the target mask M3 in a local window The i-th pixel, To guide the image In the window The average pixel value within the range, For the target mask M3 in a local window The average pixel value within the range, To guide the image In the window within variance, This is the regularization parameter.

[0009] Optionally, the local window is an 8×8 square window.

[0010] Optionally, in step S5, the target region R...T Pseudo-color processing includes: processing the target region R T The overall grayscale value is divided into multiple grayscale ranges according to the actual situation, and different RGB colors are assigned to pixels whose grayscale values ​​are in different grayscale ranges.

[0011] Optionally, the target region R T When the overall grayscale value within the range is divided into multiple grayscale intervals according to the actual situation, the grayscale thresholds between the multiple grayscale intervals are [0.1, 0.15, 0.25, 0.3, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9].

[0012] Optionally, in step S5, the target region R... T Before performing pseudo-color processing, it also includes processing the target region R. T Multiply the overall grayscale value by 2 to enhance the target area R T The brightness.

[0013] Furthermore, the present invention also provides an infrared video fluid target detection system based on spatiotemporal information fusion, comprising a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the infrared video fluid target detection method based on spatiotemporal information fusion.

[0014] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program for being programmed or configured by a microprocessor to perform the spatiotemporal information fusion infrared video fluid target detection method.

[0015] Compared with the prior art, the present invention has the following main advantages: The method of the present invention includes targeting each current frame I in the infrared video. raw The processing includes: for the current frame I raw Foreground segmentation is performed to extract the foreground as an initial fluid target mask, followed by filtering and morphological processing, centroid distance-based filtering, and then I... raw The guiding image is used for guiding filtering and dot product I. raw The invention employs pseudo-color processing to obtain pseudo-color images, which are then synthesized into the detection video. By combining the temporal variation information of individual pixels with the spatial adjacency information of the overall target, this invention achieves a highly feasible, robust, and computationally low-cost infrared video fluid target detection technology. It utilizes historical pixel changes to distinguish foreground from background, modifies the fluid target mask step-by-step, and then applies guided filtering to obtain fine details of the target region's edges. This enables accurate segmentation of fluid targets with significant temperature variations, effectively monitoring water surface temperature changes and providing data support and guidance for research and applications in related fields. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention.

[0018] Figure 2 This is a frame diagram of the original video data from an embodiment of the present invention.

[0019] Figure 3 This is the target segmentation result of an embodiment of the present invention.

[0020] Figure 4 This is the target segmentation result of the average background subtraction method used for comparison.

[0021] Figure 5 This is the target segmentation result of the Farneback dense optical flow method for comparison. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See Figure 1 The spatiotemporal information fusion infrared video fluid target detection method in this embodiment includes detecting each current frame I in the infrared video. raw The processing includes: S1: For the current frame I raw Perform foreground segmentation, and superimpose the foreground of the current frame and the foreground of the previous frame as the initial fluid target mask M1; S2: Filter and morphologically process the initial fluid target mask M1 to suppress video noise, remove small patches and smooth the target outline to obtain the denoised target mask M2. S3: Perform connectivity analysis on the denoised target mask M2, mark the connected regions and determine the centroid positions, calculate the distance from the centroid of each connected region to the specified fluid center, and retain the target mask M3 corresponding to the connected regions whose distance from the centroid to the fluid center is less than a set threshold. S4: Using the target mask M3 as the input image, and the original current frame I... raw As a guiding image, combined with the original current frame I raw The spatial information is used for guided filtering to output a target mask M4 with fine edges; S5: Multiply the target mask M4 with fine edges by the original current frame I. raw The target region R carrying the original grayscale value change information is obtained. T For the target region R T Pseudocolor processing is performed to obtain pseudocolor image I processed and the pseudo-color image I processed Synthesized into the detection video V at the same frame rate processed middle.

[0024] It should be noted that, for the current frame I raw Foreground segmentation can be performed using the desired segmentation algorithm. For example, as an optional implementation, see [link to implementation details]. Figure 1 In step S1 of this embodiment, for the current frame I raw Foreground segmentation refers to using a Gaussian Mixture Model (GMM) to segment the current frame I raw Foreground and background identification is performed on each pixel in the image, and then all foreground pixels are extracted to obtain the foreground of the current frame. After inputting the original infrared video, each pixel is processed independently. Each pixel has a signal sequence with time as the horizontal axis and pixel grayscale value as the vertical axis. Using a Gaussian mixture model, a small segment of video in which no target appears is selected to build an initial background model. The current pixel is determined to belong to the foreground or background based on the historical changes in pixel values. At the same time, the latest video frame is added to the background model for dynamic updates. Then, the current foreground is superimposed with the foreground of the previous frame to obtain the initial foreground target mask M1.

[0025] Gaussian Mixture Models (GMMs) represent the background based on pixel sample statistics. They assume that the change of each pixel in a video image sequence is a random process, meaning that the variation of each pixel can be described by a weighted sum of one or more Gaussian distributions, where the weights and parameters of each Gaussian distribution are updated over time. Let the original infrared video have pixels... The sequence of pixel values ​​that change over time is The sampling point at time t is The sampling point obeys Gaussian mixture model probability distribution: , , in, This represents the number of Gaussian submodels. Let be the weights of the Gaussian submodel at time t. represent Time of the first A Gaussian Mixture Model (GMM) Let be the mean of this Gaussian distribution. Let be the covariance matrix of the Gaussian distribution.

[0026] The specific steps of Gaussian Mixture Model (GMM) modeling are as follows: (1) First, the statistical information of the samples over a short period of time is analyzed using the Gaussian Mixture Modeling method to establish an initial background model. Then, statistical difference is used to determine whether the new pixel value is foreground or background. , in represent Time of the first The mean of a Gaussian model, represent Time of the first The standard deviation of a Gaussian model To determine whether a pixel is a multiplicative factor for the foreground or background, if the above formula holds, Pixels at any moment It is judged as background, otherwise as foreground.

[0027] (2) If the pixel value is within the mean deviation of a certain Gaussian distribution, the weights of each Gaussian model are updated as follows: , The weight learning rate is When matching models Conversely 0, and finally normalize the weights; (3) For the matched Gaussian distribution model, update its mean according to the following formula. and standard deviation : , , , (4) If the pixel value is not within the mean deviation of any Gaussian distribution, the model with the smallest weight is replaced with a new model. The mean of the new model is the current pixel value, the standard deviation is the initial larger value, and the weight is the smaller value. (5) According to Sort the Gaussian models in descending order (where...) (representing weights), select the top weights that satisfy the following formula. A Gaussian model is used as a background: , in The threshold represents the proportion of the background. The relevant parameters of the Gaussian Mixture Model (GMM) in this embodiment are as follows: initial training frames: 160 frames; number of individual Gaussian models constituting the GMM: 3; initial variance: adaptive (squared of the pixel value); initial model weights: 0.05; judgment threshold: 2.5 times the standard deviation; foreground threshold: 0.7; parameter learning rate: 0.005.

[0028] In step S2 of this embodiment, when filtering and morphological processing are performed on the initial fluid target mask M1, filtering refers to performing a median filter on the initial fluid target mask M1 to remove instantaneous video noise. A median filter means selecting a local window centered on the current pixel for each pixel and using the median value within the window as the filtered result for that pixel. The morphological processing refers to performing a morphological "opening" operation on the filtered initial fluid target mask M1. The morphological "opening" operation includes erosion followed by dilation to break narrow connections between targets and removal of small protruding areas to smooth the target contour. Specifically, in this embodiment, the filtering kernel for median filtering is a 3×3 square centered on the current pixel, and the structural element for the "opening" operation is a disk-shaped structure with a radius of 3.

[0029] In step S3, connectivity analysis is performed on the denoised target mask M2, connected regions are marked and centroid positions are determined, and the distance from the centroid of each connected region to the specified fluid center is calculated. When retaining the target mask M3 corresponding to the connected region whose distance from the centroid to the fluid center is less than a set threshold, in this embodiment, the distance threshold between the centroid of the connected region and the fluid center is set to 100 according to the maximum diffusion degree of the fluid target.

[0030] In step S4, the corresponding original video frames are... As a guiding image, the local window radius is determined based on the size of the water surface fluid target, and appropriate regularization parameters are introduced to perform guided filtering on the foreground target mask M3, maximizing the restoration of the original gradient information and contour details of the fluid target. The output target mask... It can be represented as a guide image. A local linear transformation. In step S4 of this embodiment, the original current frame I is combined. raw Guided filtering based on spatial information refers to filtering based on a set local window, combined with the original current frame I. raw The spatial information is used for guided filtering, and the functional expression of the guided filtering (the functional expression of the local linear transformation mentioned above) is: , In the above formula, This represents a local filtering window centered at k. For the target mask M4 in the local window The i-th pixel, For the current frame I raw In local window The i-th pixel, and For local windows The two corresponding constants, and The expressions for the computation functions are as follows: , , In the above formula, For local windows The total number of pixels in the image. For the target mask M3 in a local window The i-th pixel, To guide the image In the window The average pixel value within the range, For the target mask M3 in a local window The average pixel value within the range, To guide the image In the window within variance, This is the regularization parameter.

[0031] By using the corresponding original video frame as the guiding image, the size of the local window is determined according to the size of the water surface fluid target, and an appropriate regularization parameter is introduced to perform guided filtering on the foreground target mask, thereby restoring the original gradient information and contour details of the fluid target to the greatest extent. In this embodiment, the local window is an 8×8 square window, and the regularization parameter is 10. -6 .

[0032] Using grayscale layering, fine target regions R are analyzed. T Pseudo-color processing is performed, which involves dividing the overall grayscale value of the target area into multiple grayscale intervals according to the actual situation, and assigning different RGB colors to pixels whose grayscale values ​​fall within different grayscale intervals. Since the target area is generally dark, for ease of processing, a pseudo-color processing method is chosen. The overall grayscale value is multiplied by 2, and then pseudo-color processing is performed.

[0033] In step S5 of this embodiment, a grayscale layering method is used to refine the target area R. T Pseudo-color processing is performed; specifically, in step S5, the target region R... T Pseudo-color processing includes: processing the target region R T The overall grayscale value is divided into multiple grayscale ranges according to the actual situation, and different RGB colors are assigned to pixels whose grayscale values ​​are in different grayscale ranges.

[0034] This embodiment targets the region R. T When the overall grayscale value within the range is divided into multiple grayscale intervals according to the actual situation, the grayscale thresholds between the multiple grayscale intervals are [0.1, 0.15, 0.25, 0.3, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9], that is, 16 grayscale intervals are divided using 15 grayscale thresholds. In this embodiment, different pseudo-colors are designed for rendering different grayscale intervals, and the corresponding RGB values ​​are displayed in hexadecimal as follows: {'#000000', '#fbff05','#fbf215', '#fcd734', '#fcc944', '#fdae63','#fda173','#fe8692', '#fe78a1','#fe5dc1','#ff50d0','#ff42e0','#ff38a1','#ff3382','#ff2d63','#ff2634'}.

[0035] Since the target area is generally dark, the step S5 in this embodiment involves processing the target area R... T Before performing pseudo-color processing, it also includes processing the target region R. T Multiply the overall grayscale value by 2 to enhance the target area R T The brightness is then adjusted, and then pseudo-color processing is performed.

[0036] To verify the spatiotemporal information fusion-based infrared video fluid target detection method and system proposed in this example, hot water was injected into the bottom of a room-temperature pool at a certain rate. The hot water diffused upwards, forming a fluid target. An infrared camera fixed at the top continuously captured the temperature difference changes in the pool water. The total duration of the original infrared video was 27 seconds, with a total of 670 frames. Frames 281, 463, and 557 of the video were selected for algorithm comparison and verification. The original video frames are shown below. Figure 2 As shown, from left to right, (a), (b), and (c) are frames 281, 463, and 557, respectively.

[0037] The processing result of the method in this embodiment is as follows: Figure 3 As shown in (a), (b), and (c), the comparison methods employed are the average background subtraction method and the Farneback dense optical flow method, both commonly used in video moving target detection. The processing results are as follows: Figure 4 (a), (b), and (c) in the text are related to... Figure 5 As shown in (a), (b), and (c) of the figure. A qualitative evaluation from the perspective of human visual attention reveals that... Figure 2The original infrared video appears dark and blurry overall, with low contrast and obvious striped noise. The result obtained using the method in this embodiment shows a clearer boundary between the target and background, while preserving the detailed temperature variations within the target. The added pseudo-color processing also improves the target visualization. The average background subtraction method can obtain the approximate outline of the target, but due to strong video noise, a large amount of noise is incorrectly identified as the target. The Farneback dense optical flow method can obtain the instantaneous motion velocity of each pixel, but since most pixels exhibit some fluctuation over time, the optical flow method also incorrectly identifies these pixels as the target. Furthermore, the moving target in this embodiment appears diffused on the water surface; the optical flow method treats the central area of ​​the target as a "stationary" area, thus only obtaining the outer ring of the target.

[0038] In addition, this embodiment selects the following four indicators to quantitatively evaluate the target segmentation method proposed in this example: Pixel Accuracy (PA): , Class Pixel Accuracy (CPA) , Intersection over Union (IoU): , DICE coefficient: , Where TP(1 Positive) represents the number of target pixels correctly identified; FP(0 Positive) represents the number of non-target pixels incorrectly identified as targets; FN(0 Negative) represents the number of target pixels incorrectly identified as non-target pixels; and TN(1 Negative) represents the number of non-target pixels correctly identified.

[0039] Due to the large number of video frames, 10 frames ([280 300 350 400 450 500 550 600 650 670]) were extracted from the original infrared video and manually labeled. The results of the three methods are shown in Table 1.

[0040] Table 1: Quantitative evaluation results of the three methods.

[0041] Generally, in the field of semantic segmentation, higher values ​​for the above four metrics are considered better. As shown in Table 1, due to the strong noise and pixel fluctuations in infrared video, the average background subtraction method and the Farneback dense optical flow method perform poorly in terms of CPA, IoU, and DICE, and are not suitable for the target segmentation scenario described in this embodiment. However, the method described in this embodiment performs well in all four metrics, thus confirming that the method achieves accurate segmentation of water surface fluid targets with significant temperature variations.

[0042] In summary, the spatiotemporal information fusion infrared video fluid target detection method of this embodiment combines the temporal variation information of individual pixels with the spatial adjacency information of the overall target, including: inputting the original infrared video. In the time dimension, based on the historical pixel values ​​of each pixel in the video, it is determined whether the pixel at that moment belongs to the foreground or the background, and the background model is dynamically updated. The current foreground is superimposed with the foreground of the previous frame to output the initial fluid target mask. ; Initial fluid target mask for input Filtering and morphological operations are performed to suppress video noise, remove small patches, and smooth the target contour, outputting a denoised target mask. ; for the denoised target mask Connectivity analysis is performed, connected regions are marked, and their centroid positions are determined. The distance from the centroid of each connected region to the fluid center is calculated. Only connected regions whose distance falls within a certain threshold range are retained, thus outputting a target mask that retains a small number of connected regions. ;The target mask obtained in the previous step As input images, the original video frames As a guide image, spatial information from the original image is combined with guide filtering to output a target mask with finer edges. ; target mask with fine edges Dot product of original video frames To obtain the target area carrying the original grayscale value change information. Then, pseudo-color processing is performed on the target area to obtain... Finally, the video was synthesized at the same frame rate. This invention integrates temporal and spatial information from infrared video, enabling the segmentation of infrared video water surface fluid targets, highlighting the surface temperature differences of infrared video water targets, and providing better visualization effects.

[0043] Furthermore, this embodiment also provides an infrared video fluid target detection system based on spatiotemporal information fusion, including a microprocessor and a memory interconnected thereto. The microprocessor is programmed or configured to execute the infrared video fluid target detection method based on spatiotemporal information fusion. Additionally, this embodiment also provides a computer-readable storage medium storing a computer program for being programmed or configured by the microprocessor to execute the infrared video fluid target detection method based on spatiotemporal information fusion.

[0044] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0045] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A spatio-temporal information fusion infrared video stream body target detection method, characterized in that, The processing comprises for each current frame I raw of the infrared video S1: for the current frame I raw perform foreground segmentation, and superimpose the foreground of the current frame and the foreground of the previous frame as an initial fluid target mask M1; S2: Filter and morphologically process the initial fluid target mask M1 to suppress video noise, remove small patches and smooth the target outline to obtain the denoised target mask M2. S3: Perform connectivity analysis on the denoised target mask M2, mark the connected regions and determine the centroid positions, calculate the distance from the centroid of each connected region to the specified fluid center, and retain the target mask M3 corresponding to the connected regions whose distance from the centroid to the fluid center is less than a set threshold. S4: taking the target mask M3 as an input image, conducting guided filtering on the original current frame I raw as a guide image in combination with the spatial information of the original current frame I raw to output a target mask M4 with fine edges; S5: Multiply the target mask M4 with fine edges by the original current frame I. raw The target region R carrying the original grayscale value change information is obtained. T For the target region R T Pseudocolor processing is performed to obtain pseudocolor image I processed and the pseudo-color image I processed Synthesized into the detection video V at the same frame rate processed middle; In step S4, the original current frame I raw is guided filtered based on the spatial information of the original current frame I raw , and the function expression of the guided filtering is as follows: ， In the above formula, This represents a local filtering window centered at k. For the target mask M4 in the local window The i-th pixel, For the current frame I raw In local window The i-th pixel, and For local windows The two corresponding constants, and The expressions for the computation functions are as follows: ， ， In the above formula, For local windows The total number of pixels in the image. For the target mask M3 in a local window The i-th pixel, To guide the image In the window The average pixel value within the range, For the target mask M3 in a local window The average pixel value within the range, To guide the image In the window within variance, This is the regularization parameter.

2. The spatio-temporal information fused infrared video stream body target detection method according to claim 1, characterized in that, In step S1, for the current frame I raw foreground segmentation refers to using a Gaussian Mixture Model (GMM) to identify foreground and background for each pixel point in the current frame I raw and then extracting all foreground pixel points to obtain the foreground of the current frame.

3. The spatio-temporal information fused infrared video stream body target detection method of claim 1, wherein, In step S2, when filtering and morphological processing are performed on the initial fluid target mask M1, the filtering refers to performing a median filter on the initial fluid target mask M1 to filter out instantaneous noise in the video. The median filter means selecting a local window centered on the current pixel for each pixel and using the median value within the window as the result of filtering that pixel. The morphological processing refers to performing a morphological "opening" operation on the filtered initial fluid target mask M1. The morphological "opening" operation includes first eroding and then dilating to break the narrow connections between targets and removing small protruding areas to smooth the target contour.

4. The spatio-temporal information fused infrared video stream body target detection method of claim 1, wherein, The local window is an 8×8 square window.

5. The spatio-temporal information fused infrared video stream body target detection method of claim 1, wherein, The target region R in step S5 T The pseudo-color processing includes: dividing the overall gray scale value in the target region R T into multiple gray scale intervals according to actual conditions, and assigning different RGB colors to pixels with gray scale values in different gray scale intervals.

6. The spatio-temporal information fused infrared video stream body target detection method according to claim 5, characterized in that, The target area R T When the overall grayscale value within the range is divided into multiple grayscale intervals according to the actual situation, the grayscale thresholds between the multiple grayscale intervals are [0.1, 0.15, 0.25, 0.3, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9].

7. The spatio-temporal information fused infrared video stream body target detection method according to claim 6, characterized in that, The target region R in step S5 T Before pseudo-color processing, it also includes multiplying the overall gray value of the target region R T by 2 to enhance the brightness of the target region R T .

8. A spatio-temporal information fusion based infrared video stream moving target detection system comprising a microprocessor and a memory connected to each other, characterized in that, The microprocessor is programmed or configured to execute the infrared video fluid target detection method based on spatiotemporal information fusion as described in any one of claims 1 to 7.

9. A computer-readable storage medium having stored therein a computer program, characterized in that, The computer program is used to be programmed or configured by a microprocessor to execute the infrared video fluid target detection method based on spatiotemporal information fusion as described in any one of claims 1 to 7.

Images