A dynamic target recognition method for penetration imaging combined with timing information

Abstract

The application relates to a dynamic target identification method for penetration imaging combined with timing information. The method comprises the following steps: adjusting the delay time of a current gate window according to the overlapping degree between a laser emission pulse and a return signal obtained in a previous gate window; analyzing the concentration changes of a plurality of candidate smoke blocks in a penetration image obtained in the current gate window and a plurality of historical penetration images before the penetration image, so as to determine a target smoke block in the penetration image; performing gray scale updating on a background frame of the penetration image according to the deviation degree between the area of the target smoke block in the penetration image and the area of the target smoke block in a plurality of historical penetration images before the penetration image, so as to obtain a target background frame; and performing background deletion in the penetration image based on the target background frame, so as to obtain a candidate region, and performing a dynamic target identification task according to the candidate region. The application can reduce the false detection rate and the missed detection rate of the dynamic target.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method for dynamic target recognition using penetration imaging that incorporates temporal information. Background Technology

[0002] In fire rescue scenarios, laser distance-gated imaging technology can effectively penetrate dense smoke to obtain image information inside the fire scene, providing crucial support for the search and rescue of trapped personnel and fire situation assessment. However, existing technologies have significant limitations when dealing with high-concentration, dynamically changing smoke scenes caused by fire. During the propagation of the laser pulse in space, echo signals from distant areas (such as walls or ceilings behind the smoke layer) and target echo signals return to the detector within the same gating gate opening time, forming isolated visual artifacts with randomly distributed bright and dark pixels. These artifacts can cause false breaks or abnormal protrusions in the target outline, resulting in the target appearing and disappearing intermittently. Simultaneously, the smoke concentration in a fire scene dynamically changes with the combustion process. Fixed threshold segmentation cannot adapt to the differences in smoke concentration across different time frames, further exacerbating the problem of misclassifying smoke pseudo-contours as dynamic targets or misclassifying real target pixels partially obscured by smoke as background. This causes a sharp decrease in the recognition accuracy of existing methods in high-velocity dense smoke environments during the flashover phase, leading to a significant increase in the false detection and false negative rates for dynamic targets. Summary of the Invention

[0003] To address the aforementioned technical problems, the purpose of this application is to provide a method for dynamic target recognition using penetration imaging that incorporates temporal information. The specific technical solution adopted is as follows: Firstly, a method for dynamic target recognition using penetration imaging incorporating temporal information is provided, the method comprising: The delay time of the current gating window is adjusted based on the degree of overlap between the laser emission pulse and the echo signal obtained from the previous gating window; the delay time is used to suppress visual artifacts caused by echo aliasing. In the current gating window, the density changes of multiple candidate smoke patches in the image and the previous historical images are analyzed to determine the target smoke patch in the image. The density change of the pixels in the target smoke patch meets the preset density change conditions. The multiple candidate smoke patches are obtained by image segmentation of the image and the previous historical images. Based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in several previous historical penetrated images, the background frame of the penetrated image is updated in grayscale to obtain the target background frame. Background removal is performed on the penetrating image based on the target background frame to obtain candidate regions, and dynamic target recognition is performed based on the candidate regions.

[0004] Optionally, the delay time of the current gating window is adjusted based on the degree of overlap between the laser emission pulse and the echo signal acquired in the previous gating window, including: Based on the cross-correlation between the laser emission pulse and the echo signal obtained from the previous gating window, and the geometric mean relationship between the total energy of the echo signal and the total energy of the laser emission pulse, the overlap coefficient between the echo signal and the laser emission pulse is calculated; the overlap coefficient characterizes the degree of temporal overlap between the echo signal and the laser emission pulse. Adjust the delay time of the current gating window based on the overlap coefficient.

[0005] Optionally, based on the cross-correlation between the laser emission pulse and the echo signal obtained from the previous gating window, and the geometric mean relationship between the total energy of the echo signal and the total energy of the laser emission pulse, the overlap coefficient between the echo signal and the laser emission pulse is calculated, including: Calculate the peak value of the cross-correlation between the laser emission pulse and the echo signal; Calculate the sum of squares of the voltages at all sampling points in the echo signal to obtain the total echo energy; Calculate the sum of squares of the voltages at all sampling points in the laser emission pulse to obtain the total emitted energy; Dividing the cross-correlation peak value by the geometric mean of the total echo energy and the total emitted energy yields the overlap coefficient between the echo signal and the laser emission pulse.

[0006] Optionally, in the penetration image acquired in the current gating window and several previous historical penetration images, the concentration changes of multiple candidate smoke patches in the images are analyzed to determine the target smoke patch in the penetration image, including: Dynamic thresholding is performed on the penetrated image and multiple previous historical penetrated images to obtain multiple initial smoke patches in the penetrated image and multiple initial smoke patches in each historical penetrated image. Multiple initial smoke patches in the penetrated image and multiple initial smoke patches in each historical penetrated image are differentially processed to obtain multiple differential patches between each two frames of images. The intersection of multiple differential patches between multiple frames of images is then processed to obtain multiple candidate smoke patches. Calculate the standard deviation of the rate of change of area for each candidate smoke patch in the penetrated image and multiple previous historical penetrated images; Extract the grayscale time series of each pixel in the candidate smoke patch in consecutive frames of images, decompose the grayscale time series corresponding to each pixel into multiple components of different frequencies, and mark the pixels that meet the preset conditions as potential smoke pixels according to the proportion of the energy of the high-frequency components in the total oscillation energy. The candidate smoke patches are used to determine the target smoke patch based on the proportion of potential smoke pixels and the standard deviation of the area change rate in the candidate smoke patch.

[0007] Optionally, the standard deviation of the area change rate of each candidate smoke patch in the penetrated image and multiple previous historical penetrated images is calculated, including: Obtain the area value of each candidate smoke patch in the penetrated image and its multiple preceding historical penetrated images; Calculate the rate of change of area between each pair of adjacent frames; The standard deviation of the area change rate of each candidate smoke patch in multiple frames of imagery is calculated to obtain the standard deviation of the area change rate of each candidate smoke patch; the standard deviation of the area change rate is used to characterize the degree of fluctuation of the area of ​​the candidate smoke patch.

[0008] Optionally, the grayscale time series of each pixel in the candidate smoke patch is extracted in multiple consecutive frames of imagery. The grayscale time series corresponding to each pixel is decomposed into multiple components of different frequencies. Based on the proportion of the energy of the high-frequency components in the total oscillation energy, pixels that meet preset conditions are marked as potential smoke pixels, including: Extract the grayscale time series of each pixel in the candidate smoke patch in multiple consecutive frames of images, decompose the grayscale time series corresponding to each pixel into multiple components of different frequencies, and calculate the proportion of the energy of the high-frequency component in the total oscillation energy to obtain the energy proportion of the high-frequency component. When the energy proportion of high-frequency components is greater than the first preset threshold, the pixel is determined to be a smoke scattering feature point; For smoke scattering feature points, the mean and standard deviation of the high-frequency component energy ratio of all pixels in its neighborhood are obtained, and the neighborhood consistency value of the smoke scattering feature points is determined based on the mean and standard deviation. When the neighborhood consistency value of a smoke scattering feature point is greater than a second preset threshold, the smoke scattering feature point is determined to be a potential smoke pixel.

[0009] Optionally, the candidate smoke patches are used to determine the target smoke patch based on the proportion of potential smoke pixels and the standard deviation of the area change rate in the candidate smoke patch, including: The potential smoke ratio is obtained by calculating the proportion of potential smoke pixels in the candidate smoke patch to the total number of pixels in the candidate smoke patch. When the proportion of potential smoke is greater than the third preset threshold and the standard deviation of the area change rate is greater than the fourth preset threshold, the candidate smoke patch is determined as the target smoke patch.

[0010] Optionally, based on the deviation between the area of ​​the target smoke patch in the penetrated image and the areas of the target smoke patches in its previous historical penetrated images, the background frame of the penetrated image is updated in grayscale to obtain the target background frame, including: The update weight of the background frame is determined based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images; wherein, the update weight of the background frame is positively correlated with the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images. The target background frame is obtained by weighted fusion of the previous historical target background frame and the penetrated image based on the update weight of the background frame.

[0011] Optionally, the update weight of the background frame is determined based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images, including: Calculate the deviation of the area of ​​the target smoke patch in the penetrated image from the average area of ​​the target smoke patch in multiple previous historical penetrated images, and obtain the deviation value; The update weight of the background frame is determined by the weighted sum of the preset base weight and the deviation value.

[0012] Optionally, background removal is performed on the penetrated image based on the target background frame to obtain candidate regions, and a dynamic target recognition task is performed based on the candidate regions, including: The penetrating image and the target background frame are differentially processed to obtain a grayscale difference image; Binarize the grayscale difference image to obtain candidate regions; Based on the degree of overlap between the laser emission pulse and the echo signal obtained in the current gating window, determine whether there is echo aliasing; When echo aliasing exists, the artifact region is determined based on the time delay of the echo signal; The overlapping areas between the candidate regions and the artifact regions are removed to obtain the remaining candidate regions, and the dynamic target recognition task is performed based on the remaining candidate regions.

[0013] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this application.

[0014] This application offers the following advantages: By dynamically adjusting the delay time of the current gating window based on the overlap between the laser emission pulse and the echo signal obtained from the previous gating window, visual artifacts caused by the aliasing of long-distance echoes and target echoes can be effectively suppressed from the hardware source, reducing the interference of artifacts on subsequent recognition processes. By analyzing the concentration changes of multiple candidate smoke patches in the penetration image obtained in the current gating window and several previous historical penetration images, and utilizing the temporal characteristics of smoke pixel concentration changes conforming to preset concentration change conditions, the target smoke patch is accurately selected, thereby effectively distinguishing dynamic smoke from real dynamic targets. Based on the deviation between the area of ​​the target smoke patch and its historical area, the background frame is updated in grayscale, enabling the background frame to adaptively follow the dynamic changes in smoke concentration, ensuring that the smoke distribution in the background frame matches that in the current frame. Background removal based on the updated target background frame in the penetrating image can effectively filter out residual smoke interference and retain the real dynamic target candidate region. This significantly reduces the false detection rate and false negative rate of dynamic targets in the high-velocity dense smoke environment during the flashover stage, solves the problem of targets appearing and disappearing due to echo aliasing artifacts, and achieves stable and accurate dynamic target recognition. Attached Figure Description

[0015] Figure 1 This is a flowchart of a penetration imaging dynamic target recognition method that incorporates temporal information in one embodiment; Figure 2 This is a schematic diagram of the structure of a penetration imaging dynamic target recognition system that incorporates temporal information in one embodiment. Detailed Implementation

[0016] The following description, in conjunction with the accompanying drawings, details a specific scheme for a penetration imaging dynamic target recognition method incorporating temporal information provided in this application. For example... Figure 1 As shown, the method includes: S11. Adjust the delay time of the current gating window based on the degree of overlap between the laser emission pulse and the echo signal obtained from the previous gating window.

[0017] The delay time is used to suppress visual artifacts caused by echo aliasing.

[0018] This application constructs a typical building fire scenario (including flammable furniture, decorative materials, and other combustibles) in a fire simulation laboratory. A laser range-gated imaging system (operating frequency set at 25 frames / second, using burst mode, duty cycle 50%) is deployed within the imaging field of view. A lidar smoke concentration sensor (sampling frequency preset to 25Hz) is installed 10 cm from the optical axis of the imaging system. A fire is triggered by an ignition device, and a continuous time-series image sequence of the entire process is acquired, from the initial stage of fire, smoke layer formation, ceiling jet, smoke layer settling to the flashover stage. Simultaneously, the estimated smoke concentration corresponding to the acquisition time of each frame is recorded, obtaining a synchronous data time-series sequence with a duration of no less than 5 minutes and approximately 7500 frames, to collect information on several parameters and form an information database.

[0019] In a fire scenario, when a laser range-gated imaging system operates in burst mode, the time delay during laser pulse propagation can cause aliasing between echo signals from distant areas (such as walls or ceilings behind the smoke layer) and echo signals from targets (trapped personnel, moving burning materials). This aliasing results in the detector receiving reflected light from different distances within the same gate opening time, creating visual artifacts. To suppress artifact generation at its source, the delay time of the current gating window can be dynamically adjusted.

[0020] In one embodiment, adjusting the delay time of the current gating window based on the degree of overlap between the laser emission pulse and the echo signal obtained in the previous gating window includes: Based on the cross-correlation between the laser emission pulse and the echo signal obtained from the previous gating window, and the geometric mean relationship between the total energy of the echo signal and the total energy of the laser emission pulse, the overlap coefficient between the echo signal and the laser emission pulse is calculated; the overlap coefficient characterizes the degree of temporal overlap between the echo signal and the laser emission pulse. Adjust the delay time of the current gating window based on the overlap coefficient.

[0021] Specifically, based on the cross-correlation between the laser emission pulse and the echo signal obtained from the previous gating window, and the geometric mean relationship between the total energy of the echo signal and the total energy of the laser emission pulse, the overlap coefficient between the echo signal and the laser emission pulse is calculated, including: Calculate the peak value of the cross-correlation between the laser emission pulse and the echo signal; Calculate the sum of squares of the voltages at all sampling points in the echo signal to obtain the total echo energy; Calculate the sum of squares of the voltages at all sampling points in the laser emission pulse to obtain the total emitted energy; Dividing the cross-correlation peak value by the geometric mean of the total echo energy and the total emitted energy yields the overlap coefficient between the echo signal and the laser emission pulse.

[0022] For the laser emission pulse P and echo signal E synchronously collected in the previous gating window, the cross-correlation function of the two is calculated, and the cross-correlation peak C is extracted. The cross-correlation peak C reflects the alignment degree of the two signals on the time axis. Due to the difference in signal amplitude, energy normalization processing needs to be performed on the cross-correlation peak: calculate the total echo energy of the echo signal respectively (the sum of the squares of the voltages of all sampling points) and the total emission energy of the laser emission pulse (the sum of the squares of the voltages of all sampling points). Then, the cross-correlation peak is divided by the geometric mean of the total echo energy and the total emission energy to obtain the overlap coefficient R, and the calculation formula is: ; wherein R ranges from 0 to 1, and the cross-correlation peak C is obtained by calculating the temporal convolution of the laser emission pulse and the echo signal. The cross-correlation peak reflects the similarity degree of the two signals, and the geometric mean of energy can further eliminate the influence of signal amplitude difference, which intuitively describes the temporal overlap ratio between the echo signal and the laser emission pulse.

[0023] The delay time of the current gating window is dynamically adjusted according to the value of the overlap coefficient R: if R ≤ 30%, it indicates that the long-distance echo and the target echo are severely aliased. At this time, the delay time of the gating window is increased by 10 nanoseconds to shift the opening moment of the gating gate backward, so as to preferentially receive the target echo at a closer position and discard the smoke scattering energy at the leading edge of the echo. If 30% < R < 70%, it indicates that there is mild aliasing, and the delay time is increased by 5 nanoseconds. If R ≥ 70%, it indicates that there is no obvious aliasing, and the original gating window parameters remain unchanged.

[0024] Through the above dynamic adjustment at the hardware level, long-distance smoke echoes can be effectively filtered, the generation of artifacts can be reduced, and higher-quality penetration images can be provided for subsequent image processing.

[0025] S12. In the penetration image acquired in the current gating window and a plurality of historical penetration images before the penetration image, the concentration change of a plurality of candidate smoke blocks in the image is analyzed to determine the target smoke block in the penetration image.

[0026] wherein the concentration change of the pixel points in the target smoke block meets a preset concentration change condition, and the plurality of candidate smoke blocks are obtained by performing image segmentation on the penetration image and the plurality of historical penetration images before the penetration image.

[0027] In one embodiment, analyzing the concentration change of a plurality of candidate smoke blocks in the image in the penetration image acquired in the current gating window and a plurality of historical penetration images before the penetration image to determine the target smoke block in the penetration image includes: Dynamic thresholding is performed on the penetrated image and multiple previous historical penetrated images to obtain multiple initial smoke patches in the penetrated image and multiple initial smoke patches in each historical penetrated image. Multiple initial smoke patches in the penetrated image and multiple initial smoke patches in each historical penetrated image are differentially processed to obtain multiple differential patches between each two frames of images. The intersection of multiple differential patches between multiple frames of images is then processed to obtain multiple candidate smoke patches. Calculate the standard deviation of the rate of change of area for each candidate smoke patch in the penetrated image and multiple previous historical penetrated images; Extract the grayscale time series of each pixel in the candidate smoke patch in consecutive frames of images, decompose the grayscale time series corresponding to each pixel into multiple components of different frequencies, and mark the pixels that meet the preset conditions as potential smoke pixels according to the proportion of the energy of the high-frequency components in the total oscillation energy. The candidate smoke patches are used to determine the target smoke patch based on the proportion of potential smoke pixels and the standard deviation of the area change rate in the candidate smoke patch.

[0028] Dynamic thresholding is performed on the current penetration image and multiple previous historical penetration images (e.g., N consecutive frames). Specifically, the mean smoke concentration within a sliding window (the last 5 frames) is calculated. with standard deviation Smoke concentration is characterized using the grayscale ratio method: the proportion of pixels with a grayscale value greater than the baseline grayscale G0 (the average grayscale value of the image in a smoke-free scene plus 50, taken as 180) out of the total number of pixels is used to obtain the smoke concentration for each frame. Since the changes in smoke concentration are continuously distributed over a short period, its mean value... The average level and standard deviation of the current smoke concentration. This reflects the amplitude of concentration fluctuations. The threshold search range of the Otsu method (OTSU) is dynamically constrained as [ , The threshold value is then normalized to between 0 and 255. This threshold is used to segment the current penetrated image, extracting the foreground region (including smoke and moving targets) to obtain multiple initial smoke patches in the penetrated image. The above operation is performed on multiple consecutive frames (e.g., a preset 5 frames) to obtain multiple initial smoke patches in each historical penetrated image.

[0029] Multiple initial smoke patches in the penetrated image and multiple initial smoke patches in each historical penetrated image are differentially processed to obtain multiple differential patches between every two frames. The intersection of multiple differential patches between multiple frames is then processed to obtain multiple candidate smoke patches. Specifically, the inter-frame difference map of every two adjacent frames in five consecutive frames is calculated, resulting in a total of four difference maps. The intersection of these four difference maps is then taken to obtain multiple candidate smoke patches.

[0030] In one embodiment, calculating the standard deviation of the rate of change of area for each candidate smoke patch in the penetrated image and multiple previous historical penetrated images includes: Obtain the area value of each candidate smoke patch in the penetrated image and its multiple preceding historical penetrated images; Calculate the rate of change of area between each pair of adjacent frames; The standard deviation of the area change rate of each candidate smoke patch in multiple frames of imagery is calculated to obtain the standard deviation of the area change rate of each candidate smoke patch; the standard deviation of the area change rate is used to characterize the degree of fluctuation of the area of ​​the candidate smoke patch.

[0031] Candidate smoke patches contain both real dynamic targets and pseudo-changing regions formed by smoke flow. In fire scenes, smoke exhibits high-frequency pulsation characteristics: the outline area changes drastically during the rise of the smoke plume, with a large rate of change; while the outline area of ​​real dynamic targets (such as trapped personnel moving slowly) changes relatively smoothly, typically with a small rate of change per frame. Therefore, by calculating the standard deviation of the temporal area change rate of stable changing regions, the two can be distinguished: Let the area of ​​each candidate smoke patch in 5 consecutive frames be respectively , , , , Calculate the rate of change of the area of ​​the candidate smoke patch in each frame relative to the corresponding candidate smoke patch in the previous frame. , , , For example, the formula for the rate of change of area in the first frame is: Subsequent frames follow the same pattern, with each formula's denominator including... The constant offset.

[0032] Furthermore, since the area change of smoke is random and irregular, the standard deviation of its area change rate is relatively large; while the area change of the real target is continuous and stable, with a smaller standard deviation. Therefore, we first calculate the standard deviation of the area change rate for each candidate smoke patch. .

[0033] ; in, The standard deviation of the rate of change of area for each candidate smoke patch; For the first The rate of change of area of ​​each candidate smoke patch in the frame image relative to the corresponding candidate smoke patch in the previous frame image; This represents the average rate of change of area for each candidate smoke patch across multiple image frames.

[0034] In one embodiment, the grayscale time series of each pixel in the candidate smoke patch is extracted from multiple consecutive frames of imagery. The grayscale time series corresponding to each pixel is decomposed into multiple components of different frequencies. Based on the proportion of the energy of the high-frequency components in the total oscillation energy, pixels that meet preset conditions are marked as potential smoke pixels, including: Extract the grayscale time series of each pixel in the candidate smoke patch in multiple consecutive frames of images, decompose the grayscale time series corresponding to each pixel into multiple components of different frequencies, and calculate the proportion of the energy of the high-frequency component in the total oscillation energy to obtain the energy proportion of the high-frequency component. When the energy proportion of high-frequency components is greater than the first preset threshold, the pixel is determined to be a smoke scattering feature point; For smoke scattering feature points, the mean and standard deviation of the high-frequency component energy ratio of all pixels in its neighborhood are obtained, and the neighborhood consistency value of the smoke scattering feature points is determined based on the mean and standard deviation. When the neighborhood consistency value of a smoke scattering feature point is greater than a second preset threshold, the smoke scattering feature point is determined to be a potential smoke pixel.

[0035] For each pixel in each candidate smoke patch Extract its grayscale time series from consecutive D frames of imagery (e.g., D = 60). The empirical mode decomposition method is adopted to... It is decomposed into multiple intrinsic mode function (IMF) components of different frequencies and a residual term. The high-frequency random fluctuations caused by smoke are concentrated in the first few high-frequency components, while the low-frequency smooth changes caused by the target motion are reflected in the low-frequency components.

[0036] Select the first three high-frequency components and calculate the proportion of their energy in the total oscillation energy to obtain the energy proportion of the high-frequency components. The calculation formula is: ; in, This represents the mean of grayscale values ​​within the grayscale time series. This reflects the proportion of high-frequency component energy in the total oscillation energy. For the first The high-frequency component is at the ... The frame value reflects the oscillation characteristics of the signal at different times; Used to take absolute values; the numerator describes the energy of the high-frequency components, and the denominator represents the total oscillation energy.

[0037] like ≥ First preset threshold (e.g., 0.6) indicates a pixel The grayscale fluctuations are dominated by high-frequency random components, which are consistent with the characteristics of smoke, and are initially marked as smoke scattering feature points.

[0038] To further improve the accuracy of judgment for smoke scattering feature points, spatial neighborhood information needs to be combined. An 8-neighborhood window is taken for each smoke scattering feature point. Calculate the average proportion of high-frequency components of all pixels within an 8-neighborhood window. and standard deviation The neighborhood consistency value for each smoke scattering feature point is determined based on the mean and standard deviation. The calculation formula is: ;in, This is a negative correlation normalization function, whose input is the ratio of the standard deviation to the mean of the proportion of high-frequency components in the neighborhood pixels. The output is a normalized consistency score between 0 and 1. The core function of this function is to achieve negative correlation mapping—the smaller the input coefficient of variation (indicating that the smoke / target modal of each pixel in the neighborhood is highly consistent), the closer the output is to 1; the larger the input coefficient of variation (indicating that the modal differences in the neighborhood are large, such as target edges), the closer the output is to 0. This transforms the original dispersion index into an intuitive consistency measure. A larger value indicates more consistent modal characteristics among pixels within a 3×3 neighborhood centered on that pixel. This is used to obtain the smoke scattering feature points. After setting the values, a sliding window is used for spatial verification: for each smoke scattering feature point, a 3×3 decision window is formed around it, and the values ​​within the window are statistically analyzed. The number of pixels that is greater than or equal to the second preset threshold (e.g., 0.6) is determined to be a potential smoke pixel if the proportion of this number to the total number of pixels in the window exceeds 70%.

[0039] In one embodiment, determining the candidate smoke patch as a target smoke patch based on the proportion of potential smoke pixels and the standard deviation of the area change rate in the candidate smoke patch includes: The potential smoke ratio is obtained by calculating the proportion of potential smoke pixels in the candidate smoke patch to the total number of pixels in the candidate smoke patch. When the proportion of potential smoke is greater than the third preset threshold and the standard deviation of the area change rate is greater than the fourth preset threshold, the candidate smoke patch is determined as the target smoke patch.

[0040] For each candidate smoke patch, first calculate the proportion of potential smoke pixels within it to its total number of pixels. Then, the standard deviation of the area change rate for each candidate smoke patch is obtained. When ρ ≥ the third preset threshold (e.g., 0.7) and If the value is ≥ the fourth preset threshold (e.g., 0.2), the candidate smoke patch is determined to be the target smoke patch, and it is removed entirely from the foreground. Otherwise, even if... Even if the area is large, the region is preserved to avoid partially obscuring the true target outline due to misjudgment.

[0041] S13. Based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images, the background frame of the penetrated image is updated to grayscale to obtain the target background frame.

[0042] In one embodiment, the background frame of the penetrated image is updated to grayscale based on the deviation between the area of ​​the target smoke patch in the penetrated image and the areas of the target smoke patches in its previous historical penetrated images, to obtain the target background frame, including: The update weight of the background frame is determined based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images; wherein, the update weight of the background frame is positively correlated with the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images. The target background frame is obtained by weighted fusion of the previous historical target background frame and the penetrated image based on the update weight of the background frame.

[0043] Specifically, the update weights of the background frames are determined based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the areas of the target smoke patches in its previous historical penetrated images, including: The deviation value is obtained by calculating the degree of deviation of the area of ​​the target smoke patch in the penetrated image relative to the average area of ​​the target smoke patch in multiple previous historical penetrated images; The update weight of the background frame is determined by the weighted sum of the preset base weight and the deviation value.

[0044] After removing the target smoke patches, the remaining foreground region in the current penetrated image is the suspected moving target. Simultaneously, the target smoke patches can be used to calculate the rate of change in smoke concentration. The sum of the pixel areas of multiple target smoke patches in the current penetrated image is calculated as follows: The sum of the pixel areas of the target smoke patches in the first four frames are respectively , , , ,calculate , , , , average The update weight W of the background frame is constructed, and the calculation formula is as follows: ;in, The base weight can be 0.3 to ensure the stability of the background and avoid sudden changes in the background due to noise in a single frame. For deviation values, weighted weights The weights are increased with the rate of change of smoke concentration to ensure that the background responds quickly in high dynamic scenes. This is a function that takes the minimum value.

[0045] Based on the update weight W of the background frame, the previous historical target background frame is updated. With current penetration image Perform weighted fusion to obtain the target background frame of the current penetrating image. : ,in, For the first After the filtered image frame passes through the gating window, during system cold start (i.e., system startup or scene reset), the average grayscale value of the initial 5 consecutive filtered images is calculated and used as the initial target background frame. Through an adaptive update mechanism, the matching degree between the background frame and the smoke area of ​​the current frame is higher, effectively reducing the interference of smoke on target recognition.

[0046] S14. Based on the target background frame, perform background deletion in the penetrating image to obtain candidate regions, and perform dynamic target recognition task based on the candidate regions.

[0047] After adaptive background updating, the grayscale difference map between the current penetrated image and the target background frame can highlight the changing areas of the real dynamic target. However, there may still be a small number of residual artifacts caused by echo aliasing in the image. These artifacts are usually located in distant regions (such as the distant mask region pre-labeled based on the initial static background image, such as behind the ceiling jet layer), and their location is strongly correlated with the overlap coefficient R.

[0048] In one embodiment, background removal is performed on the penetrated image based on the target background frame to obtain candidate regions, and a dynamic target recognition task is performed based on the candidate regions, including: The penetrating image and the target background frame are differentially processed to obtain a grayscale difference image; Binarize the grayscale difference image to obtain candidate regions; Based on the degree of overlap between the laser emission pulse and the echo signal obtained in the current gating window, determine whether there is echo aliasing; When echo aliasing exists, the artifact region is determined based on the time delay of the echo signal; The overlapping areas between the candidate regions and the artifact regions are removed to obtain the remaining candidate regions, and the dynamic target recognition task is performed based on the remaining candidate regions.

[0049] Calculate the current penetration image With target background frame Gray-scale difference image , ;in, Coordinates in the grayscale difference image grayscale value at that location and These represent the grayscale values ​​at corresponding coordinates of the current penetrated image and the target background frame. The grayscale difference image reflects the degree of change at each pixel; a larger difference indicates a more significant change.

[0050] Gray-scale difference image Binarization is performed, and a threshold is set (calibrated by the grayscale difference generated by the movement of a real target in an artifact-free scene, preset to 30) to obtain candidate regions. Combining this with the overlap coefficient R obtained in the previous steps, if R ≤ 0.3, it indicates significant aliasing, and the artifact region needs to be located based on the temporal delay of the echo signal. Since long-distance echoes mainly originate from fixed structures behind the ceiling jet layer, their image positions are usually located in the upper half of the image. Therefore, isolated regions with an area less than a certain number of pixels (preset 50 pixels) in the upper half of the penetrating image (preset to have a vertical coordinate less than 1 / 2 of the total number of rows) are identified as artifact regions. These artifact regions are removed from the candidate regions, resulting in the remaining candidate regions. Morphological closing operations (using 3×3 structuring elements) are performed on the remaining candidate regions to fill the tiny holes in the target outline, obtaining a complete dynamic target region. The bounding box and centroid coordinates of each target are output, achieving accurate identification of real dynamic targets.

[0051] The identified dynamic target bounding boxes and centroid coordinates are output in JSON format, including target ID, timestamp, and location coordinates. The target bounding boxes and tracking trajectories are overlaid on the original time-series imagery to generate a visual video. Trapped personnel are marked with red rectangles, and moving burning materials are marked with yellow rectangles. If a trapped person is detected, an audible and visual alarm is triggered simultaneously, providing intuitive and real-time decision support for fire rescue.

[0052] This application also provides a penetration imaging dynamic target recognition system that combines temporal information, such as... Figure 2 As shown, the system includes: The adjustment module 21 is used to adjust the delay time of the current gating window according to the degree of overlap between the laser emission pulse and the echo signal obtained from the previous gating window; the delay time is used to suppress visual artifacts caused by echo aliasing. Analysis module 22 is used to analyze the concentration changes of multiple candidate smoke patches in the current gating window and multiple previous historical penetration images to determine the target smoke patch in the penetration image; the concentration change of the pixels in the target smoke patch meets the preset concentration change conditions, and the multiple candidate smoke patches are obtained by image segmentation of the penetration image and multiple previous historical penetration images. Update module 23 is used to perform grayscale update on the background frame of the penetration image based on the degree of deviation between the area of ​​the target smoke patch in the penetration image and the area of ​​the target smoke patch in multiple previous historical penetration images, so as to obtain the target background frame. The deletion module 24 is used to perform background deletion in the penetrating image based on the target background frame to obtain candidate regions, and to perform dynamic target recognition tasks based on the candidate regions.

[0053] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0054] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for dynamic target recognition using penetration imaging that incorporates temporal information, characterized in that, The method includes: The delay time of the current gating window is adjusted based on the degree of overlap between the laser emission pulse and the echo signal obtained from the previous gating window; the delay time is used to suppress visual artifacts caused by echo aliasing. In the current gating window, the density changes of multiple candidate smoke patches in the image and the previous historical images are analyzed to determine the target smoke patch in the image. The density change of the pixels in the target smoke patch meets the preset density change conditions. The multiple candidate smoke patches are obtained by image segmentation of the image and the previous historical images. Based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in several previous historical penetrated images, the background frame of the penetrated image is updated in grayscale to obtain the target background frame. Background removal is performed on the penetrating image based on the target background frame to obtain candidate regions, and dynamic target recognition is performed based on the candidate regions.

2. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 1, characterized in that, The step of adjusting the delay time of the current gating window based on the degree of overlap between the laser emission pulse and the echo signal obtained from the previous gating window includes: Based on the cross-correlation between the laser emission pulse and the echo signal obtained from the previous gating window, and the geometric mean relationship between the total energy of the echo signal and the total energy of the laser emission pulse, the overlap coefficient between the echo signal and the laser emission pulse is calculated; the overlap coefficient characterizes the degree of temporal overlap between the echo signal and the laser emission pulse. Adjust the delay time of the current gating window based on the overlap coefficient.

3. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 2, characterized in that, The calculation of the overlap coefficient between the echo signal and the laser emission pulse, based on the cross-correlation between the laser emission pulse and the echo signal obtained from the previous gating window, and the geometric mean relationship between the total energy of the echo signal and the total energy of the laser emission pulse, includes: Calculate the peak value of the cross-correlation between the laser emission pulse and the echo signal; Calculate the sum of squares of the voltages at all sampling points in the echo signal to obtain the total echo energy; Calculate the sum of squares of the voltages at all sampling points in the laser emission pulse to obtain the total emitted energy; Dividing the cross-correlation peak value by the geometric mean of the total echo energy and the total emitted energy yields the overlap coefficient between the echo signal and the laser emission pulse.

4. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 1, characterized in that, The process of analyzing the concentration changes of multiple candidate smoke patches in the current gating window and its preceding historical penetration images to determine the target smoke patch in the penetration image includes: Dynamic thresholding is performed on the penetrated image and multiple previous historical penetrated images to obtain multiple initial smoke patches in the penetrated image and multiple initial smoke patches in each historical penetrated image. Multiple initial smoke patches in the penetrated image and multiple initial smoke patches in each historical penetrated image are differentially processed to obtain multiple differential patches between each two frames of images. The intersection of multiple differential patches between multiple frames of images is then processed to obtain multiple candidate smoke patches. Calculate the standard deviation of the rate of change of area for each candidate smoke patch in the penetrated image and multiple previous historical penetrated images; Extract the grayscale time series of each pixel in the candidate smoke patch in consecutive frames of images, decompose the grayscale time series corresponding to each pixel into multiple components of different frequencies, and mark the pixels that meet the preset conditions as potential smoke pixels according to the proportion of the energy of the high-frequency components in the total oscillation energy. The candidate smoke patches are used to determine the target smoke patch based on the proportion of potential smoke pixels and the standard deviation of the area change rate in the candidate smoke patch.

5. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 4, characterized in that, The calculation of the standard deviation of the area change rate of each candidate smoke patch in the penetrated image and multiple previous historical penetrated images includes: Obtain the area value of each candidate smoke patch in the penetrated image and its multiple preceding historical penetrated images; Calculate the rate of change of area between each pair of adjacent frames; The standard deviation of the area change rate of each candidate smoke patch in multiple frames of imagery is calculated to obtain the standard deviation of the area change rate of each candidate smoke patch; the standard deviation of the area change rate is used to characterize the degree of fluctuation of the area of ​​the candidate smoke patch.

6. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 4, characterized in that, The process involves extracting the grayscale time series of each pixel in the candidate smoke patch across multiple consecutive frames, decomposing the grayscale time series corresponding to each pixel into multiple components of different frequencies, and marking pixels that meet preset conditions as potential smoke pixels based on the proportion of the energy of the high-frequency components in the total oscillation energy. This includes: Extract the grayscale time series of each pixel in the candidate smoke patch in multiple consecutive frames of images, decompose the grayscale time series corresponding to each pixel into multiple components of different frequencies, and calculate the proportion of the energy of the high-frequency component in the total oscillation energy to obtain the energy proportion of the high-frequency component. When the energy proportion of high-frequency components is greater than the first preset threshold, the pixel is determined to be a smoke scattering feature point; For smoke scattering feature points, the mean and standard deviation of the high-frequency component energy ratio of all pixels in its neighborhood are obtained, and the neighborhood consistency value of the smoke scattering feature points is determined based on the mean and standard deviation. When the neighborhood consistency value of a smoke scattering feature point is greater than a second preset threshold, the smoke scattering feature point is determined to be a potential smoke pixel.

7. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 4, characterized in that, The step of determining the target smoke patch based on the proportion of potential smoke pixels and the standard deviation of the area change rate in the candidate smoke patch includes: The potential smoke ratio is obtained by calculating the proportion of potential smoke pixels in the candidate smoke patch to the total number of pixels in the candidate smoke patch. When the proportion of potential smoke is greater than the third preset threshold and the standard deviation of the area change rate is greater than the fourth preset threshold, the candidate smoke patch is determined as the target smoke patch.

8. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 1, characterized in that, The step of updating the grayscale of the background frame of the penetrated image based on the deviation between the area of ​​the target smoke patch in the penetrated image and the areas of the target smoke patches in its previous historical penetrated images to obtain the target background frame includes: The update weight of the background frame is determined based on the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images; wherein, the update weight of the background frame is positively correlated with the degree of deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images. The target background frame is obtained by weighted fusion of the previous historical target background frame and the penetrated image based on the update weight of the background frame.

9. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 8, characterized in that, The step of determining the update weight of the background frame based on the deviation between the area of ​​the target smoke patch in the penetrated image and the area of ​​the target smoke patch in its previous historical penetrated images includes: Calculate the deviation of the area of ​​the target smoke patch in the penetrated image from the average area of ​​the target smoke patch in multiple previous historical penetrated images, and obtain the deviation value; The update weight of the background frame is determined by the weighted sum of the preset base weight and the deviation value.

10. The method for dynamic target recognition using penetration imaging combined with temporal information as described in claim 1, characterized in that, The step of removing background from the penetrated image based on the target background frame to obtain candidate regions, and performing dynamic target recognition based on the candidate regions, includes: The penetrating image and the target background frame are differentially processed to obtain a grayscale difference image; Binarize the grayscale difference image to obtain candidate regions; Based on the degree of overlap between the laser emission pulse and the echo signal obtained in the current gating window, determine whether there is echo aliasing; When echo aliasing exists, the artifact region is determined based on the time delay of the echo signal; The overlapping areas between the candidate regions and the artifact regions are removed to obtain the remaining candidate regions, and the dynamic target recognition task is performed based on the remaining candidate regions.

Images