Mountainous transmission channel fire multi-stage smoke differentiation perception method and device

Abstract

The application provides a mountainous power transmission channel fire multi-stage smoke differentiated perception method and device, relates to the technical field of data processing, and comprises the following steps: constructing a smoke-free background according to a pre-ignition image in a multispectral image sequence collected along a mountainous power transmission channel, fusing a difference between a current frame and the smoke-free background into a smoke physical feature set; arranging smoke state quantities in the smoke physical feature set in time sequence, determining stage boundaries through time sequence segmentation optimization, obtaining smoke stage labels to which each frame belongs, and constructing a corresponding perception model; inputting an absolute difference image between the current frame and the smoke-free background into the perception model, and outputting a stage perception probability of the smoke stage label to which the current frame belongs; superimposing the stage perception probability on a power transmission channel geographic information layer, and taking the stage perception probability as a trigger input to trigger an early warning instruction. According to the application, stage judgment and differentiated early warning of mountainous power transmission channel fire smoke can be realized without relying on manual labeling.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and device for differential sensing of multi-stage smoke in fires in mountainous power transmission channels. Background Technology

[0002] Power transmission lines in mountainous areas often traverse complex terrain and dense vegetation, making them vulnerable to rapid spread of fire smoke that could threaten the safety of the lines. To obtain the data foundation for smoke detection and early warning, controlled combustion experiments are typically conducted along the transmission lines. This involves image acquisition nodes at fixed locations continuously capturing images of the smoke from ignition to extinguishing, creating a multispectral image sequence for subsequent detection model training.

[0003] In the aforementioned image sequence, the coverage area, spread rate, and contrast with the background of the smoke continuously change with the combustion process, and the differences in characteristics between different combustion stages are obvious. To establish perceptual judgment criteria for each combustion stage, it is first necessary to determine the combustion stage to which each frame in the image sequence belongs. However, controlled experimental sites typically only record the ignition and extinguishing times, and the start and end frame positions of each combustion stage are not manually marked. Furthermore, dividing the image sequence into fixed-duration proportional segments cannot reflect the differences in the actual duration of the combustion process at each stage. Consequently, the stage attribution of each frame cannot be determined from the image sequence itself, and the perceptual judgment criteria for each stage cannot be established.

[0004] Therefore, the problem that needs to be solved is how to automatically determine the combustion stage of each frame based solely on the multispectral image sequence acquired in a controlled experiment and establish the corresponding perception basis for each stage. Summary of the Invention

[0005] This application provides a method and device for multi-stage smoke differentiation perception of fires in mountainous power transmission channels, in order to solve the problem of how to automatically determine the combustion stage of each frame from the image sequence itself and establish corresponding perception basis for each stage under the condition that there is a lack of manual labeling of stage boundaries in controlled combustion experimental image sequences.

[0006] The first aspect of this application provides a method for differentiated smoke perception in multiple stages of fires in power transmission corridors in mountainous areas, including: A smoke-free background is constructed from the pre-ignition images in the multispectral image sequence collected along the mountain power transmission channel. The difference between the current frame in the multispectral image sequence and the smoke-free background is fused into a smoke physical feature set. After arranging the smoke state quantities in the smoke physical feature set in time sequence, the stage boundaries are determined by time sequence segmentation optimization to obtain the smoke stage label of each frame, and a corresponding perception model is constructed for each smoke stage label. The absolute difference map between the current frame and the smoke-free background is input into the perception model, and the stage perception probability of the smoke stage label to which the current frame belongs is output. The stage perception probability is superimposed onto the power transmission channel geographic information layer based on the smoke stage label, and the stage perception probability is used as the trigger input to trigger the early warning command.

[0007] Optionally, in one possible implementation of the first aspect, constructing a smoke-free background based on pre-ignition images from a multispectral image sequence acquired along a mountainous power transmission corridor includes: Within the experimental group to which the multispectral image sequence belongs, a reference frame is selected. Parametric registration based on grayscale correlation is performed on each frame in the pre-ignition image and the reference frame to obtain the first registration mapping of each frame. When the registration quality of the first registration mapping of each frame is determined to be lower than a preset registration quality threshold, local features are extracted from the pre-ignition image and robust matching is performed to obtain the second registration mapping of the frame using homography mapping. The frames of the pre-ignition image are aligned to the reference frame coordinate system using the first registration mapping or the second registration mapping. The median time value is taken for each pixel of the aligned pre-ignition image to obtain the smokeless background.

[0008] Optionally, in one possible implementation of the first aspect, fusing the difference between the current frame in the multispectral image sequence and the smoke-free background into a smoke physical feature set includes: Calculate the pixel-by-pixel difference between each spectral channel of the current frame and the corresponding spectral channel of the smokeless background to obtain the channel difference of each spectral channel; Based on the smoke stage label corresponding to the experimental group to which the historical multispectral image sequence belongs, the ratio of the inter-class dispersion to the intra-class dispersion of the channel difference of each spectral channel among the smoke stage labels is calculated to obtain the stage discrimination contribution value corresponding to each spectral channel. The channel difference between the two spectral channels with the largest contribution value in the stage is determined as the coordinate axis to construct a smoke difference coordinate system; The corresponding differences of the current frame on each coordinate axis of the smoke difference coordinate system are marked with coordinate points according to the pixel position in the smoke difference coordinate system to obtain the difference feature points corresponding to each pixel position; The set of difference feature points at each pixel location is determined as the smoke physical feature set.

[0009] Optionally, in one possible implementation of the first aspect, the step of marking the corresponding differences of the current frame on each coordinate axis of the smoke difference coordinate system according to the pixel position in the smoke difference coordinate system to obtain the difference feature points corresponding to each pixel position includes: The difference feature points of each frame are arranged into a temporal difference feature point sequence according to the acquisition time of each frame; Based on the connection direction between the difference feature points of the current frame and the difference feature points of the previous frame in the temporal difference feature point sequence and the preset recursive retention coefficient, the coordinates of the difference feature points of the current frame and the coordinates of the difference feature points of the previous frame are weighted and summed to obtain the corrected difference feature point coordinates corresponding to the current frame. The corrected coordinates of the difference feature points are used as the coordinates of the difference feature points in the current frame to obtain the difference feature points corresponding to each pixel position.

[0010] Optionally, in one possible implementation of the first aspect, the step of arranging the smoke state variables in the smoke physical feature set in a temporal sequence and then determining the stage boundaries through temporal segmentation optimization to obtain the smoke stage label of each frame, and constructing a corresponding perception model for each smoke stage label, includes: Based on the difference feature points of each frame in the smoke physical feature set, the smoke pseudomask corresponding to each frame is obtained through adaptive threshold filtering and connectivity growth; Using the coverage area of ​​the smoke mask, the average coordinates of the differential feature points within the smoke mask, and the change in the coverage area of ​​the smoke mask between adjacent frames as components, the components corresponding to each frame are arranged into a time-series stage description vector sequence according to the acquisition time of each frame. The time-series stage description vector sequence is standardized by components to obtain the standardized time-series stage description vector sequence. Dynamic programming is performed on the post-ignition sequence in the standardized time-series stage description vector sequence to obtain the stage boundary of the smoke combustion stage; After ignition, each frame is mapped to the corresponding smoke combustion stage interval divided by the stage boundary to obtain the smoke stage label of each frame. The difference feature points of the training frames belonging to each smoke stage label are sequentially connected in the smoke difference coordinate system to obtain the stage standard trajectory graphic corresponding to each smoke stage label, and the stage standard trajectory graphic is determined as the perception model corresponding to each smoke stage label.

[0011] Optionally, in one possible implementation of the first aspect, obtaining the smoke pseudomask corresponding to each frame based on the difference feature points of each frame in the smoke physical feature set through adaptive threshold filtering and connectivity growth includes: The coordinates of the difference feature points of each frame in the smoke difference coordinate system are arranged into a time-series coordinate sequence according to the acquisition time of each frame; Based on the angle between the cumulative direction of the temporal coordinate sequence corresponding to each pixel position in the smoke difference coordinate system and the preset smoke expansion direction, the difference feature points whose angle does not exceed the preset direction deviation threshold are determined as reliable seed feature points. For the coordinates of the differential feature points in each frame before ignition in the multispectral image sequence, a preset quantile statistic is obtained. After superimposing a preset safety margin on the preset quantile statistic, a truncation process is performed between the preset lower threshold and the preset upper threshold to obtain the adaptive high threshold of the experimental group to which the multispectral image sequence belongs. The differential feature points in each frame whose coordinate values ​​are not lower than the adaptive high threshold are identified as smoke candidate feature points; Starting from the reliable seed feature points, perform connected growth under the constraint of the smoke candidate feature points, and perform morphological cleaning on the connected growth results to obtain the smoke pseudomask corresponding to each frame.

[0012] Optionally, in one possible implementation of the first aspect, the step of sequentially connecting the difference feature points of the training frames belonging to each of the smoke stage labels in the smoke difference coordinate system to obtain the stage standard trajectory graphic corresponding to each of the smoke stage labels includes: According to the preset grid density, the difference feature points of the training frames belonging to each smoke stage label are uniformly sampled in the smoke difference coordinate system to obtain the uniformly sampled difference feature points corresponding to each smoke stage label. The uniform sampling difference feature points corresponding to each smoke stage label are sequentially connected in the smoke difference coordinate system according to the acquisition time of each training frame to obtain the stage standard trajectory graphic corresponding to each smoke stage label.

[0013] Optionally, in one possible implementation of the first aspect, performing dynamic programming on the post-ignition sequence in the standardized time-series stage description vector sequence to obtain the stage boundaries of the smoke combustion stage includes: With the objective of minimizing the sum of the squared deviations of the standardized stage description vectors within each segment of the standardized stage description vector sequence relative to the mean vector of the standardized stage description vectors of all frames within each segment, dynamic programming is performed on the post-ignition sequence under the constraint that the number of frames in each segment is not less than a preset minimum duration of frames to obtain the stage boundaries.

[0014] Optionally, in one possible implementation of the first aspect, mapping each frame after ignition to the corresponding smoke combustion stage interval divided by the stage boundary to obtain the smoke stage label to which each frame belongs includes: When it is determined that the number of frames in the post-ignition sequence is lower than the preset minimum number of solvable frames, the coverage component sequence corresponding to the post-ignition sequence in the timing stage description vector sequence is divided into a first half and a second half according to the timing. The change in coverage range component is obtained by subtracting the mean of the first half of the coverage range component from the mean of the second half of the coverage range component. The coverage range change ratio is obtained by taking the ratio of the change in the coverage range component to the mean of the first half of the coverage range component. In response to the coverage change ratio exceeding a preset growth ratio threshold, all frames in the post-ignition sequence are identified as growth phase labels, and the growth phase labels are used as smoke phase labels to which each frame belongs. In response to the coverage change ratio being lower than a preset attenuation ratio threshold, all frames in the post-ignition sequence are identified as attenuation stage labels, and the attenuation stage labels are used as smoke stage labels to which each frame belongs. In response to the coverage change ratio being between the preset attenuation ratio threshold and the preset growth ratio threshold, all frames in the post-ignition sequence are identified as stable phase labels, and the stable phase labels are used as smoke phase labels to which each frame belongs.

[0015] Optionally, in one possible implementation of the first aspect, the step of outputting the stage-aware probability of the smoke stage label to which the current frame belongs includes: The current frame, the smoke-free background, and the absolute difference map are stitched together according to the channel dimension to obtain the fused input image; In the smoke difference coordinate system, the difference feature points corresponding to the fused input image are connected sequentially according to the acquisition time of each frame to obtain the difference trajectory graphic corresponding to the current frame; The centroid of the coordinates of the difference feature points of the standard trajectory graphics corresponding to each smoke stage label is used as the translation reference point. The centroid of the coordinates of the difference feature points of the difference trajectory graphic is translated to the translation reference point to obtain the translated difference trajectory graphic; The trajectory overlap degree corresponding to each smoke stage label is obtained by the ratio of the area of ​​the overlapping region between the translated difference trajectory graphic and the stage standard trajectory graphic to the smaller of the area of ​​the translated difference trajectory graphic and the area of ​​the stage standard trajectory graphic. Normalization is performed on the trajectory overlap degree corresponding to each smoke stage label to obtain the stage perception probability.

[0016] Optionally, in one possible implementation of the first aspect, the step of sequentially connecting the difference feature points corresponding to the fused input image in the smoke difference coordinate system according to the acquisition time of each frame to obtain the difference trajectory graphic corresponding to the current frame includes: Using the smoke pseudomask corresponding to each frame obtained through adaptive threshold filtering and connected growth as the boundary, the differential feature points corresponding to the fused input image are divided into differential feature points inside the smoke area and differential feature points outside the smoke area. The differential feature points within the smoke area of ​​each frame are sequentially connected in the smoke difference coordinate system according to the acquisition time of each frame to obtain the main trajectory. Sort the differential feature points outside the smoke area in each frame according to their coordinate values, and extract the differential feature points outside the smoke area whose coordinate values ​​are not lower than the quantile corresponding to the preset outer area extraction ratio to obtain the extracted differential feature points outside the smoke area. Connect the extracted feature points outside the smoke area sequentially in the smoke difference coordinate system to obtain the trajectory auxiliary line; The main trajectory and the auxiliary trajectory lines are merged to obtain the difference trajectory graphic.

[0017] Optionally, in one possible implementation of the first aspect, triggering the warning instruction using the stage perception probability as a trigger input includes: Set a corresponding preset stage risk value for each of the smoke stage labels; The smoke risk value is obtained by summing the products of the stage perception probability corresponding to each smoke stage label and the preset stage risk value. When the smoke risk value is determined to exceed a preset risk threshold, the warning command is triggered.

[0018] A second aspect of this application provides a multi-stage smoke differentiation sensing device for fires in mountainous power transmission corridors, comprising: The background modeling module is used to construct a smoke-free background based on the pre-ignition images in the multispectral image sequence collected along the mountain power transmission channel, and to fuse the difference between the current frame in the multispectral image sequence and the smoke-free background into a smoke physical feature set. The phase division module is used to arrange the smoke state quantities in the smoke physical feature set in time sequence and then determine the phase boundary through time sequence segmentation optimization to obtain the smoke phase label of each frame, and build a corresponding perception model for each smoke phase label. The probabilistic reasoning module is used to input the absolute difference map between the current frame and the smoke-free background into the perception model and output the stage perception probability of the smoke stage label to which the current frame belongs. The early warning linkage module is used to overlay the stage perception probability onto the power transmission channel geographic information layer based on the smoke stage label, and to trigger an early warning command using the stage perception probability as the trigger input.

[0019] The beneficial effects of the multi-stage smoke differentiation sensing method and device for fires in mountainous power transmission channels provided in this application are as follows: 1. This application constructs a smoke difference coordinate system based on the channel difference between the current frame and the smoke-free background in a multispectral image sequence. The difference response of each pixel position is expressed in coordinate points. Then, a temporal stage description vector sequence is constructed using the pseudo-mask coverage area formed by the difference feature points of each frame and its inter-frame variation. Dynamic programming with a minimum duration frame constraint is used to determine the stage boundaries from within the image sequence, thus automatically generating stage labels for each frame. Since the stage boundaries are determined based on the inherent temporal distribution of smoke difference features rather than manual labeling or fixed division rules, the stage assignment of each frame can correspond to the actual evolution of the combustion process, making it possible to establish a basis for perceiving each stage under unlabeled experimental data conditions.

[0020] 2. After the stage labels for each frame are determined, this application connects the difference feature points of the training frames belonging to each stage sequentially in the smoke difference coordinate system to form a standard trajectory graphic for each stage, which serves as the perception model corresponding to each stage. During inference, the difference trajectory graphic corresponding to the current frame is aligned with the standard trajectory graphic of each stage, and the contour overlap is calculated and normalized to the stage perception probability. Then, the smoke risk value is obtained by weighting the probability with the risk value specific to each stage to trigger the warning. Since the perception model uses trajectory shape rather than frame-by-frame feature thresholds as the basis for comparison, the model of each stage can accommodate the natural fluctuations of the differences between frames within the same stage, so that the stage perception probability has a continuous and comparable quantitative form, and the warning triggering thus has a traceable stage attribution basis. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the multi-stage smoke differentiation sensing method for fires in mountainous power transmission channels provided in this application embodiment; Figure 2 This is a schematic diagram of reliable seed feature points and smoke candidate feature points for the multi-stage smoke differential perception method for fires in mountain power transmission channels provided in this application embodiment; Figure 3 This is a schematic diagram of the four-neighborhood connectivity growth of the multi-stage smoke differential perception method for fires in mountain power transmission channels provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of the multi-stage smoke differentiation sensing device for fires in mountain power transmission channels provided in this application embodiment; Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0024] See Figure 1 This is a flowchart of the multi-stage smoke differentiation sensing method for fires in mountainous power transmission channels provided in this application embodiment. Figure 1 The execution entity of the method shown can be a software and / or hardware device. The execution entity of this application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. User equipment can include, but is not limited to, computers, smartphones, personal digital assistants (PDAs), and the aforementioned electronic devices. Network equipment can include, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers. This embodiment does not impose any limitations on this.

[0025] The method for differentially sensing smoke in mountain power transmission channel fires provided in this application includes steps S1 to S4.

[0026] Step S1: Construct a smoke-free background based on the pre-ignition images in the multispectral image sequence collected along the power transmission channel in the mountainous area, and fuse the differences between the current frame and the smoke-free background in the multispectral image sequence into a smoke physical feature set.

[0027] It should be noted that in the controlled combustion experiment of the power transmission channel in the mountainous area, the fixed-point image acquisition node continuously captured images throughout the ignition process. The resulting multispectral image sequence includes frames before and after ignition. Since the image acquisition node is fixed outdoors, it is affected by factors such as wind vibration, resulting in slight spatial displacement between the frames before ignition. If these frames are used directly to establish the background, they will introduce positional deviations when comparing pixels one by one.

[0028] Understandably, the smoke physical feature set stores all the difference response information between the current frame and the smoke-free background in the form of a set of coordinate points, providing a data source for pseudo-mask generation, temporal stage description vector construction, and input to the perception model.

[0029] In this embodiment of the application, a smoke-free background is constructed based on pre-ignition images from a multispectral image sequence collected along a power transmission corridor in a mountainous area, including the following steps: S11: Select a reference frame within the experimental group to which the multispectral image sequence belongs, and perform parameterized registration based on grayscale correlation between each frame in the pre-ignition image and the reference frame to obtain the first registration mapping of each frame; S12: When the registration quality of the first registration mapping of each frame is determined to be lower than the preset registration quality threshold, local features are extracted from the pre-ignition image and robust matching is performed to obtain the second registration mapping of the frame using homography mapping. S13: Align each frame of the pre-ignition image to the reference frame coordinate system using the first registration mapping or the second registration mapping, and take the median time value for each pixel of the aligned pre-ignition image to obtain a smokeless background.

[0030] It should be noted that the reference frame is the baseline frame used to establish a unified alignment coordinate system in the pre-ignition image sequence. In the experimental group, the normalized grayscale cross-correlation coefficient between each frame and all other frames in the pre-ignition image sequence was calculated. The frame with the largest sum of normalized grayscale cross-correlation coefficients with all other frames in the pre-ignition image sequence was determined as the reference frame. The normalized grayscale cross-correlation coefficient reflects the overall similarity of the grayscale distribution between two frames. Taking the frame with the largest sum of cross-correlation coefficients as the reference frame ensures that the reference frame is closest to most frames in the pre-ignition image sequence in grayscale distribution, thus reducing the deformation that needs to be compensated during registration. After the reference frame is determined, local feature points are extracted from the reference frame and stored for use in subsequent frames when affine registration is unreliable, allowing for robust local feature matching.

[0031] Specifically, for each frame in the image before ignition, using the grayscale matrix of the reference frame as a template, normalized cross-correlation calculation is performed on the grayscale matrix of the current registered frame within a preset search range. This calculates the affine transformation parameters that map the pixel coordinates of the current registered frame to the reference frame coordinate system, maximizing the normalized cross-correlation coefficient. These affine transformation parameters are recorded as the first registration mapping for the current registered frame, and the resulting normalized cross-correlation coefficient is recorded as the registration quality of the first registration mapping. The preset search range is determined based on the maximum observed inter-frame displacement in the historical experimental group at the same location. Typically, 1.5 times the maximum observed inter-frame displacement is used as the search radius to cover the inter-frame displacement range under normal wind vibration conditions. A higher registration quality value indicates more sufficient spatial displacement compensation between the current registered frame and the reference frame by the affine transformation.

[0032] The registration quality values ​​of the first registration mapping in each frame are read and compared with a preset registration quality threshold. The preset registration quality threshold is determined based on offline calibration using an experimental group at the same location. The calibration criterion is that the pixel displacement in the static background region of the superimposed image after registration should not exceed one pixel. The lower limit of the corresponding normalized cross-correlation coefficient is then used as the preset registration quality threshold. For stable nodes installed on concrete foundations, the measured normalized cross-correlation coefficient is usually not less than 0.85, and the preset registration quality threshold can be set to 0.85. For nodes installed on tower crossarms under wind-induced vibration conditions with large inter-frame displacement, the preset registration quality threshold can be set to 0.75 after calibration, with the constraint that the pixel displacement after registration should not exceed one pixel. When the registration quality value of a frame is lower than the preset registration quality threshold, it indicates that the affine transformation has failed to find a sufficiently reliable alignment result within the preset search range. At this point, local feature points are extracted from the current registered frame and robustly matched with the stored local feature points of the reference frame. After eliminating mismatched point pairs, the homography matrix mapping the pixel coordinates of the current registered frame to the coordinate system of the reference frame is solved using the remaining valid matching point pairs. This homography matrix is ​​recorded as the second registration mapping of the current registered frame. Compared to affine transformation, homography mapping can compensate for perspective distortion and achieve accurate alignment even when affine registration is unreliable.

[0033] It should be noted that when a brief period of strong light obstruction or lens contamination occurs during the experimental group's acquisition period, causing an overall grayscale anomaly in a frame, the normalized cross-correlation coefficient between the currently registered frame and the reference frame may approach zero. Local feature point extraction may also fail to obtain a sufficient number of effective matching point pairs due to grayscale distortion, resulting in invalid first and second registration mappings. For frames that cannot be registered, the currently registered frame is removed from the pre-ignition image sequence and excluded from the calculation of the median time value for each pixel, ensuring that the establishment of a smoke-free background is not affected by abnormal frames.

[0034] For each frame in the pre-ignition image, based on the comparison between the registration quality value and the preset registration quality threshold, either the first registration map or the second registration map is selected to perform coordinate transformation on the current registered frame, aligning all frames after the coordinate transformation to the reference frame coordinate system. After alignment, the pixel values ​​of all aligned frames at each pixel position are arranged along the time dimension, and the pixel value in the middle position of the arrangement is taken as the pixel value of the smokeless background at the current pixel position. This process is repeated for all pixel positions to obtain the smokeless background. Taking the pixel value in the middle position instead of the arithmetic mean of the pixel values ​​of each frame is to ensure that in the pre-ignition image sequence, if a few frames are obscured by vegetation due to occasional vegetation movement, resulting in pixel values ​​biased towards vegetation color, the consistent pixel value of most frames is used as the background estimate, avoiding the situation where a few moving frames cause the background estimate at the current pixel position to be biased towards vegetation color.

[0035] For example, a certain experimental group has 20 frames of images before ignition. After calculating the sum of normalized grayscale cross-correlation coefficients across these 20 frames, the 8th frame is determined as the reference frame, and local feature points are extracted and stored. Affine registration is performed on the 3rd frame, and the resulting normalized cross-correlation coefficient is 0.91, which is higher than the preset registration quality threshold of 0.85. The affine transformation parameters are recorded as the first registration mapping for the 3rd frame. Affine registration is performed on the 17th frame, and the resulting normalized cross-correlation coefficient is 0.72, which is lower than the preset registration quality threshold of 0.85. Local feature points are extracted from the 17th frame and robustly matched with the stored local feature points from the 8th frame. After solving the homography matrix, the homography matrix is ​​recorded as the second registration mapping for the 17th frame. After the 20 frames are aligned, the pixel value at the center of the 20 aligned frames is taken for each pixel position to obtain a smoke-free background.

[0036] In this embodiment of the application, the difference between the current frame and the smoke-free background in the multispectral image sequence is fused into a smoke physical feature set, including the following steps: S141: Calculate the pixel-by-pixel difference between each spectral channel of the current frame and the corresponding spectral channel of the smokeless background to obtain the channel difference of each spectral channel; S142: Based on the smoke stage label corresponding to the experimental group to which the historical multispectral image sequence belongs, calculate the ratio of the inter-class dispersion to the intra-class dispersion of the channel difference of each spectral channel among the smoke stage labels, and obtain the stage discrimination contribution value corresponding to each spectral channel. S143: Determine the channel difference between the two spectral channels with the largest contribution value in each stage as the coordinate axis to construct a smoke difference coordinate system; S144: Mark the corresponding differences of the current frame on each coordinate axis of the smoke difference coordinate system according to the pixel position in the smoke difference coordinate system to obtain the difference feature points corresponding to each pixel position; S145: The set of difference feature points at each pixel location is determined as the smoke physical feature set.

[0037] It should be noted that the multispectral image sequence contains several spectral channels, and each spectral channel has different response characteristics to smoke. For each spectral channel of the current frame, the pixel value matrix of the current frame in the current spectral channel is subtracted element-wise from the pixel value matrix of the smoke-free background in the current spectral channel to obtain the channel difference matrix of the current spectral channel. Each spectral channel is calculated independently, and the original difference in the response of each spectral channel to smoke is retained without cross-channel merging.

[0038] The coordinate axis selection calculation was performed offline based on the historical experimental group with completed stage divisions, using the smoke stage labels of each frame in the historical experimental group as the grouping basis. For the channel difference of each spectral channel, after grouping according to the smoke stage label of each frame, the inter-class dispersion of the current spectral channel was obtained by dividing the sum of squared deviations of the mean channel difference of each smoke stage label group relative to the mean channel difference of all historical samples by the total number of smoke stage labels; the intra-class dispersion of the current spectral channel was obtained by dividing the sum of squared deviations of the channel difference of all historical sample frames relative to the mean of the smoke stage label group of each frame by the total number of historical sample frames. Both are dispersion in the sense of mean and have consistent dimensions. The ratio of inter-class dispersion to intra-class dispersion is used as the stage differentiation contribution value of the current spectral channel. The larger the ratio, the more dispersed the channel difference of the current spectral channel is among different combustion stages and the more concentrated its distribution is within the same combustion stage, indicating a stronger ability to distinguish combustion stages. After calculating the stage-specific contribution values ​​for all spectral channels, the contribution values ​​are sorted from largest to smallest. The difference of the spectral channel ranked first is taken as the horizontal axis of the smoke difference coordinate system, and the difference of the spectral channel ranked second is taken as the vertical axis of the smoke difference coordinate system, thus constructing a two-dimensional smoke difference coordinate system.

[0039] The coordinate axis selection is an offline calculation result. Under the condition that the model of the acquisition equipment and the experimental conditions at the same deployment point do not change significantly, the coordinate axis selection result can be reused across experimental groups without recalculating for each new experimental group. If the deployment point changes or the multispectral camera model is replaced, it is necessary to re-acquire the experimental group under the new conditions and complete the stage division. The coordinate axis selection calculation should then be re-executed using historical samples from the new experimental group to ensure the effectiveness of the smoke difference coordinate system in distinguishing each combustion stage under the current acquisition conditions.

[0040] For each pixel position in the current frame, the channel difference in the horizontal spectral channel is read as the x-coordinate and the channel difference in the vertical spectral channel is read as the y-coordinate. A coordinate point is marked in the smoke difference coordinate system to obtain the difference feature point corresponding to the current pixel position. After traversing all pixel positions in the current frame, the difference feature points corresponding to all pixel positions are collected into a set, and this set is determined as the smoke physical feature set.

[0041] For example, a multispectral image sequence from a certain experimental group contains four spectral channels: visible blue, visible red, near-infrared, and short-wave infrared. After calculating the stage-differentiation contribution value of each spectral channel based on historical experimental group samples, the contribution values ​​are: near-infrared 3.21, short-wave infrared 2.87, visible red 1.43, and visible blue 0.96. Using the near-infrared channel as the horizontal axis and the short-wave infrared channel as the vertical axis, a smoke difference coordinate system is constructed. If the channel difference value of a pixel position in the current frame is 18 in the near-infrared channel and 11 in the short-wave infrared channel, then the coordinate point (18, 11) in the smoke difference coordinate system is marked as the difference feature point corresponding to the current pixel position.

[0042] In this embodiment of the application, the corresponding differences of the current frame on each coordinate axis of the smoke difference coordinate system are marked with coordinate points according to pixel positions in the smoke difference coordinate system to obtain the difference feature points corresponding to each pixel position, including the following steps: S1441: Arrange the difference feature points of each frame according to the acquisition time of each frame to form a temporal difference feature point sequence; S1442: Based on the connection direction between the difference feature points of the current frame and the difference feature points of the previous frame in the temporal difference feature point sequence and the preset recursive retention coefficient, the weighted sum of the difference feature point coordinates of the current frame and the difference feature point coordinates of the previous frame is obtained to obtain the corrected difference feature point coordinates corresponding to the current frame. S1443: Use the corrected coordinates of the difference feature points as the coordinates of the difference feature points in the current frame to obtain the difference feature points corresponding to each pixel position.

[0043] It should be noted that in mountainous power transmission corridor environments, vegetation swaying and sudden changes in light intensity can introduce random jitter in the coordinates of differential feature points in a single frame, causing jumps in the coordinates of differential feature points between adjacent frames that are unrelated to the combustion process. Specifically, for each frame in the multispectral image sequence that has completed initial coordinate point marking, the set of differential feature points for each frame is arranged sequentially from earliest to latest according to the acquisition time of each frame, resulting in a temporal differential feature point sequence. This preserves the temporal evolution order of the differential feature points during the progression of the combustion process, providing an index basis for frame-by-frame iterative correction.

[0044] For each pixel position in the temporal difference feature point sequence of the current frame, the initial difference feature point coordinates of the current pixel position in the current frame and the corrected difference feature point coordinates of the current pixel position in the previous frame are read. The direction from the corrected difference feature point coordinates of the previous frame to the initial difference feature point coordinates of the current frame is the recursive direction of the difference feature point coordinates of the current frame. A preset recursive retention coefficient is used as the proportion of the corrected difference feature point coordinates of the previous frame in the weighted summation, and 1 minus the preset recursive retention coefficient is used as the proportion of the initial difference feature point coordinates of the current frame. The two are then weighted and summed to obtain the corrected difference feature point coordinates corresponding to the current pixel position in the current frame. The preset recursive retention coefficient is calibrated based on the vegetation density and wind speed conditions in the mountainous area where the experimental group is located. The preset recursive retention coefficient can be 0.3 to 0.4 for points with more violent vegetation swaying, and 0.1 to 0.2 for points with less violent swaying. The larger the preset recursive retention coefficient, the higher the proportion of the corrected coordinates of the previous frame in the weighted summation, and the more obvious the smoothing effect on random jitter, but the greater the response delay to rapid changes in the combustion process.

[0045] It should be noted that for the first frame of the temporal difference feature point sequence, since there is no previous frame, no recursive correction is performed. The initial coordinate point marking result of the first frame is directly used as the difference feature point coordinate of the first frame, and the difference feature point coordinate of the first frame is used as the input of the corrected coordinate of the previous frame in the recursive correction of the second frame.

[0046] The initial coordinate point markings are replaced with the corrected difference feature point coordinates to update the difference feature point coordinate records for each pixel position in the current frame, thus completing the final determination of the difference feature points in the current frame. Recursive correction is performed frame-by-frame in the temporal difference feature point sequence, with the difference feature point coordinates of the previous frame (which has already been corrected) used as the weighted summation input during each frame correction, ensuring the correction results are continuously propagated throughout the temporal sequence.

[0047] For example, in a certain experimental group, the initial difference feature point coordinates of a pixel position in the 5th frame are (20, 13), and the corrected difference feature point coordinates in the 4th frame are (17, 11). The preset recursion retention coefficient is 0.3. Using 0.3 as the proportion of the corrected coordinates in the 4th frame and 0.7 as the proportion of the initial coordinates in the 5th frame, a weighted sum is performed to obtain the corrected horizontal axis coordinate value of 19.1 and the corrected vertical axis coordinate value of 12.4. The corrected difference feature point coordinates of the current pixel position in the 5th frame are (19.1, 12.4).

[0048] Preferably, step S1 can ensure that the established smokeless background stably reflects the background state before ignition at each pixel position without manual intervention, eliminating the influence of inter-frame displacement on the positional deviation of pixel-by-pixel difference calculation; at the same time, it enables the coordinate space where the subsequent difference feature points are located to have the strongest distinguishing ability for each combustion stage, rather than losing the stage distinguishing information after averaging and mixing all spectral channel information; the temporal recursive correction makes the distribution of difference feature points in each frame continuous and stable in the coordinate space, without causing coordinate jumps unrelated to the combustion process due to single-frame vegetation swaying or sudden changes in illumination, so that the pseudo-mask generation, temporal segmentation and trajectory graphic construction based on the coordinate space distribution law in step S2 can all be carried out on feature data with high signal-to-noise ratio, effectively reducing the transmission influence of complex background interference in mountainous areas on subsequent steps.

[0049] Step S2: Arrange the smoke state variables in the smoke physical feature set in time sequence, determine the stage boundary through time sequence segmentation optimization, obtain the smoke stage label of each frame, and build a corresponding perception model for each smoke stage label.

[0050] It should be noted that in the controlled combustion experiment image sequence, the image acquisition nodes continuously captured images throughout the ignition process. The smoke physical feature sets of all the obtained frames each recorded the difference response between the current frame and the smokeless background. However, the combustion stage to which each frame belongs was not manually labeled; only the ignition time and extinguishing time were recorded. In order to determine the combustion stage of each frame from the image data itself, step S2 extracts the corrected difference feature point coordinates corresponding to the current pixel position in the smoke physical feature sets of each frame across all frames for each pixel position. These coordinates are arranged from earliest to latest according to the acquisition time of each frame to form a temporal coordinate sequence of the current pixel position, which is used for pseudo-mask generation.

[0051] Understandably, after step S2 is completed, each frame has a clear smoke stage label, and each smoke stage label has a closed stage standard trajectory graphic as a perception model, providing a data source for the stage perception probability calculation in step S3.

[0052] In this embodiment of the application, based on the difference feature points of each frame in the smoke physical feature set, the smoke pseudomask corresponding to each frame is obtained through adaptive threshold filtering and connectivity growth, including the following steps: S211: Arrange the coordinates of the difference feature points of each frame in the smoke difference coordinate system according to the acquisition time of each frame to form a time-series coordinate sequence; S212: Based on the angle between the cumulative direction of the temporal coordinate sequence corresponding to each pixel position in the smoke difference coordinate system and the preset smoke expansion direction, the difference feature points whose angle does not exceed the preset direction deviation threshold are determined as reliable seed feature points. S213: Take a preset quantile statistic for the coordinates of the difference feature points in each frame before ignition in the multispectral image sequence, and perform truncation processing between the preset lower threshold and the preset upper threshold after superimposing a preset safety margin on the preset quantile statistic to obtain the adaptive high threshold of the experimental group to which the multispectral image sequence belongs. S214: Determine the differential feature points in each frame whose coordinate values ​​are not lower than the adaptive high threshold as smoke candidate feature points; S215: Starting from reliable seed feature points, perform connected growth under the constraint of smoke candidate feature points, perform morphological cleaning on the connected growth results, and obtain the smoke pseudomask corresponding to each frame.

[0053] It should be noted that in step S211, the smoke pseudomask is a binary label matrix with the same resolution as the image. Each element in the matrix corresponds to a pixel position in the image. An element value of 1 indicates that the current pixel position is identified as a smoke pixel, and an element value of 0 indicates that the current pixel position is identified as a background pixel. For each pixel position in the smoke physical feature set of all frames, the corrected difference feature point coordinates corresponding to the current pixel position in each frame are read and arranged in order from earliest to latest acquisition time of each frame. Each acquisition time corresponds to a two-dimensional coordinate point, and the sequence length is equal to the total number of frames in the current experimental group, forming a temporal coordinate sequence of the current pixel position.

[0054] In step S212, for the temporal coordinate sequence of the current pixel position, the two-dimensional increment between adjacent acquisition time coordinate points is calculated one by one. The two-dimensional increments of all adjacent time points are accumulated in the horizontal and vertical directions respectively to obtain the net cumulative displacement vector of the temporal coordinate sequence of the current pixel position in the smoke difference coordinate system. The direction pointed to by the net cumulative displacement vector is the net cumulative direction of the temporal coordinate sequence of the current pixel position. During the combustion process of smoke in mountain fires, the corrected difference feature point coordinates of the smoke pixel position move monotonically towards the high response area of ​​smoke in the smoke difference coordinate system with time. The increment direction between adjacent time coordinate points is basically consistent, and the superimposed net cumulative direction is close to the preset smoke expansion direction. However, the temporal coordinate sequence of the pixel position caused by vegetation swaying oscillates back and forth, and the increment direction of adjacent time points changes alternately. The magnitude of the superimposed net cumulative displacement vector is close to zero, and the net cumulative direction is inconsistent with the preset smoke expansion direction.

[0055] The preset smoke propagation direction is determined offline based on historical experimental groups at the same deployment point. Specifically, the temporal coordinate sequence of known smoke pixel positions is extracted from the historical experimental group. For each known smoke pixel position, a net cumulative displacement vector is calculated. The arithmetic mean of the net cumulative displacement vectors of all known smoke pixel positions is taken along the horizontal and vertical axes to obtain the average net cumulative displacement vector. The direction pointed to by the average net cumulative displacement vector is used as the preset smoke propagation direction. The preset direction deviation threshold reflects the maximum allowable deviation between the net cumulative direction and the preset smoke propagation direction. It is calibrated based on the angle distribution between the net cumulative direction and the preset smoke propagation direction of all known smoke pixel positions in the historical experimental group. The 95th percentile of the angle distribution is taken as the preset direction deviation threshold to ensure that the false exclusion rate of known smoke pixels in the historical experimental group does not exceed 5%, typically between 15 and 25 degrees. For each pixel position in the current frame, the angle between the net cumulative direction of the current pixel position's temporal coordinate sequence and the preset smoke propagation direction is calculated. The difference feature points corresponding to pixel positions with angles not exceeding the preset direction deviation threshold are determined as reliable seed feature points.

[0056] In step S213, the corrected horizontal and vertical coordinates of all pixel positions in all frames before ignition in the current experimental group's multispectral image sequence are read. These coordinates are merged into a single set, and the 95th percentile is taken as the preset quantile statistic. This means that 95% of the coordinates in the set are lower than the current estimate, thus covering the upper limit of most background interference responses. The preset quantile statistic is added to a preset safety margin to obtain the superimposed candidate threshold. The preset safety margin is 10% to 15% of the preset quantile statistic, calibrated based on the fluctuation range of the coordinates of the pre-ignition differential feature points in the same historical experimental group, aiming to ensure that the proportion of all pre-ignition differential feature point coordinates in the historical experimental group that are lower than the candidate threshold is not less than 98%. The candidate threshold is compared with the preset lower threshold. If the candidate threshold is lower than the preset lower threshold, the preset lower threshold is used; if the candidate threshold is compared with the preset upper threshold, the preset upper threshold is used; otherwise, the candidate threshold is directly used to obtain the adaptive high threshold for the current experimental group. The lower limit of the preset threshold is taken as the 10th percentile of the adaptive high threshold of all experimental groups in the historical experimental groups, and the upper limit of the preset threshold is taken as the 90th percentile of the adaptive high threshold of all experimental groups in the historical experimental groups.

[0057] In step S214, for all the differential feature points in the current frame smoke physical feature set, the coordinate values ​​of the current differential feature points on the horizontal and vertical axes of the smoke differential coordinate system are read one by one. The larger of the horizontal axis coordinate value and the vertical axis coordinate value is compared with the adaptive high threshold. The differential feature points whose larger value is not lower than the adaptive high threshold are determined as smoke candidate feature points. The admission condition is that the difference response of the current pixel position in at least one coordinate axis direction in the smoke differential coordinate system exceeds the background interference upper limit.

[0058] like Figure 2 As shown, in step S215, the reliable seed feature point set of the current frame is used as the starting queue, and reliable seed feature points are taken out one by one from the queue to read the pixel position corresponding to the current difference feature point in the image. Figure 3 As shown, the algorithm checks whether the difference feature points at the four adjacent pixel positions (top, bottom, left, and right) of the current pixel location in the image belong to the smoke candidate feature points. Adjacent difference feature points that belong to the smoke candidate feature points and have not yet been visited are added to the expansion queue and marked as visited. This operation is repeated until the expansion queue is empty. After connected component growth is completed, for independent connected regions in the growth result whose area is lower than the preset minimum connected region area, the binary label matrix element values ​​of all pixel positions within the current connected region are set from 1 to 0 to remove them. For internal holes in the growth result whose area is lower than the preset maximum hole area, the binary label matrix element values ​​of all pixel positions within the hole are set from 0 to 1 to fill them. After morphological cleaning, the smoke pseudomask corresponding to each frame is obtained. The preset minimum connected region area is taken as the 10th percentile of the connected region area of ​​the initial smoke pseudomask in the historical experimental group; connected regions lower than the current value are considered noise and removed. The preset maximum hole area is taken as the 90th percentile of the internal hole area of ​​the smoke pseudomask in the historical experimental group; internal holes lower than the current value are filled.

[0059] It should be noted that when there is extensive vegetation occlusion in a frame of the image before ignition that lasts for more than a single frame, the net cumulative direction may temporarily deviate from the preset smoke expansion direction in the temporal coordinate sequence of the current frame. For differential feature points whose net cumulative direction angle exceeds the preset direction deviation threshold but whose coordinate values ​​are significantly higher than the adaptive high threshold, the current differential feature point is included in the smoke candidate feature points, but not as a reliable seed feature point. The expansion of adjacent reliable seed feature points determines whether the current differential feature point is included in the smoke pseudomask through a connectivity growth step, so as to avoid holes in the smoke pseudomask caused by extensive vegetation occlusion.

[0060] In this embodiment of the application, steps S22, S23 and S24 are as follows: In step S22, for the smoke pseudomask of the current frame, the total number of pixel positions with an element value of 1 in the binary label matrix is ​​read and recorded as the coverage range component of the current frame; the horizontal axis coordinate values ​​of the corrected difference feature points corresponding to all pixel positions with an element value of 1 in the binary label matrix are read, and the arithmetic mean is taken to obtain the horizontal axis coordinate mean; then the vertical axis coordinate values ​​are read, and the arithmetic mean is taken to obtain the vertical axis coordinate mean; the horizontal axis coordinate mean and the vertical axis coordinate mean are combined and recorded as the coordinate mean component of the current frame. The coverage range component of the current frame is subtracted from the coverage range component of the previous frame to obtain the coverage range change component of the current frame. Since the first frame of the time-series stage description vector sequence does not have a previous frame, the coverage range change component of the first frame is set to zero. The coverage range component, the horizontal axis coordinate mean, the vertical axis coordinate mean, and the coverage range change component of the current frame are concatenated into a four-dimensional vector in a fixed order to obtain the stage description vector of the current frame. The stage description vectors of all frames are arranged from earliest to latest according to the acquisition time to form the time-series stage description vector sequence.

[0061] In step S23, the temporal stage description vector sequence has four components, each with different dimensions. The coverage component is measured in pixels, the mean of the horizontal and vertical axes is measured in the units of the smoke difference coordinate system, and the coverage variation component is measured in the difference in the number of pixels between adjacent frames. For each component, the values ​​of the current component in all frames of the temporal stage description vector sequence are iterated, and the mean and standard deviation of the entire sequence are calculated. Then, all frames are iterated again, and the standardized value of the current component in the current frame is obtained by subtracting the mean of the entire sequence from the value of the current component in the current frame and then dividing the result by the standard deviation of the entire sequence. The above operations are performed on all four components in sequence, and the four standardized values ​​of each frame are reassembled into a four-dimensional vector to obtain the standardized temporal stage description vector sequence.

[0062] In step S24, frames whose acquisition time is later than the ignition time are extracted from the standardized time-series stage description vector sequence. The ignition time is determined by the timestamp of the ignition event recorded by the experimental group. The extracted frames are arranged from earliest to latest according to their acquisition time to obtain the post-ignition sequence. Dynamic programming is performed on the post-ignition sequence to determine three segment positions in the post-ignition sequence, dividing the post-ignition sequence into four continuous segments. The goal is to minimize the sum of the squared deviations between the standardized stage description vector of all frames in each segment and the mean vector of that segment, while ensuring that the number of frames in each segment is not less than the preset minimum number of continuous frames. Once the three segment positions are determined, they become the stage boundaries of the smoke combustion stage.

[0063] In this embodiment of the application, dynamic programming is performed on the post-ignition sequence in the standardized time-series stage description vector sequence to obtain the stage boundaries of the smoke combustion stage, including the following steps: S241: With the objective of minimizing the sum of the squared deviations of the standardized stage description vectors in each segment of the standardized stage description vector sequence relative to the mean vector of the standardized stage description vectors of all frames in each segment, dynamic programming is performed on the post-ignition sequence under the constraint that the number of frames in each segment is not less than the preset minimum duration of frames to obtain the stage boundaries.

[0064] It should be noted that in step S241, dynamic programming is solved using a frame-by-frame recursive approach. Starting from the first frame of the post-ignition sequence, for each frame, the minimum sum of squared deviations is calculated for dividing the current frame and all previous frames into 1, 2, and 3 segments, provided that the number of frames in each segment is not less than the preset minimum duration constraint. The minimum sum of squared deviations for dividing the current frame and all previous frames into k segments is equal to the minimum sum of squared deviations of the standardized stage description vectors of all frames within each segment relative to the mean vector within that segment, provided that the number of frames in each segment is not less than the preset minimum duration constraint. The frame position of the last segment cut point that minimizes the sum of squared deviations of the current k segments is recorded as the optimal segment position of the current frame under the k-segment division. After traversing to the last frame of the post-ignition sequence, a step-by-step backtracking is triggered from the optimal segment position of the last frame under the 3-segment division, determining the optimal combination of segment positions for each stage in turn. The three segment positions that minimize the sum of squared deviations of the four segments are taken as the stage boundaries. The preset minimum duration frame count is determined based on the experimental group's acquisition frame rate and the physical shortest duration of each combustion stage. It is obtained by rounding down the product of the acquisition frame rate and the physical shortest duration of each combustion stage. For example, if the acquisition frame rate is 2 frames per second and the estimated physical shortest duration of each combustion stage is 10 seconds, the preset minimum duration frame count is 20 frames. The preset minimum solvable frame count is the sum of 4 times the preset minimum duration frame count and 1. When the number of frames in the post-ignition sequence is lower than the preset minimum solvable frame count, it indicates that the post-ignition sequence is insufficient to support all 4 segments that meet the preset minimum duration frame count constraint. In this case, step S241 is exited and step S251 is initiated to perform degenerate branch processing.

[0065] In this embodiment of the application, in step S25, the stage boundaries divide the post-ignition sequence into four consecutive intervals, which correspond to the four combustion stages of smoke generation, growth, stabilization, and decay in chronological order. For each frame in the post-ignition sequence, the acquisition time of the current frame is read, and it is determined which of the four intervals the current acquisition time falls into. The smoke stage label of the current frame is set as the combustion stage label of the corresponding interval. After traversing all frames in the post-ignition sequence, the smoke stage label of each frame is obtained.

[0066] In this embodiment of the application, each frame after ignition is mapped to the corresponding smoke combustion stage interval divided by the stage boundary to obtain the smoke stage label of each frame, including the following steps: S251: When the number of frames in the post-ignition sequence is determined to be lower than the preset minimum solvable frame number, the coverage component sequence corresponding to the post-ignition sequence in the timing stage description vector sequence is divided into the first half and the second half according to the timing. S252: Subtract the arithmetic mean of the first half of the coverage range from the arithmetic mean of the second half of the coverage range to obtain the increase or decrease in the coverage range. S253: The ratio of the increase or decrease in coverage area component to the arithmetic mean of the coverage area component in the first half is used to obtain the coverage area change ratio; S254: In response to the coverage change ratio exceeding the preset growth ratio threshold, all frames in the post-ignition sequence are identified as growth phase labels, and the growth phase labels are used as the smoke phase labels of each frame. S255: In response to the coverage change ratio being lower than the preset attenuation ratio threshold, all frames in the post-ignition sequence are identified as attenuation stage labels, and the attenuation stage labels are used as smoke stage labels for each frame. S256: In response to the coverage change ratio being between a preset attenuation ratio threshold and a preset growth ratio threshold, all frames in the post-ignition sequence are identified as stable phase labels, and the stable phase labels are used as smoke phase labels for each frame.

[0067] It should be noted that in step S251, the coverage components of each frame corresponding to the ignition sequence are extracted from the standardized time-series description vector sequence and arranged in ascending order of acquisition time to form a coverage component sequence. Half of the total number of frames in the coverage component sequence is taken, and the coverage components of the first half of the frames are assigned to the first half, while the coverage components of the last half are assigned to the second half. If the total number of frames is odd, the coverage components of the middle frames are assigned to the second half.

[0068] In step S252, the arithmetic mean of the coverage range component values ​​of all frames in the first half is taken to obtain the arithmetic mean of the coverage range component in the first half; the arithmetic mean of the coverage range component values ​​of all frames in the second half is taken to obtain the arithmetic mean of the coverage range component in the second half; the increase or decrease in coverage range component value is obtained by subtracting the arithmetic mean of the coverage range component in the second half from the arithmetic mean of the coverage range component in the second half. When the increase or decrease in coverage range component value is positive, it indicates that the coverage range of the smoke pseudomask is larger than that of the first half in the second half of the sequence after ignition, and the smoke is in an expanding state; when the increase or decrease in coverage range component value is negative, it indicates that the coverage range of the second half is smaller than that of the first half, and the smoke is in a contracting state.

[0069] In step S253, the arithmetic mean of the first half of the coverage range components is read. If the arithmetic mean of the first half of the coverage range components is not lower than the preset minimum coverage range mean, the coverage range change ratio is obtained by the ratio of the increase / decrease of the coverage range components to the arithmetic mean of the first half of the coverage range components. If the arithmetic mean of the first half of the coverage range components is lower than the preset minimum coverage range mean, the coverage range change ratio is obtained by the ratio of the increase / decrease of the coverage range components to the preset minimum coverage range mean. The preset minimum coverage range mean is determined based on the minimum observable value of the smoke pseudomask coverage range in the initial stage of the historical experimental group at the same location, taking the 5th percentile of the coverage range components in each frame of the initial stage in the historical experimental group. By using the ratio of the increase / decrease of the coverage range components to the reference value, the absolute increase / decrease of the coverage range is normalized to the relative degree of change, eliminating the influence of differences in the absolute value of the coverage range caused by different combustion scales in different experimental groups, making the coverage range change ratio comparable between different experimental groups.

[0070] In step S254, the preset growth ratio threshold is determined based on the distribution of coverage change ratios during the growth phase in the historical experimental group. The 10th percentile of the coverage change ratio of short sequences after ignition that are clearly in the growth phase in the historical experimental group is taken to ensure that the correct recognition rate of the growth phase is not less than 90%, typically between 0.15 and 0.25. If the coverage change ratio exceeds the preset growth ratio threshold, it indicates that the smoke coverage area in the post-ignition sequence is in a state of rapid expansion, and the smoke phase labels of all frames in the post-ignition sequence are written as growth phase labels.

[0071] In step S255, the preset attenuation ratio threshold is determined based on the distribution of coverage change ratios during the attenuation phase in the historical experimental group. The 90th percentile of the coverage change ratio of short sequences after ignition that are clearly in the attenuation phase in the historical experimental group is taken. The preset attenuation ratio threshold is negative to ensure that the correct recognition rate of the attenuation phase is not less than 90%, typically ranging from -0.15 to -0.25. If the coverage change ratio is lower than the preset attenuation ratio threshold, it indicates that the smoke coverage area in the post-ignition sequence is in a state of rapid contraction, and the smoke phase labels of all frames in the post-ignition sequence are written as attenuation phase labels.

[0072] In step S256, if the coverage change ratio is between the preset attenuation ratio threshold and the preset growth ratio threshold, it indicates that there is no significant difference between the front and back halves of the smoke coverage in the post-ignition sequence. The smoke stage labels of all frames in the post-ignition sequence are then written as stable stage labels.

[0073] In this embodiment of the application, the difference feature points of the training frames belonging to each smoke stage label are sequentially connected in the smoke difference coordinate system to obtain the stage standard trajectory graphic corresponding to each smoke stage label. The stage standard trajectory graphic is determined as the perception model corresponding to each smoke stage label, including the following steps: S261: Perform uniform sampling on the difference feature points of the training frames belonging to each smoke stage label in the smoke difference coordinate system according to the preset grid density to obtain the uniformly sampled difference feature points corresponding to each smoke stage label. S262: Connect the uniformly sampled difference feature points corresponding to each smoke stage label in the smoke difference coordinate system according to the acquisition time of each training frame to obtain the stage standard trajectory graphic corresponding to each smoke stage label.

[0074] It should be noted that in step S261, the training frames come from the same batch of historical experimental groups used in the coordinate axis selection calculation in step S141. Each frame in the historical experimental group has already completed the assignment of smoke stage labels. For each smoke stage label, the corrected difference feature points of all training frames in the historical experimental group that belong to the current smoke stage label are read. The smoke difference coordinate system is then uniformly divided into square grids of equal side length along the horizontal and vertical axes. The grid side length is determined by a preset grid density. The preset grid density aims to ensure that the total number of uniformly sampled difference feature points retained by each smoke stage label after uniform sampling is no less than 500. This is achieved by projecting the corrected difference feature points of all training frames for the current smoke stage label onto the grid and counting the hit count of each grid cell. The grid side length is adjusted until the total number of grid cells with a non-zero hit count is no less than 500, and the corresponding grid side length is determined as the preset grid density. For each grid cell, from all the corrected difference feature points that hit the current grid cell, select the corrected difference feature point whose coordinate value is closest to the center of the current grid cell as the uniform sampling difference feature point of the current grid cell, and obtain the set of uniform sampling difference feature points corresponding to each smoke stage label.

[0075] In step S262, for each smoke stage label, the acquisition time of the training frame to which each uniformly sampled difference feature point belongs in the uniformly sampled difference feature point set is read, and the uniformly sampled difference feature point set is sorted from earliest to latest according to the acquisition time. Following the sorted order, the smoke difference coordinates of two adjacent uniformly sampled difference feature points are read one by one and connected by a line segment. After connecting all adjacent uniformly sampled difference feature points in sequence, the coordinates of the last uniformly sampled difference feature point and the first uniformly sampled difference feature point after sorting are read. If the two coordinates do not coincide, the last uniformly sampled difference feature point and the first uniformly sampled difference feature point are connected by a line segment to close the polyline, obtaining a closed stage standard trajectory graphic corresponding to the current smoke stage label. The closed stage standard trajectory graphic has a clear internal region boundary, making the calculation of the area of ​​the overlapping contour region in step S3 have an operable closed boundary.

[0076] For example, in a certain experimental group, the growth phase was assigned to 60 training frames. Uniform sampling was performed on the corrected difference feature points of these 60 training frames. After determining the grid side length, a total of 520 grid cells were hit. The corrected difference feature point closest to the grid center was selected as the uniformly sampled difference feature point for each grid cell, resulting in 520 uniformly sampled difference feature points. These points were then sorted from earliest to latest according to the acquisition time of each training frame. Adjacent uniformly sampled difference feature points were connected one by one with line segments. Finally, the 520th and 1st uniformly sampled difference feature points were connected with a line segment, forming a closed standard trajectory graphic corresponding to the growth phase. The stable phase was assigned to 80 training frames. Similarly, after uniform sampling, sorting, connection, and closure, a closed standard trajectory graphic corresponding to the stable phase was formed. The distribution areas and contour directions of the standard trajectory graphics corresponding to the growth and stable phases in the smoke difference coordinate system are significantly different, providing a distinguishable morphological basis for comparing the contour overlap in step S3.

[0077] Preferably, step S2 can automatically extract stage boundaries from the inherent variation patterns of the image sequence without any manual stage annotation, so that the combustion stage attribution of each frame corresponds to the physical evolution of the actual combustion process, rather than the stage attribution distortion caused by proportional hard segmentation; the reciprocating difference response caused by vegetation swaying and the monotonic difference response caused by smoke expansion can be distinguished from the net cumulative direction of temporal coordinates, so that the smoke pseudomask is not contaminated by vegetation interference areas, and the coverage range can accurately reflect the distribution range of real smoke pixels; uniform sampling ensures the representative balance of each region of the standard trajectory graphic in the coordinate space of each stage, avoiding the overemphasis or neglect of the smoke state in a certain period of time in the perception model due to uneven frame density, so that the perception model of each stage can stably cover the complete state change range of the current stage.

[0078] Step S3: Input the absolute difference map between the current frame and the smoke-free background into the perception model, and output the stage perception probability of the smoke stage label to which the current frame belongs.

[0079] It should be noted that, after completing the perception model construction for each smoke stage label in step S2, step S3 performs stage perception probability calculation for each frame of the current experimental group. It is understandable that for early frames, the temporal range covered by the cumulative trajectory up to the current frame is relatively short, the difference trajectory graphic is not yet complete, and its comparability with the stage standard trajectory graphic is relatively low. As the number of frames increases, the temporal range covered by the difference trajectory graphic gradually expands, and the reliability of the stage perception probability gradually improves. This is a reasonable physical setting for constructing the difference trajectory graphic up to the current frame.

[0080] In this embodiment of the application, the step of outputting the stage perception probability of the smoke stage label of the current frame includes the following steps: S31: Concatenate the current frame, the smoke-free background, and the absolute difference map along the channel dimension to obtain the fused input image; S32: Connect the difference feature points corresponding to the fused input image in the smoke difference coordinate system according to the acquisition time of each frame to obtain the difference trajectory graphic corresponding to the current frame; S33: Use the centroid of the coordinates of the difference feature points of the standard trajectory graphic corresponding to each smoke stage label as the translation reference point; S34: Translate the centroid of the coordinates of the difference feature points of the difference trajectory graphic to the translation reference point to obtain the translated difference trajectory graphic; S35: The trajectory overlap degree corresponding to each smoke stage label is obtained by the ratio of the area of ​​the overlapping region between the translated difference trajectory graphic and the stage standard trajectory graphic to the smaller of the area of ​​the translated difference trajectory graphic and the area of ​​the stage standard trajectory graphic. S36: Perform normalization processing on the trajectory overlap degree corresponding to each smoke stage label to obtain the stage perception probability.

[0081] It should be noted that in step S31, the absolute difference map is calculated as follows: For each spectral channel of the current frame, the pixel value matrix of the current frame in the current spectral channel is subtracted element-wise from the pixel value matrix of the smokeless background in the current spectral channel. The absolute value of each element in the subtraction result is taken to obtain the absolute difference map of the current spectral channel. The above operation is performed sequentially for all spectral channels to obtain a set of absolute difference maps with the same number of channels as the multispectral image sequence. This set of absolute difference maps is collectively referred to as the absolute difference map. The absolute difference map has the same image resolution and number of channels as the current frame and the smokeless background. The pixel values ​​of each spectral channel of the current frame, the pixel values ​​of each spectral channel of the smokeless background, and the absolute differences of each spectral channel of the absolute difference map are arranged sequentially according to the channel dimension. The above arrangement operation is performed on each pixel position in the image to obtain the fused input image. The total number of channels of the fused input image is three times the number of channels of the current frame, and it also includes the absolute brightness information of the current frame, the reference information of the smokeless background, and the difference intensity information between the two.

[0082] In step S32, for each frame of the current experimental group up to the current frame acquisition time, the smoke pseudomask calculated in step S215 is read frame by frame. For the pixel positions with an element value of 1 in the smoke pseudomask, the corrected difference feature point coordinates of each time point up to the current frame acquisition time in the temporal coordinate sequence constructed in step S211 are read and arranged from earliest to latest according to the acquisition time of each frame. For the arranged set of corrected difference feature point coordinates, uniform sampling is performed in the smoke difference coordinate system according to the preset grid density. For each grid cell, the corrected difference feature point closest to the center of the grid cell is selected and retained, and redundant points are removed to obtain the difference trajectory sampling points of the current frame. The preset grid density is the same as the preset grid density used when constructing the stage standard trajectory graphic in step S261 to ensure that the point density of the difference trajectory graphic and the stage standard trajectory graphic are comparable. According to the acquisition time of each frame from early to late, the difference trajectory sampling points are connected sequentially with line segments in the smoke difference coordinate system. Finally, the last difference trajectory sampling point is connected with the first and second difference trajectory sampling points with line segments to close the polyline, thus obtaining the closed difference trajectory graphic corresponding to the current frame.

[0083] It should be noted that when the total number of corrected difference feature points within the smoke pseudomask up to the current frame is less than 3, the difference trajectory graphic cannot form a valid closed polygon. For a total of 1 point, a square with a preset grid density is constructed centered on the corrected difference feature point coordinates at the current pixel position as the degenerate difference trajectory graphic, for subsequent contour overlap calculations. For a total of 2 points, rectangles are constructed by extending outwards by half a preset grid density from the coordinates of each of the two corrected difference feature points as the degenerate difference trajectory graphic.

[0084] In this embodiment of the application, the step of sequentially connecting the difference feature points corresponding to the fused input image in the smoke difference coordinate system according to the acquisition time of each frame to obtain the difference trajectory graphic corresponding to the current frame includes the following steps: S321: Using the smoke pseudomask corresponding to each frame obtained by adaptive thresholding and connected growth as the boundary, the differential feature points corresponding to the fused input image are divided into differential feature points inside the smoke region and differential feature points outside the smoke region. S322: Connect the difference feature points in the smoke area of ​​each frame in the smoke difference coordinate system according to the acquisition time of each frame to obtain the main body of the trajectory; S323: Sort the differential feature points outside the smoke area of ​​each frame according to their coordinate values, and extract the differential feature points outside the smoke area whose coordinate values ​​are not lower than the quantile corresponding to the preset outer area extraction ratio to obtain the extracted differential feature points outside the smoke area. S324: Connect the differential feature points outside the extracted smoke area sequentially in the smoke difference coordinate system to obtain the trajectory auxiliary line; S325: Merge the main trajectory line with the auxiliary trajectory line to obtain the difference trajectory graphic.

[0085] It should be noted that in step S321, for each frame up to the current frame acquisition time, the smoke pseudomask calculated in real time for each frame of the current experimental group in step S215 is read. The binary label matrix of the smoke pseudomask of the current frame is read frame by frame. For the corrected difference feature point corresponding to each pixel position in the fused input image, the element value of the current pixel position in the smoke pseudomask of the current frame is read. If the element value is 1, the current corrected difference feature point is classified as a difference feature point within the smoke area; if the element value is 0, it is classified as a difference feature point outside the smoke area. The above operation is performed sequentially for all frames up to the current frame to obtain the set of difference feature points within the smoke area and the set of difference feature points outside the smoke area for each pixel position in each frame.

[0086] In step S322, for all frames up to the current frame, the set of differential feature points in the smoke area of ​​the current frame is read frame by frame from early to late according to the acquisition time. The set of differential feature points in the smoke area of ​​each frame is connected sequentially by line segments in the smoke difference coordinate system according to the acquisition time of each frame from early to late. The differential feature points in the smoke area of ​​the last frame are connected to the differential feature points in the smoke area of ​​the first frame by line segments to close the loop and obtain the closed trajectory body.

[0087] In step S323, for all frames up to the current frame, all corrected differential feature points in the set of differential feature points outside the smoke area in each frame are summarized. For each corrected differential feature point, the arithmetic square root of the sum of the squares of its horizontal and vertical coordinate values ​​is calculated. The current value is used as the distance from the current corrected differential feature point to the origin of the smoke difference coordinate system. All differential feature points outside the smoke area are sorted from largest to smallest based on the current distance value. The preset outer area truncation ratio is the upper limit of the proportion of high-response points retained among the differential feature points outside the smoke area. It is determined based on the distribution of the proportion of high-response points in the background area to the total number of differential feature points outside the smoke area in the historical experimental group. It is taken as the 75th percentile of the proportion of high-response points in the background in the historical experimental group, usually between 10% and 20%. The corrected differential feature points whose distance value after sorting is not less than the ranking corresponding to the total number of differential feature points outside the smoke area multiplied by the preset outer area truncation ratio are read, and the read corrected differential feature points are determined as the truncated differential feature points outside the smoke area.

[0088] In step S324, the differential feature points outside the extracted smoke area are connected one by one with line segments in the smoke difference coordinate system according to the sorting order of distance values ​​from largest to smallest as in step S323, to obtain a non-closed trajectory auxiliary line. The trajectory auxiliary line starts from the differential feature point outside the extracted smoke area that is farthest from the origin of the smoke difference coordinate system and gradually extends towards the differential feature points outside the extracted smoke area that are closer to the origin, reflecting the distribution trend of high-response differential feature points in the background area in the smoke difference coordinate system.

[0089] In step S325, the set of closed polyline segments of the main trajectory body and the set of non-closed polyline segments of the trajectory auxiliary lines are combined into a single line segment set, and the graphic corresponding to the current line segment set is determined as the differential trajectory graphic.

[0090] For example, in an experimental group, the current frame is frame 30. The difference feature points within the smoke area of ​​each frame up to frame 30 are connected sequentially in frame order, forming a closed trajectory body. There are 8000 difference feature points outside the smoke area up to frame 30. The distances from each difference feature point to the origin of the smoke difference coordinate system are calculated and sorted from largest to smallest. With a preset outer area truncation ratio of 15%, the top 1200 corrected difference feature points are retained as the truncated smoke area difference feature points. These are then connected sequentially in descending order of distance value to obtain trajectory auxiliary lines. The trajectory body and the trajectory auxiliary lines are merged to obtain the difference trajectory graphic corresponding to frame 30.

[0091] Step S33: Use the centroid of the coordinates of the difference feature points of the standard trajectory graphics corresponding to each smoke stage label as the translation reference point.

[0092] It should be noted that in step S33, for each smoke stage label, the horizontal axis coordinate values ​​of all uniformly sampled difference feature points in the closed stage standard trajectory graph constructed in step S262 are read, and the arithmetic mean is taken to obtain the horizontal axis centroid value; then, the vertical axis coordinate values ​​of all uniformly sampled difference feature points are read, and the arithmetic mean is taken to obtain the vertical axis centroid value; the horizontal axis centroid value and the vertical axis centroid value are combined into a coordinate pair, which is used as the centroid of the difference feature point coordinates of the stage standard trajectory graph corresponding to the current smoke stage label, and the current coordinate pair is recorded as the translation reference point of the current smoke stage label. The above operation is performed sequentially for all smoke stage labels to obtain the translation reference point corresponding to each smoke stage label. The translation reference point remains unchanged between experimental groups, and is calculated and stored offline in the training phase of step S2 along with the stage standard trajectory graph, and is directly read in step S3 without recalculation.

[0093] In step S34, for the difference trajectory graphic corresponding to the current frame, the horizontal axis coordinate values ​​of all line segment nodes in the difference trajectory graphic are read, and the arithmetic mean is taken to obtain the horizontal axis centroid value of the difference trajectory graphic; the vertical axis coordinate values ​​of all line segment nodes are read, and the arithmetic mean is taken to obtain the vertical axis centroid value of the difference trajectory graphic; the horizontal axis centroid value and the vertical axis centroid value are combined into a coordinate pair, which is used as the centroid of the difference feature point coordinates of the difference trajectory graphic. For each smoke stage label, the horizontal axis translation offset is obtained by subtracting the horizontal axis centroid value of the difference trajectory graphic from the horizontal axis coordinate value of the current smoke stage label's translation reference point; the vertical axis translation offset is obtained by subtracting the vertical axis centroid value of the difference trajectory graphic from the vertical axis coordinate value of the current smoke stage label's translation reference point. The coordinates of all line segment nodes in the difference trajectory graphic are obtained by adding the horizontal axis translation offset to the horizontal axis coordinates and the vertical axis translation offset to the vertical axis coordinates. The reconstructed graphic using the coordinates of all line segment nodes after translation is then used as the translated difference trajectory graphic. The translation aligns the centroid of the translated difference trajectory graphic with the centroid of the standard trajectory graphic corresponding to the current smoke stage label, eliminating interference from the overall response intensity shift during the combustion process on the contour shape comparison.

[0094] In step S35, for each smoke stage label, the translated difference trajectory graphic and the stage standard trajectory graphic corresponding to the current smoke stage label are respectively regarded as polygons in the smoke difference coordinate system. An intersection operation is performed on the two polygons to obtain intersecting polygons, and the area of ​​the intersecting polygons is calculated as the area of ​​the contour overlap region. The area of ​​the polygon enclosed by the translated difference trajectory graphic and the area of ​​the polygon enclosed by the stage standard trajectory graphic corresponding to the current smoke stage label are calculated, and the smaller of the two is taken as the reference area. The trajectory overlap degree corresponding to the current smoke stage label is obtained by the ratio of the contour overlap region area to the reference area. Taking the smaller of the two as the reference area is to avoid the trajectory overlap degree being artificially inflated when the area of ​​the difference trajectory graphic is much larger than the area of ​​the stage standard trajectory graphic, resulting in the difference trajectory graphic completely covering the stage standard trajectory graphic. This ensures that the trajectory overlap degree truly reflects the similarity in shape between the two polygons.

[0095] It should be noted that when the sum of the trajectory overlap rates corresponding to all smoke stage labels is zero, it indicates that the difference trajectory graphic corresponding to the current frame has no overlapping area with the standard trajectory graphics of all stages. In the current case, setting the stage perception probability of all smoke stage labels to the reciprocal of the total number of smoke stage labels, i.e., the perception probability of each stage is equal, indicates that the current frame cannot be determined to belong to a stage through contour comparison. This usually occurs when the difference trajectory graphics of early frames after ignition are not yet complete.

[0096] In step S36, the overlap ratio of each trajectory to the sum of the overlap ratios of all smoke stage labels is used to obtain the stage perception probability corresponding to each smoke stage label. Normalization makes the sum of the stage perception probabilities of each smoke stage label equal to one. The higher the stage perception probability, the closer the contour shape of the difference trajectory graphic of the current frame is to the standard trajectory graphic of the current smoke stage label.

[0097] For example, in a certain experimental group, there are four smoke stage labels. The overlap of the trajectory for the initial stage is calculated to be 0.12, for the growth stage to be 0.63, for the stable stage to be 0.21, and for the decay stage to be 0.04. The sum of the overlap of the four trajectories is 1.00. The stage perception probabilities of each smoke stage label are 0.12, 0.63, 0.21, and 0.04, respectively. The growth stage has the highest stage perception probability, indicating that the difference trajectory pattern of the current frame is most similar to the standard trajectory pattern of the growth stage in terms of outline shape.

[0098] Preferably, step S3 eliminates the interference of overall response intensity offset on contour comparison by aligning the coordinate centroids, so that the trajectory overlap truly reflects the structural similarity between the current frame's smoke state evolution and the standard form of each stage, rather than the positional similarity affected by response offset; using the smaller of the two areas as the benchmark ensures that the trajectory overlap calculation result is not artificially inflated due to the large area of ​​the difference trajectory graphic, so that the relative magnitude relationship between the perception probabilities of each stage can stably reflect the similarity between the current frame and the actual form of each stage; after normalization, the sum of the perception probabilities of each stage is one, so that the perception result has a directly comparable probability form, which is convenient for weighted fusion with the risk level of each stage in step S4.

[0099] Step S4: Based on the smoke stage label, overlay the stage perception probability onto the power transmission channel geographic information layer, and use the stage perception probability as the trigger input to trigger the early warning command.

[0100] It should be noted that, in step S4, after outputting the stage perception probability of each frame in step S3, the stage perception probability is multiplied by the preset stage risk value corresponding to each smoke stage label and then summed to obtain the smoke risk value of the current frame. The smoke risk value is then compared with a preset risk threshold to determine whether to trigger an early warning command. The smoke risk value integrates the probability that the current frame belongs to each smoke stage label and the degree of hazard of each stage in a power transmission channel fire scenario. This ensures that the early warning trigger is jointly determined by the perception probability and risk level of all stages, avoiding missed or false alarms due to the uncertainty of judging a single stage when the stage perception probabilities are close.

[0101] It is understandable that the output of step S4 is an early warning instruction, which carries the acquisition time of the current frame and the geographic coordinates of the image acquisition node, for use in the early warning display in the geographic information layer of the power transmission channel.

[0102] In this embodiment of the application, triggering the early warning command using the stage perception probability as the trigger input includes the following steps: S41: Set corresponding preset risk values ​​for each smoke stage label; S42: The smoke risk value is obtained by summing the products of the stage perception probability corresponding to each smoke stage label and the preset stage risk value; S43: When the smoke risk value is determined to exceed the preset risk threshold, an early warning command is triggered.

[0103] It should be noted that in step S41, the preset stage risk value reflects the degree of hazard to the transmission line safety of each smoke stage label in a mountainous transmission channel fire scenario. This value is determined by the transmission channel maintenance personnel based on historical fire accident data and the spread rate of each combustion stage. The initial stage corresponds to a smaller fire intensity and slower spread rate, with a lower preset stage risk value, typically 0.2 to 0.4. The growth stage sees rapid fire expansion and accelerated spread, with a higher preset stage risk value, typically 0.7 to 0.9. The stable stage experiences a persistently strong fire, with a higher preset stage risk value, typically 0.8 to 1.0. The decay stage sees a gradually weakening fire, with a moderate preset stage risk value, typically 0.4 to 0.6. The sum of the preset stage risk values ​​for each stage is not required to be normalized; it only reflects the relative degree of hazard. The maintenance personnel adjust the numerical relationships of each stage based on the historical accident data of the current transmission channel.

[0104] In step S42, the stage perception probability corresponding to each smoke stage label output in step S36 is read. For each smoke stage label, the preset stage risk value corresponding to the current smoke stage label is read. The stage perception probability of the current smoke stage label is multiplied by the preset stage risk value to obtain the risk contribution value of the current smoke stage label. The risk contribution values ​​of all smoke stage labels are accumulated to obtain the smoke risk value of the current frame.

[0105] In step S43, if the smoke risk value of the current frame exceeds the preset risk threshold, an early warning command is triggered. The acquisition time of the current frame, the geographical coordinates of the image acquisition node, and the stage perception probability of each smoke stage label in the current frame are written into the early warning command and sent to the power transmission channel geographic information layer for overlay display. If the smoke risk value of the current frame does not exceed the preset risk threshold, the early warning command is not triggered. Steps S42 and S43 are executed again after waiting for the stage perception probability of the next frame to be input.

[0106] It should be noted that when the image acquisition node fails to transmit the stage perception probability of the current frame to the early warning processing terminal in a timely manner due to a communication failure, the calculation of the smoke risk value of the current frame in step S43 will not be completed. For communication interruptions exceeding the preset frame loss limit consecutively, the early warning processing terminal maintains the early warning status of the current frame using the smoke risk value calculated from the most recently successfully received stage perception probability, and simultaneously sends a communication failure alarm to the maintenance personnel, prompting them to manually check the communication status of the current image acquisition node. The preset frame loss limit is determined based on the product of the image acquisition node's acquisition frame rate and the maximum allowed communication interruption duration. For example, if the acquisition frame rate is 2 frames per second and the maximum allowed communication interruption duration is 30 seconds, the preset frame loss limit is 60 frames.

[0107] For example, in a certain experimental group, the perception probabilities of the four smoke stage labels in the current frame are 0.12, 0.63, 0.21, and 0.04, respectively, with corresponding preset stage risk values ​​of 0.3, 0.8, 0.9, and 0.5. The risk contribution values ​​for each stage are 0.036, 0.504, 0.189, and 0.020, respectively. The sum of the four risk contribution values ​​is 0.749, and the smoke risk value of the current frame is 0.749. The preset risk threshold is 0.45. The smoke risk value of the current frame, 0.749, exceeds the preset risk threshold of 0.45, triggering an early warning command. The acquisition time of the current frame, the geographical coordinates of the image acquisition node, and the stage perception probability are written into the early warning command and overlaid on the transmission channel geographic information layer.

[0108] Step S4 enables early warning decisions to avoid relying on rigid judgments of a single stage in the current frame. Instead, the smoke risk value is determined by the perception probability of each stage and the actual degree of hazard at each stage. Even when the perception probabilities of each stage are relatively dispersed, it can still output continuous and smooth risk quantification results. This avoids frequent jumps or missed reports near the stage boundaries when a single stage with the highest probability corresponds to a fixed early warning level, due to the close probability. The early warning command carries the perception probability information of each stage and is superimposed on the geographic information layer, so that maintenance personnel can know the current stage status distribution of the smoke when they receive the early warning, providing a basis for prioritizing on-site emergency response.

[0109] Based on the above steps, this application also includes the following embodiments: See Figure 4 This is a schematic diagram of the structure of the multi-stage smoke differentiation sensing device for fires in mountain power transmission channels provided in this application embodiment, including: The background modeling module is used to construct a smoke-free background based on the pre-ignition images in the multispectral image sequence collected along the mountain power transmission channel, and to fuse the difference between the current frame and the smoke-free background in the multispectral image sequence into a smoke physical feature set. The phase division module is used to arrange the smoke state variables in the smoke physical feature set in time sequence, determine the phase boundary through time sequence segmentation optimization, obtain the smoke phase label of each frame, and build a corresponding perception model for each smoke phase label. The probabilistic reasoning module is used to input the absolute difference map between the current frame and the smoke-free background into the perception model and output the stage perception probability of the smoke stage label to which the current frame belongs. The early warning linkage module is used to overlay the stage perception probability onto the geographic information layer of the power transmission channel based on the smoke stage label, and to trigger early warning commands using the stage perception probability as the trigger input.

[0110] Figure 4 The apparatus of the illustrated embodiment can be used to perform corresponding actions. Figure 1 The steps in the method embodiments shown are implemented in a similar manner and have similar technical effects, and will not be repeated here.

[0111] See Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 40 includes: a processor 41, a memory 42, and a computer program; wherein, The memory 42 is used to store computer programs, and the memory may also be flash memory. Computer programs may be, for example, application programs or functional modules that implement the methods described above.

[0112] The processor 41 is used to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0113] Alternatively, the memory 42 can be either standalone or integrated with the processor 41.

[0114] When the memory 42 is a device independent of the processor 41, the device may also include: Bus 43 is used to connect memory 42 and processor 41.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for differentiated smoke perception in multi-stage fires in mountainous power transmission corridors, characterized in that, include: A smoke-free background is constructed based on pre-ignition images from a multispectral image sequence collected along a mountainous power transmission corridor. The differences between the current frame and the smoke-free background in the multispectral image sequence are fused into a smoke physical feature set. This includes: calculating the pixel-by-pixel difference between each spectral channel of the current frame and the corresponding spectral channel of the smoke-free background to obtain the channel difference of each spectral channel; based on the smoke stage label corresponding to the experimental group to which the historical multispectral image sequence belongs, calculating the ratio of the inter-class dispersion to the intra-class dispersion of the channel difference of each spectral channel among the smoke stage labels to obtain the stage differentiation contribution value corresponding to each spectral channel; determining the channel difference of the two spectral channels with the largest stage differentiation contribution value as coordinate axes to construct a smoke difference coordinate system; marking the corresponding differences of the current frame on each coordinate axis of the smoke difference coordinate system according to the pixel position in the smoke difference coordinate system to obtain the difference feature points corresponding to each pixel position; and determining the set of the difference feature points at each pixel position as the smoke physical feature set. After arranging the smoke state variables in the smoke physical feature set in a temporal sequence, the stage boundaries are determined through temporal segmentation optimization to obtain the smoke stage label for each frame. A corresponding perception model is then constructed for each smoke stage label, including: based on the difference feature points of each frame in the smoke physical feature set, adaptive threshold filtering and connectivity growth are used to obtain the smoke pseudomask corresponding to each frame; using the coverage area of ​​the smoke pseudomask, the average coordinates of the difference feature points within the smoke pseudomask, and the change in the coverage area of ​​the smoke pseudomask between adjacent frames as components, the components corresponding to each frame are arranged into a temporal stage description vector sequence according to the acquisition time of each frame; the temporal stage... The description vector sequence is standardized by components to obtain the standardized temporal stage description vector sequence; dynamic programming is performed on the post-ignition sequence in the standardized temporal stage description vector sequence to obtain the stage boundary of the smoke combustion stage; each post-ignition frame is mapped to the corresponding smoke combustion stage interval divided by the stage boundary to obtain the smoke stage label to which each frame belongs; the difference feature points of the training frames belonging to each smoke stage label are sequentially connected in the smoke difference coordinate system to obtain the stage standard trajectory graphic corresponding to each smoke stage label, and the stage standard trajectory graphic is determined as the perception model corresponding to each smoke stage label. The absolute difference map between the current frame and the smoke-free background is input into the perception model, and the stage perception probability of the smoke stage label to which the current frame belongs is output. The stage perception probability is superimposed onto the power transmission channel geographic information layer based on the smoke stage label, and the stage perception probability is used as the trigger input to trigger the early warning command.

2. The method according to claim 1, characterized in that, The construction of a smoke-free background based on pre-ignition images from a multispectral image sequence collected along a power transmission corridor in the mountainous area includes: Within the experimental group to which the multispectral image sequence belongs, a reference frame is selected. Parametric registration based on grayscale correlation is performed on each frame in the pre-ignition image and the reference frame to obtain the first registration mapping of each frame. When the registration quality of the first registration mapping of each frame is determined to be lower than a preset registration quality threshold, local features are extracted from the pre-ignition image and robust matching is performed to obtain the second registration mapping of the frame using homography mapping. The frames of the pre-ignition image are aligned to the reference frame coordinate system using the first registration mapping or the second registration mapping. The median time value is taken for each pixel of the aligned pre-ignition image to obtain the smokeless background.

3. The method according to claim 1, characterized in that, The step of marking the corresponding differences of the current frame on each coordinate axis of the smoke difference coordinate system according to the pixel position in the smoke difference coordinate system to obtain the difference feature points corresponding to each pixel position includes: The difference feature points of each frame are arranged into a temporal difference feature point sequence according to the acquisition time of each frame; Based on the connection direction between the difference feature points of the current frame and the difference feature points of the previous frame in the temporal difference feature point sequence and the preset recursive retention coefficient, the coordinates of the difference feature points of the current frame and the coordinates of the difference feature points of the previous frame are weighted and summed to obtain the corrected difference feature point coordinates corresponding to the current frame. The corrected coordinates of the difference feature points are used as the coordinates of the difference feature points in the current frame to obtain the difference feature points corresponding to each pixel position.

4. The method according to claim 1, characterized in that, The process of obtaining the smoke pseudomask corresponding to each frame based on the differential feature points of each frame in the smoke physical feature set, through adaptive threshold filtering and connectivity growth, includes: The coordinates of the difference feature points of each frame in the smoke difference coordinate system are arranged into a time-series coordinate sequence according to the acquisition time of each frame; Based on the angle between the cumulative direction of the temporal coordinate sequence corresponding to each pixel position in the smoke difference coordinate system and the preset smoke expansion direction, the difference feature points whose angle does not exceed the preset direction deviation threshold are determined as reliable seed feature points. For the coordinates of the differential feature points in each frame before ignition in the multispectral image sequence, a preset quantile statistic is obtained. After superimposing a preset safety margin on the preset quantile statistic, a truncation process is performed between the preset lower threshold and the preset upper threshold to obtain the adaptive high threshold of the experimental group to which the multispectral image sequence belongs. The differential feature points in each frame whose coordinate values ​​are not lower than the adaptive high threshold are identified as smoke candidate feature points; Starting from the reliable seed feature points, perform connected growth under the constraint of the smoke candidate feature points, and perform morphological cleaning on the connected growth results to obtain the smoke pseudomask corresponding to each frame.

5. The method according to claim 1, characterized in that, The differential feature points of the training frames belonging to each of the smoke stage labels are sequentially connected in the smoke difference coordinate system to obtain the stage standard trajectory graphic corresponding to each smoke stage label, including: According to the preset grid density, the difference feature points of the training frames belonging to each smoke stage label are uniformly sampled in the smoke difference coordinate system to obtain the uniformly sampled difference feature points corresponding to each smoke stage label. The uniform sampling difference feature points corresponding to each smoke stage label are sequentially connected in the smoke difference coordinate system according to the acquisition time of each training frame to obtain the stage standard trajectory graphic corresponding to each smoke stage label.

6. The method according to claim 1, characterized in that, The step of performing dynamic programming on the post-ignition sequence in the standardized time-series stage description vector sequence to obtain the stage boundary of the smoke combustion stage includes: With the objective of minimizing the sum of the squared deviations of the standardized stage description vectors in each segment of the standardized stage description vector sequence relative to the mean vector of the standardized stage description vectors of all frames in each segment, dynamic programming is performed on the post-ignition sequence under the constraint that the number of frames in each segment is not less than a preset minimum duration of frames to obtain the stage boundary.

7. The method according to claim 1, characterized in that, The step of mapping each frame after ignition to the corresponding smoke combustion stage interval divided by the stage boundary to obtain the smoke stage label of each frame includes: When it is determined that the number of frames in the post-ignition sequence is lower than the preset minimum number of solvable frames, the coverage component sequence corresponding to the post-ignition sequence in the timing stage description vector sequence is divided into a first half and a second half according to the timing. The change in coverage range component is obtained by subtracting the mean of the first half of the coverage range component from the mean of the second half of the coverage range component. The coverage range change ratio is obtained by taking the ratio of the change in the coverage range component to the mean of the first half of the coverage range component. In response to the coverage change ratio exceeding a preset growth ratio threshold, all frames in the post-ignition sequence are identified as growth phase labels, and the growth phase labels are used as smoke phase labels to which each frame belongs. In response to the coverage change ratio being lower than a preset attenuation ratio threshold, all frames in the post-ignition sequence are identified as attenuation stage labels, and the attenuation stage labels are used as smoke stage labels to which each frame belongs. In response to the coverage change ratio being between the preset attenuation ratio threshold and the preset growth ratio threshold, all frames in the post-ignition sequence are identified as stable phase labels, and the stable phase labels are used as smoke phase labels to which each frame belongs.

8. The method according to claim 1, characterized in that, The step-aware probability of outputting the smoke stage label of the current frame includes: The current frame, the smoke-free background, and the absolute difference map are stitched together according to the channel dimension to obtain the fused input image; In the smoke difference coordinate system, the difference feature points corresponding to the fused input image are connected sequentially according to the acquisition time of each frame to obtain the difference trajectory graphic corresponding to the current frame; The centroid of the coordinates of the difference feature points of the standard trajectory graphics corresponding to each smoke stage label is used as the translation reference point. The centroid of the coordinates of the difference feature points of the difference trajectory graphic is translated to the translation reference point to obtain the translated difference trajectory graphic; The trajectory overlap degree corresponding to each smoke stage label is obtained by the ratio of the area of ​​the overlapping region between the translated difference trajectory graphic and the stage standard trajectory graphic to the smaller of the area of ​​the translated difference trajectory graphic and the area of ​​the stage standard trajectory graphic. Normalization is performed on the trajectory overlap degree corresponding to each smoke stage label to obtain the stage perception probability.

9. The method according to claim 8, characterized in that, The step of sequentially connecting the difference feature points corresponding to the fused input image in the smoke difference coordinate system according to the acquisition time of each frame to obtain the difference trajectory graphic corresponding to the current frame includes: Using the smoke pseudomask corresponding to each frame obtained through adaptive threshold filtering and connected growth as the boundary, the differential feature points corresponding to the fused input image are divided into differential feature points inside the smoke area and differential feature points outside the smoke area. The differential feature points within the smoke area of ​​each frame are sequentially connected in the smoke difference coordinate system according to the acquisition time of each frame to obtain the main trajectory. Sort the differential feature points outside the smoke area in each frame according to their coordinate values, and extract the differential feature points outside the smoke area whose coordinate values ​​are not lower than the quantile corresponding to the preset outer area extraction ratio to obtain the extracted differential feature points outside the smoke area. Connect the extracted feature points outside the smoke area sequentially in the smoke difference coordinate system to obtain the trajectory auxiliary line; The main trajectory and the auxiliary trajectory lines are merged to obtain the difference trajectory graphic.

10. The method according to claim 1, characterized in that, The method of triggering an early warning instruction using the stage perception probability as a trigger input includes: Set a corresponding preset stage risk value for each of the smoke stage labels; The smoke risk value is obtained by summing the products of the stage perception probability corresponding to each smoke stage label and the preset stage risk value. When the smoke risk value is determined to exceed a preset risk threshold, the warning command is triggered.

11. A multi-stage smoke differentiation sensing device for fires in mountainous power transmission channels, characterized in that, include: The background modeling module is used to construct a smoke-free background based on pre-ignition images from a multispectral image sequence collected along a mountainous power transmission corridor. It integrates the differences between the current frame and the smoke-free background in the multispectral image sequence into a smoke physical feature set. This includes: calculating the pixel-by-pixel difference between each spectral channel of the current frame and the corresponding spectral channel of the smoke-free background to obtain the channel difference for each spectral channel; using the smoke stage label corresponding to the experimental group to which the historical multispectral image sequence belongs as a basis, calculating the ratio of the inter-class dispersion to the intra-class dispersion of the channel difference for each spectral channel among the smoke stage labels to obtain the stage differentiation contribution value corresponding to each spectral channel; determining the channel difference between the two spectral channels with the largest stage differentiation contribution values ​​as coordinate axes to construct a smoke difference coordinate system; marking the corresponding differences of the current frame on each coordinate axis of the smoke difference coordinate system according to pixel position in the smoke difference coordinate system to obtain the difference feature points corresponding to each pixel position; and determining the set of difference feature points at each pixel position as the smoke physical feature set. The phase division module is used to arrange the smoke state variables in the smoke physical feature set in a temporal sequence, determine the phase boundaries through temporal segmentation optimization, obtain the smoke phase label of each frame, and construct a corresponding perception model for each smoke phase label. This includes: based on the difference feature points of each frame in the smoke physical feature set, obtaining the smoke pseudomask corresponding to each frame through adaptive threshold filtering and connectivity growth; using the coverage area of ​​the smoke pseudomask, the average coordinates of the difference feature points within the smoke pseudomask, and the change in the coverage area of ​​the smoke pseudomask between adjacent frames as components, arranging the components corresponding to each frame according to the acquisition time of each frame into a temporal phase description vector sequence; and further... The temporal stage description vector sequence is standardized by components to obtain the standardized temporal stage description vector sequence; dynamic programming is performed on the post-ignition sequence in the standardized temporal stage description vector sequence to obtain the stage boundary of the smoke combustion stage; each post-ignition frame is mapped to the corresponding smoke combustion stage interval divided by the stage boundary to obtain the smoke stage label to which each frame belongs; the difference feature points of the training frames belonging to each smoke stage label are sequentially connected in the smoke difference coordinate system to obtain the stage standard trajectory graphic corresponding to each smoke stage label, and the stage standard trajectory graphic is determined as the perception model corresponding to each smoke stage label. The probabilistic reasoning module is used to input the absolute difference map between the current frame and the smoke-free background into the perception model and output the stage perception probability of the smoke stage label to which the current frame belongs. The early warning linkage module is used to overlay the stage perception probability onto the power transmission channel geographic information layer based on the smoke stage label, and to trigger an early warning command using the stage perception probability as the trigger input.

Images