A video stream-based intelligent detection method for smoke and fire in a building passage

Abstract

The present application relates to the technical field of image detection, in particular to a building passage fire and smoke intelligent detection method based on video stream, comprising the following steps: extracting each frame picture according to building passage video stream input signal, executing complex multi-direction decomposition processing, and generating spatial feature sub-band set; extracting complex pixel parameters corresponding to horizontal and vertical directions of the spatial feature sub-band set, and separating phase values to obtain sub-band phase sequence. Through continuous screening of distortion threshold threshold value, gravity reverse direction vector interval, variance lower limit threshold value and rigid body motion tolerance interval, the present application can successively exclude disturbances with insufficient amplitude, disturbances with directions not complying with the heat convection upward rule, disturbances with insufficient phase fluctuation stability and displacement disturbances caused by rigid body motion, so that the finally established hidden heat source positioning coordinate is closer to the early heat source position of fire and smoke, and the anti-interference ability of building passage fire and smoke detection is improved.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to an intelligent method for detecting smoke and fire in building passageways based on video streams. Background Technology

[0002] Image detection technology involves collecting, analyzing, filtering, and judging visual information such as targets, regions, textures, edges, colors, motion states, and spatial relationships in images or videos. It typically identifies the presence of specific targets or abnormal states in a scene by processing pixel values, grayscale changes, color changes, local morphology, and temporal changes in consecutive image frames.

[0003] Current image detection technologies primarily focus on acquiring, analyzing, filtering, and judging targets, regions, textures, edges, colors, motion states, and spatial relationships. In actual operation, they often pay more attention to visible shapes or significant pixel changes in the image. In the early stages of smoke and fire, before obvious smoke clouds, flame outlines, or color anomalies have formed, pixel grayscale changes, color changes, and local shape changes often lack stable and distinguishable characteristics, easily misattributing early thermal disturbances to background fluctuations. Building corridors also present complex interferences such as lighting switching, reflective walls, rapid passage of people, door opening, and equipment shadow movement. When judging solely based on the brightness, edges, texture, or motion state of consecutive image frames, it is difficult to distinguish between non-rigid air distortion caused by thermal convection and rigid body displacement caused by the movement of physical objects. For example, when a person passes through the camera's field of view, the edges of clothing and shadows may produce obvious motion characteristics, increasing the risk of misjudgment. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an intelligent smoke and fire detection method for building passageways based on video streams.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a smart smoke and fire detection method for building passageways based on video streams, comprising the following steps:

[0006] Extract each frame from the video stream input signal of the building passageway, perform complex multi-directional decomposition processing, and generate a set of spatial feature subbands;

[0007] Extract the complex pixel parameters corresponding to the horizontal and vertical directions of the spatial feature sub-band set, and separate the phase values ​​to obtain the sub-band phase sequence;

[0008] Extract the component values ​​within a set frequency band range from the sub-band phase sequence to establish the target frequency band phase sequence;

[0009] The phase component values ​​in the vertical and horizontal directions of the target frequency band phase sequence are multiplied by the amplification factor to generate phase amplification sequences in the vertical and horizontal directions, respectively.

[0010] Calculate the first-order time-domain derivatives of both to obtain the vertical and horizontal distortion rate parameters, and calculate the amplitude of the generation distortion rate and the spatiotemporal distortion direction angle.

[0011] Extract elements from the distortion rate amplitude that exceed the set distortion threshold to generate distortion spatiotemporal candidate parameters;

[0012] Extract parameters whose spatial vector direction falls within the interval of the opposite direction of gravity vector from the candidate parameters of the distorted spatiotemporal space, and establish an upward thermal convection feature sequence;

[0013] Determine whether the phase fluctuation variance parameter mapped by the upward thermal convection feature sequence exceeds the lower limit threshold value, and filter to generate a high-frequency distortion coordinate set;

[0014] The relative rigid body displacement parameters of the region pointed to by the high-frequency distortion coordinate set relative to the local background are obtained, and two-dimensional spatiotemporal coordinate points that do not exceed the rigid body motion tolerance range are extracted to establish the location coordinates of the hidden heat source.

[0015] Preferably, the step of obtaining the spatial feature subset is as follows:

[0016] Based on the video stream input signal of the building passage, continuous channel images are extracted according to the frame order of the video stream input signal. The frame number, sampling time, pixel position coordinates and pixel grayscale value of each continuous channel image are recorded. For each continuous channel image, complex multi-directional decomposition is performed at the spatial level according to the pixel change direction in the horizontal, vertical and diagonal directions. The complex responses of each direction obtained by decomposition are collected according to the frame number and pixel position coordinates to generate a spatial feature sub-band set.

[0017] Preferably, the step of obtaining the target frequency band phase sequence is as follows:

[0018] Extract the complex pixel parameters corresponding to the horizontal direction and the complex pixel parameters in the vertical direction in the spatial feature sub-band set. Read the real part value, imaginary part value, frame number and pixel position coordinates of each complex pixel parameter. Separate the amplitude value and phase value according to the numerical mapping relationship between the real part value and the imaginary part value. Retain the direction identifier, frame number and pixel position coordinates corresponding to the phase value to obtain the sub-band phase sequence.

[0019] Based on the sub-band phase sequence, the continuous phase change trajectory is arranged according to the frame number corresponding to the phase value. The frequency band belonging identifier corresponding to the continuous phase change trajectory in the time dimension is calculated. The phase component values ​​of the frequency band belonging identifier falling into the 2 Hz to 5 Hz frequency band range are retained, and the phase component values ​​of the frequency band belonging identifier not falling into the 2 Hz to 5 Hz frequency band range are removed. The retained phase component values ​​are rearranged according to the direction identifier, frame number, and pixel position coordinates to establish the target frequency band phase sequence.

[0020] Preferably, the step of obtaining the spatiotemporal distortion direction angle is as follows:

[0021] Read the direction identifier, frame number, sampling time and pixel position coordinates corresponding to each phase component value in the target frequency band phase sequence. Arrange the phase component values ​​with the direction identifier in the vertical direction continuously according to the frame number, and arrange the phase component values ​​with the direction identifier in the horizontal direction continuously according to the frame number. Call the vertical direction amplification factor and the horizontal direction amplification factor respectively, and multiply the phase component values ​​under the same frame number and the same pixel position coordinates one by one to generate the vertical direction phase amplification sequence and the horizontal direction phase amplification sequence.

[0022] Based on the vertical phase amplification sequence and the horizontal phase amplification sequence, extract the phase amplification value, sampling time difference, and pixel position coordinates corresponding to adjacent frame numbers. Divide the change in the vertical phase amplification value between adjacent sampling times by the sampling time difference, and divide the change in the horizontal phase amplification value between adjacent sampling times by the sampling time difference. Retain the frame number corresponding to each change rate according to the same pixel position coordinates to obtain the vertical distortion rate parameter and the horizontal distortion rate parameter.

[0023] Based on the vertical and horizontal distortion rate parameters, the vertical and horizontal distortion rate values ​​under the same frame number and the same pixel position coordinates are read. The squares of the vertical and horizontal distortion rate values ​​are summed and squared to obtain the distortion rate amplitude. Then, the quadrant to which the distortion direction belongs is determined according to the coordinate axis signs of the horizontal and vertical distortion rate values. The direction angle value corresponding to each distortion rate amplitude is calculated to generate the spatiotemporal distortion direction angle.

[0024] Preferably, the steps for obtaining the candidate parameters of the distorted spatiotemporal spacetime are as follows:

[0025] The amplitude records are read item by item according to the frame number, sampling time and pixel position coordinates corresponding to the distortion rate amplitude. The distortion rate amplitudes of the same pixel position coordinates at consecutive sampling times are arranged into a time dimension change sequence. The independent amplitude records corresponding to each sampling time in the time dimension change sequence are extracted, and the frame number, sampling time and pixel position coordinates corresponding to the independent amplitude records are retained to obtain discrete variable elements.

[0026] The distortion rate amplitude, frame number, sampling time, and pixel position coordinates of the discrete variable elements are read one by one. Each distortion rate amplitude is compared with a set distortion threshold. If the distortion rate amplitude is greater than the distortion threshold, the corresponding discrete variable element is retained. If the distortion rate amplitude is less than or equal to the distortion threshold, the corresponding discrete variable element is removed. The retained discrete variable elements are rearranged according to the frame number and pixel position coordinates to generate distortion spatiotemporal candidate parameters.

[0027] Preferably, the step of obtaining the upward thermal convection characteristic sequence is as follows:

[0028] The frame number, sampling time, and pixel position coordinates corresponding to the distorted spatiotemporal candidate parameters are called. The direction angle values ​​corresponding to the same frame number, sampling time, and pixel position coordinates are retrieved in the spatiotemporal distortion direction angle. The direction angle values ​​are compared with the angle boundaries of the gravity anti-direction vector interval. If the direction angle value falls inside the gravity anti-direction vector interval, the corresponding distorted spatiotemporal candidate parameter is extracted. If the direction angle value does not fall inside the gravity anti-direction vector interval, the corresponding distorted spatiotemporal candidate parameter is removed, and an upward thermal convection feature sequence is established.

[0029] Preferably, the steps for obtaining the high-frequency distortion coordinate set are as follows:

[0030] The frame number, sampling time, pixel position coordinates, and phase component values ​​corresponding to each discrete element within the upward thermal convection feature sequence are read. The phase component values ​​of the same pixel position coordinates at consecutive sampling times are arranged into a phase fluctuation trajectory. The degree of discrete deviation of the phase fluctuation trajectory relative to the phase mean is calculated to form a phase fluctuation variance parameter. The phase fluctuation variance parameter is compared item by item with a set lower limit threshold value. If the phase fluctuation variance parameter is greater than the lower limit threshold value, the corresponding discrete element is retained. If the phase fluctuation variance parameter is less than or equal to the lower limit threshold value, the corresponding discrete element is discarded. The retained discrete elements are arranged according to the frame number, sampling time, and pixel position coordinates to generate a high-frequency distortion coordinate set.

[0031] Preferably, the step of obtaining the location coordinates of the concealed heat source is as follows:

[0032] Read the pixel position coordinates, frame number, and sampling time pointed to by the high-frequency distortion coordinate set, extract the pixel region around the pixel position coordinates, extract the coordinate offset of the pixel region between adjacent sampling times, read the background coordinate offset of the local background under the same frame number, perform item-by-item difference processing on the coordinate offset of the pixel region and the background coordinate offset of the local background, retain the horizontal displacement value, vertical displacement value, frame number, sampling time, and pixel position coordinates after the difference processing, and obtain the relative rigid body displacement parameter;

[0033] The lateral displacement value, longitudinal displacement value, frame number, sampling time, and pixel position coordinates in the relative rigid body displacement parameters are read item by item. The lateral displacement value and longitudinal displacement value are compared with the preset rigid body motion tolerance interval boundary. If the lateral displacement value and longitudinal displacement value do not exceed the rigid body motion tolerance interval, the corresponding two-dimensional spatiotemporal coordinate point is extracted. If the lateral displacement value or longitudinal displacement value exceeds the rigid body motion tolerance interval, the corresponding two-dimensional spatiotemporal coordinate point is discarded. The retained two-dimensional spatiotemporal coordinate points are output as coordinate signals according to the frame number and sampling time to establish the hidden heat source location coordinates.

[0034] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0035] In this invention, images are extracted frame by frame from the input signal of the building passage video stream, and complex multi-directional decomposition is performed on each frame. This decomposes the horizontal texture, vertical texture, edge disturbances, and local spatial changes in the building passage image into spatial feature sub-bands. Furthermore, phase values ​​are separated from the complex pixel parameters in the horizontal and vertical directions, avoiding the problem of being easily interfered with by lighting changes when relying solely on color, brightness, or grayscale changes for recognition. By selecting component values ​​within a set frequency band from the sub-band phase sequence, a target frequency band phase sequence is established. This concentrates and preserves the periodic phase fluctuations corresponding to the early thermal disturbances of fireworks, while weakening irrelevant changes in non-target frequency bands caused by personnel passage, light flickering, and camera shake. By analyzing the vertical phase component values ​​and... By amplifying the values ​​of the horizontal phase components and calculating their first-order time derivatives, subtle image distortions caused by rising hot air can be transformed into vertical and horizontal distortion rate parameters. Combined with the distortion rate amplitude and the spatiotemporal distortion direction angle, the detection basis is expanded from static image features to spatiotemporal motion features with directional attributes. Through continuous filtering of distortion threshold, gravity reverse direction vector interval, variance lower limit threshold, and rigid body motion tolerance interval, disturbances with insufficient amplitude, directions that do not conform to the upward thermal convection, insufficient phase fluctuation stability, and displacement disturbances caused by rigid body motion can be eliminated in sequence. This makes the final established coordinates of the hidden heat source closer to the early heat source location of the fire, improving the anti-interference capability of fire detection in building passageways. Attached Figure Description

[0036] Figure 1 A frequency band interval selection diagram for the target frequency band phase sequence;

[0037] Figure 2 This is a screening diagram of spatiotemporal distortion direction angle and upward thermal convection characteristic sequence. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0039] Please see Figure 1-2 This invention provides a technical solution: an intelligent smoke and fire detection method for building passageways based on video streams, comprising the following steps:

[0040] Based on the video stream input signal of the building passage, extract each frame, perform complex multi-directional decomposition processing to generate a spatial feature sub-band set; extract the complex pixel parameters corresponding to the horizontal and vertical directions of the spatial feature sub-band set, separate the phase values ​​to obtain the sub-band phase sequence; extract the component values ​​in the sub-band phase sequence that are within the set frequency band interval, and establish the target frequency band phase sequence;

[0041] The phase component values ​​in the vertical and horizontal directions of the target frequency band phase sequence are multiplied by the amplification factor to generate the vertical and horizontal phase amplification sequences respectively; the first derivatives in the time domain of the two are calculated to obtain the vertical and horizontal distortion rate parameters, and the amplitude of the generated distortion rate and the spatiotemporal distortion direction angle are calculated.

[0042] Extract elements from the distortion rate amplitude that exceed the set distortion threshold to generate distortion spatiotemporal candidate parameters; extract parameters from the distortion spatiotemporal candidate parameters whose spatial vector direction falls within the interval of the opposite direction of gravity vector to establish an upward thermal convection feature sequence.

[0043] Determine whether the phase fluctuation variance parameter of the upward thermal convection feature sequence mapping exceeds the lower limit threshold value, and filter to generate a high-frequency distortion coordinate set; obtain the relative rigid body displacement parameter of the region pointed to by the high-frequency distortion coordinate set relative to the local background, extract the two-dimensional spatiotemporal coordinate points that do not exceed the rigid body motion tolerance range, and establish the location coordinates of the hidden heat source.

[0044] The steps for obtaining the spatial feature subset are as follows:

[0045] Based on the video stream input signal of the building passage, continuous channel images are extracted according to the frame order of the video stream input signal. The frame number, sampling time, pixel position coordinates and pixel grayscale value of each continuous channel image are recorded. For each continuous channel image, complex multi-directional decomposition is performed at the spatial level according to the pixel change direction in the horizontal, vertical and diagonal directions. The complex responses of each direction obtained by decomposition are collected according to the frame number and pixel position coordinates to generate a spatial feature sub-band set.

[0046] Specifically, based on the input signal of the building corridor video stream, continuous corridor images are first captured frame by frame at the inherent frame rate of the video stream, such as 25 or 30 frames per second. For each frame, a data record is created containing a unique frame number, a sampling time accurate to milliseconds, the two-dimensional coordinates (x, y) of the pixel in the image, and the corresponding 8-bit grayscale value (0-255). Then, for each grayscale image frame, the Dual-Tree Complex Wavelet Transform (DTCWT) technique is applied for complex multi-directional decomposition. This transformation is achieved through two parallel real wavelet transforms, one of which uses a Hilbert transform as its filter bank, thus generating complex wavelet coefficients with approximately shift invariance and excellent direction selectivity. Specifically, DTCWT is applied to each image frame, decomposing it into multiple scales and six predetermined directions (e.g., ±15°, ±45°, ±75°). The responses are combined to characterize pixel changes in the horizontal, vertical, and diagonal directions. For example, the horizontal changes are mainly composed of sub-band responses in the ±15° direction, the vertical changes are composed of sub-band responses in the ±75° direction, and the diagonal changes are composed of sub-band responses in the ±45° direction. The complex coefficients (including real and imaginary parts) obtained after decomposition in each direction and at each scale are strictly mapped and bound to their respective frame numbers and pixel position coordinates. Finally, the complex response data of all frames, all pixels, and all directions are structurally aggregated to generate a spatial feature sub-band set.

[0047] The steps for obtaining the target frequency band phase sequence are as follows:

[0048] Extract the complex pixel parameters corresponding to the horizontal and vertical directions in the spatial feature sub-band set. Read the real part, imaginary part, frame number and pixel position coordinates of each complex pixel parameter. Separate the amplitude value and phase value according to the numerical mapping relationship between the real part and the imaginary part. Retain the direction identifier, frame number and pixel position coordinates corresponding to the phase value to obtain the sub-band phase sequence.

[0049] Based on the sub-band phase sequence, arrange the continuous phase change trajectory according to the frame number corresponding to the phase value, calculate the frequency band assignment identifier corresponding to the continuous phase change trajectory in the time dimension, retain the phase component values ​​that fall within the 2 Hz to 5 Hz frequency band, remove the phase component values ​​that do not fall within the 2 Hz to 5 Hz frequency band, and rearrange the retained phase component values ​​according to the direction identifier, frame number, and pixel position coordinates to establish the target frequency band phase sequence.

[0050] Specifically, complex pixel parameters corresponding to the horizontal and vertical directions are extracted from the spatial feature sub-band set. Each complex pixel parameter is processed, as it contains a real part and an imaginary part, and is associated with its source frame number and pixel position coordinates. Then, using the representation of complex numbers in polar coordinates, each complex pixel parameter is transformed from Cartesian coordinates (real part, imaginary part) to polar coordinates (amplitude, phase). Specifically, the phase value is calculated using the arctangent function, which is a two-parameter arctangent function atan2. The input is the imaginary and real parts of the complex pixel parameter. This function can correctly determine the quadrant of the phase angle based on the signs of the two inputs, thus avoiding the ambiguity of the traditional arctangent function around ±90°, and obtaining an accurate phase value within the range of (-π, π]. In this process, the separated amplitude value is discarded, and only the calculated phase value is retained. At the same time, each phase value is appended with its original direction identifier (horizontal or vertical), frame number, and pixel position coordinates. All the processed phase data are collected to obtain the sub-band phase sequence.

[0051] Based on the sub-band phase sequence, for each independent pixel location coordinate, its phase values ​​across consecutive frames are arranged in ascending order of frame number, forming a continuous phase change trajectory for that pixel. This is a discrete-time signal with time (frame number) as the independent variable. To analyze the frequency characteristics of this trajectory, a fixed-length time window, such as 1 second (corresponding to 25 or 30 consecutive phase values), is extracted. A Fast Fourier Transform (FFT) is applied to the phase sequence within this window to convert the time-domain signal into a frequency-domain representation, thereby calculating the energy or amplitude of different frequency components. Based on the calculated spectrum, the frequency band in which the main energy of the phase change trajectory is concentrated is determined; this frequency band is the frequency band assignment. Next, a target frequency band screening standard is established, which is defined as a frequency band range of 2 Hz to 5 Hz. This range is set based on empirical observations of the frequency of hot air disturbances caused by early fires. These disturbances usually manifest as low-to-mid-frequency flickering or fluctuations. The frequency band assignment of each pixel phase trajectory is compared with this range. If its main energy falls within the 2 Hz to 5 Hz frequency band range, all phase component values ​​constituting the trajectory are retained. If its main energy does not fall within this range, all these phase component values ​​are discarded. Finally, all the phase component values ​​that have been screened and retained are reorganized and arranged according to their original direction identifier, frame number, and pixel position coordinates to establish the target frequency band phase sequence.

[0052] The steps for obtaining the spatiotemporal distortion direction angle are as follows:

[0053] Read the direction identifier, frame number, sampling time and pixel position coordinates corresponding to each phase component value in the target frequency band phase sequence. Arrange the phase component values ​​with the direction identifier in the vertical direction continuously according to the frame number, and arrange the phase component values ​​with the direction identifier in the horizontal direction continuously according to the frame number. Call the vertical direction amplification factor and the horizontal direction amplification factor respectively, and multiply the phase component values ​​under the same frame number and the same pixel position coordinates one by one to generate the vertical direction phase amplification sequence and the horizontal direction phase amplification sequence.

[0054] Based on the vertical and horizontal phase amplification sequences, extract the phase amplification values, sampling time differences, and pixel position coordinates corresponding to adjacent frame numbers. Divide the change in vertical phase amplification values ​​between adjacent sampling times by the sampling time difference, and divide the change in horizontal phase amplification values ​​between adjacent sampling times by the sampling time difference. Retain the frame number corresponding to each change rate according to the same pixel position coordinates to obtain the vertical distortion rate parameter and the horizontal distortion rate parameter.

[0055] Based on the vertical and horizontal distortion rate parameters, the vertical and horizontal distortion rate values ​​under the same frame number and pixel position coordinates are read. The squares of the vertical and horizontal distortion rate values ​​are summed and squared to obtain the distortion rate amplitude. Then, the quadrant to which the distortion direction belongs is determined according to the coordinate axis signs of the horizontal and vertical distortion rate values. The direction angle value corresponding to each distortion rate amplitude is calculated to generate the spatiotemporal distortion direction angle.

[0056] Specifically, the process reads the value of each phase component in the target frequency band phase sequence and simultaneously acquires its corresponding direction identifier (vertical or horizontal), frame number, sampling time, and pixel position coordinates. All phase component values ​​with vertical direction identifiers are categorized and arranged according to frame number and pixel position coordinates to form a vertical phase dataset. Similarly, all phase component values ​​with horizontal direction identifiers are categorized and arranged to form a horizontal phase dataset. Next, phase amplification processing is introduced to enhance the sensitivity to weak signals in subsequent analysis. This process uses vertical and horizontal amplification coefficients, which are empirical values ​​set based on prior knowledge and experimental testing. For example, based on the typical distance between the camera and a potential fire source, the amplification coefficient can be set to 50 to effectively amplify the small phase changes caused by thermal buoyancy. Specifically, for each data point in the vertical and horizontal phase datasets, while ensuring that the frame number and pixel position coordinates are completely consistent, its phase component value is multiplied item by item by the corresponding amplification coefficient (e.g., 50). Through this operation, vertical and horizontal phase amplification sequences are generated respectively.

[0057] Based on the vertical and horizontal phase amplification sequences, the temporal evolution of each pixel in the video stream is analyzed. Specifically, for any fixed pixel position coordinate, the phase amplification values ​​corresponding to two temporally adjacent consecutive frames (e.g., frame n and frame (n+1)) are extracted. Simultaneously, the sampling times corresponding to these two frames are recorded, and the difference in sampling times between the two frames is calculated. This difference is typically the reciprocal of the video frame rate, for example, 1 / 25 of a second. Then, the change in the vertical phase amplification value between these two adjacent sampling times is calculated, which is the value of frame (n+1) minus the value of frame n. The difference is divided by the sampling time difference to obtain the vertical phase change rate of the pixel at that time, i.e., the vertical distortion rate parameter. The same calculation method is used to process the horizontal phase amplification sequence, that is, the horizontal phase amplification value of the (n+1)th frame is subtracted from the value of the nth frame, and then divided by the sampling time difference to obtain the horizontal distortion rate parameter. The vertical and horizontal change rate values ​​of all pixels calculated at all times are stored together with their corresponding frame numbers and pixel position coordinates to finally obtain the vertical distortion rate parameter and horizontal distortion rate parameter covering the entire spatiotemporal dimension.

[0058] Based on the vertical and horizontal distortion rate parameters, the distortion vector characteristics of each spatiotemporal point are further calculated. Specifically, for any frame in the video stream and any pixel position coordinate within that frame, the corresponding vertical and horizontal distortion rate values ​​are read. These two values ​​are considered as components of a two-dimensional vector on the vertical and horizontal axes. First, the magnitude of this two-dimensional vector, i.e., the distortion rate amplitude, is obtained by calculating the square root of the sum of the squares of these two components. This amplitude reflects the total intensity of phase distortion at that pixel. Then, the direction of the vector, i.e., the spatiotemporal distortion direction angle, is calculated. The process involves determining the Cartesian coordinate quadrant to which the distortion direction vector belongs based on the sign (positive or negative) of the horizontal and vertical distortion rate values. For example, if both the horizontal and vertical values ​​are positive, the vector belongs to the first quadrant. Then, using the two-parameter arctangent function atan2, the vertical and horizontal distortion rate values ​​are input to calculate the direction angle. This angle uniquely represents the direction of the distortion in two-dimensional space. Each calculated distortion rate amplitude and its corresponding direction angle value, along with its original frame number and pixel position coordinates, are associated and stored to generate a set of spatiotemporal distortion direction angles.

[0059] The steps for obtaining candidate parameters for distorted spatiotemporal space are as follows:

[0060] Read the amplitude records item by item according to the frame number, sampling time and pixel position coordinates corresponding to the distortion rate amplitude. Arrange the distortion rate amplitudes of the same pixel position coordinates at consecutive sampling times into a time dimension change sequence. Extract the independent amplitude records corresponding to each sampling time in the time dimension change sequence, and retain the frame number, sampling time and pixel position coordinates corresponding to the independent amplitude records to obtain discrete variable elements.

[0061] The distortion rate amplitude, frame number, sampling time, and pixel position coordinates of each discrete variable element are read one by one. Each distortion rate amplitude is compared with the set distortion threshold. If the distortion rate amplitude is greater than the distortion threshold, the corresponding discrete variable element is retained. If the distortion rate amplitude is less than or equal to the distortion threshold, the corresponding discrete variable element is removed. The retained discrete variable elements are rearranged according to the frame number and pixel position coordinates to generate distortion spatiotemporal candidate parameters.

[0062] Specifically, according to the frame number, sampling time, and pixel position coordinates corresponding to the distortion rate amplitude, the previously calculated amplitude records are read item by item. The focus is on the behavior of a single pixel in the time dimension. Specifically, any pixel position coordinate is selected, and its distortion rate amplitude at all consecutive sampling times is arranged in chronological order to form a time-dimensional change sequence representing the change of the pixel's distortion intensity over time. Since the original calculation is based on consecutive frames, each sampling time in this sequence corresponds to an independent amplitude record. Next, the independent amplitude records are extracted one by one from this time-dimensional change sequence, ensuring that each record carries its original frame number, sampling time, and pixel position coordinate information. This process is equivalent to converting the data from a time-based sequence structure into a series of independent discrete data points containing complete spatiotemporal labels. Each of these discrete data points represents the intensity of a phase distortion event that occurs at a specific time and location. All such discrete data points of all pixels at all times are collected to obtain a set of discrete variable elements.

[0063] The discrete variable elements are read item by item. Each element contains the distortion rate amplitude, frame number, sampling time, and pixel location coordinates. Each distortion rate amplitude is compared with a dynamically set distortion threshold. The threshold is set as follows: First, a normal building passageway video without smoke or significant disturbance is acquired as a baseline sample. The distortion rate amplitude of all pixels in this sample at all times is calculated to form a baseline distribution. Then, the average value of this baseline distribution is calculated. and standard deviation Distortion threshold It is then set to ,in This is a sensitivity adjustment coefficient, empirically 3, to cover most normal fluctuations. For example, if the average value is obtained by analyzing a benchmark video. rad / s, standard deviation If the distortion threshold is set to rad / s, then the distortion threshold is... In the comparison process, if the distortion rate amplitude of a discrete variable element is greater than 0.085 rad / s, it is determined to be a significant distortion event and the discrete variable element is retained. Conversely, if its amplitude is less than or equal to 0.085 rad / s, it is considered to be background noise or normal disturbance and is discarded. Finally, all the discrete variable elements that have passed the screening are reorganized according to their frame number and pixel position coordinates to generate candidate parameters for distortion spatiotemporal parameters.

[0064] The steps for obtaining the upward thermal convection characteristic sequence are as follows:

[0065] The frame number, sampling time, and pixel position coordinates corresponding to the distorted spatiotemporal candidate parameters are called. The directional angle values ​​corresponding to the same frame number, sampling time, and pixel position coordinates are searched in the spatiotemporal distortion directional angle. The directional angle values ​​are compared with the angular boundaries of the gravity anti-direction vector interval. If the directional angle value falls inside the gravity anti-direction vector interval, the corresponding distorted spatiotemporal candidate parameter is extracted. If the directional angle value does not fall inside the gravity anti-direction vector interval, the corresponding distorted spatiotemporal candidate parameter is removed, and an upward thermal convection feature sequence is established.

[0066] Specifically, the frame number, sampling time, and pixel position coordinates attached to each element in the spatiotemporal distortion candidate parameters are called. Using this information as an index, the corresponding orientation angle value at the same spatiotemporal point is precisely retrieved from the previously generated set of spatiotemporal distortion orientation angles. Next, the retrieved orientation angle value is compared with a preset gravity-opposite vector interval. This interval is defined based on physical common sense: hot air moves vertically upwards due to buoyancy because its density is lower than cold air. In typical video surveillance footage, the direction of gravity usually points to the bottom of the image (positive Y-axis direction), therefore the opposite direction of gravity is the top of the image (negative Y-axis direction). Considering airflow disturbances, this direction is not strictly -90 degrees, but rather an interval, for example, set to a range of -120 degrees to -60 degrees (with horizontal to the right as 0 degrees and counterclockwise as positive). This interval is the gravity-opposite vector interval. The specific comparison process is as follows: determine whether each retrieved orientation angle value falls within [-120°, ... Within the angular boundary of -60°, if the direction angle value is within this range, it indicates that the distortion event has a significant upward movement trend, which is consistent with the basic characteristics of thermal convection. Therefore, the corresponding distortion spatiotemporal candidate parameters are extracted and retained. If the direction angle value does not fall into this range, it is considered that it may be caused by other non-thermal source disturbances (such as people walking or objects moving) and is removed. All the parameters retained after screening are gathered to establish an upward thermal convection characteristic sequence.

[0067] The steps for obtaining the high-frequency distortion coordinate set are as follows:

[0068] Read the frame number, sampling time, pixel position coordinates, and phase component value corresponding to each discrete element within the upward thermal convection feature sequence. Arrange the phase component values ​​of the same pixel position coordinates at consecutive sampling times to form a phase fluctuation trajectory. Calculate the degree of discrete deviation of the phase fluctuation trajectory from the phase mean to form a phase fluctuation variance parameter. Compare the phase fluctuation variance parameter item by item with the set lower limit threshold value. If the phase fluctuation variance parameter is greater than the lower limit threshold value, retain the corresponding discrete element. If the phase fluctuation variance parameter is less than or equal to the lower limit threshold value, discard the corresponding discrete element. Arrange the retained discrete elements according to the frame number, sampling time, and pixel position coordinates to generate a high-frequency distortion coordinate set.

[0069] Specifically, each discrete element within the upward thermal convection feature sequence is read, and its frame number, sampling time, pixel position coordinates, and original phase component values ​​are obtained. For each unique pixel position coordinate appearing in the sequence, its phase component values ​​at consecutive sampling times (e.g., within a time window encompassing 10 frames before and after the current time) are collected and arranged chronologically to form a short-term phase fluctuation trajectory. To quantify the intensity of this trajectory's fluctuations, the variance of this trajectory is calculated, i.e., the degree of discrete deviation of all values ​​in the phase fluctuation trajectory from their own mean, resulting in a phase fluctuation variance parameter. This variance parameter is then compared with a set lower variance threshold. This threshold is also based on statistical analysis of normal environments. Specifically, in a smoke-free baseline video, the phase fluctuation variance of a large number of randomly selected pixels is calculated to form a "normal variance" distribution. The 98th percentile of this distribution is taken as the lower variance threshold. For example, if 98% of the calculated normal variance values ​​are below 0.05... If the variance is less than or equal to 0.05 rad², then the lower limit threshold of the variance is set to 0.05 rad². During comparison, if the calculated phase fluctuation variance parameter is greater than 0.05 rad², then the pixel is considered to exhibit high-frequency jitter characteristics at that moment, which is consistent with the characteristics of rapid changes in air refractive index caused by thermal radiation, and the discrete element is retained. If the variance is less than or equal to 0.05 rad², then the element is discarded. Finally, all the retained discrete elements are rearranged according to their spatiotemporal coordinate information to generate a high-frequency distortion coordinate set.

[0070] The steps to obtain the location coordinates of a hidden heat source are as follows:

[0071] Read the pixel position coordinates, frame number, and sampling time pointed to by the high-frequency distortion coordinate set, extract the pixel region around the pixel position coordinates, extract the coordinate offset of the pixel region between adjacent sampling times, read the background coordinate offset of the local background under the same frame number, perform item-by-item difference processing on the coordinate offset of the pixel region and the background coordinate offset of the local background, retain the horizontal displacement value, vertical displacement value, frame number, sampling time, and pixel position coordinates after the difference processing, and obtain the relative rigid body displacement parameter;

[0072] The system reads the lateral displacement value, longitudinal displacement value, frame number, sampling time, and pixel position coordinates from the relative rigid body displacement parameters one by one. The lateral displacement value and longitudinal displacement value are compared with the preset rigid body motion tolerance interval boundary. If the lateral displacement value and longitudinal displacement value do not exceed the rigid body motion tolerance interval, the corresponding two-dimensional spatiotemporal coordinate point is extracted. If the lateral displacement value or longitudinal displacement value exceeds the rigid body motion tolerance interval, the corresponding two-dimensional spatiotemporal coordinate point is removed. The remaining two-dimensional spatiotemporal coordinate points are output as coordinate signals according to the frame number and sampling time to establish the hidden heat source location coordinates.

[0073] Specifically, each element in the high-frequency distortion coordinate set is read to obtain its pointed-to pixel position coordinates, frame number, and sampling time. A predetermined pixel region, such as a 5x5 pixel neighborhood, is extracted centered on this pixel position coordinate. Then, the motion of this pixel region is calculated through inter-frame comparison. Specifically, optical flow methods, such as the Lucas-Kanade method, are used to calculate the coordinate offset (lateral and vertical displacement) of this region between the current and previous frames. Simultaneously, to distinguish between real heat source disturbances and camera jitter or minor overall scene movement, the motion of the local background needs to be estimated. A position is selected far from the region pointed to by the high-frequency distortion coordinate set. The reference area, considered as a static background, is used to calculate the background coordinate offset of the local background between the same two frames using the same optical flow method. This yields the horizontal and vertical displacements of the background. Next, motion compensation is performed by subtracting the horizontal displacement of the pixel area from the horizontal displacement of the background, and subtracting the vertical displacement of the pixel area from the vertical displacement of the background. The result of the difference processing is the relative motion of the pixel area relative to its local background. These calculated relative horizontal and vertical displacement values, along with the corresponding frame number, sampling time, and pixel position coordinates, are saved to obtain the relative rigid body displacement parameters.

[0074] The relative rigid body displacement parameters are read item by item. Each item includes the relative lateral displacement value, the longitudinal displacement value, and the corresponding frame number, sampling time, and pixel position coordinates. To ultimately confirm the location of the hidden heat source, false alarms caused by independent moving objects other than heat sources (such as flying insects or fluttering pieces of paper) need to be eliminated. This step is accomplished by judging the amplitude of the relative motion. Specifically, the lateral and longitudinal displacement values ​​in each relative rigid body displacement parameter are compared with a preset rigid body motion tolerance range. The tolerance range is set based on the fact that pixel displacement caused by pure heat waves is usually very small, manifesting as pixel intensity jitter rather than large-scale physical displacement. Therefore, a very small tolerance can be set based on experience. For example, the rigid body motion tolerance range can be set to [-0.5, [0.5] pixels, which means that if the movement of an area relative to the background is less than half a pixel in both the horizontal and vertical directions, it is more likely to be caused by thermal disturbance. Otherwise, it may be the movement of a physical object. When comparing, if the absolute values ​​of the horizontal and vertical displacement values ​​of a parameter are both less than 0.5 (i.e., both fall within the range of [-0.5, 0.5]), the disturbance is considered to originate from non-rigid body motion, which is consistent with the characteristics of a hidden heat source. The corresponding two-dimensional spatiotemporal coordinate points are extracted. If the absolute value of the horizontal or vertical displacement exceeds 0.5, it is judged to be rigid body motion interference and is removed. Finally, all the retained two-dimensional spatiotemporal coordinate points are summarized and output according to the frame number and sampling time to establish the hidden heat source location coordinates.

[0075] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for intelligent detection of smoke and fire in building passageways based on video streams, characterized in that, Includes the following steps: Extract each frame from the video stream input signal of the building passageway, perform complex multi-directional decomposition processing, and generate a set of spatial feature subbands; Extract the complex pixel parameters corresponding to the horizontal and vertical directions of the spatial feature sub-band set, and separate the phase values ​​to obtain the sub-band phase sequence; Extract the component values ​​within a set frequency band range from the sub-band phase sequence to establish the target frequency band phase sequence; The phase component values ​​in the vertical and horizontal directions of the target frequency band phase sequence are multiplied by the amplification factor to generate phase amplification sequences in the vertical and horizontal directions, respectively. Calculate the first-order time-domain derivatives of both to obtain the vertical and horizontal distortion rate parameters, and calculate the amplitude of the generation distortion rate and the spatiotemporal distortion direction angle. Extract elements from the distortion rate amplitude that exceed the set distortion threshold to generate distortion spatiotemporal candidate parameters; Extract parameters whose spatial vector direction falls within the interval of the opposite direction of gravity vector from the candidate parameters of the distorted spatiotemporal space, and establish an upward thermal convection feature sequence; Determine whether the phase fluctuation variance parameter mapped by the upward thermal convection feature sequence exceeds the lower limit threshold value, and filter to generate a high-frequency distortion coordinate set; The relative rigid body displacement parameters of the region pointed to by the high-frequency distortion coordinate set relative to the local background are obtained, and two-dimensional spatiotemporal coordinate points that do not exceed the rigid body motion tolerance range are extracted to establish the location coordinates of the hidden heat source.

2. The intelligent smoke and fire detection method for building passageways based on video stream according to claim 1, characterized in that, The steps for obtaining the spatial feature sub-band set are as follows: Based on the video stream input signal of the building passage, continuous channel images are extracted according to the frame order of the video stream input signal. The frame number, sampling time, pixel position coordinates and pixel grayscale value of each continuous channel image are recorded. For each continuous channel image, complex multi-directional decomposition is performed at the spatial level according to the pixel change direction in the horizontal, vertical and diagonal directions. The complex responses of each direction obtained by decomposition are collected according to the frame number and pixel position coordinates to generate a spatial feature sub-band set.

3. The intelligent smoke and fire detection method for building passageways based on video stream according to claim 1, characterized in that, The steps for obtaining the target frequency band phase sequence are as follows: Extract the complex pixel parameters corresponding to the horizontal direction and the complex pixel parameters in the vertical direction in the spatial feature sub-band set. Read the real part value, imaginary part value, frame number and pixel position coordinates of each complex pixel parameter. Separate the amplitude value and phase value according to the numerical mapping relationship between the real part value and the imaginary part value. Retain the direction identifier, frame number and pixel position coordinates corresponding to the phase value to obtain the sub-band phase sequence. Based on the sub-band phase sequence, the continuous phase change trajectory is arranged according to the frame number corresponding to the phase value. The frequency band belonging identifier corresponding to the continuous phase change trajectory in the time dimension is calculated. The phase component values ​​of the frequency band belonging identifier falling into the 2 Hz to 5 Hz frequency band range are retained, and the phase component values ​​of the frequency band belonging identifier not falling into the 2 Hz to 5 Hz frequency band range are removed. The retained phase component values ​​are rearranged according to the direction identifier, frame number, and pixel position coordinates to establish the target frequency band phase sequence.

4. The intelligent smoke and fire detection method for building passageways based on video stream according to claim 1, characterized in that, The steps for obtaining the spatiotemporal distortion direction angle are as follows: Read the direction identifier, frame number, sampling time and pixel position coordinates corresponding to each phase component value in the target frequency band phase sequence. Arrange the phase component values ​​with the direction identifier in the vertical direction continuously according to the frame number, and arrange the phase component values ​​with the direction identifier in the horizontal direction continuously according to the frame number. Call the vertical direction amplification factor and the horizontal direction amplification factor respectively, and multiply the phase component values ​​under the same frame number and the same pixel position coordinates one by one to generate the vertical direction phase amplification sequence and the horizontal direction phase amplification sequence. Based on the vertical phase amplification sequence and the horizontal phase amplification sequence, extract the phase amplification value, sampling time difference, and pixel position coordinates corresponding to adjacent frame numbers. Divide the change in the vertical phase amplification value between adjacent sampling times by the sampling time difference, and divide the change in the horizontal phase amplification value between adjacent sampling times by the sampling time difference. Retain the frame number corresponding to each change rate according to the same pixel position coordinates to obtain the vertical distortion rate parameter and the horizontal distortion rate parameter. Based on the vertical and horizontal distortion rate parameters, the vertical and horizontal distortion rate values ​​under the same frame number and the same pixel position coordinates are read. The squares of the vertical and horizontal distortion rate values ​​are summed and squared to obtain the distortion rate amplitude. Then, the quadrant to which the distortion direction belongs is determined according to the coordinate axis signs of the horizontal and vertical distortion rate values. The direction angle value corresponding to each distortion rate amplitude is calculated to generate the spatiotemporal distortion direction angle.

5. The intelligent smoke and fire detection method for building passageways based on video stream according to claim 1, characterized in that, The steps for obtaining the candidate parameters of the distorted spatiotemporal spacetime are as follows: The amplitude records are read item by item according to the frame number, sampling time and pixel position coordinates corresponding to the distortion rate amplitude. The distortion rate amplitudes of the same pixel position coordinates at consecutive sampling times are arranged into a time dimension change sequence. The independent amplitude records corresponding to each sampling time in the time dimension change sequence are extracted, and the frame number, sampling time and pixel position coordinates corresponding to the independent amplitude records are retained to obtain discrete variable elements. The distortion rate amplitude, frame number, sampling time, and pixel position coordinates of the discrete variable elements are read one by one. Each distortion rate amplitude is compared with a set distortion threshold. If the distortion rate amplitude is greater than the distortion threshold, the corresponding discrete variable element is retained. If the distortion rate amplitude is less than or equal to the distortion threshold, the corresponding discrete variable element is removed. The retained discrete variable elements are rearranged according to the frame number and pixel position coordinates to generate distortion spatiotemporal candidate parameters.

6. The intelligent smoke and fire detection method for building passageways based on video stream according to claim 1, characterized in that, The steps for obtaining the upward thermal convection feature sequence are as follows: The frame number, sampling time, and pixel position coordinates corresponding to the distorted spatiotemporal candidate parameters are called. The direction angle values ​​corresponding to the same frame number, sampling time, and pixel position coordinates are retrieved in the spatiotemporal distortion direction angle. The direction angle values ​​are compared with the angle boundaries of the gravity anti-direction vector interval. If the direction angle value falls inside the gravity anti-direction vector interval, the corresponding distorted spatiotemporal candidate parameter is extracted. If the direction angle value does not fall inside the gravity anti-direction vector interval, the corresponding distorted spatiotemporal candidate parameter is removed, and an upward thermal convection feature sequence is established.

7. The intelligent smoke and fire detection method for building passageways based on video stream according to claim 1, characterized in that, The steps for obtaining the high-frequency distortion coordinate set are as follows: The frame number, sampling time, pixel position coordinates, and phase component values ​​corresponding to each discrete element within the upward thermal convection feature sequence are read. The phase component values ​​of the same pixel position coordinates at consecutive sampling times are arranged into a phase fluctuation trajectory. The degree of discrete deviation of the phase fluctuation trajectory relative to the phase mean is calculated to form a phase fluctuation variance parameter. The phase fluctuation variance parameter is compared item by item with a set lower limit threshold value. If the phase fluctuation variance parameter is greater than the lower limit threshold value, the corresponding discrete element is retained. If the phase fluctuation variance parameter is less than or equal to the lower limit threshold value, the corresponding discrete element is discarded. The retained discrete elements are arranged according to the frame number, sampling time, and pixel position coordinates to generate a high-frequency distortion coordinate set.

8. The intelligent smoke and fire detection method for building passageways based on video stream according to claim 1, characterized in that, The steps for obtaining the location coordinates of the concealed heat source are as follows: Read the pixel position coordinates, frame number, and sampling time pointed to by the high-frequency distortion coordinate set, extract the pixel region around the pixel position coordinates, extract the coordinate offset of the pixel region between adjacent sampling times, read the background coordinate offset of the local background under the same frame number, perform item-by-item difference processing on the coordinate offset of the pixel region and the background coordinate offset of the local background, retain the horizontal displacement value, vertical displacement value, frame number, sampling time, and pixel position coordinates after the difference processing, and obtain the relative rigid body displacement parameter; The lateral displacement value, longitudinal displacement value, frame number, sampling time, and pixel position coordinates in the relative rigid body displacement parameters are read item by item. The lateral displacement value and longitudinal displacement value are compared with the preset rigid body motion tolerance interval boundary. If the lateral displacement value and longitudinal displacement value do not exceed the rigid body motion tolerance interval, the corresponding two-dimensional spatiotemporal coordinate point is extracted. If the lateral displacement value or longitudinal displacement value exceeds the rigid body motion tolerance interval, the corresponding two-dimensional spatiotemporal coordinate point is discarded. The retained two-dimensional spatiotemporal coordinate points are output as coordinate signals according to the frame number and sampling time to establish the hidden heat source location coordinates.

Images